/brz/remove-bazaar

To get this branch, use:
bzr branch http://gegoxaren.bato24.eu/bzr/brz/remove-bazaar

« back to all changes in this revision

Viewing changes to bzrlib/tuned_gzip.py

  • Committer: John Arbash Meinel
  • Date: 2009-06-01 18:13:46 UTC
  • mto: (4360.4.11 1.15-pack-source)
  • mto: This revision was merged to the branch mainline in revision 4396.
  • Revision ID: john@arbash-meinel.com-20090601181346-2fxsd3o977j5bj5b
Add tests that ensure we can fetch branches with ghosts in their ancestry.

Also added similar tests when stacking is involved.
Then fixed both the StreamSource and GroupCHKStreamSource to handle these cases.
Andrew's fix didn't work in the case of Stacked, as it only worked if the
initial fetch created a fully complete target. Not if there was a ghost
involved with the transmitted revisions, and stacking was also involved.

Basic fix is just to allow absent records during 'get_stream_for_missing_keys',
the StreamSink is then responsible for ensuring no content is actually missing.

Show diffs side-by-side

added added

removed removed

Lines of Context:
bzrlib/tuned_gzip.py
52
52
    width=-zlib.MAX_WBITS, mem=zlib.DEF_MEM_LEVEL,
53
53
    crc32=zlib.crc32):
54
54
    """Create a gzip file containing bytes and return its content."""
55
 
    return chunks_to_gzip([bytes])
56
 
 
57
 
 
58
 
def chunks_to_gzip(chunks, factory=zlib.compressobj,
59
 
    level=zlib.Z_DEFAULT_COMPRESSION, method=zlib.DEFLATED,
60
 
    width=-zlib.MAX_WBITS, mem=zlib.DEF_MEM_LEVEL,
61
 
    crc32=zlib.crc32):
62
 
    """Create a gzip file containing chunks and return its content.
63
 
 
64
 
    :param chunks: An iterable of strings. Each string can have arbitrary
65
 
        layout.
66
 
    """
67
55
    result = [
68
56
        '\037\213'  # self.fileobj.write('\037\213')  # magic header
69
57
        '\010'      # self.fileobj.write('\010')      # compression method
81
69
    # using a compressobj avoids a small header and trailer that the compress()
82
70
    # utility function adds.
83
71
    compress = factory(level, method, width, mem, 0)
84
 
    crc = 0
85
 
    total_len = 0
86
 
    for chunk in chunks:
87
 
        crc = crc32(chunk, crc)
88
 
        total_len += len(chunk)
89
 
        zbytes = compress.compress(chunk)
90
 
        if zbytes:
91
 
            result.append(zbytes)
 
72
    result.append(compress.compress(bytes))
92
73
    result.append(compress.flush())
 
74
    result.append(struct.pack("<L", LOWU32(crc32(bytes))))
93
75
    # size may exceed 2GB, or even 4GB
94
 
    result.append(struct.pack("<LL", LOWU32(crc), LOWU32(total_len)))
 
76
    result.append(struct.pack("<L", LOWU32(len(bytes))))
95
77
    return ''.join(result)
96
78
 
97
79