/brz/remove-bazaar : changes

» Changes from revision 0.22.4

To get this branch, use:

bzr branch
http://gegoxaren.bato24.eu/bzr/brz/remove-bazaar

From Revision 0.22.4 to 0.20.2

expand all

Rev	Summary	Authors	Date
0.22.4	Bring in the missing update from 'trunk' Bring in the missing update from 'trunk'	John Arbash Meinel	16 years ago
0.22.3	Play with some experimental alternate hashes, comment them o... Play with some experimental alternate hashes, comment them out for now.	John Arbash Meinel	16 years ago
0.22.2	experiment with removing the label and sha1 fields. Seems to... experiment with removing the label and sha1 fields. Seems to shrink texts by 10-30%.	John Arbash Meinel	16 years ago
0.22.1	A first-cut at implementing an auto-pack by copying everythi... A first-cut at implementing an auto-pack by copying everything.	John Arbash Meinel	16 years ago
0.20.17	Fix the test suite now that we don't match short lines Fix the test suite now that we don't match short lines	John Arbash Meinel	16 years ago
0.20.16	Adding a 'soft' flag, to make the minimum match 200 bytes Adding a 'soft' flag, to make the minimum match 200 bytes comp time is 9m46s, comp size is improved across the board 11.3MB. So max group 8MB, max inter-file-id 4MB, 'soft' matching with a new file_id gives good compression at equivalent speed.	John Arbash Meinel	16 years ago
0.20.15	Change so that regions that have lots of copies get converte... Change so that regions that have lots of copies get converted back into an insertion. This does get triggered, but it doesn't help. The total compression is 17MB, and the conversion time is 10min. Which is equivalent to the original values. Even further, don't match blocks that are shorter than XX bytes (currently 10). With a value of 5, we still get trivial blocks inserted. With a value of 10, everything changes to copies. Dropping the max block size to 8MB decreases the total bytes to 14MB (presumably because the copy records now have 1 fewer byte per record). It also makes it 9m versus 10m. Preferentially splitting based on file-id (at >= 4MB) stays at 9min, but drops it to 13MB	John Arbash Meinel	16 years ago
0.20.14	Factor out _get_group_and_delta_lines. Factor out _get_group_and_delta_lines. The previous change (to ignore empty texts, and start new compressors) dropped the conversion time to 11m43s at a modest expansion to 13.4MB. The time difference is surprising, we should check if it is the no-newlines or the new-compressors. (my guess is the latter).	John Arbash Meinel	16 years ago
0.20.13	Play around a bit. Play around a bit. 1) Empty texts are no-op inserted, to avoid ever trying to match against their text. 2) If we find a new file-id and the compressor is more than half full, we go ahead and start a new compressor.	John Arbash Meinel	16 years ago
0.20.12	Change the code a little bit. Change the code a little bit. If a given text has not been seen before, insert all lines for that text. At present, we are doing worse than knit compression, because we have so many matching groups from various locations. Which causes us to just have huge swaths of copies. By inserting the full lines, we get more regions that we are able to generate a larger match against. This slows down the processing (10m => 24m), but improves compression (16MB => 12MB).	John Arbash Meinel	16 years ago
0.20.11	start experimenting with gc-optimal ordering. start experimenting with gc-optimal ordering.	John Arbash Meinel	16 years ago
0.20.10	Change the extraction ordering for 'unordered'. Change the extraction ordering for 'unordered'. Instead of using a random ordering, use the ordering defined by the index memos. This should give us the best group-locality. This gives a rather large performance improvement. Like 30s versus 10min.	John Arbash Meinel	16 years ago
0.20.9	Revert previous change. Revert previous change.	John Arbash Meinel	16 years ago
0.20.8	Allow writing negative offsets. Turns out not to actually co... Allow writing negative offsets. Turns out not to actually compress better. After zlib compression, negative offsets are a loss. Presumably because there is redundancy that zlib can factor out from bytes-since-start.	John Arbash Meinel	16 years ago
0.20.7	(ugly hack) autopacking doesn't work, so don't do it. (ugly hack) autopacking doesn't work, so don't do it. Force the fetch order and delta logic to use fulltexts in topological order. It isn't great, but it means things work.	John Arbash Meinel	16 years ago
0.20.6	Merge in the dev5 formats. Merge in the dev5 formats.	John Arbash Meinel	16 years ago
0.20.5	Finish the Fulltext => Chunked conversions so that we work i... Finish the Fulltext => Chunked conversions so that we work in the more-efficient Chunks.	John Arbash Meinel	16 years ago
0.20.4	Simplify the internals. We've already checked 'chk_support' ... Simplify the internals. We've already checked 'chk_support' so we don't need to check again.	John Arbash Meinel	17 years ago
0.20.3	Properly add GCPlainCHK to the pack_incompatible list. Properly add GCPlainCHK to the pack_incompatible list.	John Arbash Meinel	17 years ago
0.20.2	Teach groupcompress about 'chunked' encoding Teach groupcompress about 'chunked' encoding	John Arbash Meinel	17 years ago

Older »