/brz/remove-bazaar : changes

» Changes from revision 0.20.26

To get this branch, use:

bzr branch
http://gegoxaren.bato24.eu/bzr/brz/remove-bazaar

From Revision 0.20.26 to 0.20.7

expand all

Rev	Summary	Authors	Date
0.20.26	Try even harder, now with even more streams. Try even harder, now with even more streams. The compressed size drops by another 4x. Turn the data for each layer into a different stream. With this change, gc255 has compressed inventory drop to 1.5MB which is finally smaller than the source 'knit' format.	John Arbash Meinel	16 years ago
0.20.25	As expected, splitting things up into streams of streams As expected, splitting things up into streams of streams gives even better compression. (Down to 4.4MB for inventories). Probably the big win is that parent_id_basename content doesn't compress well at all versus id_to_entry content, and this way you don't get large offsets.	John Arbash Meinel	16 years ago
0.20.24	Add a general progress indicator for other parts of copy. Add a general progress indicator for other parts of copy.	John Arbash Meinel	16 years ago
0.20.23	Add a progress indicator for chk pages. Add a progress indicator for chk pages. Fix a bug with handling signatures, which don't have a parent graph	John Arbash Meinel	16 years ago
0.20.22	Make it clear that the bits you get from 'apply_delta' are c... Make it clear that the bits you get from 'apply_delta' are chunks, not lines.	John Arbash Meinel	16 years ago
0.20.21	Merge the chk sorting code. Merge the chk sorting code. Restore labels and sha1s in the stored data. Leave the 'extra' formats commented out for now.	John Arbash Meinel	16 years ago
0.20.20	Setting _fetch_order='topological' gives sub-optimal orderin... Setting _fetch_order='topological' gives sub-optimal ordering for gc=>gc fetches. This is because the 'autopack' code will convert to 'gc-optimal', which means that 'unordered' will then continue the 'gc-optimal' route.	John Arbash Meinel	16 years ago
0.20.19	Groupcompress now supports 'autopack' and 'pack'. Groupcompress now supports 'autopack' and 'pack'. It does this by just creating a new pack file, wrapping a GCVersionedFiles around it, and streaming in the data in 'gc-optimal' ordering. This actually seems to work fairly well.	John Arbash Meinel	16 years ago
0.20.18	Implement new handling of get_bytes_as(), and get_missing_co... Implement new handling of get_bytes_as(), and get_missing_compression_parent_keys() Now works on bzr.dev's new streaming code.	John Arbash Meinel	16 years ago
0.20.17	Fix the test suite now that we don't match short lines Fix the test suite now that we don't match short lines	John Arbash Meinel	16 years ago
0.20.16	Adding a 'soft' flag, to make the minimum match 200 bytes Adding a 'soft' flag, to make the minimum match 200 bytes comp time is 9m46s, comp size is improved across the board 11.3MB. So max group 8MB, max inter-file-id 4MB, 'soft' matching with a new file_id gives good compression at equivalent speed.	John Arbash Meinel	16 years ago
0.20.15	Change so that regions that have lots of copies get converte... Change so that regions that have lots of copies get converted back into an insertion. This does get triggered, but it doesn't help. The total compression is 17MB, and the conversion time is 10min. Which is equivalent to the original values. Even further, don't match blocks that are shorter than XX bytes (currently 10). With a value of 5, we still get trivial blocks inserted. With a value of 10, everything changes to copies. Dropping the max block size to 8MB decreases the total bytes to 14MB (presumably because the copy records now have 1 fewer byte per record). It also makes it 9m versus 10m. Preferentially splitting based on file-id (at >= 4MB) stays at 9min, but drops it to 13MB	John Arbash Meinel	16 years ago
0.20.14	Factor out _get_group_and_delta_lines. Factor out _get_group_and_delta_lines. The previous change (to ignore empty texts, and start new compressors) dropped the conversion time to 11m43s at a modest expansion to 13.4MB. The time difference is surprising, we should check if it is the no-newlines or the new-compressors. (my guess is the latter).	John Arbash Meinel	16 years ago
0.20.13	Play around a bit. Play around a bit. 1) Empty texts are no-op inserted, to avoid ever trying to match against their text. 2) If we find a new file-id and the compressor is more than half full, we go ahead and start a new compressor.	John Arbash Meinel	16 years ago
0.20.12	Change the code a little bit. Change the code a little bit. If a given text has not been seen before, insert all lines for that text. At present, we are doing worse than knit compression, because we have so many matching groups from various locations. Which causes us to just have huge swaths of copies. By inserting the full lines, we get more regions that we are able to generate a larger match against. This slows down the processing (10m => 24m), but improves compression (16MB => 12MB).	John Arbash Meinel	16 years ago
0.20.11	start experimenting with gc-optimal ordering. start experimenting with gc-optimal ordering.	John Arbash Meinel	16 years ago
0.20.10	Change the extraction ordering for 'unordered'. Change the extraction ordering for 'unordered'. Instead of using a random ordering, use the ordering defined by the index memos. This should give us the best group-locality. This gives a rather large performance improvement. Like 30s versus 10min.	John Arbash Meinel	16 years ago
0.20.9	Revert previous change. Revert previous change.	John Arbash Meinel	17 years ago
0.20.8	Allow writing negative offsets. Turns out not to actually co... Allow writing negative offsets. Turns out not to actually compress better. After zlib compression, negative offsets are a loss. Presumably because there is redundancy that zlib can factor out from bytes-since-start.	John Arbash Meinel	17 years ago
0.20.7	(ugly hack) autopacking doesn't work, so don't do it. (ugly hack) autopacking doesn't work, so don't do it. Force the fetch order and delta logic to use fulltexts in topological order. It isn't great, but it means things work.	John Arbash Meinel	17 years ago

Older »