/brz/remove-bazaar

To get this branch, use:
bzr branch http://gegoxaren.bato24.eu/bzr/brz/remove-bazaar

« back to all changes in this revision

Viewing changes to bzrlib/bencode.py

  • Committer: John Arbash Meinel
  • Date: 2009-09-09 18:52:56 UTC
  • mto: (4634.52.16 2.0)
  • mto: This revision was merged to the branch mainline in revision 4738.
  • Revision ID: john@arbash-meinel.com-20090909185256-rdaxy872xauoem46
https://launchpad.net/bugs/402623
Work around bug #402623 by allowing BTreeGraphIndex(...,unlimited_cache=True).

The basic issue is that the access pattern for chk pages is fully random,
because the keys are 'sha1' handles. As such, we have no locality of
reference, and downloading a large project over HTTP can cause us to
redownload all of the .cix pages multiple times. The bug report
noticed the pages getting downloaded 4-5 times.
This was causing a significant increase in the total bytes downloaded.
(For Launchpad, downloading the 10MB cix file 5 times was 50MB, out of
around 160MB total download.)

Show diffs side-by-side

added added

removed removed

Lines of Context:
bzrlib/bencode.py
16
16
 
17
17
"""Wrapper around the bencode pyrex and python implementation"""
18
18
 
19
 
from bzrlib import osutils
20
 
 
21
19
try:
22
20
    from bzrlib._bencode_pyx import bdecode, bdecode_as_tuple, bencode, Bencached
23
 
except ImportError, e:
24
 
    osutils.failed_to_load_extension(e)
 
21
except ImportError:
25
22
    from bzrlib.util._bencode_py import bdecode, bdecode_as_tuple, bencode, Bencached