/brz/remove-bazaar

To get this branch, use:
bzr branch http://gegoxaren.bato24.eu/bzr/brz/remove-bazaar

« back to all changes in this revision

Viewing changes to bzrlib/rules.py

  • Committer: John Arbash Meinel
  • Date: 2009-09-09 18:52:56 UTC
  • mto: (4634.52.16 2.0)
  • mto: This revision was merged to the branch mainline in revision 4738.
  • Revision ID: john@arbash-meinel.com-20090909185256-rdaxy872xauoem46
https://launchpad.net/bugs/402623
Work around bug #402623 by allowing BTreeGraphIndex(...,unlimited_cache=True).

The basic issue is that the access pattern for chk pages is fully random,
because the keys are 'sha1' handles. As such, we have no locality of
reference, and downloading a large project over HTTP can cause us to
redownload all of the .cix pages multiple times. The bug report
noticed the pages getting downloaded 4-5 times.
This was causing a significant increase in the total bytes downloaded.
(For Launchpad, downloading the 10MB cix file 5 times was 50MB, out of
around 160MB total download.)

Show diffs side-by-side

added added

removed removed

Lines of Context:
bzrlib/rules.py
21
21
 
22
22
from bzrlib import (
23
23
    config,
24
 
    cmdline,
 
24
    commands,
25
25
    errors,
26
26
    globbing,
27
27
    osutils,
81
81
        self.pattern_to_section = {}
82
82
        for s in sections:
83
83
            if s.startswith(FILE_PREFS_PREFIX):
84
 
                file_patterns = cmdline.split(s[FILE_PREFS_PREFIX_LEN:])
 
84
                file_patterns = commands.shlex_split_unicode(
 
85
                    s[FILE_PREFS_PREFIX_LEN:])
85
86
                patterns.extend(file_patterns)
86
87
                for fp in file_patterns:
87
88
                    self.pattern_to_section[fp] = s