/brz/remove-bazaar

To get this branch, use:
bzr branch http://gegoxaren.bato24.eu/bzr/brz/remove-bazaar

« back to all changes in this revision

Viewing changes to bzrlib/groupcompress.py

  • Committer: John Arbash Meinel
  • Date: 2009-11-06 18:05:03 UTC
  • mto: This revision was merged to the branch mainline in revision 4814.
  • Revision ID: john@arbash-meinel.com-20091106180503-q3hcg1nxmut5b2cb
Stop holding the gil while extracting data.

Previous experience has shown that we don't really spend a lot
of time here, but there isn't any reason to hold the lock while
we do it anyway.

Performance testing has shown that for bzr operations 'with nogil'
doesn't really do much, which was generally expected. Any variation
observed is within the noise margins.

This is mostly about 'the right thing to do', and potential
future work where we may actually use threads.

Show diffs side-by-side

added added

removed removed

Lines of Context:
bzrlib/groupcompress.py
1
 
# Copyright (C) 2008, 2009, 2010 Canonical Ltd
 
1
# Copyright (C) 2008, 2009 Canonical Ltd
2
2
#
3
3
# This program is free software; you can redistribute it and/or modify
4
4
# it under the terms of the GNU General Public License as published by
31
31
    knit,
32
32
    osutils,
33
33
    pack,
34
 
    static_tuple,
35
34
    trace,
36
35
    )
37
36
from bzrlib.btree_index import BTreeBuilder
1631
1630
        keys_to_add = []
1632
1631
        def flush():
1633
1632
            bytes = self._compressor.flush().to_bytes()
1634
 
            self._compressor = GroupCompressor()
1635
1633
            index, start, length = self._access.add_raw_records(
1636
1634
                [(None, len(bytes))], bytes)[0]
1637
1635
            nodes = []
1640
1638
            self._index.add_records(nodes, random_id=random_id)
1641
1639
            self._unadded_refs = {}
1642
1640
            del keys_to_add[:]
 
1641
            self._compressor = GroupCompressor()
1643
1642
 
1644
1643
        last_prefix = None
1645
1644
        max_fulltext_len = 0
1747
1746
                key = record.key
1748
1747
            self._unadded_refs[key] = record.parents
1749
1748
            yield found_sha1
1750
 
            as_st = static_tuple.StaticTuple.from_sequence
1751
 
            if record.parents is not None:
1752
 
                parents = as_st([as_st(p) for p in record.parents])
1753
 
            else:
1754
 
                parents = None
1755
 
            refs = static_tuple.StaticTuple(parents)
1756
 
            keys_to_add.append((key, '%d %d' % (start_point, end_point), refs))
 
1749
            keys_to_add.append((key, '%d %d' % (start_point, end_point),
 
1750
                (record.parents,)))
1757
1751
        if len(keys_to_add):
1758
1752
            flush()
1759
1753
        self._compressor = None
1883
1877
        if not random_id:
1884
1878
            present_nodes = self._get_entries(keys)
1885
1879
            for (index, key, value, node_refs) in present_nodes:
1886
 
                # Sometimes these are passed as a list rather than a tuple
1887
 
                node_refs = static_tuple.as_tuples(node_refs)
1888
 
                passed = static_tuple.as_tuples(keys[key])
1889
 
                if node_refs != passed[1]:
1890
 
                    details = '%s %s %s' % (key, (value, node_refs), passed)
 
1880
                if node_refs != keys[key][1]:
 
1881
                    details = '%s %s %s' % (key, (value, node_refs), keys[key])
1891
1882
                    if self._inconsistency_fatal:
1892
1883
                        raise errors.KnitCorrupt(self, "inconsistent details"
1893
1884
                                                 " in add_records: %s" %