bzr branch
http://gegoxaren.bato24.eu/bzr/brz/remove-bazaar
| 
974.1.27
by aaron.bentley at utoronto
 Initial greedy fetch work  | 
1  | 
# Copyright (C) 2005 by Canonical Ltd
 | 
2  | 
||
3  | 
# This program is free software; you can redistribute it and/or modify
 | 
|
4  | 
# it under the terms of the GNU General Public License as published by
 | 
|
5  | 
# the Free Software Foundation; either version 2 of the License, or
 | 
|
6  | 
# (at your option) any later version.
 | 
|
7  | 
||
8  | 
# This program is distributed in the hope that it will be useful,
 | 
|
9  | 
# but WITHOUT ANY WARRANTY; without even the implied warranty of
 | 
|
10  | 
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 | 
|
11  | 
# GNU General Public License for more details.
 | 
|
12  | 
||
13  | 
# You should have received a copy of the GNU General Public License
 | 
|
14  | 
# along with this program; if not, write to the Free Software
 | 
|
15  | 
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 | 
|
| 
1218
by Martin Pool
 - fix up import  | 
16  | 
|
| 
1231
by Martin Pool
 - more progress on fetch on top of weaves  | 
17  | 
|
18  | 
"""Copying of history from one branch to another.
 | 
|
19  | 
||
20  | 
The basic plan is that every branch knows the history of everything
 | 
|
21  | 
that has merged into it.  As the first step of a merge, pull, or
 | 
|
22  | 
branch operation we copy history from the source into the destination
 | 
|
23  | 
branch.
 | 
|
24  | 
||
25  | 
The copying is done in a slightly complicated order.  We don't want to
 | 
|
26  | 
add a revision to the store until everything it refers to is also
 | 
|
27  | 
stored, so that if a revision is present we can totally recreate it.
 | 
|
28  | 
However, we can't know what files are included in a revision until we
 | 
|
29  | 
read its inventory.  Therefore, we first pull the XML and hold it in
 | 
|
30  | 
memory until we've updated all of the files referenced.
 | 
|
31  | 
"""
 | 
|
32  | 
||
| 
1534.1.31
by Robert Collins
 Deprecated fetch.fetch and fetch.greedy_fetch for branch.fetch, and move the Repository.fetch internals to InterRepo and InterWeaveRepo.  | 
33  | 
import bzrlib  | 
34  | 
import bzrlib.errors as errors  | 
|
35  | 
from bzrlib.errors import (InstallFailed, NoSuchRevision, WeaveError,  | 
|
36  | 
MissingText)  | 
|
37  | 
from bzrlib.trace import mutter  | 
|
38  | 
from bzrlib.progress import ProgressBar  | 
|
39  | 
from bzrlib.revision import NULL_REVISION  | 
|
40  | 
from bzrlib.symbol_versioning import *  | 
|
41  | 
||
42  | 
||
| 
1231
by Martin Pool
 - more progress on fetch on top of weaves  | 
43  | 
# TODO: Avoid repeatedly opening weaves so many times.
 | 
| 
974.1.27
by aaron.bentley at utoronto
 Initial greedy fetch work  | 
44  | 
|
| 
1238
by Martin Pool
 - remove a lot of dead code from fetch  | 
45  | 
# XXX: This doesn't handle ghost (not present in branch) revisions at
 | 
| 
1240
by Martin Pool
 - clean up fetch code and add progress bar  | 
46  | 
# all yet.  I'm not sure they really should be supported.
 | 
47  | 
||
| 
1262
by Martin Pool
 - fetch should also copy ancestry records  | 
48  | 
# NOTE: This doesn't copy revisions which may be present but not
 | 
49  | 
# merged into the last revision.  I'm not sure we want to do that.
 | 
|
| 
1238
by Martin Pool
 - remove a lot of dead code from fetch  | 
50  | 
|
51  | 
# - get a list of revisions that need to be pulled in
 | 
|
52  | 
# - for each one, pull in that revision file
 | 
|
53  | 
#   and get the inventory, and store the inventory with right
 | 
|
54  | 
#   parents.
 | 
|
55  | 
# - and get the ancestry, and store that with right parents too
 | 
|
56  | 
# - and keep a note of all file ids and version seen
 | 
|
57  | 
# - then go through all files; for each one get the weave,
 | 
|
58  | 
#   and add in all file versions
 | 
|
59  | 
||
60  | 
||
| 
1534.1.31
by Robert Collins
 Deprecated fetch.fetch and fetch.greedy_fetch for branch.fetch, and move the Repository.fetch internals to InterRepo and InterWeaveRepo.  | 
61  | 
@deprecated_function(zero_eight)  | 
| 
1260
by Martin Pool
 - some updates for fetch/update function  | 
62  | 
def greedy_fetch(to_branch, from_branch, revision=None, pb=None):  | 
| 
1534.1.32
by Robert Collins
 Deprecate direct use of fetch.Fetcher.  | 
63  | 
"""Legacy API, please see branch.fetch(from_branch, last_revision, pb)."""  | 
| 
1219
by Martin Pool
 - BROKEN: start refactoring fetch code to work well with weaves  | 
64  | 
f = Fetcher(to_branch, from_branch, revision, pb)  | 
65  | 
return f.count_copied, f.failed_revisions  | 
|
66  | 
||
| 
1534.1.32
by Robert Collins
 Deprecate direct use of fetch.Fetcher.  | 
67  | 
fetch = greedy_fetch  | 
68  | 
||
| 
1219
by Martin Pool
 - BROKEN: start refactoring fetch code to work well with weaves  | 
69  | 
|
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
70  | 
class RepoFetcher(object):  | 
71  | 
"""Pull revisions and texts from one repository to another.  | 
|
72  | 
||
73  | 
    last_revision
 | 
|
74  | 
        if set, try to limit to the data this revision references.
 | 
|
75  | 
||
76  | 
    after running:
 | 
|
| 
1260
by Martin Pool
 - some updates for fetch/update function  | 
77  | 
    count_copied -- number of revisions copied
 | 
| 
1534.1.33
by Robert Collins
 Move copy_content_into into InterRepository and InterWeaveRepo, and disable the default codepath test as we have optimised paths for all current combinations.  | 
78  | 
|
79  | 
    This should not be used directory, its essential a object to encapsulate
 | 
|
80  | 
    the logic in InterRepository.fetch().
 | 
|
| 
1260
by Martin Pool
 - some updates for fetch/update function  | 
81  | 
    """
 | 
| 
1534.4.50
by Robert Collins
 Got the bzrdir api straightened out, plenty of refactoring to use it pending, but the api is up and running.  | 
82  | 
def __init__(self, to_repository, from_repository, last_revision=None, pb=None):  | 
| 
1534.1.31
by Robert Collins
 Deprecated fetch.fetch and fetch.greedy_fetch for branch.fetch, and move the Repository.fetch internals to InterRepo and InterWeaveRepo.  | 
83  | 
        # result variables.
 | 
84  | 
self.failed_revisions = []  | 
|
85  | 
self.count_copied = 0  | 
|
| 
1556.1.4
by Robert Collins
 Add a new format for what will become knit, and the surrounding logic to upgrade repositories within metadirs, and tests for the same.  | 
86  | 
if to_repository.control_files._transport.base == from_repository.control_files._transport.base:  | 
| 
1534.6.13
by Robert Collins
 Allow push/pull and branch between branches in the same shared repository.  | 
87  | 
            # check that last_revision is in 'from' and then return a no-operation.
 | 
88  | 
if last_revision not in (None, NULL_REVISION):  | 
|
89  | 
from_repository.get_revision(last_revision)  | 
|
90  | 
            return
 | 
|
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
91  | 
self.to_repository = to_repository  | 
92  | 
self.from_repository = from_repository  | 
|
93  | 
        # must not mutate self._last_revision as its potentially a shared instance
 | 
|
| 
1185.65.27
by Robert Collins
 Tweak storage towards mergability.  | 
94  | 
self._last_revision = last_revision  | 
95  | 
if pb is None:  | 
|
96  | 
self.pb = bzrlib.ui.ui_factory.progress_bar()  | 
|
97  | 
else:  | 
|
98  | 
self.pb = pb  | 
|
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
99  | 
self.from_repository.lock_read()  | 
| 
1185.65.27
by Robert Collins
 Tweak storage towards mergability.  | 
100  | 
try:  | 
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
101  | 
self.to_repository.lock_write()  | 
| 
1185.65.27
by Robert Collins
 Tweak storage towards mergability.  | 
102  | 
try:  | 
103  | 
self.__fetch()  | 
|
104  | 
finally:  | 
|
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
105  | 
self.to_repository.unlock()  | 
| 
1185.65.27
by Robert Collins
 Tweak storage towards mergability.  | 
106  | 
finally:  | 
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
107  | 
self.from_repository.unlock()  | 
| 
1185.65.27
by Robert Collins
 Tweak storage towards mergability.  | 
108  | 
|
109  | 
def __fetch(self):  | 
|
110  | 
"""Primary worker function.  | 
|
111  | 
||
112  | 
        This initialises all the needed variables, and then fetches the 
 | 
|
113  | 
        requested revisions, finally clearing the progress bar.
 | 
|
114  | 
        """
 | 
|
| 
1185.67.2
by Aaron Bentley
 Renamed Branch.storage to Branch.repository  | 
115  | 
self.to_weaves = self.to_repository.weave_store  | 
116  | 
self.to_control = self.to_repository.control_weaves  | 
|
117  | 
self.from_weaves = self.from_repository.weave_store  | 
|
118  | 
self.from_control = self.from_repository.control_weaves  | 
|
| 
1240
by Martin Pool
 - clean up fetch code and add progress bar  | 
119  | 
self.count_total = 0  | 
| 
1185.33.55
by Martin Pool
 [patch] weave fetch optimizations (Goffredo Baroncelli)  | 
120  | 
self.file_ids_names = {}  | 
| 
1392
by Robert Collins
 reinstate testfetch test case  | 
121  | 
try:  | 
| 
1185.65.30
by Robert Collins
 Merge integration.  | 
122  | 
revs = self._revids_to_fetch()  | 
| 
1185.64.3
by Goffredo Baroncelli
 This patch changes the fetch code. Before, the original code expanded every inventory and  | 
123  | 
            # nothing to do
 | 
124  | 
if revs:  | 
|
| 
1534.1.11
by Robert Collins
 Format tweaks for latest merges.  | 
125  | 
self._fetch_weave_texts(revs)  | 
126  | 
self._fetch_inventory_weave(revs)  | 
|
127  | 
self._fetch_revision_texts(revs)  | 
|
| 
1185.64.3
by Goffredo Baroncelli
 This patch changes the fetch code. Before, the original code expanded every inventory and  | 
128  | 
self.count_copied += len(revs)  | 
| 
1185.12.15
by Aaron Bentley
 Cleared progress bar properly after fetch  | 
129  | 
finally:  | 
130  | 
self.pb.clear()  | 
|
| 
1231
by Martin Pool
 - more progress on fetch on top of weaves  | 
131  | 
|
| 
1185.65.30
by Robert Collins
 Merge integration.  | 
132  | 
def _revids_to_fetch(self):  | 
| 
1534.1.31
by Robert Collins
 Deprecated fetch.fetch and fetch.greedy_fetch for branch.fetch, and move the Repository.fetch internals to InterRepo and InterWeaveRepo.  | 
133  | 
self.pb.update('get destination history')  | 
| 
1185.65.27
by Robert Collins
 Tweak storage towards mergability.  | 
134  | 
mutter('fetch up to rev {%s}', self._last_revision)  | 
| 
1534.4.50
by Robert Collins
 Got the bzrdir api straightened out, plenty of refactoring to use it pending, but the api is up and running.  | 
135  | 
if self._last_revision is NULL_REVISION:  | 
136  | 
            # explicit limit of no revisions needed
 | 
|
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
137  | 
return None  | 
| 
1534.4.50
by Robert Collins
 Got the bzrdir api straightened out, plenty of refactoring to use it pending, but the api is up and running.  | 
138  | 
if (self._last_revision != None and  | 
| 
1185.65.27
by Robert Collins
 Tweak storage towards mergability.  | 
139  | 
self.to_repository.has_revision(self._last_revision)):  | 
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
140  | 
return None  | 
141  | 
||
| 
1417.1.13
by Robert Collins
 do not download remote ancestry.weave if the target revision we are stopping at is in our local store  | 
142  | 
try:  | 
| 
1534.4.50
by Robert Collins
 Got the bzrdir api straightened out, plenty of refactoring to use it pending, but the api is up and running.  | 
143  | 
return self.to_repository.missing_revision_ids(self.from_repository,  | 
144  | 
self._last_revision)  | 
|
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
145  | 
except errors.NoSuchRevision:  | 
| 
1185.65.27
by Robert Collins
 Tweak storage towards mergability.  | 
146  | 
raise InstallFailed([self._last_revision])  | 
| 
1185.64.3
by Goffredo Baroncelli
 This patch changes the fetch code. Before, the original code expanded every inventory and  | 
147  | 
|
| 
1534.1.11
by Robert Collins
 Format tweaks for latest merges.  | 
148  | 
def _fetch_revision_texts(self, revs):  | 
| 
1185.65.30
by Robert Collins
 Merge integration.  | 
149  | 
self.to_repository.revision_store.copy_multi(  | 
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
150  | 
self.from_repository.revision_store,  | 
151  | 
revs,  | 
|
152  | 
pb=self.pb)  | 
|
| 
1185.64.3
by Goffredo Baroncelli
 This patch changes the fetch code. Before, the original code expanded every inventory and  | 
153  | 
|
| 
1534.1.11
by Robert Collins
 Format tweaks for latest merges.  | 
154  | 
def _fetch_weave_texts(self, revs):  | 
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
155  | 
file_ids = self.from_repository.fileid_involved_by_set(revs)  | 
| 
1185.64.3
by Goffredo Baroncelli
 This patch changes the fetch code. Before, the original code expanded every inventory and  | 
156  | 
count = 0  | 
157  | 
num_file_ids = len(file_ids)  | 
|
158  | 
for file_id in file_ids:  | 
|
| 
1185.77.1
by John Arbash Meinel
 Simplifying what bzr displays when merging  | 
159  | 
self.pb.update("merge weaves", count, num_file_ids)  | 
| 
1185.64.3
by Goffredo Baroncelli
 This patch changes the fetch code. Before, the original code expanded every inventory and  | 
160  | 
count +=1  | 
161  | 
to_weave = self.to_weaves.get_weave_or_empty(file_id,  | 
|
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
162  | 
self.to_repository.get_transaction())  | 
| 
1185.64.3
by Goffredo Baroncelli
 This patch changes the fetch code. Before, the original code expanded every inventory and  | 
163  | 
from_weave = self.from_weaves.get_weave(file_id,  | 
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
164  | 
self.from_repository.get_transaction())  | 
| 
1185.64.3
by Goffredo Baroncelli
 This patch changes the fetch code. Before, the original code expanded every inventory and  | 
165  | 
|
166  | 
if to_weave.numversions() > 0:  | 
|
167  | 
                # destination has contents, must merge
 | 
|
168  | 
try:  | 
|
169  | 
to_weave.join(from_weave)  | 
|
170  | 
except errors.WeaveParentMismatch:  | 
|
171  | 
to_weave.reweave(from_weave)  | 
|
172  | 
else:  | 
|
173  | 
                # destination is empty, just replace it
 | 
|
| 
1534.1.11
by Robert Collins
 Format tweaks for latest merges.  | 
174  | 
to_weave = from_weave.copy()  | 
| 
1185.64.3
by Goffredo Baroncelli
 This patch changes the fetch code. Before, the original code expanded every inventory and  | 
175  | 
|
176  | 
self.to_weaves.put_weave(file_id, to_weave,  | 
|
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
177  | 
self.to_repository.get_transaction())  | 
| 
1534.1.11
by Robert Collins
 Format tweaks for latest merges.  | 
178  | 
self.pb.clear()  | 
| 
1185.64.3
by Goffredo Baroncelli
 This patch changes the fetch code. Before, the original code expanded every inventory and  | 
179  | 
|
| 
1534.1.11
by Robert Collins
 Format tweaks for latest merges.  | 
180  | 
def _fetch_inventory_weave(self, revs):  | 
| 
1551.3.11
by Aaron Bentley
 Merge from Robert  | 
181  | 
self.pb.update("inventory fetch", 0, 2)  | 
| 
1185.65.30
by Robert Collins
 Merge integration.  | 
182  | 
from_weave = self.from_repository.get_inventory_weave()  | 
183  | 
to_weave = self.to_repository.get_inventory_weave()  | 
|
| 
1551.3.11
by Aaron Bentley
 Merge from Robert  | 
184  | 
self.pb.update("inventory fetch", 1, 2)  | 
185  | 
to_weave = self.to_control.get_weave('inventory',  | 
|
| 
1534.4.43
by Robert Collins
 Merge from integration  | 
186  | 
self.to_repository.get_transaction())  | 
| 
1551.3.11
by Aaron Bentley
 Merge from Robert  | 
187  | 
self.pb.update("inventory fetch", 2, 2)  | 
| 
1185.64.4
by Goffredo Baroncelli
 Apply some style changes on the basis of Robert suggest  | 
188  | 
|
| 
1185.64.3
by Goffredo Baroncelli
 This patch changes the fetch code. Before, the original code expanded every inventory and  | 
189  | 
if to_weave.numversions() > 0:  | 
190  | 
            # destination has contents, must merge
 | 
|
191  | 
try:  | 
|
| 
1551.3.11
by Aaron Bentley
 Merge from Robert  | 
192  | 
to_weave.join(from_weave, pb=self.pb, msg='merge inventory')  | 
| 
1185.64.3
by Goffredo Baroncelli
 This patch changes the fetch code. Before, the original code expanded every inventory and  | 
193  | 
except errors.WeaveParentMismatch:  | 
| 
1551.3.11
by Aaron Bentley
 Merge from Robert  | 
194  | 
to_weave.reweave(from_weave, pb=self.pb, msg='reweave inventory')  | 
| 
1185.64.3
by Goffredo Baroncelli
 This patch changes the fetch code. Before, the original code expanded every inventory and  | 
195  | 
else:  | 
196  | 
            # destination is empty, just replace it
 | 
|
| 
1534.1.11
by Robert Collins
 Format tweaks for latest merges.  | 
197  | 
to_weave = from_weave.copy()  | 
| 
1185.64.3
by Goffredo Baroncelli
 This patch changes the fetch code. Before, the original code expanded every inventory and  | 
198  | 
|
199  | 
self.to_control.put_weave('inventory', to_weave,  | 
|
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
200  | 
self.to_repository.get_transaction())  | 
| 
1185.64.4
by Goffredo Baroncelli
 Apply some style changes on the basis of Robert suggest  | 
201  | 
|
| 
1534.1.11
by Robert Collins
 Format tweaks for latest merges.  | 
202  | 
self.pb.clear()  | 
| 
1240
by Martin Pool
 - clean up fetch code and add progress bar  | 
203  | 
|
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
204  | 
|
205  | 
class Fetcher(object):  | 
|
| 
1534.1.32
by Robert Collins
 Deprecate direct use of fetch.Fetcher.  | 
206  | 
"""Backwards compatability glue for branch.fetch()."""  | 
207  | 
||
208  | 
@deprecated_method(zero_eight)  | 
|
| 
1534.4.41
by Robert Collins
 Branch now uses BzrDir reasonably sanely.  | 
209  | 
def __init__(self, to_branch, from_branch, last_revision=None, pb=None):  | 
| 
1534.1.32
by Robert Collins
 Deprecate direct use of fetch.Fetcher.  | 
210  | 
"""Please see branch.fetch()."""  | 
211  | 
to_branch.fetch(from_branch, last_revision, pb)  |