/brz/remove-bazaar

To get this branch, use:
bzr branch http://gegoxaren.bato24.eu/bzr/brz/remove-bazaar
2697.2.2 by Martin Pool
deprecate Branch.append_revision
1
# Copyright (C) 2005, 2007 Canonical Ltd
1887.1.1 by Adeodato Simó
Do not separate paragraphs in the copyright statement with blank lines,
2
#
1115 by Martin Pool
- split fetch tests into a separate file
3
# This program is free software; you can redistribute it and/or modify
4
# it under the terms of the GNU General Public License as published by
5
# the Free Software Foundation; either version 2 of the License, or
6
# (at your option) any later version.
1887.1.1 by Adeodato Simó
Do not separate paragraphs in the copyright statement with blank lines,
7
#
1115 by Martin Pool
- split fetch tests into a separate file
8
# This program is distributed in the hope that it will be useful,
9
# but WITHOUT ANY WARRANTY; without even the implied warranty of
10
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
11
# GNU General Public License for more details.
1887.1.1 by Adeodato Simó
Do not separate paragraphs in the copyright statement with blank lines,
12
#
1115 by Martin Pool
- split fetch tests into a separate file
13
# You should have received a copy of the GNU General Public License
14
# along with this program; if not, write to the Free Software
15
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
1238 by Martin Pool
- remove a lot of dead code from fetch
16
1185.1.41 by Robert Collins
massive patch from Alexander Belchenko - many PEP8 fixes, removes unused function uuid
17
import os
2004.1.22 by v.ladeuil+lp at free
Implements Range header handling for GET requests. Fix a test.
18
import re
1238 by Martin Pool
- remove a lot of dead code from fetch
19
import sys
20
2697.2.2 by Martin Pool
deprecate Branch.append_revision
21
import bzrlib
2323.8.2 by Aaron Bentley
Give a nicer error on fetch when repos are in incompatible formats
22
from bzrlib import (
23
    bzrdir,
24
    errors,
1551.15.70 by Aaron Bentley
Avoid using builtins.merge
25
    merge,
2323.8.2 by Aaron Bentley
Give a nicer error on fetch when repos are in incompatible formats
26
    repository,
3606.7.7 by John Arbash Meinel
Add tests for the fetching behavior.
27
    versionedfile,
2323.8.2 by Aaron Bentley
Give a nicer error on fetch when repos are in incompatible formats
28
    )
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
29
from bzrlib.branch import Branch
1534.4.41 by Robert Collins
Branch now uses BzrDir reasonably sanely.
30
from bzrlib.bzrdir import BzrDir
2241.1.5 by Martin Pool
Move KnitFormat2 into repofmt
31
from bzrlib.repofmt import knitrepo
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
32
from bzrlib.tests import TestCaseWithTransport
3102.1.1 by Vincent Ladeuil
Rename bzrlib/test/HTTPTestUtils.py to bzrlib/tests/http_utils.py and fix
33
from bzrlib.tests.http_utils import TestCaseWithWebserver
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
34
from bzrlib.tests.test_revision import make_branches
35
from bzrlib.trace import mutter
1551.8.42 by Aaron Bentley
Ensure that fetch properly updates inventory root knit
36
from bzrlib.upgrade import Convert
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
37
from bzrlib.workingtree import WorkingTree
1185.1.41 by Robert Collins
massive patch from Alexander Belchenko - many PEP8 fixes, removes unused function uuid
38
2696.3.3 by Martin Pool
Start setting the default format to dirstate-tags
39
# These tests are a bit old; please instead add new tests into
40
# interrepository_implementations/ so they'll run on all relevant
41
# combinations.
42
1115 by Martin Pool
- split fetch tests into a separate file
43
1238 by Martin Pool
- remove a lot of dead code from fetch
44
def has_revision(branch, revision_id):
1534.4.41 by Robert Collins
Branch now uses BzrDir reasonably sanely.
45
    return branch.repository.has_revision(revision_id)
1534.4.5 by Robert Collins
Turn branch format.open into a factory.
46
1393 by Robert Collins
reenable remotebranch tests
47
def fetch_steps(self, br_a, br_b, writable_a):
48
    """A foreign test method for testing fetch locally and remotely."""
1534.4.41 by Robert Collins
Branch now uses BzrDir reasonably sanely.
49
     
50
    # TODO RBC 20060201 make this a repository test.
51
    repo_b = br_b.repository
52
    self.assertFalse(repo_b.has_revision(br_a.revision_history()[3]))
53
    self.assertTrue(repo_b.has_revision(br_a.revision_history()[2]))
1185.16.145 by Martin Pool
Remove all assert statements from test cases.
54
    self.assertEquals(len(br_b.revision_history()), 7)
1534.1.31 by Robert Collins
Deprecated fetch.fetch and fetch.greedy_fetch for branch.fetch, and move the Repository.fetch internals to InterRepo and InterWeaveRepo.
55
    self.assertEquals(br_b.fetch(br_a, br_a.revision_history()[2])[0], 0)
56
    # branch.fetch is not supposed to alter the revision history
1185.16.145 by Martin Pool
Remove all assert statements from test cases.
57
    self.assertEquals(len(br_b.revision_history()), 7)
1534.4.41 by Robert Collins
Branch now uses BzrDir reasonably sanely.
58
    self.assertFalse(repo_b.has_revision(br_a.revision_history()[3]))
1393 by Robert Collins
reenable remotebranch tests
59
1534.4.50 by Robert Collins
Got the bzrdir api straightened out, plenty of refactoring to use it pending, but the api is up and running.
60
    # fetching the next revision up in sample data copies one revision
1534.1.31 by Robert Collins
Deprecated fetch.fetch and fetch.greedy_fetch for branch.fetch, and move the Repository.fetch internals to InterRepo and InterWeaveRepo.
61
    self.assertEquals(br_b.fetch(br_a, br_a.revision_history()[3])[0], 1)
1534.4.41 by Robert Collins
Branch now uses BzrDir reasonably sanely.
62
    self.assertTrue(repo_b.has_revision(br_a.revision_history()[3]))
1185.16.145 by Martin Pool
Remove all assert statements from test cases.
63
    self.assertFalse(has_revision(br_a, br_b.revision_history()[6]))
1534.4.41 by Robert Collins
Branch now uses BzrDir reasonably sanely.
64
    self.assertTrue(br_a.repository.has_revision(br_b.revision_history()[5]))
1393 by Robert Collins
reenable remotebranch tests
65
1092.2.28 by Robert Collins
reenable test of fetching a branch with ghosts
66
    # When a non-branch ancestor is missing, it should be unlisted...
1415 by Robert Collins
remove the ancestry weave file
67
    # as its not reference from the inventory weave.
1534.4.50 by Robert Collins
Got the bzrdir api straightened out, plenty of refactoring to use it pending, but the api is up and running.
68
    br_b4 = self.make_branch('br_4')
1534.1.31 by Robert Collins
Deprecated fetch.fetch and fetch.greedy_fetch for branch.fetch, and move the Repository.fetch internals to InterRepo and InterWeaveRepo.
69
    count, failures = br_b4.fetch(br_b)
1092.2.28 by Robert Collins
reenable test of fetching a branch with ghosts
70
    self.assertEqual(count, 7)
71
    self.assertEqual(failures, [])
1393 by Robert Collins
reenable remotebranch tests
72
1534.1.31 by Robert Collins
Deprecated fetch.fetch and fetch.greedy_fetch for branch.fetch, and move the Repository.fetch internals to InterRepo and InterWeaveRepo.
73
    self.assertEqual(writable_a.fetch(br_b)[0], 1)
1534.4.41 by Robert Collins
Branch now uses BzrDir reasonably sanely.
74
    self.assertTrue(has_revision(br_a, br_b.revision_history()[3]))
75
    self.assertTrue(has_revision(br_a, br_b.revision_history()[4]))
1393 by Robert Collins
reenable remotebranch tests
76
        
1534.4.50 by Robert Collins
Got the bzrdir api straightened out, plenty of refactoring to use it pending, but the api is up and running.
77
    br_b2 = self.make_branch('br_b2')
1534.1.31 by Robert Collins
Deprecated fetch.fetch and fetch.greedy_fetch for branch.fetch, and move the Repository.fetch internals to InterRepo and InterWeaveRepo.
78
    self.assertEquals(br_b2.fetch(br_b)[0], 7)
1534.4.41 by Robert Collins
Branch now uses BzrDir reasonably sanely.
79
    self.assertTrue(has_revision(br_b2, br_b.revision_history()[4]))
80
    self.assertTrue(has_revision(br_b2, br_a.revision_history()[2]))
1185.16.145 by Martin Pool
Remove all assert statements from test cases.
81
    self.assertFalse(has_revision(br_b2, br_a.revision_history()[3]))
1393 by Robert Collins
reenable remotebranch tests
82
1534.4.50 by Robert Collins
Got the bzrdir api straightened out, plenty of refactoring to use it pending, but the api is up and running.
83
    br_a2 = self.make_branch('br_a2')
1534.1.31 by Robert Collins
Deprecated fetch.fetch and fetch.greedy_fetch for branch.fetch, and move the Repository.fetch internals to InterRepo and InterWeaveRepo.
84
    self.assertEquals(br_a2.fetch(br_a)[0], 9)
1534.4.41 by Robert Collins
Branch now uses BzrDir reasonably sanely.
85
    self.assertTrue(has_revision(br_a2, br_b.revision_history()[4]))
86
    self.assertTrue(has_revision(br_a2, br_a.revision_history()[3]))
87
    self.assertTrue(has_revision(br_a2, br_a.revision_history()[2]))
1393 by Robert Collins
reenable remotebranch tests
88
1534.4.50 by Robert Collins
Got the bzrdir api straightened out, plenty of refactoring to use it pending, but the api is up and running.
89
    br_a3 = self.make_branch('br_a3')
1534.4.41 by Robert Collins
Branch now uses BzrDir reasonably sanely.
90
    # pulling a branch with no revisions grabs nothing, regardless of 
91
    # whats in the inventory.
1534.1.31 by Robert Collins
Deprecated fetch.fetch and fetch.greedy_fetch for branch.fetch, and move the Repository.fetch internals to InterRepo and InterWeaveRepo.
92
    self.assertEquals(br_a3.fetch(br_a2)[0], 0)
1393 by Robert Collins
reenable remotebranch tests
93
    for revno in range(4):
1534.4.41 by Robert Collins
Branch now uses BzrDir reasonably sanely.
94
        self.assertFalse(
95
            br_a3.repository.has_revision(br_a.revision_history()[revno]))
1534.1.31 by Robert Collins
Deprecated fetch.fetch and fetch.greedy_fetch for branch.fetch, and move the Repository.fetch internals to InterRepo and InterWeaveRepo.
96
    self.assertEqual(br_a3.fetch(br_a2, br_a.revision_history()[2])[0], 3)
1534.4.41 by Robert Collins
Branch now uses BzrDir reasonably sanely.
97
    # pull the 3 revisions introduced by a@u-0-3
1534.1.31 by Robert Collins
Deprecated fetch.fetch and fetch.greedy_fetch for branch.fetch, and move the Repository.fetch internals to InterRepo and InterWeaveRepo.
98
    fetched = br_a3.fetch(br_a2, br_a.revision_history()[3])[0]
1534.4.41 by Robert Collins
Branch now uses BzrDir reasonably sanely.
99
    self.assertEquals(fetched, 3, "fetched %d instead of 3" % fetched)
1393 by Robert Collins
reenable remotebranch tests
100
    # InstallFailed should be raised if the branch is missing the revision
101
    # that was requested.
2697.2.2 by Martin Pool
deprecate Branch.append_revision
102
    self.assertRaises(errors.InstallFailed, br_a3.fetch, br_a2, 'pizza')
103
2697.2.5 by Martin Pool
Kill off append_revision
104
    # TODO: Test trying to fetch from a branch that points to a revision not
105
    # actually present in its repository.  Not every branch format allows you
106
    # to directly point to such revisions, so it's a bit complicated to
107
    # construct.  One way would be to uncommit and gc the revision, but not
108
    # every branch supports that.  -- mbp 20070814
1638.1.2 by Robert Collins
Change the basis-inventory file to not have the revision-id in the file name.
109
1185.13.4 by Robert Collins
make reweave visible as a weave method, and quickly integrate into fetch
110
    #TODO: test that fetch correctly does reweaving when needed. RBC 20051008
1534.4.41 by Robert Collins
Branch now uses BzrDir reasonably sanely.
111
    # Note that this means - updating the weave when ghosts are filled in to 
112
    # add the right parents.
113
1185.13.4 by Robert Collins
make reweave visible as a weave method, and quickly integrate into fetch
114
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
115
class TestFetch(TestCaseWithTransport):
1392 by Robert Collins
reinstate testfetch test case
116
117
    def test_fetch(self):
1115 by Martin Pool
- split fetch tests into a separate file
118
        #highest indices a: 5, b: 7
2696.3.3 by Martin Pool
Start setting the default format to dirstate-tags
119
        br_a, br_b = make_branches(self, format='dirstate-tags')
1393 by Robert Collins
reenable remotebranch tests
120
        fetch_steps(self, br_a, br_b, br_a)
1404 by Robert Collins
only pull remote text weaves once per fetch operation
121
1558.4.11 by Aaron Bentley
Allow merge against self, make fetching self a noop
122
    def test_fetch_self(self):
123
        wt = self.make_branch_and_tree('br')
124
        self.assertEqual(wt.branch.fetch(wt.branch), (0, []))
125
1551.8.42 by Aaron Bentley
Ensure that fetch properly updates inventory root knit
126
    def test_fetch_root_knit(self):
1551.8.43 by Aaron Bentley
Update from reviews
127
        """Ensure that knit2.fetch() updates the root knit
1551.8.42 by Aaron Bentley
Ensure that fetch properly updates inventory root knit
128
        
1551.8.43 by Aaron Bentley
Update from reviews
129
        This tests the case where the root has a new revision, but there are no
130
        corresponding filename, parent, contents or other changes.
1551.8.42 by Aaron Bentley
Ensure that fetch properly updates inventory root knit
131
        """
1551.8.43 by Aaron Bentley
Update from reviews
132
        knit1_format = bzrdir.BzrDirMetaFormat1()
2241.1.6 by Martin Pool
Move Knit repositories into the submodule bzrlib.repofmt.knitrepo and
133
        knit1_format.repository_format = knitrepo.RepositoryFormatKnit1()
1551.8.43 by Aaron Bentley
Update from reviews
134
        knit2_format = bzrdir.BzrDirMetaFormat1()
2255.2.211 by Robert Collins
Remove knit2 repository format- it has never been supported.
135
        knit2_format.repository_format = knitrepo.RepositoryFormatKnit3()
1551.8.43 by Aaron Bentley
Update from reviews
136
        # we start with a knit1 repository because that causes the
137
        # root revision to change for each commit, even though the content,
138
        # parent, name, and other attributes are unchanged.
139
        tree = self.make_branch_and_tree('tree', knit1_format)
1551.8.42 by Aaron Bentley
Ensure that fetch properly updates inventory root knit
140
        tree.set_root_id('tree-root')
141
        tree.commit('rev1', rev_id='rev1')
142
        tree.commit('rev2', rev_id='rev2')
1551.8.43 by Aaron Bentley
Update from reviews
143
144
        # Now we convert it to a knit2 repository so that it has a root knit
145
        Convert(tree.basedir, knit2_format)
1551.8.42 by Aaron Bentley
Ensure that fetch properly updates inventory root knit
146
        tree = WorkingTree.open(tree.basedir)
1551.8.43 by Aaron Bentley
Update from reviews
147
        branch = self.make_branch('branch', format=knit2_format)
1551.8.42 by Aaron Bentley
Ensure that fetch properly updates inventory root knit
148
        branch.pull(tree.branch, stop_revision='rev1')
149
        repo = branch.repository
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
150
        repo.lock_read()
151
        try:
152
            # Make sure fetch retrieved only what we requested
153
            self.assertEqual({('tree-root', 'rev1'):()},
154
                repo.texts.get_parent_map(
155
                    [('tree-root', 'rev1'), ('tree-root', 'rev2')]))
156
        finally:
157
            repo.unlock()
1551.8.42 by Aaron Bentley
Ensure that fetch properly updates inventory root knit
158
        branch.pull(tree.branch)
1551.8.43 by Aaron Bentley
Update from reviews
159
        # Make sure that the next revision in the root knit was retrieved,
160
        # even though the text, name, parent_id, etc., were unchanged.
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
161
        repo.lock_read()
162
        try:
163
            # Make sure fetch retrieved only what we requested
164
            self.assertEqual({('tree-root', 'rev2'):(('tree-root', 'rev1'),)},
165
                repo.texts.get_parent_map([('tree-root', 'rev2')]))
166
        finally:
167
            repo.unlock()
1551.8.42 by Aaron Bentley
Ensure that fetch properly updates inventory root knit
168
2323.8.2 by Aaron Bentley
Give a nicer error on fetch when repos are in incompatible formats
169
    def test_fetch_incompatible(self):
170
        knit_tree = self.make_branch_and_tree('knit', format='knit')
171
        knit3_tree = self.make_branch_and_tree('knit3',
172
            format='dirstate-with-subtree')
173
        knit3_tree.commit('blah')
3582.1.2 by Martin Pool
Default InterRepository.fetch raises IncompatibleRepositories
174
        e = self.assertRaises(errors.IncompatibleRepositories,
175
                              knit_tree.branch.fetch, knit3_tree.branch)
176
        self.assertContainsRe(str(e),
177
            r"(?m).*/knit.*\nis not compatible with\n.*/knit3/.*\n"
178
            r"different rich-root support")
2323.8.2 by Aaron Bentley
Give a nicer error on fetch when repos are in incompatible formats
179
1185.16.96 by mbp at sourcefrog
More merge/fetch tests
180
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
181
class TestMergeFetch(TestCaseWithTransport):
1185.16.96 by mbp at sourcefrog
More merge/fetch tests
182
183
    def test_merge_fetches_unrelated(self):
184
        """Merge brings across history from unrelated source"""
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
185
        wt1 = self.make_branch_and_tree('br1')
186
        br1 = wt1.branch
187
        wt1.commit(message='rev 1-1', rev_id='1-1')
188
        wt1.commit(message='rev 1-2', rev_id='1-2')
189
        wt2 = self.make_branch_and_tree('br2')
190
        br2 = wt2.branch
191
        wt2.commit(message='rev 2-1', rev_id='2-1')
1551.15.70 by Aaron Bentley
Avoid using builtins.merge
192
        wt2.merge_from_branch(br1, from_revision='null:')
1185.16.96 by mbp at sourcefrog
More merge/fetch tests
193
        self._check_revs_present(br2)
194
1185.16.94 by mbp at sourcefrog
New test that merge fetches revisions from source
195
    def test_merge_fetches(self):
196
        """Merge brings across history from source"""
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
197
        wt1 = self.make_branch_and_tree('br1')
198
        br1 = wt1.branch
199
        wt1.commit(message='rev 1-1', rev_id='1-1')
1534.4.50 by Robert Collins
Got the bzrdir api straightened out, plenty of refactoring to use it pending, but the api is up and running.
200
        dir_2 = br1.bzrdir.sprout('br2')
201
        br2 = dir_2.open_branch()
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
202
        wt1.commit(message='rev 1-2', rev_id='1-2')
1551.15.70 by Aaron Bentley
Avoid using builtins.merge
203
        wt2 = dir_2.open_workingtree()
204
        wt2.commit(message='rev 2-1', rev_id='2-1')
205
        wt2.merge_from_branch(br1)
1185.16.96 by mbp at sourcefrog
More merge/fetch tests
206
        self._check_revs_present(br2)
207
208
    def _check_revs_present(self, br2):
1185.16.94 by mbp at sourcefrog
New test that merge fetches revisions from source
209
        for rev_id in '1-1', '1-2', '2-1':
1185.67.2 by Aaron Bentley
Renamed Branch.storage to Branch.repository
210
            self.assertTrue(br2.repository.has_revision(rev_id))
211
            rev = br2.repository.get_revision(rev_id)
1185.16.94 by mbp at sourcefrog
New test that merge fetches revisions from source
212
            self.assertEqual(rev.revision_id, rev_id)
1185.67.2 by Aaron Bentley
Renamed Branch.storage to Branch.repository
213
            self.assertTrue(br2.repository.get_inventory(rev_id))
1185.16.94 by mbp at sourcefrog
New test that merge fetches revisions from source
214
1185.16.96 by mbp at sourcefrog
More merge/fetch tests
215
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
216
class TestMergeFileHistory(TestCaseWithTransport):
217
1185.16.96 by mbp at sourcefrog
More merge/fetch tests
218
    def setUp(self):
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
219
        super(TestMergeFileHistory, self).setUp()
220
        wt1 = self.make_branch_and_tree('br1')
221
        br1 = wt1.branch
1185.16.96 by mbp at sourcefrog
More merge/fetch tests
222
        self.build_tree_contents([('br1/file', 'original contents\n')])
1534.4.28 by Robert Collins
first cut at merge from integration.
223
        wt1.add('file', 'this-file-id')
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
224
        wt1.commit(message='rev 1-1', rev_id='1-1')
1534.4.50 by Robert Collins
Got the bzrdir api straightened out, plenty of refactoring to use it pending, but the api is up and running.
225
        dir_2 = br1.bzrdir.sprout('br2')
226
        br2 = dir_2.open_branch()
227
        wt2 = dir_2.open_workingtree()
1185.16.96 by mbp at sourcefrog
More merge/fetch tests
228
        self.build_tree_contents([('br1/file', 'original from 1\n')])
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
229
        wt1.commit(message='rev 1-2', rev_id='1-2')
1185.16.96 by mbp at sourcefrog
More merge/fetch tests
230
        self.build_tree_contents([('br1/file', 'agreement\n')])
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
231
        wt1.commit(message='rev 1-3', rev_id='1-3')
1185.16.96 by mbp at sourcefrog
More merge/fetch tests
232
        self.build_tree_contents([('br2/file', 'contents in 2\n')])
1534.4.28 by Robert Collins
first cut at merge from integration.
233
        wt2.commit(message='rev 2-1', rev_id='2-1')
1185.16.96 by mbp at sourcefrog
More merge/fetch tests
234
        self.build_tree_contents([('br2/file', 'agreement\n')])
1534.4.28 by Robert Collins
first cut at merge from integration.
235
        wt2.commit(message='rev 2-2', rev_id='2-2')
1185.16.96 by mbp at sourcefrog
More merge/fetch tests
236
237
    def test_merge_fetches_file_history(self):
238
        """Merge brings across file histories"""
239
        br2 = Branch.open('br2')
1551.15.70 by Aaron Bentley
Avoid using builtins.merge
240
        br1 = Branch.open('br1')
241
        wt2 = WorkingTree.open('br2').merge_from_branch(br1)
3010.1.4 by Robert Collins
Make the knit specific fetch tests knit specific, and lock the branch when looking at historical texts in test_fetch.
242
        br2.lock_read()
243
        self.addCleanup(br2.unlock)
1185.16.96 by mbp at sourcefrog
More merge/fetch tests
244
        for rev_id, text in [('1-2', 'original from 1\n'),
245
                             ('1-3', 'agreement\n'),
246
                             ('2-1', 'contents in 2\n'),
247
                             ('2-2', 'agreement\n')]:
1185.65.1 by Aaron Bentley
Refactored out ControlFiles and RevisionStore from _Branch
248
            self.assertEqualDiff(
1185.67.2 by Aaron Bentley
Renamed Branch.storage to Branch.repository
249
                br2.repository.revision_tree(
1185.65.1 by Aaron Bentley
Refactored out ControlFiles and RevisionStore from _Branch
250
                    rev_id).get_file_text('this-file-id'), text)
1185.16.96 by mbp at sourcefrog
More merge/fetch tests
251
252
1404 by Robert Collins
only pull remote text weaves once per fetch operation
253
class TestHttpFetch(TestCaseWithWebserver):
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
254
    # FIXME RBC 20060124 this really isn't web specific, perhaps an
255
    # instrumented readonly transport? Can we do an instrumented
256
    # adapter and use self.get_readonly_url ?
1404 by Robert Collins
only pull remote text weaves once per fetch operation
257
258
    def test_fetch(self):
259
        #highest indices a: 5, b: 7
1185.16.145 by Martin Pool
Remove all assert statements from test cases.
260
        br_a, br_b = make_branches(self)
1534.4.50 by Robert Collins
Got the bzrdir api straightened out, plenty of refactoring to use it pending, but the api is up and running.
261
        br_rem_a = Branch.open(self.get_readonly_url('branch1'))
1404 by Robert Collins
only pull remote text weaves once per fetch operation
262
        fetch_steps(self, br_rem_a, br_b, br_a)
263
1540.3.15 by Martin Pool
[merge] large merge to sync with bzr.dev
264
    def _count_log_matches(self, target, logs):
265
        """Count the number of times the target file pattern was fetched in an http log"""
2004.1.22 by v.ladeuil+lp at free
Implements Range header handling for GET requests. Fix a test.
266
        get_succeeds_re = re.compile(
267
            '.*"GET .*%s HTTP/1.1" 20[06] - "-" "bzr/%s' %
268
            (     target,                    bzrlib.__version__))
1540.3.15 by Martin Pool
[merge] large merge to sync with bzr.dev
269
        c = 0
270
        for line in logs:
2004.1.22 by v.ladeuil+lp at free
Implements Range header handling for GET requests. Fix a test.
271
            if get_succeeds_re.match(line):
1540.3.15 by Martin Pool
[merge] large merge to sync with bzr.dev
272
                c += 1
273
        return c
274
1404 by Robert Collins
only pull remote text weaves once per fetch operation
275
    def test_weaves_are_retrieved_once(self):
276
        self.build_tree(("source/", "source/file", "target/"))
3010.1.4 by Robert Collins
Make the knit specific fetch tests knit specific, and lock the branch when looking at historical texts in test_fetch.
277
        # This test depends on knit dasta storage.
278
        wt = self.make_branch_and_tree('source', format='dirstate-tags')
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
279
        branch = wt.branch
280
        wt.add(["file"], ["id"])
281
        wt.commit("added file")
2911.6.1 by Blake Winton
Change 'print >> f,'s to 'f.write('s.
282
        open("source/file", 'w').write("blah\n")
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
283
        wt.commit("changed file")
1534.4.41 by Robert Collins
Branch now uses BzrDir reasonably sanely.
284
        target = BzrDir.create_branch_and_repo("target/")
1534.4.50 by Robert Collins
Got the bzrdir api straightened out, plenty of refactoring to use it pending, but the api is up and running.
285
        source = Branch.open(self.get_readonly_url("source/"))
1534.1.31 by Robert Collins
Deprecated fetch.fetch and fetch.greedy_fetch for branch.fetch, and move the Repository.fetch internals to InterRepo and InterWeaveRepo.
286
        self.assertEqual(target.fetch(source), (2, []))
1430 by Robert Collins
touchup the prefixed-store patch
287
        # this is the path to the literal file. As format changes 
288
        # occur it needs to be updated. FIXME: ask the store for the
289
        # path.
1540.3.15 by Martin Pool
[merge] large merge to sync with bzr.dev
290
        self.log("web server logs are:")
291
        http_logs = self.get_readonly_server().logs
292
        self.log('\n'.join(http_logs))
1666.1.6 by Robert Collins
Make knit the default format.
293
        # unfortunately this log entry is branch format specific. We could 
294
        # factor out the 'what files does this format use' to a method on the 
295
        # repository, which would let us to this generically. RBC 20060419
3422.1.1 by John Arbash Meinel
merge in bzr-1.5rc1, revert the transaction cache change
296
        # RBC 20080408: Or perhaps we can assert that no files are fully read
297
        # twice?
1666.1.6 by Robert Collins
Make knit the default format.
298
        self.assertEqual(1, self._count_log_matches('/ce/id.kndx', http_logs))
299
        self.assertEqual(1, self._count_log_matches('/ce/id.knit', http_logs))
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
300
        self.assertEqual(1, self._count_log_matches('inventory.kndx', http_logs))
1417.1.12 by Robert Collins
cache revision history during read transactions
301
        # this r-h check test will prevent regressions, but it currently already 
302
        # passes, before the patch to cache-rh is applied :[
2230.3.10 by Aaron Bentley
Genericised test_fetch to handle branch 6 retrieval patterns
303
        self.assertTrue(1 >= self._count_log_matches('revision-history',
304
                                                     http_logs))
305
        self.assertTrue(1 >= self._count_log_matches('last-revision',
306
                                                     http_logs))
1530.1.18 by Robert Collins
unbreak test_fetch
307
        # FIXME naughty poking in there.
1534.4.50 by Robert Collins
Got the bzrdir api straightened out, plenty of refactoring to use it pending, but the api is up and running.
308
        self.get_readonly_server().logs = []
3241.1.4 by Andrew Bennetts
Use get_smart_medium as suggested by Robert, and deal with the fallout.
309
        # check there is nothing more to fetch.  We take care to re-use the
310
        # existing transport so that the request logs we're about to examine
311
        # aren't cluttered with redundant probes for a smart server.
3241.1.5 by Andrew Bennetts
Add comment suggested by Robert.
312
        # XXX: Perhaps this further parameterisation: test http with smart
313
        # server, and test http without smart server?
3241.1.4 by Andrew Bennetts
Use get_smart_medium as suggested by Robert, and deal with the fallout.
314
        source = Branch.open(
315
            self.get_readonly_url("source/"),
316
            possible_transports=[source.bzrdir.root_transport])
1534.1.31 by Robert Collins
Deprecated fetch.fetch and fetch.greedy_fetch for branch.fetch, and move the Repository.fetch internals to InterRepo and InterWeaveRepo.
317
        self.assertEqual(target.fetch(source), (0, []))
1540.3.15 by Martin Pool
[merge] large merge to sync with bzr.dev
318
        # should make just two requests
319
        http_logs = self.get_readonly_server().logs
320
        self.log("web server logs are:")
321
        self.log('\n'.join(http_logs))
1666.1.4 by Robert Collins
* 'Metadir' is now the default disk format. This improves behaviour in
322
        self.assertEqual(1, self._count_log_matches('branch-format', http_logs))
323
        self.assertEqual(1, self._count_log_matches('branch/format', http_logs))
3422.1.1 by John Arbash Meinel
merge in bzr-1.5rc1, revert the transaction cache change
324
        self.assertEqual(1, self._count_log_matches('repository/format',
325
            http_logs))
2230.3.11 by Aaron Bentley
Fix line endings
326
        self.assertTrue(1 >= self._count_log_matches('revision-history',
327
                                                     http_logs))
328
        self.assertTrue(1 >= self._count_log_matches('last-revision',
329
                                                     http_logs))
1666.1.4 by Robert Collins
* 'Metadir' is now the default disk format. This improves behaviour in
330
        self.assertEqual(4, len(http_logs))
3380.1.1 by Aaron Bentley
Fix inventory insertion to work in topological order
331
332
3606.7.7 by John Arbash Meinel
Add tests for the fetching behavior.
333
class TestKnitToPackFetch(TestCaseWithTransport):
334
335
    def find_get_record_stream(self, calls):
336
        """In a list of calls, find 'get_record_stream' calls.
337
338
        This also ensures that there is only one get_record_stream call.
339
        """
340
        get_record_call = None
341
        for call in calls:
342
            if call[0] == 'get_record_stream':
343
                self.assertIs(None, get_record_call,
344
                              "there should only be one call to"
345
                              " get_record_stream")
346
                get_record_call = call
347
        self.assertIsNot(None, get_record_call,
348
                         "there should be exactly one call to "
349
                         " get_record_stream")
350
        return get_record_call
351
352
    def test_fetch_with_deltas_no_delta_closure(self):
353
        tree = self.make_branch_and_tree('source', format='dirstate')
354
        target = self.make_repository('target', format='pack-0.92')
355
        self.build_tree(['source/file'])
356
        tree.set_root_id('root-id')
357
        tree.add('file', 'file-id')
358
        tree.commit('one', rev_id='rev-one')
359
        source = tree.branch.repository
360
        source.texts = versionedfile.RecordingVersionedFilesDecorator(
361
                        source.texts)
362
        source.signatures = versionedfile.RecordingVersionedFilesDecorator(
363
                        source.signatures)
364
        source.revisions = versionedfile.RecordingVersionedFilesDecorator(
365
                        source.revisions)
366
        source.inventories = versionedfile.RecordingVersionedFilesDecorator(
367
                        source.inventories)
368
        # precondition
369
        self.assertTrue(target._fetch_uses_deltas)
370
        target.fetch(source, revision_id='rev-one')
371
        self.assertEqual(('get_record_stream', [('file-id', 'rev-one')],
372
                          target._fetch_order, False),
373
                         self.find_get_record_stream(source.texts.calls))
374
        self.assertEqual(('get_record_stream', [('rev-one',)],
375
                          target._fetch_order, False),
376
                         self.find_get_record_stream(source.inventories.calls))
377
        self.assertEqual(('get_record_stream', [('rev-one',)],
3849.3.1 by John Arbash Meinel
Part of bug #300289, stop requiring plain fulltexts for revisions.
378
                          target._fetch_order, False),
3606.7.7 by John Arbash Meinel
Add tests for the fetching behavior.
379
                         self.find_get_record_stream(source.revisions.calls))
380
        # XXX: Signatures is special, and slightly broken. The
381
        # standard item_keys_introduced_by actually does a lookup for every
382
        # signature to see if it exists, rather than waiting to do them all at
383
        # once at the end. The fetch code then does an all-at-once and just
384
        # allows for some of them to be missing.
385
        # So we know there will be extra calls, but the *last* one is the one
386
        # we care about.
387
        signature_calls = source.signatures.calls[-1:]
388
        self.assertEqual(('get_record_stream', [('rev-one',)],
3849.3.1 by John Arbash Meinel
Part of bug #300289, stop requiring plain fulltexts for revisions.
389
                          target._fetch_order, False),
3606.7.7 by John Arbash Meinel
Add tests for the fetching behavior.
390
                         self.find_get_record_stream(signature_calls))
391
392
    def test_fetch_no_deltas_with_delta_closure(self):
393
        tree = self.make_branch_and_tree('source', format='dirstate')
394
        target = self.make_repository('target', format='pack-0.92')
395
        self.build_tree(['source/file'])
396
        tree.set_root_id('root-id')
397
        tree.add('file', 'file-id')
398
        tree.commit('one', rev_id='rev-one')
399
        source = tree.branch.repository
400
        source.texts = versionedfile.RecordingVersionedFilesDecorator(
401
                        source.texts)
402
        source.signatures = versionedfile.RecordingVersionedFilesDecorator(
403
                        source.signatures)
404
        source.revisions = versionedfile.RecordingVersionedFilesDecorator(
405
                        source.revisions)
406
        source.inventories = versionedfile.RecordingVersionedFilesDecorator(
407
                        source.inventories)
408
        target._fetch_uses_deltas = False
409
        target.fetch(source, revision_id='rev-one')
410
        self.assertEqual(('get_record_stream', [('file-id', 'rev-one')],
411
                          target._fetch_order, True),
412
                         self.find_get_record_stream(source.texts.calls))
413
        self.assertEqual(('get_record_stream', [('rev-one',)],
414
                          target._fetch_order, True),
415
                         self.find_get_record_stream(source.inventories.calls))
416
        self.assertEqual(('get_record_stream', [('rev-one',)],
417
                          target._fetch_order, True),
418
                         self.find_get_record_stream(source.revisions.calls))
419
        # XXX: Signatures is special, and slightly broken. The
420
        # standard item_keys_introduced_by actually does a lookup for every
421
        # signature to see if it exists, rather than waiting to do them all at
422
        # once at the end. The fetch code then does an all-at-once and just
423
        # allows for some of them to be missing.
424
        # So we know there will be extra calls, but the *last* one is the one
425
        # we care about.
426
        signature_calls = source.signatures.calls[-1:]
427
        self.assertEqual(('get_record_stream', [('rev-one',)],
428
                          target._fetch_order, True),
429
                         self.find_get_record_stream(signature_calls))
430
3849.3.1 by John Arbash Meinel
Part of bug #300289, stop requiring plain fulltexts for revisions.
431
    def test_fetch_revisions_with_deltas_into_pack(self):
432
        # See BUG #261339, dev versions of bzr could accidentally create deltas
433
        # in revision texts in knit branches (when fetching from packs). So we
434
        # ensure that *if* a knit repository has a delta in revisions, that it
435
        # gets properly expanded back into a fulltext when stored in the pack
436
        # file.
437
        tree = self.make_branch_and_tree('source', format='dirstate')
438
        target = self.make_repository('target', format='pack-0.92')
439
        self.build_tree(['source/file'])
440
        tree.set_root_id('root-id')
441
        tree.add('file', 'file-id')
442
        tree.commit('one', rev_id='rev-one')
443
        # Hack the KVF for revisions so that it "accidentally" allows a delta
444
        tree.branch.repository.revisions._max_delta_chain = 200
445
        tree.commit('two', rev_id='rev-two')
446
        source = tree.branch.repository
447
        # Ensure that we stored a delta
448
        source.lock_read()
449
        self.addCleanup(source.unlock)
450
        record = source.revisions.get_record_stream([('rev-two',)],
451
            'unordered', False).next()
452
        self.assertEqual('knit-delta-gz', record.storage_kind)
453
        target.fetch(tree.branch.repository, revision_id='rev-two')
454
        # The record should get expanded back to a fulltext
455
        target.lock_read()
456
        self.addCleanup(target.unlock)
457
        record = target.revisions.get_record_stream([('rev-two',)],
458
            'unordered', False).next()
459
        self.assertEqual('knit-ft-gz', record.storage_kind)
460
461
3606.7.7 by John Arbash Meinel
Add tests for the fetching behavior.
462
3380.1.1 by Aaron Bentley
Fix inventory insertion to work in topological order
463
class Test1To2Fetch(TestCaseWithTransport):
3380.1.3 by Aaron Bentley
Fix model-change fetching with ghosts and when fetch is resumed
464
    """Tests for Model1To2 failure modes"""
3380.1.1 by Aaron Bentley
Fix inventory insertion to work in topological order
465
3380.2.4 by Aaron Bentley
Updates from review
466
    def make_tree_and_repo(self):
467
        self.tree = self.make_branch_and_tree('tree', format='pack-0.92')
468
        self.repo = self.make_repository('rich-repo', format='rich-root-pack')
469
        self.repo.lock_write()
470
        self.addCleanup(self.repo.unlock)
471
472
    def do_fetch_order_test(self, first, second):
3380.1.3 by Aaron Bentley
Fix model-change fetching with ghosts and when fetch is resumed
473
        """Test that fetch works no matter what the set order of revision is.
3380.1.1 by Aaron Bentley
Fix inventory insertion to work in topological order
474
475
        This test depends on the order of items in a set, which is
476
        implementation-dependant, so we test A, B and then B, A.
477
        """
3380.1.3 by Aaron Bentley
Fix model-change fetching with ghosts and when fetch is resumed
478
        self.make_tree_and_repo()
479
        self.tree.commit('Commit 1', rev_id=first)
480
        self.tree.commit('Commit 2', rev_id=second)
481
        self.repo.fetch(self.tree.branch.repository, second)
482
3380.2.4 by Aaron Bentley
Updates from review
483
    def test_fetch_order_AB(self):
3380.2.7 by Aaron Bentley
Update docs
484
        """See do_fetch_order_test"""
3380.2.4 by Aaron Bentley
Updates from review
485
        self.do_fetch_order_test('A', 'B')
486
487
    def test_fetch_order_BA(self):
3380.2.7 by Aaron Bentley
Update docs
488
        """See do_fetch_order_test"""
3380.2.4 by Aaron Bentley
Updates from review
489
        self.do_fetch_order_test('B', 'A')
490
3380.1.3 by Aaron Bentley
Fix model-change fetching with ghosts and when fetch is resumed
491
    def get_parents(self, file_id, revision_id):
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
492
        self.repo.lock_read()
493
        try:
494
            parent_map = self.repo.texts.get_parent_map([(file_id, revision_id)])
495
            return parent_map[(file_id, revision_id)]
496
        finally:
497
            self.repo.unlock()
3380.1.3 by Aaron Bentley
Fix model-change fetching with ghosts and when fetch is resumed
498
3380.1.2 by Aaron Bentley
Improve handling ghosts and changing root_ids
499
    def test_fetch_ghosts(self):
3380.1.3 by Aaron Bentley
Fix model-change fetching with ghosts and when fetch is resumed
500
        self.make_tree_and_repo()
501
        self.tree.commit('first commit', rev_id='left-parent')
502
        self.tree.add_parent_tree_id('ghost-parent')
503
        fork = self.tree.bzrdir.sprout('fork', 'null:').open_workingtree()
3380.1.2 by Aaron Bentley
Improve handling ghosts and changing root_ids
504
        fork.commit('not a ghost', rev_id='not-ghost-parent')
3380.1.3 by Aaron Bentley
Fix model-change fetching with ghosts and when fetch is resumed
505
        self.tree.branch.repository.fetch(fork.branch.repository,
3380.1.2 by Aaron Bentley
Improve handling ghosts and changing root_ids
506
                                     'not-ghost-parent')
3380.1.3 by Aaron Bentley
Fix model-change fetching with ghosts and when fetch is resumed
507
        self.tree.add_parent_tree_id('not-ghost-parent')
508
        self.tree.commit('second commit', rev_id='second-id')
509
        self.repo.fetch(self.tree.branch.repository, 'second-id')
510
        root_id = self.tree.get_root_id()
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
511
        self.assertEqual(
512
            ((root_id, 'left-parent'), (root_id, 'ghost-parent'),
513
             (root_id, 'not-ghost-parent')),
514
            self.get_parents(root_id, 'second-id'))
3380.1.3 by Aaron Bentley
Fix model-change fetching with ghosts and when fetch is resumed
515
516
    def make_two_commits(self, change_root, fetch_twice):
517
        self.make_tree_and_repo()
518
        self.tree.commit('first commit', rev_id='first-id')
519
        if change_root:
520
            self.tree.set_root_id('unique-id')
521
        self.tree.commit('second commit', rev_id='second-id')
522
        if fetch_twice:
523
            self.repo.fetch(self.tree.branch.repository, 'first-id')
524
        self.repo.fetch(self.tree.branch.repository, 'second-id')
3380.1.2 by Aaron Bentley
Improve handling ghosts and changing root_ids
525
526
    def test_fetch_changed_root(self):
3380.1.3 by Aaron Bentley
Fix model-change fetching with ghosts and when fetch is resumed
527
        self.make_two_commits(change_root=True, fetch_twice=False)
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
528
        self.assertEqual((), self.get_parents('unique-id', 'second-id'))
3380.1.3 by Aaron Bentley
Fix model-change fetching with ghosts and when fetch is resumed
529
530
    def test_two_fetch_changed_root(self):
531
        self.make_two_commits(change_root=True, fetch_twice=True)
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
532
        self.assertEqual((), self.get_parents('unique-id', 'second-id'))
3380.1.3 by Aaron Bentley
Fix model-change fetching with ghosts and when fetch is resumed
533
534
    def test_two_fetches(self):
535
        self.make_two_commits(change_root=False, fetch_twice=True)
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
536
        self.assertEqual((('TREE_ROOT', 'first-id'),),
537
            self.get_parents('TREE_ROOT', 'second-id'))