/brz/remove-bazaar

To get this branch, use:
bzr branch http://gegoxaren.bato24.eu/bzr/brz/remove-bazaar
4988.10.3 by John Arbash Meinel
Merge bzr.dev 5007, resolve conflict, update NEWS
1
# Copyright (C) 2006-2010 Canonical Ltd
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
2
#
3
# This program is free software; you can redistribute it and/or modify
4
# it under the terms of the GNU General Public License as published by
5
# the Free Software Foundation; either version 2 of the License, or
6
# (at your option) any later version.
7
#
8
# This program is distributed in the hope that it will be useful,
9
# but WITHOUT ANY WARRANTY; without even the implied warranty of
10
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
11
# GNU General Public License for more details.
12
#
13
# You should have received a copy of the GNU General Public License
14
# along with this program; if not, write to the Free Software
4183.7.1 by Sabin Iacob
update FSF mailing address
15
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
16
3211.5.2 by Robert Collins
Change RemoteRepository.get_parent_map to use bz2 not gzip for compression.
17
import bz2
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
18
2490.2.5 by Aaron Bentley
Use GraphWalker.unique_ancestor to determine merge base
19
from bzrlib import (
2694.5.4 by Jelmer Vernooij
Move bzrlib.util.bencode to bzrlib._bencode_py.
20
    bencode,
2490.2.5 by Aaron Bentley
Use GraphWalker.unique_ancestor to determine merge base
21
    branch,
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
22
    bzrdir,
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
23
    config,
3192.1.1 by Andrew Bennetts
Add some -Dhpss debugging to get_parent_map.
24
    debug,
2490.2.5 by Aaron Bentley
Use GraphWalker.unique_ancestor to determine merge base
25
    errors,
3172.5.1 by Robert Collins
Create a RemoteRepository get_graph implementation and delegate get_parents_map to the real repository.
26
    graph,
4509.3.28 by Martin Pool
Add missing import
27
    lock,
2490.2.5 by Aaron Bentley
Use GraphWalker.unique_ancestor to determine merge base
28
    lockdir,
29
    repository,
2948.3.1 by John Arbash Meinel
Fix bug #158333, make sure that Repository.fetch(self) is properly a no-op for all Repository implementations.
30
    revision,
4307.2.3 by Robert Collins
Change RemoteRepository.has_revision to use get_parent_map to leverage the caching.
31
    revision as _mod_revision,
4913.4.2 by Jelmer Vernooij
Add Repository.get_known_graph_ancestry.
32
    static_tuple,
3228.4.11 by John Arbash Meinel
Deprecations abound.
33
    symbol_versioning,
2490.2.5 by Aaron Bentley
Use GraphWalker.unique_ancestor to determine merge base
34
)
2535.4.27 by Andrew Bennetts
Remove some unused imports.
35
from bzrlib.branch import BranchReferenceFormat
2018.5.174 by Andrew Bennetts
Various nits discovered by pyflakes.
36
from bzrlib.bzrdir import BzrDir, RemoteBzrDirFormat
4634.85.9 by Andrew Bennetts
Add some experimental decorators: @only_raises(..) and @cleanup_method.
37
from bzrlib.decorators import needs_read_lock, needs_write_lock, only_raises
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
38
from bzrlib.errors import (
39
    NoSuchRevision,
40
    SmartProtocolError,
41
    )
2018.5.127 by Andrew Bennetts
Fix most of the lockable_files tests for RemoteBranchLockableFiles.
42
from bzrlib.lockable_files import LockableFiles
4060.1.4 by Robert Collins
Streaming fetch from remote servers.
43
from bzrlib.smart import client, vfs, repository as smart_repo
3297.4.1 by Andrew Bennetts
Merge 'Add Branch.set_last_revision_info smart method'.
44
from bzrlib.revision import ensure_null, NULL_REVISION
3441.5.5 by Andrew Bennetts
Some small tweaks and comments.
45
from bzrlib.trace import mutter, note, warning
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
46
3445.1.5 by John Arbash Meinel
allow passing a 'graph' object into Branch.update_revisions.
47
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
48
class _RpcHelper(object):
49
    """Mixin class that helps with issuing RPCs."""
50
51
    def _call(self, method, *args, **err_context):
52
        try:
53
            return self._client.call(method, *args)
54
        except errors.ErrorFromSmartServer, err:
55
            self._translate_error(err, **err_context)
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
56
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
57
    def _call_expecting_body(self, method, *args, **err_context):
58
        try:
59
            return self._client.call_expecting_body(method, *args)
60
        except errors.ErrorFromSmartServer, err:
61
            self._translate_error(err, **err_context)
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
62
4556.2.1 by Andrew Bennetts
Add Branch.set_tags_bytes RPC, with HPSS call count acceptance test. Also fixes serialisation of LockDir, and uses external_url() in LockDir's repr and contention message.
63
    def _call_with_body_bytes(self, method, args, body_bytes, **err_context):
64
        try:
65
            return self._client.call_with_body_bytes(method, args, body_bytes)
66
        except errors.ErrorFromSmartServer, err:
67
            self._translate_error(err, **err_context)
68
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
69
    def _call_with_body_bytes_expecting_body(self, method, args, body_bytes,
70
                                             **err_context):
71
        try:
72
            return self._client.call_with_body_bytes_expecting_body(
73
                method, args, body_bytes)
74
        except errors.ErrorFromSmartServer, err:
75
            self._translate_error(err, **err_context)
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
76
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
77
78
def response_tuple_to_repo_format(response):
79
    """Convert a response tuple describing a repository format to a format."""
80
    format = RemoteRepositoryFormat()
4118.1.1 by Andrew Bennetts
Fix performance regression (many small round-trips) when pushing to a remote pack, and tidy the tests.
81
    format._rich_root_data = (response[0] == 'yes')
82
    format._supports_tree_reference = (response[1] == 'yes')
83
    format._supports_external_lookups = (response[2] == 'yes')
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
84
    format._network_name = response[3]
85
    return format
86
87
2018.5.25 by Andrew Bennetts
Make sure RemoteBzrDirFormat is always registered (John Arbash Meinel, Robert Collins, Andrew Bennetts).
88
# Note: RemoteBzrDirFormat is in bzrdir.py
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
89
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
90
class RemoteBzrDir(BzrDir, _RpcHelper):
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
91
    """Control directory on a remote server, accessed via bzr:// or similar."""
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
92
4634.47.5 by Andrew Bennetts
Add tests, and fix BzrDirMeta1.has_workingtree which was failing if the local transport is decorated with a ChrootTransport or similar.
93
    def __init__(self, transport, format, _client=None, _force_probe=False):
2018.5.51 by Wouter van Heyst
Test and implement RemoteBranch.last_revision_info()
94
        """Construct a RemoteBzrDir.
95
96
        :param _client: Private parameter for testing. Disables probing and the
97
            use of a real bzrdir.
98
        """
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
99
        BzrDir.__init__(self, transport, format)
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
100
        # this object holds a delegated bzrdir that uses file-level operations
101
        # to talk to the other side
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
102
        self._real_bzrdir = None
4634.47.3 by Andrew Bennetts
Add a BzrDir.open_2.1 verb that indicates if there is a workingtree present. Removes the last 2 VFS calls from incremental pushes.
103
        self._has_working_tree = None
4044.1.3 by Robert Collins
Create a one-shot cache of the result of RemoteBzrDir.create_branch, eliminating 3 round trips for nonstacked branches and 5 for stacked.
104
        # 1-shot cache for the call pattern 'create_branch; open_branch' - see
105
        # create_branch for details.
106
        self._next_open_branch_result = None
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
107
108
        if _client is None:
3313.2.3 by Andrew Bennetts
Deprecate Transport.get_shared_medium.
109
            medium = transport.get_smart_medium()
3431.3.2 by Andrew Bennetts
Remove 'base' from _SmartClient entirely, now that the medium has it.
110
            self._client = client._SmartClient(medium)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
111
        else:
112
            self._client = _client
4634.47.5 by Andrew Bennetts
Add tests, and fix BzrDirMeta1.has_workingtree which was failing if the local transport is decorated with a ChrootTransport or similar.
113
            if not _force_probe:
114
                return
115
116
        self._probe_bzrdir()
117
4964.2.1 by Martin Pool
Add RemoteBzrDir repr
118
    def __repr__(self):
119
        return '%s(%r)' % (self.__class__.__name__, self._client)
120
4634.47.5 by Andrew Bennetts
Add tests, and fix BzrDirMeta1.has_workingtree which was failing if the local transport is decorated with a ChrootTransport or similar.
121
    def _probe_bzrdir(self):
122
        medium = self._client._medium
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
123
        path = self._path_for_remote_call(self._client)
4634.47.3 by Andrew Bennetts
Add a BzrDir.open_2.1 verb that indicates if there is a workingtree present. Removes the last 2 VFS calls from incremental pushes.
124
        if medium._is_remote_before((2, 1)):
125
            self._rpc_open(path)
126
            return
127
        try:
128
            self._rpc_open_2_1(path)
129
            return
130
        except errors.UnknownSmartMethod:
131
            medium._remember_remote_is_before((2, 1))
132
            self._rpc_open(path)
133
134
    def _rpc_open_2_1(self, path):
135
        response = self._call('BzrDir.open_2.1', path)
136
        if response == ('no',):
137
            raise errors.NotBranchError(path=self.root_transport.base)
138
        elif response[0] == 'yes':
139
            if response[1] == 'yes':
140
                self._has_working_tree = True
141
            elif response[1] == 'no':
142
                self._has_working_tree = False
143
            else:
144
                raise errors.UnexpectedSmartServerResponse(response)
145
        else:
146
            raise errors.UnexpectedSmartServerResponse(response)
147
148
    def _rpc_open(self, path):
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
149
        response = self._call('BzrDir.open', path)
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
150
        if response not in [('yes',), ('no',)]:
151
            raise errors.UnexpectedSmartServerResponse(response)
2018.5.26 by Andrew Bennetts
Extract a simple SmartClient class from RemoteTransport, and a hack to avoid VFS operations when probing for a bzrdir over a smart transport.
152
        if response == ('no',):
4634.47.3 by Andrew Bennetts
Add a BzrDir.open_2.1 verb that indicates if there is a workingtree present. Removes the last 2 VFS calls from incremental pushes.
153
            raise errors.NotBranchError(path=self.root_transport.base)
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
154
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
155
    def _ensure_real(self):
156
        """Ensure that there is a _real_bzrdir set.
157
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
158
        Used before calls to self._real_bzrdir.
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
159
        """
160
        if not self._real_bzrdir:
4789.2.1 by Robert Collins
Trigger hpssvfs backtrace dumps with RemoteBzrDir._ensure_real.
161
            if 'hpssvfs' in debug.debug_flags:
162
                import traceback
163
                warning('VFS BzrDir access triggered\n%s',
164
                    ''.join(traceback.format_stack()))
2018.5.169 by Andrew Bennetts
Add a _server_formats flag to BzrDir.open_from_transport and BzrDirFormat.find_format, make RemoteBranch.control_files into a property.
165
            self._real_bzrdir = BzrDir.open_from_transport(
166
                self.root_transport, _server_formats=False)
4070.2.1 by Robert Collins
Add a BzrDirFormat.network_name.
167
            self._format._network_name = \
168
                self._real_bzrdir._format.network_name()
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
169
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
170
    def _translate_error(self, err, **context):
171
        _translate_error(err, bzrdir=self, **context)
172
4044.1.3 by Robert Collins
Create a one-shot cache of the result of RemoteBzrDir.create_branch, eliminating 3 round trips for nonstacked branches and 5 for stacked.
173
    def break_lock(self):
174
        # Prevent aliasing problems in the next_open_branch_result cache.
175
        # See create_branch for rationale.
176
        self._next_open_branch_result = None
177
        return BzrDir.break_lock(self)
178
4070.2.3 by Robert Collins
Get BzrDir.cloning_metadir working.
179
    def _vfs_cloning_metadir(self, require_stacking=False):
3242.3.28 by Aaron Bentley
Use repository acquisition policy for sprouting
180
        self._ensure_real()
4070.2.3 by Robert Collins
Get BzrDir.cloning_metadir working.
181
        return self._real_bzrdir.cloning_metadir(
182
            require_stacking=require_stacking)
183
184
    def cloning_metadir(self, require_stacking=False):
185
        medium = self._client._medium
186
        if medium._is_remote_before((1, 13)):
187
            return self._vfs_cloning_metadir(require_stacking=require_stacking)
188
        verb = 'BzrDir.cloning_metadir'
189
        if require_stacking:
190
            stacking = 'True'
191
        else:
192
            stacking = 'False'
193
        path = self._path_for_remote_call(self._client)
194
        try:
195
            response = self._call(verb, path, stacking)
196
        except errors.UnknownSmartMethod:
4094.1.1 by Andrew Bennetts
Add some medium._remember_is_before((1, 13)) calls.
197
            medium._remember_remote_is_before((1, 13))
4070.2.3 by Robert Collins
Get BzrDir.cloning_metadir working.
198
            return self._vfs_cloning_metadir(require_stacking=require_stacking)
4160.2.9 by Andrew Bennetts
Fix BzrDir.cloning_metadir RPC to fail on branch references, and make
199
        except errors.UnknownErrorFromSmartServer, err:
200
            if err.error_tuple != ('BranchReference',):
201
                raise
202
            # We need to resolve the branch reference to determine the
203
            # cloning_metadir.  This causes unnecessary RPCs to open the
204
            # referenced branch (and bzrdir, etc) but only when the caller
205
            # didn't already resolve the branch reference.
206
            referenced_branch = self.open_branch()
207
            return referenced_branch.bzrdir.cloning_metadir()
4070.2.3 by Robert Collins
Get BzrDir.cloning_metadir working.
208
        if len(response) != 3:
209
            raise errors.UnexpectedSmartServerResponse(response)
4070.7.4 by Andrew Bennetts
Deal with branch references better in BzrDir.cloning_metadir RPC (changes protocol).
210
        control_name, repo_name, branch_info = response
211
        if len(branch_info) != 2:
212
            raise errors.UnexpectedSmartServerResponse(response)
213
        branch_ref, branch_name = branch_info
4075.2.1 by Robert Collins
Audit and make sure we are registering network_name's as factories, not instances.
214
        format = bzrdir.network_format_registry.get(control_name)
4070.2.3 by Robert Collins
Get BzrDir.cloning_metadir working.
215
        if repo_name:
216
            format.repository_format = repository.network_format_registry.get(
217
                repo_name)
4084.2.2 by Robert Collins
Review feedback.
218
        if branch_ref == 'ref':
4070.7.4 by Andrew Bennetts
Deal with branch references better in BzrDir.cloning_metadir RPC (changes protocol).
219
            # XXX: we need possible_transports here to avoid reopening the
4070.7.5 by Andrew Bennetts
Tweak comment.
220
            # connection to the referenced location
4070.7.4 by Andrew Bennetts
Deal with branch references better in BzrDir.cloning_metadir RPC (changes protocol).
221
            ref_bzrdir = BzrDir.open(branch_name)
222
            branch_format = ref_bzrdir.cloning_metadir().get_branch_format()
223
            format.set_branch_format(branch_format)
4084.2.2 by Robert Collins
Review feedback.
224
        elif branch_ref == 'branch':
4070.7.4 by Andrew Bennetts
Deal with branch references better in BzrDir.cloning_metadir RPC (changes protocol).
225
            if branch_name:
226
                format.set_branch_format(
227
                    branch.network_format_registry.get(branch_name))
228
        else:
229
            raise errors.UnexpectedSmartServerResponse(response)
4070.2.3 by Robert Collins
Get BzrDir.cloning_metadir working.
230
        return format
3242.3.28 by Aaron Bentley
Use repository acquisition policy for sprouting
231
1752.2.39 by Martin Pool
[broken] implement upgrade apis on remote bzrdirs
232
    def create_repository(self, shared=False):
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
233
        # as per meta1 formats - just delegate to the format object which may
234
        # be parameterised.
235
        result = self._format.repository_format.initialize(self, shared)
236
        if not isinstance(result, RemoteRepository):
237
            return self.open_repository()
238
        else:
239
            return result
1752.2.50 by Andrew Bennetts
Implement RemoteBzrDir.create_{branch,workingtree}
240
2796.2.19 by Aaron Bentley
Support reconfigure --lightweight-checkout
241
    def destroy_repository(self):
242
        """See BzrDir.destroy_repository"""
243
        self._ensure_real()
244
        self._real_bzrdir.destroy_repository()
245
5051.3.3 by Jelmer Vernooij
Add tests for colo branches.
246
    def create_branch(self, name=None):
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
247
        # as per meta1 formats - just delegate to the format object which may
248
        # be parameterised.
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
249
        real_branch = self._format.get_branch_format().initialize(self,
250
            name=name)
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
251
        if not isinstance(real_branch, RemoteBranch):
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
252
            result = RemoteBranch(self, self.find_repository(), real_branch,
253
                                  name=name)
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
254
        else:
4044.1.3 by Robert Collins
Create a one-shot cache of the result of RemoteBzrDir.create_branch, eliminating 3 round trips for nonstacked branches and 5 for stacked.
255
            result = real_branch
256
        # BzrDir.clone_on_transport() uses the result of create_branch but does
257
        # not return it to its callers; we save approximately 8% of our round
258
        # trips by handing the branch we created back to the first caller to
259
        # open_branch rather than probing anew. Long term we need a API in
260
        # bzrdir that doesn't discard result objects (like result_branch).
261
        # RBC 20090225
262
        self._next_open_branch_result = result
263
        return result
1752.2.50 by Andrew Bennetts
Implement RemoteBzrDir.create_{branch,workingtree}
264
5051.3.3 by Jelmer Vernooij
Add tests for colo branches.
265
    def destroy_branch(self, name=None):
2796.2.16 by Aaron Bentley
Documentation updates from review
266
        """See BzrDir.destroy_branch"""
2796.2.6 by Aaron Bentley
Implement destroy_branch
267
        self._ensure_real()
5051.3.3 by Jelmer Vernooij
Add tests for colo branches.
268
        self._real_bzrdir.destroy_branch(name=name)
4044.1.3 by Robert Collins
Create a one-shot cache of the result of RemoteBzrDir.create_branch, eliminating 3 round trips for nonstacked branches and 5 for stacked.
269
        self._next_open_branch_result = None
2796.2.6 by Aaron Bentley
Implement destroy_branch
270
2955.5.3 by Vincent Ladeuil
Fix second unwanted connection by providing the right branch to create_checkout.
271
    def create_workingtree(self, revision_id=None, from_branch=None):
2018.5.174 by Andrew Bennetts
Various nits discovered by pyflakes.
272
        raise errors.NotLocalUrl(self.transport.base)
1752.2.39 by Martin Pool
[broken] implement upgrade apis on remote bzrdirs
273
5147.4.1 by Jelmer Vernooij
Pass branch names in more places.
274
    def find_branch_format(self, name=None):
2018.5.124 by Robert Collins
Fix test_format_initialize_find_open by delegating Branch formt lookup to the BzrDir, where it should have stayed from the start.
275
        """Find the branch 'format' for this bzrdir.
276
277
        This might be a synthetic object for e.g. RemoteBranch and SVN.
278
        """
5147.4.3 by Jelmer Vernooij
Support branch name argument to BzrDir.get_branch_reference.
279
        b = self.open_branch(name=name)
2018.5.124 by Robert Collins
Fix test_format_initialize_find_open by delegating Branch formt lookup to the BzrDir, where it should have stayed from the start.
280
        return b._format
281
5147.4.3 by Jelmer Vernooij
Support branch name argument to BzrDir.get_branch_reference.
282
    def get_branch_reference(self, name=None):
2018.5.132 by Robert Collins
Make all BzrDir implementation tests pass on RemoteBzrDir - fix some things, and remove the incomplete_with_basis tests as cruft.
283
        """See BzrDir.get_branch_reference()."""
5147.4.3 by Jelmer Vernooij
Support branch name argument to BzrDir.get_branch_reference.
284
        if name is not None:
285
            # XXX JRV20100304: Support opening colocated branches
286
            raise errors.NoColocatedBranchSupport(self)
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
287
        response = self._get_branch_reference()
288
        if response[0] == 'ref':
289
            return response[1]
290
        else:
291
            return None
292
293
    def _get_branch_reference(self):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
294
        path = self._path_for_remote_call(self._client)
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
295
        medium = self._client._medium
4734.4.8 by Andrew Bennetts
Fix HPSS tests; pass 'location is a repository' message via smart server when possible (adds BzrDir.open_branchV3 verb).
296
        candidate_calls = [
297
            ('BzrDir.open_branchV3', (2, 1)),
298
            ('BzrDir.open_branchV2', (1, 13)),
299
            ('BzrDir.open_branch', None),
300
            ]
301
        for verb, required_version in candidate_calls:
302
            if required_version and medium._is_remote_before(required_version):
303
                continue
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
304
            try:
4734.4.8 by Andrew Bennetts
Fix HPSS tests; pass 'location is a repository' message via smart server when possible (adds BzrDir.open_branchV3 verb).
305
                response = self._call(verb, path)
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
306
            except errors.UnknownSmartMethod:
4734.4.8 by Andrew Bennetts
Fix HPSS tests; pass 'location is a repository' message via smart server when possible (adds BzrDir.open_branchV3 verb).
307
                if required_version is None:
308
                    raise
309
                medium._remember_remote_is_before(required_version)
310
            else:
311
                break
312
        if verb == 'BzrDir.open_branch':
313
            if response[0] != 'ok':
314
                raise errors.UnexpectedSmartServerResponse(response)
315
            if response[1] != '':
316
                return ('ref', response[1])
317
            else:
318
                return ('branch', '')
319
        if response[0] not in ('ref', 'branch'):
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
320
            raise errors.UnexpectedSmartServerResponse(response)
4734.4.8 by Andrew Bennetts
Fix HPSS tests; pass 'location is a repository' message via smart server when possible (adds BzrDir.open_branchV3 verb).
321
        return response
2018.5.132 by Robert Collins
Make all BzrDir implementation tests pass on RemoteBzrDir - fix some things, and remove the incomplete_with_basis tests as cruft.
322
5147.4.1 by Jelmer Vernooij
Pass branch names in more places.
323
    def _get_tree_branch(self, name=None):
3211.4.1 by Robert Collins
* ``RemoteBzrDir._get_tree_branch`` no longer triggers ``_ensure_real``,
324
        """See BzrDir._get_tree_branch()."""
5147.4.1 by Jelmer Vernooij
Pass branch names in more places.
325
        return None, self.open_branch(name=name)
3211.4.1 by Robert Collins
* ``RemoteBzrDir._get_tree_branch`` no longer triggers ``_ensure_real``,
326
5051.3.4 by Jelmer Vernooij
Support name to BzrDir.open_branch.
327
    def open_branch(self, name=None, unsupported=False,
328
                    ignore_fallbacks=False):
329
        if unsupported:
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
330
            raise NotImplementedError('unsupported flag support not implemented yet.')
4044.1.3 by Robert Collins
Create a one-shot cache of the result of RemoteBzrDir.create_branch, eliminating 3 round trips for nonstacked branches and 5 for stacked.
331
        if self._next_open_branch_result is not None:
332
            # See create_branch for details.
333
            result = self._next_open_branch_result
334
            self._next_open_branch_result = None
335
            return result
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
336
        response = self._get_branch_reference()
337
        if response[0] == 'ref':
2018.5.132 by Robert Collins
Make all BzrDir implementation tests pass on RemoteBzrDir - fix some things, and remove the incomplete_with_basis tests as cruft.
338
            # a branch reference, use the existing BranchReference logic.
339
            format = BranchReferenceFormat()
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
340
            return format.open(self, name=name, _found=True,
341
                location=response[1], ignore_fallbacks=ignore_fallbacks)
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
342
        branch_format_name = response[1]
343
        if not branch_format_name:
344
            branch_format_name = None
345
        format = RemoteBranchFormat(network_name=branch_format_name)
4160.2.6 by Andrew Bennetts
Add ignore_fallbacks flag.
346
        return RemoteBranch(self, self.find_repository(), format=format,
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
347
            setup_stacking=not ignore_fallbacks, name=name)
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
348
4053.1.1 by Robert Collins
New version of the BzrDir.find_repository verb supporting _network_name to support removing more _ensure_real calls.
349
    def _open_repo_v1(self, path):
350
        verb = 'BzrDir.find_repository'
351
        response = self._call(verb, path)
352
        if response[0] != 'ok':
353
            raise errors.UnexpectedSmartServerResponse(response)
354
        # servers that only support the v1 method don't support external
355
        # references either.
356
        self._ensure_real()
357
        repo = self._real_bzrdir.open_repository()
358
        response = response + ('no', repo._format.network_name())
359
        return response, repo
360
361
    def _open_repo_v2(self, path):
362
        verb = 'BzrDir.find_repositoryV2'
363
        response = self._call(verb, path)
364
        if response[0] != 'ok':
365
            raise errors.UnexpectedSmartServerResponse(response)
366
        self._ensure_real()
367
        repo = self._real_bzrdir.open_repository()
368
        response = response + (repo._format.network_name(),)
369
        return response, repo
370
371
    def _open_repo_v3(self, path):
372
        verb = 'BzrDir.find_repositoryV3'
4053.1.2 by Robert Collins
Actually make this branch work.
373
        medium = self._client._medium
374
        if medium._is_remote_before((1, 13)):
375
            raise errors.UnknownSmartMethod(verb)
4094.1.1 by Andrew Bennetts
Add some medium._remember_is_before((1, 13)) calls.
376
        try:
377
            response = self._call(verb, path)
378
        except errors.UnknownSmartMethod:
379
            medium._remember_remote_is_before((1, 13))
380
            raise
4053.1.1 by Robert Collins
New version of the BzrDir.find_repository verb supporting _network_name to support removing more _ensure_real calls.
381
        if response[0] != 'ok':
382
            raise errors.UnexpectedSmartServerResponse(response)
383
        return response, None
384
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
385
    def open_repository(self):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
386
        path = self._path_for_remote_call(self._client)
4053.1.1 by Robert Collins
New version of the BzrDir.find_repository verb supporting _network_name to support removing more _ensure_real calls.
387
        response = None
388
        for probe in [self._open_repo_v3, self._open_repo_v2,
389
            self._open_repo_v1]:
390
            try:
391
                response, real_repo = probe(path)
392
                break
393
            except errors.UnknownSmartMethod:
394
                pass
395
        if response is None:
396
            raise errors.UnknownSmartMethod('BzrDir.find_repository{3,2,}')
3245.4.24 by Andrew Bennetts
Consistently raise errors from the server as ErrorFromSmartServer exceptions.
397
        if response[0] != 'ok':
398
            raise errors.UnexpectedSmartServerResponse(response)
4053.1.1 by Robert Collins
New version of the BzrDir.find_repository verb supporting _network_name to support removing more _ensure_real calls.
399
        if len(response) != 6:
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
400
            raise SmartProtocolError('incorrect response length %s' % (response,))
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
401
        if response[1] == '':
4053.1.1 by Robert Collins
New version of the BzrDir.find_repository verb supporting _network_name to support removing more _ensure_real calls.
402
            # repo is at this dir.
403
            format = response_tuple_to_repo_format(response[2:])
3221.15.10 by Robert Collins
Add test that we can stack on a smart server from Jonathan Lange.
404
            # Used to support creating a real format instance when needed.
405
            format._creating_bzrdir = self
3990.5.1 by Andrew Bennetts
Add network_name() to RepositoryFormat.
406
            remote_repo = RemoteRepository(self, format)
407
            format._creating_repo = remote_repo
4053.1.1 by Robert Collins
New version of the BzrDir.find_repository verb supporting _network_name to support removing more _ensure_real calls.
408
            if real_repo is not None:
409
                remote_repo._set_real_repository(real_repo)
3990.5.1 by Andrew Bennetts
Add network_name() to RepositoryFormat.
410
            return remote_repo
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
411
        else:
412
            raise errors.NoRepositoryPresent(self)
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
413
4634.47.5 by Andrew Bennetts
Add tests, and fix BzrDirMeta1.has_workingtree which was failing if the local transport is decorated with a ChrootTransport or similar.
414
    def has_workingtree(self):
4634.47.3 by Andrew Bennetts
Add a BzrDir.open_2.1 verb that indicates if there is a workingtree present. Removes the last 2 VFS calls from incremental pushes.
415
        if self._has_working_tree is None:
416
            self._ensure_real()
417
            self._has_working_tree = self._real_bzrdir.has_workingtree()
4634.47.5 by Andrew Bennetts
Add tests, and fix BzrDirMeta1.has_workingtree which was failing if the local transport is decorated with a ChrootTransport or similar.
418
        return self._has_working_tree
419
420
    def open_workingtree(self, recommend_upgrade=True):
421
        if self.has_workingtree():
2445.1.1 by Andrew Bennetts
Make RemoteBzrDir.open_workingtree raise NoWorkingTree rather than NotLocalUrl
422
            raise errors.NotLocalUrl(self.root_transport)
423
        else:
424
            raise errors.NoWorkingTree(self.root_transport.base)
1752.2.50 by Andrew Bennetts
Implement RemoteBzrDir.create_{branch,workingtree}
425
2018.5.42 by Robert Collins
Various hopefully improvements, but wsgi is broken, handing over to spiv :).
426
    def _path_for_remote_call(self, client):
2018.6.1 by Robert Collins
Implement a BzrDir.open_branch smart server method for opening a branch without VFS.
427
        """Return the path to be used for this bzrdir in a remote call."""
2018.5.42 by Robert Collins
Various hopefully improvements, but wsgi is broken, handing over to spiv :).
428
        return client.remote_path_from_transport(self.root_transport)
2018.6.1 by Robert Collins
Implement a BzrDir.open_branch smart server method for opening a branch without VFS.
429
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
430
    def get_branch_transport(self, branch_format, name=None):
2018.5.162 by Andrew Bennetts
Add some missing _ensure_real calls, and a missing import.
431
        self._ensure_real()
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
432
        return self._real_bzrdir.get_branch_transport(branch_format, name=name)
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
433
1752.2.43 by Andrew Bennetts
Fix get_{branch,repository,workingtree}_transport.
434
    def get_repository_transport(self, repository_format):
2018.5.162 by Andrew Bennetts
Add some missing _ensure_real calls, and a missing import.
435
        self._ensure_real()
1752.2.43 by Andrew Bennetts
Fix get_{branch,repository,workingtree}_transport.
436
        return self._real_bzrdir.get_repository_transport(repository_format)
437
438
    def get_workingtree_transport(self, workingtree_format):
2018.5.162 by Andrew Bennetts
Add some missing _ensure_real calls, and a missing import.
439
        self._ensure_real()
1752.2.43 by Andrew Bennetts
Fix get_{branch,repository,workingtree}_transport.
440
        return self._real_bzrdir.get_workingtree_transport(workingtree_format)
441
1752.2.39 by Martin Pool
[broken] implement upgrade apis on remote bzrdirs
442
    def can_convert_format(self):
443
        """Upgrading of remote bzrdirs is not supported yet."""
444
        return False
445
446
    def needs_format_conversion(self, format=None):
447
        """Upgrading of remote bzrdirs is not supported yet."""
3943.2.5 by Martin Pool
deprecate needs_format_conversion(format=None)
448
        if format is None:
449
            symbol_versioning.warn(symbol_versioning.deprecated_in((1, 13, 0))
450
                % 'needs_format_conversion(format=None)')
1752.2.39 by Martin Pool
[broken] implement upgrade apis on remote bzrdirs
451
        return False
452
3242.3.37 by Aaron Bentley
Updates from reviews
453
    def clone(self, url, revision_id=None, force_new_repo=False,
454
              preserve_stacking=False):
2018.5.94 by Andrew Bennetts
Various small changes in aid of making tests pass (including deleting one invalid test).
455
        self._ensure_real()
456
        return self._real_bzrdir.clone(url, revision_id=revision_id,
3242.3.37 by Aaron Bentley
Updates from reviews
457
            force_new_repo=force_new_repo, preserve_stacking=preserve_stacking)
2018.5.94 by Andrew Bennetts
Various small changes in aid of making tests pass (including deleting one invalid test).
458
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
459
    def _get_config(self):
460
        return RemoteBzrDirConfig(self)
3567.1.3 by Michael Hudson
fix problem
461
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
462
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
463
class RemoteRepositoryFormat(repository.RepositoryFormat):
2018.5.159 by Andrew Bennetts
Rename SmartClient to _SmartClient.
464
    """Format for repositories accessed over a _SmartClient.
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
465
1752.2.50 by Andrew Bennetts
Implement RemoteBzrDir.create_{branch,workingtree}
466
    Instances of this repository are represented by RemoteRepository
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
467
    instances.
2018.5.118 by Robert Collins
Fix RemoteRepositoryFormat to have appropriate rich_root_data and support_tree_reference.
468
3128.1.3 by Vincent Ladeuil
Since we are there s/parameteris.*/parameteriz&/.
469
    The RemoteRepositoryFormat is parameterized during construction
2018.5.118 by Robert Collins
Fix RemoteRepositoryFormat to have appropriate rich_root_data and support_tree_reference.
470
    to reflect the capabilities of the real, remote format. Specifically
2018.5.138 by Robert Collins
Merge bzr.dev.
471
    the attributes rich_root_data and supports_tree_reference are set
2018.5.118 by Robert Collins
Fix RemoteRepositoryFormat to have appropriate rich_root_data and support_tree_reference.
472
    on a per instance basis, and are not set (and should not be) at
473
    the class level.
3990.5.3 by Robert Collins
Docs and polish on RepositoryFormat.network_name.
474
4032.1.1 by John Arbash Meinel
Merge the removal of all trailing whitespace, and resolve conflicts.
475
    :ivar _custom_format: If set, a specific concrete repository format that
3990.5.3 by Robert Collins
Docs and polish on RepositoryFormat.network_name.
476
        will be used when initializing a repository with this
477
        RemoteRepositoryFormat.
478
    :ivar _creating_repo: If set, the repository object that this
479
        RemoteRepositoryFormat was created for: it can be called into
3990.5.4 by Robert Collins
Review feedback.
480
        to obtain data like the network name.
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
481
    """
482
3543.1.2 by Michael Hudson
the two character fix
483
    _matchingbzrdir = RemoteBzrDirFormat()
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
484
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
485
    def __init__(self):
486
        repository.RepositoryFormat.__init__(self)
487
        self._custom_format = None
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
488
        self._network_name = None
4017.3.3 by Robert Collins
Review feedback - make RemoteRepository.initialize use helpers, and version-lock the new method to not attempt the method on older servers.
489
        self._creating_bzrdir = None
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
490
        self._supports_chks = None
4118.1.1 by Andrew Bennetts
Fix performance regression (many small round-trips) when pushing to a remote pack, and tidy the tests.
491
        self._supports_external_lookups = None
492
        self._supports_tree_reference = None
493
        self._rich_root_data = None
494
4608.1.2 by Martin Pool
Add RemoteRepositoryFormat repr
495
    def __repr__(self):
496
        return "%s(_network_name=%r)" % (self.__class__.__name__,
497
            self._network_name)
498
4118.1.1 by Andrew Bennetts
Fix performance regression (many small round-trips) when pushing to a remote pack, and tidy the tests.
499
    @property
4183.5.1 by Robert Collins
Add RepositoryFormat.fast_deltas to signal fast delta creation.
500
    def fast_deltas(self):
501
        self._ensure_real()
502
        return self._custom_format.fast_deltas
503
504
    @property
4118.1.1 by Andrew Bennetts
Fix performance regression (many small round-trips) when pushing to a remote pack, and tidy the tests.
505
    def rich_root_data(self):
506
        if self._rich_root_data is None:
507
            self._ensure_real()
508
            self._rich_root_data = self._custom_format.rich_root_data
509
        return self._rich_root_data
510
511
    @property
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
512
    def supports_chks(self):
513
        if self._supports_chks is None:
514
            self._ensure_real()
515
            self._supports_chks = self._custom_format.supports_chks
516
        return self._supports_chks
517
518
    @property
4118.1.1 by Andrew Bennetts
Fix performance regression (many small round-trips) when pushing to a remote pack, and tidy the tests.
519
    def supports_external_lookups(self):
520
        if self._supports_external_lookups is None:
521
            self._ensure_real()
522
            self._supports_external_lookups = \
4104.4.2 by Robert Collins
Fix test_source for 1.13 landing.
523
                self._custom_format.supports_external_lookups
4118.1.1 by Andrew Bennetts
Fix performance regression (many small round-trips) when pushing to a remote pack, and tidy the tests.
524
        return self._supports_external_lookups
525
526
    @property
527
    def supports_tree_reference(self):
528
        if self._supports_tree_reference is None:
529
            self._ensure_real()
530
            self._supports_tree_reference = \
531
                self._custom_format.supports_tree_reference
532
        return self._supports_tree_reference
4017.3.3 by Robert Collins
Review feedback - make RemoteRepository.initialize use helpers, and version-lock the new method to not attempt the method on older servers.
533
534
    def _vfs_initialize(self, a_bzrdir, shared):
535
        """Helper for common code in initialize."""
536
        if self._custom_format:
537
            # Custom format requested
538
            result = self._custom_format.initialize(a_bzrdir, shared=shared)
539
        elif self._creating_bzrdir is not None:
540
            # Use the format that the repository we were created to back
541
            # has.
542
            prior_repo = self._creating_bzrdir.open_repository()
543
            prior_repo._ensure_real()
544
            result = prior_repo._real_repository._format.initialize(
545
                a_bzrdir, shared=shared)
546
        else:
547
            # assume that a_bzr is a RemoteBzrDir but the smart server didn't
548
            # support remote initialization.
549
            # We delegate to a real object at this point (as RemoteBzrDir
550
            # delegate to the repository format which would lead to infinite
551
            # recursion if we just called a_bzrdir.create_repository.
552
            a_bzrdir._ensure_real()
553
            result = a_bzrdir._real_bzrdir.create_repository(shared=shared)
554
        if not isinstance(result, RemoteRepository):
555
            return self.open(a_bzrdir)
556
        else:
557
            return result
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
558
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
559
    def initialize(self, a_bzrdir, shared=False):
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
560
        # Being asked to create on a non RemoteBzrDir:
561
        if not isinstance(a_bzrdir, RemoteBzrDir):
4017.3.3 by Robert Collins
Review feedback - make RemoteRepository.initialize use helpers, and version-lock the new method to not attempt the method on older servers.
562
            return self._vfs_initialize(a_bzrdir, shared)
563
        medium = a_bzrdir._client._medium
564
        if medium._is_remote_before((1, 13)):
565
            return self._vfs_initialize(a_bzrdir, shared)
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
566
        # Creating on a remote bzr dir.
567
        # 1) get the network name to use.
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
568
        if self._custom_format:
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
569
            network_name = self._custom_format.network_name()
4294.2.5 by Robert Collins
Reasonable unit test coverage for initialize_on_transport_ex.
570
        elif self._network_name:
571
            network_name = self._network_name
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
572
        else:
573
            # Select the current bzrlib default and ask for that.
574
            reference_bzrdir_format = bzrdir.format_registry.get('default')()
575
            reference_format = reference_bzrdir_format.repository_format
576
            network_name = reference_format.network_name()
577
        # 2) try direct creation via RPC
578
        path = a_bzrdir._path_for_remote_call(a_bzrdir._client)
579
        verb = 'BzrDir.create_repository'
580
        if shared:
581
            shared_str = 'True'
582
        else:
583
            shared_str = 'False'
584
        try:
585
            response = a_bzrdir._call(verb, path, network_name, shared_str)
586
        except errors.UnknownSmartMethod:
4017.3.3 by Robert Collins
Review feedback - make RemoteRepository.initialize use helpers, and version-lock the new method to not attempt the method on older servers.
587
            # Fallback - use vfs methods
4094.1.1 by Andrew Bennetts
Add some medium._remember_is_before((1, 13)) calls.
588
            medium._remember_remote_is_before((1, 13))
4017.3.3 by Robert Collins
Review feedback - make RemoteRepository.initialize use helpers, and version-lock the new method to not attempt the method on older servers.
589
            return self._vfs_initialize(a_bzrdir, shared)
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
590
        else:
591
            # Turn the response into a RemoteRepository object.
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
592
            format = response_tuple_to_repo_format(response[1:])
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
593
            # Used to support creating a real format instance when needed.
594
            format._creating_bzrdir = a_bzrdir
595
            remote_repo = RemoteRepository(a_bzrdir, format)
596
            format._creating_repo = remote_repo
597
            return remote_repo
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
598
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
599
    def open(self, a_bzrdir):
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
600
        if not isinstance(a_bzrdir, RemoteBzrDir):
601
            raise AssertionError('%r is not a RemoteBzrDir' % (a_bzrdir,))
1752.2.72 by Andrew Bennetts
Make Remote* classes in remote.py more consistent and remove some dead code.
602
        return a_bzrdir.open_repository()
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
603
4053.1.4 by Robert Collins
Move the fetch control attributes from Repository to RepositoryFormat.
604
    def _ensure_real(self):
605
        if self._custom_format is None:
606
            self._custom_format = repository.network_format_registry.get(
607
                self._network_name)
608
609
    @property
610
    def _fetch_order(self):
611
        self._ensure_real()
612
        return self._custom_format._fetch_order
613
614
    @property
615
    def _fetch_uses_deltas(self):
616
        self._ensure_real()
617
        return self._custom_format._fetch_uses_deltas
618
619
    @property
620
    def _fetch_reconcile(self):
621
        self._ensure_real()
622
        return self._custom_format._fetch_reconcile
623
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
624
    def get_format_description(self):
4792.1.1 by Andrew Bennetts
Show real branch/repo format description in 'info -v' over HPSS.
625
        self._ensure_real()
626
        return 'Remote: ' + self._custom_format.get_format_description()
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
627
628
    def __eq__(self, other):
4088.3.1 by Benjamin Peterson
compare types with 'is' not ==
629
        return self.__class__ is other.__class__
1752.2.87 by Andrew Bennetts
Make tests pass.
630
3990.5.1 by Andrew Bennetts
Add network_name() to RepositoryFormat.
631
    def network_name(self):
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
632
        if self._network_name:
633
            return self._network_name
3990.5.1 by Andrew Bennetts
Add network_name() to RepositoryFormat.
634
        self._creating_repo._ensure_real()
635
        return self._creating_repo._real_repository._format.network_name()
636
4022.1.1 by Robert Collins
Refactoring of fetch to have a sender and sink component enabling splitting the logic over a network stream. (Robert Collins, Andrew Bennetts)
637
    @property
4431.3.7 by Jonathan Lange
Cherrypick bzr.dev 4470, resolving conflicts.
638
    def pack_compresses(self):
639
        self._ensure_real()
640
        return self._custom_format.pack_compresses
641
642
    @property
4022.1.1 by Robert Collins
Refactoring of fetch to have a sender and sink component enabling splitting the logic over a network stream. (Robert Collins, Andrew Bennetts)
643
    def _serializer(self):
4053.1.4 by Robert Collins
Move the fetch control attributes from Repository to RepositoryFormat.
644
        self._ensure_real()
645
        return self._custom_format._serializer
4022.1.1 by Robert Collins
Refactoring of fetch to have a sender and sink component enabling splitting the logic over a network stream. (Robert Collins, Andrew Bennetts)
646
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
647
4731.1.2 by Andrew Bennetts
Refactor to reduce duplication.
648
class RemoteRepository(_RpcHelper, lock._RelockDebugMixin):
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
649
    """Repository accessed over rpc.
650
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
651
    For the moment most operations are performed using local transport-backed
652
    Repository objects.
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
653
    """
654
2018.5.118 by Robert Collins
Fix RemoteRepositoryFormat to have appropriate rich_root_data and support_tree_reference.
655
    def __init__(self, remote_bzrdir, format, real_repository=None, _client=None):
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
656
        """Create a RemoteRepository instance.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
657
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
658
        :param remote_bzrdir: The bzrdir hosting this repository.
2018.5.118 by Robert Collins
Fix RemoteRepositoryFormat to have appropriate rich_root_data and support_tree_reference.
659
        :param format: The RemoteFormat object to use.
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
660
        :param real_repository: If not None, a local implementation of the
661
            repository logic for the repository, usually accessing the data
662
            via the VFS.
2018.5.57 by Robert Collins
Implement RemoteRepository.is_shared (Robert Collins, Vincent Ladeuil).
663
        :param _client: Private testing parameter - override the smart client
664
            to be used by the repository.
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
665
        """
666
        if real_repository:
2018.5.36 by Andrew Bennetts
Fix typo, and clean up some ununsed import warnings from pyflakes at the same time.
667
            self._real_repository = real_repository
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
668
        else:
669
            self._real_repository = None
1752.2.50 by Andrew Bennetts
Implement RemoteBzrDir.create_{branch,workingtree}
670
        self.bzrdir = remote_bzrdir
2018.5.57 by Robert Collins
Implement RemoteRepository.is_shared (Robert Collins, Vincent Ladeuil).
671
        if _client is None:
3313.2.1 by Andrew Bennetts
Change _SmartClient's API to accept a medium and a base, rather than a _SharedConnection.
672
            self._client = remote_bzrdir._client
2018.5.57 by Robert Collins
Implement RemoteRepository.is_shared (Robert Collins, Vincent Ladeuil).
673
        else:
674
            self._client = _client
2018.5.118 by Robert Collins
Fix RemoteRepositoryFormat to have appropriate rich_root_data and support_tree_reference.
675
        self._format = format
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
676
        self._lock_mode = None
677
        self._lock_token = None
678
        self._lock_count = 0
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
679
        self._leave_lock = False
4307.2.4 by Robert Collins
Enable caching of negative revision lookups in RemoteRepository write locks when no _real_repository has been constructed.
680
        # Cache of revision parents; misses are cached during read locks, and
681
        # write locks when no _real_repository has been set.
3835.1.12 by Aaron Bentley
Unify CachingExtraParentsProvider and CachingParentsProvider.
682
        self._unstacked_provider = graph.CachingParentsProvider(
3896.1.1 by Andrew Bennetts
Remove broken debugging cruft, and some unused imports.
683
            get_parent_map=self._get_parent_map_rpc)
3835.1.12 by Aaron Bentley
Unify CachingExtraParentsProvider and CachingParentsProvider.
684
        self._unstacked_provider.disable_cache()
2951.1.10 by Robert Collins
Peer review feedback with Ian.
685
        # For tests:
686
        # These depend on the actual remote format, so force them off for
687
        # maximum compatibility. XXX: In future these should depend on the
688
        # remote repository instance, but this is irrelevant until we perform
689
        # reconcile via an RPC call.
2951.1.5 by Robert Collins
Some work towards including the correct changes for TREE_ROOT in check parameterised tests.
690
        self._reconcile_does_inventory_gc = False
691
        self._reconcile_fixes_text_parents = False
2951.1.3 by Robert Collins
Partial support for native reconcile with packs.
692
        self._reconcile_backsup_inventory = False
2592.4.5 by Martin Pool
Add Repository.base on all repositories.
693
        self.base = self.bzrdir.transport.base
3221.12.1 by Robert Collins
Backport development1 format (stackable packs) to before-shallow-branches.
694
        # Additional places to query for data.
695
        self._fallback_repositories = []
2592.4.5 by Martin Pool
Add Repository.base on all repositories.
696
697
    def __str__(self):
698
        return "%s(%s)" % (self.__class__.__name__, self.base)
699
700
    __repr__ = __str__
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
701
3825.4.1 by Andrew Bennetts
Add suppress_errors to abort_write_group.
702
    def abort_write_group(self, suppress_errors=False):
2617.6.7 by Robert Collins
More review feedback.
703
        """Complete a write group on the decorated repository.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
704
4031.3.3 by Matt Nordhoff
Review tweaks from Ben Finney
705
        Smart methods perform operations in a single step so this API
2617.6.6 by Robert Collins
Some review feedback.
706
        is not really applicable except as a compatibility thunk
2617.6.2 by Robert Collins
Add abort_write_group and wire write_groups into fetch and commit.
707
        for older plugins that don't use e.g. the CommitBuilder
708
        facility.
3825.4.6 by Andrew Bennetts
Document the suppress_errors flag in the docstring.
709
710
        :param suppress_errors: see Repository.abort_write_group.
2617.6.2 by Robert Collins
Add abort_write_group and wire write_groups into fetch and commit.
711
        """
712
        self._ensure_real()
3825.4.1 by Andrew Bennetts
Add suppress_errors to abort_write_group.
713
        return self._real_repository.abort_write_group(
714
            suppress_errors=suppress_errors)
2617.6.2 by Robert Collins
Add abort_write_group and wire write_groups into fetch and commit.
715
4253.1.1 by Robert Collins
Add chk_bytes property to RemoteRepository
716
    @property
717
    def chk_bytes(self):
718
        """Decorate the real repository for now.
719
720
        In the long term a full blown network facility is needed to avoid
721
        creating a real repository object locally.
722
        """
723
        self._ensure_real()
724
        return self._real_repository.chk_bytes
725
2617.6.2 by Robert Collins
Add abort_write_group and wire write_groups into fetch and commit.
726
    def commit_write_group(self):
2617.6.7 by Robert Collins
More review feedback.
727
        """Complete a write group on the decorated repository.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
728
4031.3.3 by Matt Nordhoff
Review tweaks from Ben Finney
729
        Smart methods perform operations in a single step so this API
2617.6.6 by Robert Collins
Some review feedback.
730
        is not really applicable except as a compatibility thunk
2617.6.2 by Robert Collins
Add abort_write_group and wire write_groups into fetch and commit.
731
        for older plugins that don't use e.g. the CommitBuilder
732
        facility.
733
        """
734
        self._ensure_real()
735
        return self._real_repository.commit_write_group()
2617.6.1 by Robert Collins
* New method on Repository - ``start_write_group``, ``end_write_group``
736
4002.1.1 by Andrew Bennetts
Implement suspend_write_group/resume_write_group.
737
    def resume_write_group(self, tokens):
738
        self._ensure_real()
739
        return self._real_repository.resume_write_group(tokens)
740
741
    def suspend_write_group(self):
742
        self._ensure_real()
743
        return self._real_repository.suspend_write_group()
744
4343.3.29 by John Arbash Meinel
Add 'check_for_missing_texts' flag to get_missing_parent_inv..
745
    def get_missing_parent_inventories(self, check_for_missing_texts=True):
4257.4.6 by Andrew Bennetts
Make get_missing_parent_inventories work for all repo formats (it's a no-op for unstackable formats).
746
        self._ensure_real()
4343.3.29 by John Arbash Meinel
Add 'check_for_missing_texts' flag to get_missing_parent_inv..
747
        return self._real_repository.get_missing_parent_inventories(
748
            check_for_missing_texts=check_for_missing_texts)
4257.4.6 by Andrew Bennetts
Make get_missing_parent_inventories work for all repo formats (it's a no-op for unstackable formats).
749
4419.2.9 by Andrew Bennetts
Add per_repository_reference test for get_rev_id_for_revno, fix the bugs it revealed.
750
    def _get_rev_id_for_revno_vfs(self, revno, known_pair):
751
        self._ensure_real()
752
        return self._real_repository.get_rev_id_for_revno(
753
            revno, known_pair)
754
4419.2.5 by Andrew Bennetts
Add Repository.get_rev_id_for_revno, and use it both as the _ensure_real fallback and as the server-side implementation.
755
    def get_rev_id_for_revno(self, revno, known_pair):
756
        """See Repository.get_rev_id_for_revno."""
757
        path = self.bzrdir._path_for_remote_call(self._client)
758
        try:
4476.3.33 by Andrew Bennetts
Revert some accidental s/17/18/ in remote.py.
759
            if self._client._medium._is_remote_before((1, 17)):
4419.2.9 by Andrew Bennetts
Add per_repository_reference test for get_rev_id_for_revno, fix the bugs it revealed.
760
                return self._get_rev_id_for_revno_vfs(revno, known_pair)
4419.2.5 by Andrew Bennetts
Add Repository.get_rev_id_for_revno, and use it both as the _ensure_real fallback and as the server-side implementation.
761
            response = self._call(
762
                'Repository.get_rev_id_for_revno', path, revno, known_pair)
763
        except errors.UnknownSmartMethod:
4476.3.33 by Andrew Bennetts
Revert some accidental s/17/18/ in remote.py.
764
            self._client._medium._remember_remote_is_before((1, 17))
4419.2.9 by Andrew Bennetts
Add per_repository_reference test for get_rev_id_for_revno, fix the bugs it revealed.
765
            return self._get_rev_id_for_revno_vfs(revno, known_pair)
4419.2.5 by Andrew Bennetts
Add Repository.get_rev_id_for_revno, and use it both as the _ensure_real fallback and as the server-side implementation.
766
        if response[0] == 'ok':
767
            return True, response[1]
768
        elif response[0] == 'history-incomplete':
4419.2.8 by Andrew Bennetts
Add unit test for RemoteRepository.get_rev_id_for_revno using fallbacks if it gets a history-incomplete response.
769
            known_pair = response[1:3]
770
            for fallback in self._fallback_repositories:
771
                found, result = fallback.get_rev_id_for_revno(revno, known_pair)
772
                if found:
773
                    return True, result
774
                else:
775
                    known_pair = result
776
            # Not found in any fallbacks
777
            return False, known_pair
4419.2.5 by Andrew Bennetts
Add Repository.get_rev_id_for_revno, and use it both as the _ensure_real fallback and as the server-side implementation.
778
        else:
779
            raise errors.UnexpectedSmartServerResponse(response)
780
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
781
    def _ensure_real(self):
782
        """Ensure that there is a _real_repository set.
783
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
784
        Used before calls to self._real_repository.
4145.1.1 by Robert Collins
Explicitly prevent fetching while the target repository is in a write group.
785
786
        Note that _ensure_real causes many roundtrips to the server which are
787
        not desirable, and prevents the use of smart one-roundtrip RPC's to
788
        perform complex operations (such as accessing parent data, streaming
789
        revisions etc). Adding calls to _ensure_real should only be done when
790
        bringing up new functionality, adding fallbacks for smart methods that
791
        require a fallback path, and never to replace an existing smart method
792
        invocation. If in doubt chat to the bzr network team.
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
793
        """
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
794
        if self._real_repository is None:
4509.2.2 by Martin Pool
Use only -Dhpssvfs for tracebacks, and document -Dhpssdetail
795
            if 'hpssvfs' in debug.debug_flags:
4347.1.1 by Robert Collins
Show a traceback when VFS operations are started on a smart server hosted repository.
796
                import traceback
797
                warning('VFS Repository access triggered\n%s',
798
                    ''.join(traceback.format_stack()))
4307.2.4 by Robert Collins
Enable caching of negative revision lookups in RemoteRepository write locks when no _real_repository has been constructed.
799
            self._unstacked_provider.missing_keys.clear()
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
800
            self.bzrdir._ensure_real()
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
801
            self._set_real_repository(
802
                self.bzrdir._real_bzrdir.open_repository())
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
803
3533.3.1 by Andrew Bennetts
Remove duplication of error translation in bzrlib/remote.py.
804
    def _translate_error(self, err, **context):
805
        self.bzrdir._translate_error(err, repository=self, **context)
806
2988.1.2 by Robert Collins
New Repository API find_text_key_references for use by reconcile and check.
807
    def find_text_key_references(self):
808
        """Find the text key references within the repository.
809
810
        :return: a dictionary mapping (file_id, revision_id) tuples to altered file-ids to an iterable of
811
        revision_ids. Each altered file-ids has the exact revision_ids that
812
        altered it listed explicitly.
813
        :return: A dictionary mapping text keys ((fileid, revision_id) tuples)
814
            to whether they were referred to by the inventory of the
815
            revision_id that they contain. The inventory texts from all present
816
            revision ids are assessed to generate this report.
817
        """
818
        self._ensure_real()
819
        return self._real_repository.find_text_key_references()
820
2988.1.3 by Robert Collins
Add a new repositoy method _generate_text_key_index for use by reconcile/check.
821
    def _generate_text_key_index(self):
822
        """Generate a new text key index for the repository.
823
824
        This is an expensive function that will take considerable time to run.
825
826
        :return: A dict mapping (file_id, revision_id) tuples to a list of
827
            parents, also (file_id, revision_id) tuples.
828
        """
829
        self._ensure_real()
830
        return self._real_repository._generate_text_key_index()
831
3287.6.4 by Robert Collins
Fix up deprecation warnings for get_revision_graph.
832
    def _get_revision_graph(self, revision_id):
833
        """Private method for using with old (< 1.2) servers to fallback."""
2018.5.67 by Wouter van Heyst
Implement RemoteRepository.get_revision_graph (Wouter van Heyst, Robert Collins)
834
        if revision_id is None:
835
            revision_id = ''
2948.3.1 by John Arbash Meinel
Fix bug #158333, make sure that Repository.fetch(self) is properly a no-op for all Repository implementations.
836
        elif revision.is_null(revision_id):
2018.5.67 by Wouter van Heyst
Implement RemoteRepository.get_revision_graph (Wouter van Heyst, Robert Collins)
837
            return {}
838
839
        path = self.bzrdir._path_for_remote_call(self._client)
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
840
        response = self._call_expecting_body(
841
            'Repository.get_revision_graph', path, revision_id)
3245.4.58 by Andrew Bennetts
Unpack call_expecting_body's return value into variables, to avoid lots of ugly subscripting.
842
        response_tuple, response_handler = response
843
        if response_tuple[0] != 'ok':
844
            raise errors.UnexpectedSmartServerResponse(response_tuple)
845
        coded = response_handler.read_body_bytes()
3245.4.24 by Andrew Bennetts
Consistently raise errors from the server as ErrorFromSmartServer exceptions.
846
        if coded == '':
847
            # no revisions in this repository!
848
            return {}
849
        lines = coded.split('\n')
850
        revision_graph = {}
851
        for line in lines:
852
            d = tuple(line.split())
853
            revision_graph[d[0]] = d[1:]
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
854
3245.4.24 by Andrew Bennetts
Consistently raise errors from the server as ErrorFromSmartServer exceptions.
855
        return revision_graph
2018.5.67 by Wouter van Heyst
Implement RemoteRepository.get_revision_graph (Wouter van Heyst, Robert Collins)
856
4022.1.1 by Robert Collins
Refactoring of fetch to have a sender and sink component enabling splitting the logic over a network stream. (Robert Collins, Andrew Bennetts)
857
    def _get_sink(self):
858
        """See Repository._get_sink()."""
4022.1.6 by Robert Collins
Cherrypick and polish the RemoteSink for streaming push.
859
        return RemoteStreamSink(self)
4022.1.1 by Robert Collins
Refactoring of fetch to have a sender and sink component enabling splitting the logic over a network stream. (Robert Collins, Andrew Bennetts)
860
4060.1.3 by Robert Collins
Implement the separate source component for fetch - repository.StreamSource.
861
    def _get_source(self, to_format):
862
        """Return a source for streaming from this repository."""
863
        return RemoteStreamSource(self, to_format)
864
4307.2.3 by Robert Collins
Change RemoteRepository.has_revision to use get_parent_map to leverage the caching.
865
    @needs_read_lock
2018.5.40 by Robert Collins
Implement a remote Repository.has_revision method.
866
    def has_revision(self, revision_id):
4307.2.3 by Robert Collins
Change RemoteRepository.has_revision to use get_parent_map to leverage the caching.
867
        """True if this repository has a copy of the revision."""
868
        # Copy of bzrlib.repository.Repository.has_revision
869
        return revision_id in self.has_revisions((revision_id,))
2018.5.40 by Robert Collins
Implement a remote Repository.has_revision method.
870
4307.2.3 by Robert Collins
Change RemoteRepository.has_revision to use get_parent_map to leverage the caching.
871
    @needs_read_lock
3172.3.1 by Robert Collins
Repository has a new method ``has_revisions`` which signals the presence
872
    def has_revisions(self, revision_ids):
4307.2.3 by Robert Collins
Change RemoteRepository.has_revision to use get_parent_map to leverage the caching.
873
        """Probe to find out the presence of multiple revisions.
874
875
        :param revision_ids: An iterable of revision_ids.
876
        :return: A set of the revision_ids that were present.
877
        """
878
        # Copy of bzrlib.repository.Repository.has_revisions
879
        parent_map = self.get_parent_map(revision_ids)
880
        result = set(parent_map)
881
        if _mod_revision.NULL_REVISION in revision_ids:
882
            result.add(_mod_revision.NULL_REVISION)
3172.3.1 by Robert Collins
Repository has a new method ``has_revisions`` which signals the presence
883
        return result
884
4509.3.37 by Martin Pool
Remove RepositoryBase; make _has_same_location private
885
    def _has_same_fallbacks(self, other_repo):
886
        """Returns true if the repositories have the same fallbacks."""
887
        # XXX: copied from Repository; it should be unified into a base class
888
        # <https://bugs.edge.launchpad.net/bzr/+bug/401622>
889
        my_fb = self._fallback_repositories
890
        other_fb = other_repo._fallback_repositories
891
        if len(my_fb) != len(other_fb):
892
            return False
893
        for f, g in zip(my_fb, other_fb):
894
            if not f.has_same_location(g):
895
                return False
896
        return True
897
2617.6.9 by Robert Collins
Merge bzr.dev.
898
    def has_same_location(self, other):
4509.3.23 by Martin Pool
Comment on has_same_location variation
899
        # TODO: Move to RepositoryBase and unify with the regular Repository
900
        # one; unfortunately the tests rely on slightly different behaviour at
901
        # present -- mbp 20090710
4088.3.1 by Benjamin Peterson
compare types with 'is' not ==
902
        return (self.__class__ is other.__class__ and
2592.3.162 by Robert Collins
Remove some arbitrary differences from bzr.dev.
903
                self.bzrdir.transport.base == other.bzrdir.transport.base)
3835.1.1 by Aaron Bentley
Stack get_parent_map on fallback repos
904
2490.2.21 by Aaron Bentley
Rename graph to deprecated_graph
905
    def get_graph(self, other_repository=None):
906
        """Return the graph for this repository format"""
3835.1.17 by Aaron Bentley
Fix stacking bug
907
        parents_provider = self._make_parents_provider(other_repository)
3441.5.24 by Andrew Bennetts
Remove RemoteGraph experiment.
908
        return graph.Graph(parents_provider)
2490.2.5 by Aaron Bentley
Use GraphWalker.unique_ancestor to determine merge base
909
4913.4.2 by Jelmer Vernooij
Add Repository.get_known_graph_ancestry.
910
    @needs_read_lock
911
    def get_known_graph_ancestry(self, revision_ids):
912
        """Return the known graph for a set of revision ids and their ancestors.
913
        """
914
        st = static_tuple.StaticTuple
915
        revision_keys = [st(r_id).intern() for r_id in revision_ids]
916
        known_graph = self.revisions.get_known_graph_ancestry(revision_keys)
917
        return graph.GraphThunkIdsToKeys(known_graph)
918
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
919
    def gather_stats(self, revid=None, committers=None):
2018.5.62 by Robert Collins
Stub out RemoteRepository.gather_stats while its implemented in parallel.
920
        """See Repository.gather_stats()."""
2018.10.3 by v.ladeuil+lp at free
more tests for gather_stats
921
        path = self.bzrdir._path_for_remote_call(self._client)
2948.3.4 by John Arbash Meinel
Repository.gather_stats() validly can get None for the revid.
922
        # revid can be None to indicate no revisions, not just NULL_REVISION
923
        if revid is None or revision.is_null(revid):
2018.10.3 by v.ladeuil+lp at free
more tests for gather_stats
924
            fmt_revid = ''
925
        else:
2018.5.83 by Andrew Bennetts
Fix some test failures caused by the switch from unicode to UTF-8-encoded strs for revision IDs.
926
            fmt_revid = revid
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
927
        if committers is None or not committers:
2018.10.3 by v.ladeuil+lp at free
more tests for gather_stats
928
            fmt_committers = 'no'
929
        else:
930
            fmt_committers = 'yes'
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
931
        response_tuple, response_handler = self._call_expecting_body(
2018.5.153 by Andrew Bennetts
Rename call2 to call_expecting_body, and other small changes prompted by review.
932
            'Repository.gather_stats', path, fmt_revid, fmt_committers)
3245.4.58 by Andrew Bennetts
Unpack call_expecting_body's return value into variables, to avoid lots of ugly subscripting.
933
        if response_tuple[0] != 'ok':
934
            raise errors.UnexpectedSmartServerResponse(response_tuple)
2018.10.3 by v.ladeuil+lp at free
more tests for gather_stats
935
3245.4.58 by Andrew Bennetts
Unpack call_expecting_body's return value into variables, to avoid lots of ugly subscripting.
936
        body = response_handler.read_body_bytes()
2018.10.3 by v.ladeuil+lp at free
more tests for gather_stats
937
        result = {}
938
        for line in body.split('\n'):
939
            if not line:
940
                continue
941
            key, val_text = line.split(':')
942
            if key in ('revisions', 'size', 'committers'):
943
                result[key] = int(val_text)
944
            elif key in ('firstrev', 'latestrev'):
945
                values = val_text.split(' ')[1:]
946
                result[key] = (float(values[0]), long(values[1]))
947
948
        return result
2018.5.62 by Robert Collins
Stub out RemoteRepository.gather_stats while its implemented in parallel.
949
3140.1.2 by Aaron Bentley
Add ability to find branches inside repositories
950
    def find_branches(self, using=False):
951
        """See Repository.find_branches()."""
952
        # should be an API call to the server.
953
        self._ensure_real()
954
        return self._real_repository.find_branches(using=using)
955
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
956
    def get_physical_lock_status(self):
957
        """See Repository.get_physical_lock_status()."""
3015.2.10 by Robert Collins
Fix regression due to other pack related fixes in tests with packs not-default.
958
        # should be an API call to the server.
959
        self._ensure_real()
960
        return self._real_repository.get_physical_lock_status()
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
961
2617.6.1 by Robert Collins
* New method on Repository - ``start_write_group``, ``end_write_group``
962
    def is_in_write_group(self):
963
        """Return True if there is an open write group.
964
965
        write groups are only applicable locally for the smart server..
966
        """
967
        if self._real_repository:
968
            return self._real_repository.is_in_write_group()
969
970
    def is_locked(self):
971
        return self._lock_count >= 1
972
2018.5.57 by Robert Collins
Implement RemoteRepository.is_shared (Robert Collins, Vincent Ladeuil).
973
    def is_shared(self):
974
        """See Repository.is_shared()."""
975
        path = self.bzrdir._path_for_remote_call(self._client)
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
976
        response = self._call('Repository.is_shared', path)
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
977
        if response[0] not in ('yes', 'no'):
978
            raise SmartProtocolError('unexpected response code %s' % (response,))
2018.5.57 by Robert Collins
Implement RemoteRepository.is_shared (Robert Collins, Vincent Ladeuil).
979
        return response[0] == 'yes'
980
2904.1.1 by Robert Collins
* New method ``bzrlib.repository.Repository.is_write_locked`` useful for
981
    def is_write_locked(self):
982
        return self._lock_mode == 'w'
983
4840.2.7 by Vincent Ladeuil
Move the _warn_if_deprecated call from repo.__init__ to
984
    def _warn_if_deprecated(self, branch=None):
985
        # If we have a real repository, the check will be done there, if we
986
        # don't the check will be done remotely.
987
        pass
988
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
989
    def lock_read(self):
990
        # wrong eventually - want a local lock cache context
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
991
        if not self._lock_mode:
4731.1.2 by Andrew Bennetts
Refactor to reduce duplication.
992
            self._note_lock('r')
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
993
            self._lock_mode = 'r'
994
            self._lock_count = 1
4190.1.1 by Robert Collins
Negatively cache misses during read-locks in RemoteRepository.
995
            self._unstacked_provider.enable_cache(cache_misses=True)
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
996
            if self._real_repository is not None:
997
                self._real_repository.lock_read()
4379.2.1 by John Arbash Meinel
Change the fallback repository code to only lock/unlock on transition.
998
            for repo in self._fallback_repositories:
999
                repo.lock_read()
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1000
        else:
1001
            self._lock_count += 1
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1002
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1003
    def _remote_lock_write(self, token):
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1004
        path = self.bzrdir._path_for_remote_call(self._client)
1005
        if token is None:
1006
            token = ''
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
1007
        err_context = {'token': token}
1008
        response = self._call('Repository.lock_write', path, token,
1009
                              **err_context)
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1010
        if response[0] == 'ok':
1011
            ok, token = response
1012
            return token
1013
        else:
2555.1.1 by Martin Pool
Remove use of 'assert False' to raise an exception unconditionally
1014
            raise errors.UnexpectedSmartServerResponse(response)
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
1015
3692.1.1 by Andrew Bennetts
Make RemoteBranch.lock_write lock the repository too.
1016
    def lock_write(self, token=None, _skip_rpc=False):
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1017
        if not self._lock_mode:
4731.1.2 by Andrew Bennetts
Refactor to reduce duplication.
1018
            self._note_lock('w')
3692.1.1 by Andrew Bennetts
Make RemoteBranch.lock_write lock the repository too.
1019
            if _skip_rpc:
1020
                if self._lock_token is not None:
1021
                    if token != self._lock_token:
3695.1.1 by Andrew Bennetts
Remove some unused imports and fix a couple of trivially broken raise statements.
1022
                        raise errors.TokenMismatch(token, self._lock_token)
3692.1.1 by Andrew Bennetts
Make RemoteBranch.lock_write lock the repository too.
1023
                self._lock_token = token
1024
            else:
1025
                self._lock_token = self._remote_lock_write(token)
3015.2.9 by Robert Collins
Handle repositories that do not allow remote locking, like pack repositories, in the client side remote server proxy objects.
1026
            # if self._lock_token is None, then this is something like packs or
1027
            # svn where we don't get to lock the repo, or a weave style repository
1028
            # where we cannot lock it over the wire and attempts to do so will
1029
            # fail.
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1030
            if self._real_repository is not None:
1031
                self._real_repository.lock_write(token=self._lock_token)
1032
            if token is not None:
1033
                self._leave_lock = True
1034
            else:
1035
                self._leave_lock = False
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1036
            self._lock_mode = 'w'
1037
            self._lock_count = 1
4307.2.4 by Robert Collins
Enable caching of negative revision lookups in RemoteRepository write locks when no _real_repository has been constructed.
1038
            cache_misses = self._real_repository is None
1039
            self._unstacked_provider.enable_cache(cache_misses=cache_misses)
4379.2.1 by John Arbash Meinel
Change the fallback repository code to only lock/unlock on transition.
1040
            for repo in self._fallback_repositories:
1041
                # Writes don't affect fallback repos
1042
                repo.lock_read()
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1043
        elif self._lock_mode == 'r':
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1044
            raise errors.ReadOnlyError(self)
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1045
        else:
1046
            self._lock_count += 1
3015.2.9 by Robert Collins
Handle repositories that do not allow remote locking, like pack repositories, in the client side remote server proxy objects.
1047
        return self._lock_token or None
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1048
1049
    def leave_lock_in_place(self):
3015.2.9 by Robert Collins
Handle repositories that do not allow remote locking, like pack repositories, in the client side remote server proxy objects.
1050
        if not self._lock_token:
1051
            raise NotImplementedError(self.leave_lock_in_place)
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1052
        self._leave_lock = True
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1053
1054
    def dont_leave_lock_in_place(self):
3015.2.9 by Robert Collins
Handle repositories that do not allow remote locking, like pack repositories, in the client side remote server proxy objects.
1055
        if not self._lock_token:
3015.2.15 by Robert Collins
Review feedback.
1056
            raise NotImplementedError(self.dont_leave_lock_in_place)
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1057
        self._leave_lock = False
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1058
1059
    def _set_real_repository(self, repository):
1060
        """Set the _real_repository for this repository.
1061
1062
        :param repository: The repository to fallback to for non-hpss
1063
            implemented operations.
1064
        """
4053.1.5 by Robert Collins
Review feedback on RemoteRepository._set_real_revision.
1065
        if self._real_repository is not None:
1066
            # Replacing an already set real repository.
1067
            # We cannot do this [currently] if the repository is locked -
1068
            # synchronised state might be lost.
1069
            if self.is_locked():
1070
                raise AssertionError('_real_repository is already set')
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
1071
        if isinstance(repository, RemoteRepository):
1072
            raise AssertionError()
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1073
        self._real_repository = repository
4226.2.5 by Robert Collins
Fix handling of fallback repositories some more.
1074
        # three code paths happen here:
1075
        # 1) old servers, RemoteBranch.open() calls _ensure_real before setting
1076
        # up stacking. In this case self._fallback_repositories is [], and the
1077
        # real repo is already setup. Preserve the real repo and
1078
        # RemoteRepository.add_fallback_repository will avoid adding
1079
        # duplicates.
1080
        # 2) new servers, RemoteBranch.open() sets up stacking, and when
1081
        # ensure_real is triggered from a branch, the real repository to
1082
        # set already has a matching list with separate instances, but
1083
        # as they are also RemoteRepositories we don't worry about making the
1084
        # lists be identical.
1085
        # 3) new servers, RemoteRepository.ensure_real is triggered before
1086
        # RemoteBranch.ensure real, in this case we get a repo with no fallbacks
1087
        # and need to populate it.
1088
        if (self._fallback_repositories and
1089
            len(self._real_repository._fallback_repositories) !=
4226.2.2 by Robert Collins
Fix setting config options to support unicode values and don't attempt to reset repositories _fallback_repositories as the simple approach fails to work.
1090
            len(self._fallback_repositories)):
1091
            if len(self._real_repository._fallback_repositories):
1092
                raise AssertionError(
1093
                    "cannot cleanly remove existing _fallback_repositories")
3691.2.12 by Martin Pool
Add test for coping without Branch.get_stacked_on_url
1094
        for fb in self._fallback_repositories:
1095
            self._real_repository.add_fallback_repository(fb)
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1096
        if self._lock_mode == 'w':
1097
            # if we are already locked, the real repository must be able to
1098
            # acquire the lock with our token.
1099
            self._real_repository.lock_write(self._lock_token)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1100
        elif self._lock_mode == 'r':
1101
            self._real_repository.lock_read()
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
1102
2617.6.1 by Robert Collins
* New method on Repository - ``start_write_group``, ``end_write_group``
1103
    def start_write_group(self):
1104
        """Start a write group on the decorated repository.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
1105
4031.3.3 by Matt Nordhoff
Review tweaks from Ben Finney
1106
        Smart methods perform operations in a single step so this API
2617.6.6 by Robert Collins
Some review feedback.
1107
        is not really applicable except as a compatibility thunk
2617.6.1 by Robert Collins
* New method on Repository - ``start_write_group``, ``end_write_group``
1108
        for older plugins that don't use e.g. the CommitBuilder
1109
        facility.
1110
        """
1111
        self._ensure_real()
1112
        return self._real_repository.start_write_group()
1113
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1114
    def _unlock(self, token):
1115
        path = self.bzrdir._path_for_remote_call(self._client)
3015.2.9 by Robert Collins
Handle repositories that do not allow remote locking, like pack repositories, in the client side remote server proxy objects.
1116
        if not token:
1117
            # with no token the remote repository is not persistently locked.
1118
            return
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
1119
        err_context = {'token': token}
1120
        response = self._call('Repository.unlock', path, token,
1121
                              **err_context)
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1122
        if response == ('ok',):
1123
            return
1124
        else:
2555.1.1 by Martin Pool
Remove use of 'assert False' to raise an exception unconditionally
1125
            raise errors.UnexpectedSmartServerResponse(response)
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1126
4634.85.9 by Andrew Bennetts
Add some experimental decorators: @only_raises(..) and @cleanup_method.
1127
    @only_raises(errors.LockNotHeld, errors.LockBroken)
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
1128
    def unlock(self):
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
1129
        if not self._lock_count:
4509.3.25 by Martin Pool
Add an option for unlock errors to be non-fatal
1130
            return lock.cant_unlock_not_held(self)
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1131
        self._lock_count -= 1
2592.3.244 by Martin Pool
unlock while in a write group now aborts the write group, unlocks, and errors.
1132
        if self._lock_count > 0:
1133
            return
3835.1.8 by Aaron Bentley
Make UnstackedParentsProvider manage the cache
1134
        self._unstacked_provider.disable_cache()
2592.3.244 by Martin Pool
unlock while in a write group now aborts the write group, unlocks, and errors.
1135
        old_mode = self._lock_mode
1136
        self._lock_mode = None
1137
        try:
1138
            # The real repository is responsible at present for raising an
1139
            # exception if it's in an unfinished write group.  However, it
1140
            # normally will *not* actually remove the lock from disk - that's
1141
            # done by the server on receiving the Repository.unlock call.
1142
            # This is just to let the _real_repository stay up to date.
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1143
            if self._real_repository is not None:
1144
                self._real_repository.unlock()
2592.3.244 by Martin Pool
unlock while in a write group now aborts the write group, unlocks, and errors.
1145
        finally:
1146
            # The rpc-level lock should be released even if there was a
1147
            # problem releasing the vfs-based lock.
1148
            if old_mode == 'w':
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
1149
                # Only write-locked repositories need to make a remote method
4031.3.1 by Frank Aspell
Fixing various typos
1150
                # call to perform the unlock.
2592.3.244 by Martin Pool
unlock while in a write group now aborts the write group, unlocks, and errors.
1151
                old_token = self._lock_token
1152
                self._lock_token = None
1153
                if not self._leave_lock:
1154
                    self._unlock(old_token)
4379.2.1 by John Arbash Meinel
Change the fallback repository code to only lock/unlock on transition.
1155
        # Fallbacks are always 'lock_read()' so we don't pay attention to
1156
        # self._leave_lock
1157
        for repo in self._fallback_repositories:
1158
            repo.unlock()
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
1159
1160
    def break_lock(self):
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1161
        # should hand off to the network
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
1162
        self._ensure_real()
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
1163
        return self._real_repository.break_lock()
1164
2018.18.8 by Ian Clatworthy
Tarball proxy code & tests
1165
    def _get_tarball(self, compression):
2814.10.2 by Andrew Bennetts
Make the fallback a little tidier.
1166
        """Return a TemporaryFile containing a repository tarball.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
1167
2814.10.2 by Andrew Bennetts
Make the fallback a little tidier.
1168
        Returns None if the server does not support sending tarballs.
1169
        """
2018.18.25 by Martin Pool
Repository.tarball fixes for python2.4
1170
        import tempfile
2018.18.8 by Ian Clatworthy
Tarball proxy code & tests
1171
        path = self.bzrdir._path_for_remote_call(self._client)
3297.3.3 by Andrew Bennetts
SmartClientRequestProtocol*.read_response_tuple can now raise UnknownSmartMethod. Callers no longer need to do their own ad hoc unknown smart method error detection.
1172
        try:
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
1173
            response, protocol = self._call_expecting_body(
3297.3.3 by Andrew Bennetts
SmartClientRequestProtocol*.read_response_tuple can now raise UnknownSmartMethod. Callers no longer need to do their own ad hoc unknown smart method error detection.
1174
                'Repository.tarball', path, compression)
1175
        except errors.UnknownSmartMethod:
1176
            protocol.cancel_read_body()
1177
            return None
2018.18.8 by Ian Clatworthy
Tarball proxy code & tests
1178
        if response[0] == 'ok':
1179
            # Extract the tarball and return it
2018.18.25 by Martin Pool
Repository.tarball fixes for python2.4
1180
            t = tempfile.NamedTemporaryFile()
1181
            # TODO: rpc layer should read directly into it...
1182
            t.write(protocol.read_body_bytes())
1183
            t.seek(0)
1184
            return t
2814.10.1 by Andrew Bennetts
Cope gracefully if the server doesn't support the Repository.tarball smart request.
1185
        raise errors.UnexpectedSmartServerResponse(response)
2018.18.8 by Ian Clatworthy
Tarball proxy code & tests
1186
2440.1.1 by Martin Pool
Add new Repository.sprout,
1187
    def sprout(self, to_bzrdir, revision_id=None):
1188
        # TODO: Option to control what format is created?
3047.1.1 by Andrew Bennetts
Fix for bug 164626, add test that Repository.sprout preserves format.
1189
        self._ensure_real()
3047.1.4 by Andrew Bennetts
Simplify RemoteRepository.sprout thanks to review comments.
1190
        dest_repo = self._real_repository._format.initialize(to_bzrdir,
1191
                                                             shared=False)
2535.3.17 by Andrew Bennetts
[broken] Closer to a working Repository.fetch_revisions smart request.
1192
        dest_repo.fetch(self, revision_id=revision_id)
1193
        return dest_repo
2440.1.1 by Martin Pool
Add new Repository.sprout,
1194
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1195
    ### These methods are just thin shims to the VFS object for now.
1196
1197
    def revision_tree(self, revision_id):
1198
        self._ensure_real()
1199
        return self._real_repository.revision_tree(revision_id)
1200
2520.4.113 by Aaron Bentley
Avoid peeking at Repository._serializer
1201
    def get_serializer_format(self):
1202
        self._ensure_real()
1203
        return self._real_repository.get_serializer_format()
1204
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1205
    def get_commit_builder(self, branch, parents, config, timestamp=None,
1206
                           timezone=None, committer=None, revprops=None,
1207
                           revision_id=None):
1208
        # FIXME: It ought to be possible to call this without immediately
1209
        # triggering _ensure_real.  For now it's the easiest thing to do.
1210
        self._ensure_real()
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
1211
        real_repo = self._real_repository
1212
        builder = real_repo.get_commit_builder(branch, parents,
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1213
                config, timestamp=timestamp, timezone=timezone,
1214
                committer=committer, revprops=revprops, revision_id=revision_id)
1215
        return builder
1216
3221.12.1 by Robert Collins
Backport development1 format (stackable packs) to before-shallow-branches.
1217
    def add_fallback_repository(self, repository):
1218
        """Add a repository to use for looking up data not held locally.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
1219
3221.12.1 by Robert Collins
Backport development1 format (stackable packs) to before-shallow-branches.
1220
        :param repository: A repository.
1221
        """
4118.1.1 by Andrew Bennetts
Fix performance regression (many small round-trips) when pushing to a remote pack, and tidy the tests.
1222
        if not self._format.supports_external_lookups:
1223
            raise errors.UnstackableRepositoryFormat(
1224
                self._format.network_name(), self.base)
3221.12.1 by Robert Collins
Backport development1 format (stackable packs) to before-shallow-branches.
1225
        # We need to accumulate additional repositories here, to pass them in
1226
        # on various RPC's.
4035.2.3 by Robert Collins
Fix trailing whitespace.
1227
        #
4379.2.2 by John Arbash Meinel
Change the Repository.add_fallback_repository() contract slightly.
1228
        if self.is_locked():
1229
            # We will call fallback.unlock() when we transition to the unlocked
1230
            # state, so always add a lock here. If a caller passes us a locked
1231
            # repository, they are responsible for unlocking it later.
1232
            repository.lock_read()
3221.12.1 by Robert Collins
Backport development1 format (stackable packs) to before-shallow-branches.
1233
        self._fallback_repositories.append(repository)
4035.2.2 by Robert Collins
Minor tweaks to fix failing tests.
1234
        # If self._real_repository was parameterised already (e.g. because a
1235
        # _real_branch had its get_stacked_on_url method called), then the
1236
        # repository to be added may already be in the _real_repositories list.
4035.2.1 by Andrew Bennetts
Fix unnecessary get_parent_map calls after insert_stream during push.
1237
        if self._real_repository is not None:
4226.2.5 by Robert Collins
Fix handling of fallback repositories some more.
1238
            fallback_locations = [repo.bzrdir.root_transport.base for repo in
1239
                self._real_repository._fallback_repositories]
1240
            if repository.bzrdir.root_transport.base not in fallback_locations:
4035.2.2 by Robert Collins
Minor tweaks to fix failing tests.
1241
                self._real_repository.add_fallback_repository(repository)
3221.12.1 by Robert Collins
Backport development1 format (stackable packs) to before-shallow-branches.
1242
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1243
    def add_inventory(self, revid, inv, parents):
1244
        self._ensure_real()
1245
        return self._real_repository.add_inventory(revid, inv, parents)
1246
3879.2.2 by John Arbash Meinel
Rename add_inventory_delta to add_inventory_by_delta.
1247
    def add_inventory_by_delta(self, basis_revision_id, delta, new_revision_id,
5076.1.1 by Jelmer Vernooij
Allow additional arguments to RemoteRepository.add_inventory_by_delta().
1248
            parents, basis_inv=None, propagate_caches=False):
3775.2.1 by Robert Collins
Create bzrlib.repository.Repository.add_inventory_delta for adding inventories via deltas.
1249
        self._ensure_real()
3879.2.2 by John Arbash Meinel
Rename add_inventory_delta to add_inventory_by_delta.
1250
        return self._real_repository.add_inventory_by_delta(basis_revision_id,
5076.1.2 by Jelmer Vernooij
pass the keyword args as keyword args, per Rob's review.
1251
            delta, new_revision_id, parents, basis_inv=basis_inv,
1252
            propagate_caches=propagate_caches)
3775.2.1 by Robert Collins
Create bzrlib.repository.Repository.add_inventory_delta for adding inventories via deltas.
1253
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1254
    def add_revision(self, rev_id, rev, inv=None, config=None):
1255
        self._ensure_real()
1256
        return self._real_repository.add_revision(
1257
            rev_id, rev, inv=inv, config=config)
1258
1259
    @needs_read_lock
1260
    def get_inventory(self, revision_id):
1261
        self._ensure_real()
1262
        return self._real_repository.get_inventory(revision_id)
1263
4476.3.86 by Andrew Bennetts
Fix bug in declaration of RemoteRepository.iter_inventories that was causing intermittent failures in test_iter_inventories_is_ordered.
1264
    def iter_inventories(self, revision_ids, ordering=None):
3169.2.1 by Robert Collins
New method ``iter_inventories`` on Repository for access to many
1265
        self._ensure_real()
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1266
        return self._real_repository.iter_inventories(revision_ids, ordering)
3169.2.1 by Robert Collins
New method ``iter_inventories`` on Repository for access to many
1267
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1268
    @needs_read_lock
1269
    def get_revision(self, revision_id):
1270
        self._ensure_real()
1271
        return self._real_repository.get_revision(revision_id)
1272
1273
    def get_transaction(self):
1274
        self._ensure_real()
1275
        return self._real_repository.get_transaction()
1276
1277
    @needs_read_lock
2018.5.138 by Robert Collins
Merge bzr.dev.
1278
    def clone(self, a_bzrdir, revision_id=None):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1279
        self._ensure_real()
2018.5.138 by Robert Collins
Merge bzr.dev.
1280
        return self._real_repository.clone(a_bzrdir, revision_id=revision_id)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1281
1282
    def make_working_trees(self):
3349.1.1 by Aaron Bentley
Enable setting and getting make_working_trees for all repositories
1283
        """See Repository.make_working_trees"""
1284
        self._ensure_real()
1285
        return self._real_repository.make_working_trees()
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1286
4145.1.2 by Robert Collins
Add a refresh_data method on Repository allowing cleaner handling of insertions into RemoteRepository objects with _real_repository instances.
1287
    def refresh_data(self):
1288
        """Re-read any data needed to to synchronise with disk.
1289
1290
        This method is intended to be called after another repository instance
1291
        (such as one used by a smart server) has inserted data into the
1292
        repository. It may not be called during a write group, but may be
1293
        called at any other time.
1294
        """
1295
        if self.is_in_write_group():
1296
            raise errors.InternalBzrError(
1297
                "May not refresh_data while in a write group.")
1298
        if self._real_repository is not None:
1299
            self._real_repository.refresh_data()
1300
3184.1.9 by Robert Collins
* ``Repository.get_data_stream`` is now deprecated in favour of
1301
    def revision_ids_to_search_result(self, result_set):
1302
        """Convert a set of revision ids to a graph SearchResult."""
1303
        result_parents = set()
1304
        for parents in self.get_graph().get_parent_map(
1305
            result_set).itervalues():
1306
            result_parents.update(parents)
1307
        included_keys = result_set.intersection(result_parents)
1308
        start_keys = result_set.difference(included_keys)
1309
        exclude_keys = result_parents.difference(result_set)
1310
        result = graph.SearchResult(start_keys, exclude_keys,
1311
            len(result_set), result_set)
1312
        return result
1313
1314
    @needs_read_lock
1315
    def search_missing_revision_ids(self, other, revision_id=None, find_ghosts=True):
1316
        """Return the revision ids that other has that this does not.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
1317
3184.1.9 by Robert Collins
* ``Repository.get_data_stream`` is now deprecated in favour of
1318
        These are returned in topological order.
1319
1320
        revision_id: only return revision ids included by revision_id.
1321
        """
1322
        return repository.InterRepository.get(
1323
            other, self).search_missing_revision_ids(revision_id, find_ghosts)
1324
4070.9.2 by Andrew Bennetts
Rough prototype of allowing a SearchResult to be passed to fetch, and using that to improve network conversations.
1325
    def fetch(self, source, revision_id=None, pb=None, find_ghosts=False,
1326
            fetch_spec=None):
4145.1.1 by Robert Collins
Explicitly prevent fetching while the target repository is in a write group.
1327
        # No base implementation to use as RemoteRepository is not a subclass
1328
        # of Repository; so this is a copy of Repository.fetch().
4070.9.2 by Andrew Bennetts
Rough prototype of allowing a SearchResult to be passed to fetch, and using that to improve network conversations.
1329
        if fetch_spec is not None and revision_id is not None:
1330
            raise AssertionError(
1331
                "fetch_spec and revision_id are mutually exclusive.")
4145.1.1 by Robert Collins
Explicitly prevent fetching while the target repository is in a write group.
1332
        if self.is_in_write_group():
4145.1.3 by Robert Collins
NEWS conflicts.
1333
            raise errors.InternalBzrError(
1334
                "May not fetch while in a write group.")
4145.1.1 by Robert Collins
Explicitly prevent fetching while the target repository is in a write group.
1335
        # fast path same-url fetch operations
4509.3.19 by Martin Pool
RemoteRepository.fetch is not a no-op if there's different stacking
1336
        if (self.has_same_location(source)
1337
            and fetch_spec is None
4509.3.37 by Martin Pool
Remove RepositoryBase; make _has_same_location private
1338
            and self._has_same_fallbacks(source)):
2881.4.1 by Robert Collins
Move responsibility for detecting same-repo fetching from the
1339
            # check that last_revision is in 'from' and then return a
1340
            # no-operation.
1341
            if (revision_id is not None and
4145.1.5 by Robert Collins
More fixes from grabbing the Repository implementation of fetch for RemoteRepository.
1342
                not revision.is_null(revision_id)):
2881.4.1 by Robert Collins
Move responsibility for detecting same-repo fetching from the
1343
                self.get_revision(revision_id)
2592.4.5 by Martin Pool
Add Repository.base on all repositories.
1344
            return 0, []
4145.1.1 by Robert Collins
Explicitly prevent fetching while the target repository is in a write group.
1345
        # if there is no specific appropriate InterRepository, this will get
1346
        # the InterRepository base class, which raises an
1347
        # IncompatibleRepositories when asked to fetch.
4145.1.2 by Robert Collins
Add a refresh_data method on Repository allowing cleaner handling of insertions into RemoteRepository objects with _real_repository instances.
1348
        inter = repository.InterRepository.get(source, self)
4145.1.1 by Robert Collins
Explicitly prevent fetching while the target repository is in a write group.
1349
        return inter.fetch(revision_id=revision_id, pb=pb,
1350
            find_ghosts=find_ghosts, fetch_spec=fetch_spec)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1351
2520.4.54 by Aaron Bentley
Hang a create_bundle method off repository
1352
    def create_bundle(self, target, base, fileobj, format=None):
1353
        self._ensure_real()
1354
        self._real_repository.create_bundle(target, base, fileobj, format)
1355
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1356
    @needs_read_lock
2530.1.1 by Aaron Bentley
Make topological sorting optional for get_ancestry
1357
    def get_ancestry(self, revision_id, topo_sorted=True):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1358
        self._ensure_real()
2530.1.1 by Aaron Bentley
Make topological sorting optional for get_ancestry
1359
        return self._real_repository.get_ancestry(revision_id, topo_sorted)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1360
1361
    def fileids_altered_by_revision_ids(self, revision_ids):
1362
        self._ensure_real()
1363
        return self._real_repository.fileids_altered_by_revision_ids(revision_ids)
1364
3036.1.3 by Robert Collins
Privatise VersionedFileChecker.
1365
    def _get_versioned_file_checker(self, revisions, revision_versions_cache):
2745.6.1 by Aaron Bentley
Initial checking of knit graphs
1366
        self._ensure_real()
3036.1.3 by Robert Collins
Privatise VersionedFileChecker.
1367
        return self._real_repository._get_versioned_file_checker(
2745.6.50 by Andrew Bennetts
Remove find_bad_ancestors; it's not needed anymore.
1368
            revisions, revision_versions_cache)
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
1369
2708.1.7 by Aaron Bentley
Rename extract_files_bytes to iter_files_bytes
1370
    def iter_files_bytes(self, desired_files):
2708.1.9 by Aaron Bentley
Clean-up docs and imports
1371
        """See Repository.iter_file_bytes.
2708.1.3 by Aaron Bentley
Implement extract_files_bytes on Repository
1372
        """
1373
        self._ensure_real()
2708.1.7 by Aaron Bentley
Rename extract_files_bytes to iter_files_bytes
1374
        return self._real_repository.iter_files_bytes(desired_files)
2708.1.3 by Aaron Bentley
Implement extract_files_bytes on Repository
1375
3835.1.1 by Aaron Bentley
Stack get_parent_map on fallback repos
1376
    def get_parent_map(self, revision_ids):
3835.1.6 by Aaron Bentley
Reduce inefficiency when doing make_parents_provider frequently
1377
        """See bzrlib.Graph.get_parent_map()."""
3835.1.5 by Aaron Bentley
Fix make_parents_provider
1378
        return self._make_parents_provider().get_parent_map(revision_ids)
3835.1.1 by Aaron Bentley
Stack get_parent_map on fallback repos
1379
1380
    def _get_parent_map_rpc(self, keys):
3172.5.6 by Robert Collins
Create new smart server verb Repository.get_parent_map.
1381
        """Helper for get_parent_map that performs the RPC."""
3313.2.1 by Andrew Bennetts
Change _SmartClient's API to accept a medium and a base, rather than a _SharedConnection.
1382
        medium = self._client._medium
3453.4.10 by Andrew Bennetts
Change _is_remote_at_least to _is_remote_before.
1383
        if medium._is_remote_before((1, 2)):
3213.1.1 by Andrew Bennetts
Recover (by reconnecting) if the server turns out not to understand the new requests in 1.2 that send bodies.
1384
            # We already found out that the server can't understand
3213.1.3 by Andrew Bennetts
Fix typo in comment.
1385
            # Repository.get_parent_map requests, so just fetch the whole
3213.1.1 by Andrew Bennetts
Recover (by reconnecting) if the server turns out not to understand the new requests in 1.2 that send bodies.
1386
            # graph.
3948.3.7 by Martin Pool
Updated tests for RemoteRepository.get_parent_map on old servers.
1387
            #
1388
            # Note that this reads the whole graph, when only some keys are
1389
            # wanted.  On this old server there's no way (?) to get them all
1390
            # in one go, and the user probably will have seen a warning about
1391
            # the server being old anyhow.
1392
            rg = self._get_revision_graph(None)
4031.3.3 by Matt Nordhoff
Review tweaks from Ben Finney
1393
            # There is an API discrepancy between get_parent_map and
3389.1.1 by John Arbash Meinel
Fix bug #214894. Fix RemoteRepository.get_parent_map() when server is <v1.2
1394
            # get_revision_graph. Specifically, a "key:()" pair in
1395
            # get_revision_graph just means a node has no parents. For
1396
            # "get_parent_map" it means the node is a ghost. So fix up the
1397
            # graph to correct this.
1398
            #   https://bugs.launchpad.net/bzr/+bug/214894
1399
            # There is one other "bug" which is that ghosts in
1400
            # get_revision_graph() are not returned at all. But we won't worry
1401
            # about that for now.
1402
            for node_id, parent_ids in rg.iteritems():
1403
                if parent_ids == ():
1404
                    rg[node_id] = (NULL_REVISION,)
1405
            rg[NULL_REVISION] = ()
1406
            return rg
3213.1.1 by Andrew Bennetts
Recover (by reconnecting) if the server turns out not to understand the new requests in 1.2 that send bodies.
1407
3172.5.6 by Robert Collins
Create new smart server verb Repository.get_parent_map.
1408
        keys = set(keys)
3373.5.2 by John Arbash Meinel
Add repository_implementation tests for get_parent_map
1409
        if None in keys:
1410
            raise ValueError('get_parent_map(None) is not valid')
3172.5.6 by Robert Collins
Create new smart server verb Repository.get_parent_map.
1411
        if NULL_REVISION in keys:
1412
            keys.discard(NULL_REVISION)
1413
            found_parents = {NULL_REVISION:()}
1414
            if not keys:
1415
                return found_parents
1416
        else:
1417
            found_parents = {}
3211.5.1 by Robert Collins
Change the smart server get_parents method to take a graph search to exclude already recieved parents from. This prevents history shortcuts causing huge numbers of duplicates.
1418
        # TODO(Needs analysis): We could assume that the keys being requested
1419
        # from get_parent_map are in a breadth first search, so typically they
1420
        # will all be depth N from some common parent, and we don't have to
1421
        # have the server iterate from the root parent, but rather from the
1422
        # keys we're searching; and just tell the server the keyspace we
1423
        # already have; but this may be more traffic again.
1424
1425
        # Transform self._parents_map into a search request recipe.
1426
        # TODO: Manage this incrementally to avoid covering the same path
1427
        # repeatedly. (The server will have to on each request, but the less
1428
        # work done the better).
4190.1.3 by Robert Collins
Allow optional inclusion of ghost data in server get_parent_map calls.
1429
        #
1430
        # Negative caching notes:
1431
        # new server sends missing when a request including the revid
1432
        # 'include-missing:' is present in the request.
1433
        # missing keys are serialised as missing:X, and we then call
1434
        # provider.note_missing(X) for-all X
3835.1.8 by Aaron Bentley
Make UnstackedParentsProvider manage the cache
1435
        parents_map = self._unstacked_provider.get_cached_map()
3213.1.8 by Andrew Bennetts
Merge from bzr.dev.
1436
        if parents_map is None:
1437
            # Repository is not locked, so there's no cache.
1438
            parents_map = {}
4190.1.3 by Robert Collins
Allow optional inclusion of ghost data in server get_parent_map calls.
1439
        # start_set is all the keys in the cache
3213.1.8 by Andrew Bennetts
Merge from bzr.dev.
1440
        start_set = set(parents_map)
4190.1.3 by Robert Collins
Allow optional inclusion of ghost data in server get_parent_map calls.
1441
        # result set is all the references to keys in the cache
3211.5.1 by Robert Collins
Change the smart server get_parents method to take a graph search to exclude already recieved parents from. This prevents history shortcuts causing huge numbers of duplicates.
1442
        result_parents = set()
3213.1.8 by Andrew Bennetts
Merge from bzr.dev.
1443
        for parents in parents_map.itervalues():
3211.5.1 by Robert Collins
Change the smart server get_parents method to take a graph search to exclude already recieved parents from. This prevents history shortcuts causing huge numbers of duplicates.
1444
            result_parents.update(parents)
1445
        stop_keys = result_parents.difference(start_set)
4190.1.4 by Robert Collins
Cache ghosts when we can get them from a RemoteRepository in get_parent_map.
1446
        # We don't need to send ghosts back to the server as a position to
1447
        # stop either.
1448
        stop_keys.difference_update(self._unstacked_provider.missing_keys)
4214.2.5 by Andrew Bennetts
Fix the bug.
1449
        key_count = len(parents_map)
1450
        if (NULL_REVISION in result_parents
1451
            and NULL_REVISION in self._unstacked_provider.missing_keys):
1452
            # If we pruned NULL_REVISION from the stop_keys because it's also
1453
            # in our cache of "missing" keys we need to increment our key count
1454
            # by 1, because the reconsitituted SearchResult on the server will
1455
            # still consider NULL_REVISION to be an included key.
1456
            key_count += 1
3211.5.1 by Robert Collins
Change the smart server get_parents method to take a graph search to exclude already recieved parents from. This prevents history shortcuts causing huge numbers of duplicates.
1457
        included_keys = start_set.intersection(result_parents)
1458
        start_set.difference_update(included_keys)
4214.2.5 by Andrew Bennetts
Fix the bug.
1459
        recipe = ('manual', start_set, stop_keys, key_count)
3842.3.20 by Andrew Bennetts
Re-revert changes from another thread that accidentally got reinstated here.
1460
        body = self._serialise_search_recipe(recipe)
3172.5.6 by Robert Collins
Create new smart server verb Repository.get_parent_map.
1461
        path = self.bzrdir._path_for_remote_call(self._client)
1462
        for key in keys:
3360.2.8 by Martin Pool
Change assertion to a plain raise
1463
            if type(key) is not str:
1464
                raise ValueError(
1465
                    "key %r not a plain string" % (key,))
3172.5.8 by Robert Collins
Review feedback.
1466
        verb = 'Repository.get_parent_map'
4190.1.4 by Robert Collins
Cache ghosts when we can get them from a RemoteRepository in get_parent_map.
1467
        args = (path, 'include-missing:') + tuple(keys)
3297.3.3 by Andrew Bennetts
SmartClientRequestProtocol*.read_response_tuple can now raise UnknownSmartMethod. Callers no longer need to do their own ad hoc unknown smart method error detection.
1468
        try:
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
1469
            response = self._call_with_body_bytes_expecting_body(
3842.3.20 by Andrew Bennetts
Re-revert changes from another thread that accidentally got reinstated here.
1470
                verb, args, body)
3297.3.3 by Andrew Bennetts
SmartClientRequestProtocol*.read_response_tuple can now raise UnknownSmartMethod. Callers no longer need to do their own ad hoc unknown smart method error detection.
1471
        except errors.UnknownSmartMethod:
3213.1.2 by Andrew Bennetts
Add test for reconnection if get_parent_map is unknown by the server.
1472
            # Server does not support this method, so get the whole graph.
3213.1.1 by Andrew Bennetts
Recover (by reconnecting) if the server turns out not to understand the new requests in 1.2 that send bodies.
1473
            # Worse, we have to force a disconnection, because the server now
1474
            # doesn't realise it has a body on the wire to consume, so the
1475
            # only way to recover is to abandon the connection.
3213.1.6 by Andrew Bennetts
Emit warnings when forcing a reconnect.
1476
            warning(
1477
                'Server is too old for fast get_parent_map, reconnecting.  '
1478
                '(Upgrade the server to Bazaar 1.2 to avoid this)')
3213.1.1 by Andrew Bennetts
Recover (by reconnecting) if the server turns out not to understand the new requests in 1.2 that send bodies.
1479
            medium.disconnect()
1480
            # To avoid having to disconnect repeatedly, we keep track of the
1481
            # fact the server doesn't understand remote methods added in 1.2.
3453.4.9 by Andrew Bennetts
Rename _remote_is_not to _remember_remote_is_before.
1482
            medium._remember_remote_is_before((1, 2))
3948.3.7 by Martin Pool
Updated tests for RemoteRepository.get_parent_map on old servers.
1483
            # Recurse just once and we should use the fallback code.
1484
            return self._get_parent_map_rpc(keys)
3245.4.58 by Andrew Bennetts
Unpack call_expecting_body's return value into variables, to avoid lots of ugly subscripting.
1485
        response_tuple, response_handler = response
1486
        if response_tuple[0] not in ['ok']:
1487
            response_handler.cancel_read_body()
1488
            raise errors.UnexpectedSmartServerResponse(response_tuple)
1489
        if response_tuple[0] == 'ok':
1490
            coded = bz2.decompress(response_handler.read_body_bytes())
3172.5.6 by Robert Collins
Create new smart server verb Repository.get_parent_map.
1491
            if coded == '':
1492
                # no revisions found
1493
                return {}
1494
            lines = coded.split('\n')
1495
            revision_graph = {}
1496
            for line in lines:
1497
                d = tuple(line.split())
1498
                if len(d) > 1:
1499
                    revision_graph[d[0]] = d[1:]
1500
                else:
4190.1.4 by Robert Collins
Cache ghosts when we can get them from a RemoteRepository in get_parent_map.
1501
                    # No parents:
1502
                    if d[0].startswith('missing:'):
1503
                        revid = d[0][8:]
1504
                        self._unstacked_provider.note_missing_key(revid)
1505
                    else:
1506
                        # no parents - so give the Graph result
1507
                        # (NULL_REVISION,).
1508
                        revision_graph[d[0]] = (NULL_REVISION,)
3172.5.6 by Robert Collins
Create new smart server verb Repository.get_parent_map.
1509
            return revision_graph
1510
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1511
    @needs_read_lock
1512
    def get_signature_text(self, revision_id):
1513
        self._ensure_real()
1514
        return self._real_repository.get_signature_text(revision_id)
1515
1516
    @needs_read_lock
4988.5.1 by Jelmer Vernooij
Rename Repository.get_inventory_xml -> Repository._get_inventory_xml.
1517
    def _get_inventory_xml(self, revision_id):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1518
        self._ensure_real()
4988.5.1 by Jelmer Vernooij
Rename Repository.get_inventory_xml -> Repository._get_inventory_xml.
1519
        return self._real_repository._get_inventory_xml(revision_id)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1520
1521
    def reconcile(self, other=None, thorough=False):
1522
        self._ensure_real()
1523
        return self._real_repository.reconcile(other=other, thorough=thorough)
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
1524
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1525
    def all_revision_ids(self):
1526
        self._ensure_real()
1527
        return self._real_repository.all_revision_ids()
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
1528
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1529
    @needs_read_lock
4137.3.2 by Ian Clatworthy
Repository.get_deltas_for_revisions() now supports file-id filtering
1530
    def get_deltas_for_revisions(self, revisions, specific_fileids=None):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1531
        self._ensure_real()
4137.3.2 by Ian Clatworthy
Repository.get_deltas_for_revisions() now supports file-id filtering
1532
        return self._real_repository.get_deltas_for_revisions(revisions,
1533
            specific_fileids=specific_fileids)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1534
1535
    @needs_read_lock
4137.3.2 by Ian Clatworthy
Repository.get_deltas_for_revisions() now supports file-id filtering
1536
    def get_revision_delta(self, revision_id, specific_fileids=None):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1537
        self._ensure_real()
4137.3.2 by Ian Clatworthy
Repository.get_deltas_for_revisions() now supports file-id filtering
1538
        return self._real_repository.get_revision_delta(revision_id,
1539
            specific_fileids=specific_fileids)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1540
1541
    @needs_read_lock
1542
    def revision_trees(self, revision_ids):
1543
        self._ensure_real()
1544
        return self._real_repository.revision_trees(revision_ids)
1545
1546
    @needs_read_lock
1547
    def get_revision_reconcile(self, revision_id):
1548
        self._ensure_real()
1549
        return self._real_repository.get_revision_reconcile(revision_id)
1550
1551
    @needs_read_lock
4332.3.35 by Robert Collins
Fix failing tests.
1552
    def check(self, revision_ids=None, callback_refs=None, check_repo=True):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1553
        self._ensure_real()
4332.3.11 by Robert Collins
Move tree and back callbacks into the repository check core.
1554
        return self._real_repository.check(revision_ids=revision_ids,
4332.3.35 by Robert Collins
Fix failing tests.
1555
            callback_refs=callback_refs, check_repo=check_repo)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1556
2018.5.138 by Robert Collins
Merge bzr.dev.
1557
    def copy_content_into(self, destination, revision_id=None):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1558
        self._ensure_real()
1559
        return self._real_repository.copy_content_into(
2018.5.138 by Robert Collins
Merge bzr.dev.
1560
            destination, revision_id=revision_id)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1561
2814.10.2 by Andrew Bennetts
Make the fallback a little tidier.
1562
    def _copy_repository_tarball(self, to_bzrdir, revision_id=None):
2018.18.10 by Martin Pool
copy_content_into from Remote repositories by using temporary directories on both ends.
1563
        # get a tarball of the remote repository, and copy from that into the
1564
        # destination
1565
        from bzrlib import osutils
2018.18.9 by Martin Pool
remote Repository.tarball builds a temporary directory and tars that
1566
        import tarfile
2018.18.20 by Martin Pool
Route branch operations through remote copy_content_into
1567
        # TODO: Maybe a progress bar while streaming the tarball?
1568
        note("Copying repository content as tarball...")
2018.18.25 by Martin Pool
Repository.tarball fixes for python2.4
1569
        tar_file = self._get_tarball('bz2')
2814.10.2 by Andrew Bennetts
Make the fallback a little tidier.
1570
        if tar_file is None:
1571
            return None
1572
        destination = to_bzrdir.create_repository()
2018.18.10 by Martin Pool
copy_content_into from Remote repositories by using temporary directories on both ends.
1573
        try:
2018.18.25 by Martin Pool
Repository.tarball fixes for python2.4
1574
            tar = tarfile.open('repository', fileobj=tar_file,
1575
                mode='r|bz2')
3638.3.2 by Vincent Ladeuil
Fix all calls to tempfile.mkdtemp to osutils.mkdtemp.
1576
            tmpdir = osutils.mkdtemp()
2018.18.25 by Martin Pool
Repository.tarball fixes for python2.4
1577
            try:
1578
                _extract_tar(tar, tmpdir)
1579
                tmp_bzrdir = BzrDir.open(tmpdir)
1580
                tmp_repo = tmp_bzrdir.open_repository()
1581
                tmp_repo.copy_content_into(destination, revision_id)
1582
            finally:
1583
                osutils.rmtree(tmpdir)
2018.18.10 by Martin Pool
copy_content_into from Remote repositories by using temporary directories on both ends.
1584
        finally:
2018.18.25 by Martin Pool
Repository.tarball fixes for python2.4
1585
            tar_file.close()
2814.10.2 by Andrew Bennetts
Make the fallback a little tidier.
1586
        return destination
2018.18.23 by Martin Pool
review cleanups
1587
        # TODO: Suggestion from john: using external tar is much faster than
1588
        # python's tarfile library, but it may not work on windows.
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1589
3842.3.20 by Andrew Bennetts
Re-revert changes from another thread that accidentally got reinstated here.
1590
    @property
1591
    def inventories(self):
1592
        """Decorate the real repository for now.
1593
1594
        In the long term a full blown network facility is needed to
1595
        avoid creating a real repository object locally.
1596
        """
1597
        self._ensure_real()
1598
        return self._real_repository.inventories
1599
2604.2.1 by Robert Collins
(robertc) Introduce a pack command.
1600
    @needs_write_lock
4431.3.7 by Jonathan Lange
Cherrypick bzr.dev 4470, resolving conflicts.
1601
    def pack(self, hint=None):
2604.2.1 by Robert Collins
(robertc) Introduce a pack command.
1602
        """Compress the data within the repository.
1603
1604
        This is not currently implemented within the smart server.
1605
        """
1606
        self._ensure_real()
4431.3.7 by Jonathan Lange
Cherrypick bzr.dev 4470, resolving conflicts.
1607
        return self._real_repository.pack(hint=hint)
2604.2.1 by Robert Collins
(robertc) Introduce a pack command.
1608
3842.3.20 by Andrew Bennetts
Re-revert changes from another thread that accidentally got reinstated here.
1609
    @property
1610
    def revisions(self):
1611
        """Decorate the real repository for now.
1612
1613
        In the short term this should become a real object to intercept graph
1614
        lookups.
1615
1616
        In the long term a full blown network facility is needed.
1617
        """
1618
        self._ensure_real()
1619
        return self._real_repository.revisions
1620
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1621
    def set_make_working_trees(self, new_value):
4017.3.4 by Robert Collins
Create a verb for Repository.set_make_working_trees.
1622
        if new_value:
1623
            new_value_str = "True"
1624
        else:
1625
            new_value_str = "False"
1626
        path = self.bzrdir._path_for_remote_call(self._client)
1627
        try:
1628
            response = self._call(
1629
                'Repository.set_make_working_trees', path, new_value_str)
1630
        except errors.UnknownSmartMethod:
1631
            self._ensure_real()
1632
            self._real_repository.set_make_working_trees(new_value)
1633
        else:
1634
            if response[0] != 'ok':
1635
                raise errors.UnexpectedSmartServerResponse(response)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1636
3842.3.20 by Andrew Bennetts
Re-revert changes from another thread that accidentally got reinstated here.
1637
    @property
1638
    def signatures(self):
1639
        """Decorate the real repository for now.
1640
1641
        In the long term a full blown network facility is needed to avoid
1642
        creating a real repository object locally.
1643
        """
1644
        self._ensure_real()
1645
        return self._real_repository.signatures
1646
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1647
    @needs_write_lock
1648
    def sign_revision(self, revision_id, gpg_strategy):
1649
        self._ensure_real()
1650
        return self._real_repository.sign_revision(revision_id, gpg_strategy)
1651
3842.3.20 by Andrew Bennetts
Re-revert changes from another thread that accidentally got reinstated here.
1652
    @property
1653
    def texts(self):
1654
        """Decorate the real repository for now.
1655
1656
        In the long term a full blown network facility is needed to avoid
1657
        creating a real repository object locally.
1658
        """
1659
        self._ensure_real()
1660
        return self._real_repository.texts
1661
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1662
    @needs_read_lock
1663
    def get_revisions(self, revision_ids):
1664
        self._ensure_real()
1665
        return self._real_repository.get_revisions(revision_ids)
1666
1667
    def supports_rich_root(self):
4053.1.4 by Robert Collins
Move the fetch control attributes from Repository to RepositoryFormat.
1668
        return self._format.rich_root_data
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1669
2018.5.83 by Andrew Bennetts
Fix some test failures caused by the switch from unicode to UTF-8-encoded strs for revision IDs.
1670
    def iter_reverse_revision_history(self, revision_id):
1671
        self._ensure_real()
1672
        return self._real_repository.iter_reverse_revision_history(revision_id)
1673
2018.5.96 by Andrew Bennetts
Merge from bzr.dev, resolving the worst of the semantic conflicts, but there's
1674
    @property
1675
    def _serializer(self):
4053.1.4 by Robert Collins
Move the fetch control attributes from Repository to RepositoryFormat.
1676
        return self._format._serializer
2018.5.96 by Andrew Bennetts
Merge from bzr.dev, resolving the worst of the semantic conflicts, but there's
1677
2018.5.97 by Andrew Bennetts
Fix more tests.
1678
    def store_revision_signature(self, gpg_strategy, plaintext, revision_id):
1679
        self._ensure_real()
1680
        return self._real_repository.store_revision_signature(
1681
            gpg_strategy, plaintext, revision_id)
1682
2996.2.8 by Aaron Bentley
Fix add_signature discrepancies
1683
    def add_signature_text(self, revision_id, signature):
2996.2.3 by Aaron Bentley
Add tests for install_revisions and add_signature
1684
        self._ensure_real()
2996.2.8 by Aaron Bentley
Fix add_signature discrepancies
1685
        return self._real_repository.add_signature_text(revision_id, signature)
2996.2.3 by Aaron Bentley
Add tests for install_revisions and add_signature
1686
2018.5.97 by Andrew Bennetts
Fix more tests.
1687
    def has_signature_for_revision_id(self, revision_id):
1688
        self._ensure_real()
1689
        return self._real_repository.has_signature_for_revision_id(revision_id)
1690
2535.3.45 by Andrew Bennetts
Add item_keys_introduced_by to RemoteRepository.
1691
    def item_keys_introduced_by(self, revision_ids, _files_pb=None):
1692
        self._ensure_real()
1693
        return self._real_repository.item_keys_introduced_by(revision_ids,
1694
            _files_pb=_files_pb)
1695
2819.2.4 by Andrew Bennetts
Add a 'revision_graph_can_have_wrong_parents' method to repository.
1696
    def revision_graph_can_have_wrong_parents(self):
1697
        # The answer depends on the remote repo format.
1698
        self._ensure_real()
1699
        return self._real_repository.revision_graph_can_have_wrong_parents()
1700
4332.3.34 by Robert Collins
Delete obsolete pack tuned find_inconsistent_revisions as the generic code path permits the same optimisation.
1701
    def _find_inconsistent_revision_parents(self, revisions_iterator=None):
2819.2.5 by Andrew Bennetts
Make reconcile abort gracefully if the revision index has bad parents.
1702
        self._ensure_real()
4332.3.34 by Robert Collins
Delete obsolete pack tuned find_inconsistent_revisions as the generic code path permits the same optimisation.
1703
        return self._real_repository._find_inconsistent_revision_parents(
1704
            revisions_iterator)
2819.2.5 by Andrew Bennetts
Make reconcile abort gracefully if the revision index has bad parents.
1705
1706
    def _check_for_inconsistent_revision_parents(self):
1707
        self._ensure_real()
1708
        return self._real_repository._check_for_inconsistent_revision_parents()
1709
3835.1.17 by Aaron Bentley
Fix stacking bug
1710
    def _make_parents_provider(self, other=None):
3835.1.8 by Aaron Bentley
Make UnstackedParentsProvider manage the cache
1711
        providers = [self._unstacked_provider]
3835.1.17 by Aaron Bentley
Fix stacking bug
1712
        if other is not None:
1713
            providers.insert(0, other)
3835.1.7 by Aaron Bentley
Updates from review
1714
        providers.extend(r._make_parents_provider() for r in
1715
                         self._fallback_repositories)
4379.3.3 by Gary van der Merwe
Rename and add doc string for StackedParentsProvider.
1716
        return graph.StackedParentsProvider(providers)
3172.5.1 by Robert Collins
Create a RemoteRepository get_graph implementation and delegate get_parents_map to the real repository.
1717
3842.3.20 by Andrew Bennetts
Re-revert changes from another thread that accidentally got reinstated here.
1718
    def _serialise_search_recipe(self, recipe):
1719
        """Serialise a graph search recipe.
1720
1721
        :param recipe: A search recipe (start, stop, count).
1722
        :return: Serialised bytes.
1723
        """
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1724
        start_keys = ' '.join(recipe[1])
1725
        stop_keys = ' '.join(recipe[2])
1726
        count = str(recipe[3])
3842.3.20 by Andrew Bennetts
Re-revert changes from another thread that accidentally got reinstated here.
1727
        return '\n'.join((start_keys, stop_keys, count))
1728
4070.9.5 by Andrew Bennetts
Better wire protocol: don't shoehorn MiniSearchResult serialisation into previous serialisation format.
1729
    def _serialise_search_result(self, search_result):
4070.9.14 by Andrew Bennetts
Tweaks requested by Robert's review.
1730
        if isinstance(search_result, graph.PendingAncestryResult):
1731
            parts = ['ancestry-of']
1732
            parts.extend(search_result.heads)
4070.9.5 by Andrew Bennetts
Better wire protocol: don't shoehorn MiniSearchResult serialisation into previous serialisation format.
1733
        else:
1734
            recipe = search_result.get_recipe()
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1735
            parts = [recipe[0], self._serialise_search_recipe(recipe)]
4070.9.5 by Andrew Bennetts
Better wire protocol: don't shoehorn MiniSearchResult serialisation into previous serialisation format.
1736
        return '\n'.join(parts)
1737
3842.3.2 by Andrew Bennetts
Revert the RemoteVersionedFiles.get_parent_map implementation, leaving just the skeleton of RemoteVersionedFiles.
1738
    def autopack(self):
1739
        path = self.bzrdir._path_for_remote_call(self._client)
1740
        try:
1741
            response = self._call('PackRepository.autopack', path)
1742
        except errors.UnknownSmartMethod:
1743
            self._ensure_real()
1744
            self._real_repository._pack_collection.autopack()
1745
            return
4145.1.2 by Robert Collins
Add a refresh_data method on Repository allowing cleaner handling of insertions into RemoteRepository objects with _real_repository instances.
1746
        self.refresh_data()
3842.3.2 by Andrew Bennetts
Revert the RemoteVersionedFiles.get_parent_map implementation, leaving just the skeleton of RemoteVersionedFiles.
1747
        if response[0] != 'ok':
1748
            raise errors.UnexpectedSmartServerResponse(response)
1749
1750
4022.1.6 by Robert Collins
Cherrypick and polish the RemoteSink for streaming push.
1751
class RemoteStreamSink(repository.StreamSink):
1752
4032.3.7 by Robert Collins
Move write locking and write group responsibilities into the Sink objects themselves, allowing complete avoidance of unnecessary calls when the sink is a RemoteSink.
1753
    def _insert_real(self, stream, src_format, resume_tokens):
4022.1.6 by Robert Collins
Cherrypick and polish the RemoteSink for streaming push.
1754
        self.target_repo._ensure_real()
1755
        sink = self.target_repo._real_repository._get_sink()
4032.3.7 by Robert Collins
Move write locking and write group responsibilities into the Sink objects themselves, allowing complete avoidance of unnecessary calls when the sink is a RemoteSink.
1756
        result = sink.insert_stream(stream, src_format, resume_tokens)
4029.2.1 by Robert Collins
Support streaming push to stacked branches.
1757
        if not result:
1758
            self.target_repo.autopack()
1759
        return result
4022.1.6 by Robert Collins
Cherrypick and polish the RemoteSink for streaming push.
1760
4032.3.7 by Robert Collins
Move write locking and write group responsibilities into the Sink objects themselves, allowing complete avoidance of unnecessary calls when the sink is a RemoteSink.
1761
    def insert_stream(self, stream, src_format, resume_tokens):
4144.3.2 by Andrew Bennetts
Use Repository.insert_stream_locked if there is a lock_token for the remote repo.
1762
        target = self.target_repo
4307.2.4 by Robert Collins
Enable caching of negative revision lookups in RemoteRepository write locks when no _real_repository has been constructed.
1763
        target._unstacked_provider.missing_keys.clear()
4476.3.82 by Andrew Bennetts
Mention another bug fix in NEWS, and update verb name, comments, and NEWS additions for landing on 1.19 rather than 1.18.
1764
        candidate_calls = [('Repository.insert_stream_1.19', (1, 19))]
4144.3.2 by Andrew Bennetts
Use Repository.insert_stream_locked if there is a lock_token for the remote repo.
1765
        if target._lock_token:
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1766
            candidate_calls.append(('Repository.insert_stream_locked', (1, 14)))
1767
            lock_args = (target._lock_token or '',)
4144.3.2 by Andrew Bennetts
Use Repository.insert_stream_locked if there is a lock_token for the remote repo.
1768
        else:
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1769
            candidate_calls.append(('Repository.insert_stream', (1, 13)))
1770
            lock_args = ()
4144.3.2 by Andrew Bennetts
Use Repository.insert_stream_locked if there is a lock_token for the remote repo.
1771
        client = target._client
4022.1.9 by Robert Collins
Fix critical issue in bzr.dev - pushing to an old bzr:// server fails because the stream being consumed before the fallback code occurs, which makes it fail to do the fetch. (Robert Collins, Andrew Bennetts, #332314)
1772
        medium = client._medium
4144.3.2 by Andrew Bennetts
Use Repository.insert_stream_locked if there is a lock_token for the remote repo.
1773
        path = target.bzrdir._path_for_remote_call(client)
4476.3.70 by Andrew Bennetts
Review tweaks.
1774
        # Probe for the verb to use with an empty stream before sending the
1775
        # real stream to it.  We do this both to avoid the risk of sending a
1776
        # large request that is then rejected, and because we don't want to
1777
        # implement a way to buffer, rewind, or restart the stream.
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1778
        found_verb = False
1779
        for verb, required_version in candidate_calls:
1780
            if medium._is_remote_before(required_version):
1781
                continue
4476.3.25 by Andrew Bennetts
Reinstate don't-reprobe-with-resume-tokens optimisation, fixing acceptance test.
1782
            if resume_tokens:
1783
                # We've already done the probing (and set _is_remote_before) on
1784
                # a previous insert.
1785
                found_verb = True
1786
                break
4060.1.4 by Robert Collins
Streaming fetch from remote servers.
1787
            byte_stream = smart_repo._stream_to_byte_stream([], src_format)
4029.2.1 by Robert Collins
Support streaming push to stacked branches.
1788
            try:
1789
                response = client.call_with_body_stream(
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1790
                    (verb, path, '') + lock_args, byte_stream)
4029.2.1 by Robert Collins
Support streaming push to stacked branches.
1791
            except errors.UnknownSmartMethod:
4144.3.2 by Andrew Bennetts
Use Repository.insert_stream_locked if there is a lock_token for the remote repo.
1792
                medium._remember_remote_is_before(required_version)
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1793
            else:
1794
                found_verb = True
1795
                break
1796
        if not found_verb:
1797
            # Have to use VFS.
1798
            return self._insert_real(stream, src_format, resume_tokens)
1799
        self._last_inv_record = None
1800
        self._last_substream = None
4476.3.82 by Andrew Bennetts
Mention another bug fix in NEWS, and update verb name, comments, and NEWS additions for landing on 1.19 rather than 1.18.
1801
        if required_version < (1, 19):
4476.3.35 by Andrew Bennetts
Expand comment.
1802
            # Remote side doesn't support inventory deltas.  Wrap the stream to
1803
            # make sure we don't send any.  If the stream contains inventory
1804
            # deltas we'll interrupt the smart insert_stream request and
1805
            # fallback to VFS.
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1806
            stream = self._stop_stream_if_inventory_delta(stream)
4060.1.4 by Robert Collins
Streaming fetch from remote servers.
1807
        byte_stream = smart_repo._stream_to_byte_stream(
1808
            stream, src_format)
4032.3.7 by Robert Collins
Move write locking and write group responsibilities into the Sink objects themselves, allowing complete avoidance of unnecessary calls when the sink is a RemoteSink.
1809
        resume_tokens = ' '.join(resume_tokens)
4022.1.9 by Robert Collins
Fix critical issue in bzr.dev - pushing to an old bzr:// server fails because the stream being consumed before the fallback code occurs, which makes it fail to do the fetch. (Robert Collins, Andrew Bennetts, #332314)
1810
        response = client.call_with_body_stream(
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1811
            (verb, path, resume_tokens) + lock_args, byte_stream)
4029.2.1 by Robert Collins
Support streaming push to stacked branches.
1812
        if response[0][0] not in ('ok', 'missing-basis'):
4022.1.9 by Robert Collins
Fix critical issue in bzr.dev - pushing to an old bzr:// server fails because the stream being consumed before the fallback code occurs, which makes it fail to do the fetch. (Robert Collins, Andrew Bennetts, #332314)
1813
            raise errors.UnexpectedSmartServerResponse(response)
4476.3.54 by Andrew Bennetts
Update fallback-to-VFS in remote.py for the inventory-deltas substream.
1814
        if self._last_substream is not None:
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1815
            # The stream included an inventory-delta record, but the remote
1816
            # side isn't new enough to support them.  So we need to send the
1817
            # rest of the stream via VFS.
4634.35.15 by Andrew Bennetts
Fix 'fallback to vfs if RPC does not support inventory-deltas' case to refresh the vfs pack names before starting the fallback.
1818
            self.target_repo.refresh_data()
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1819
            return self._resume_stream_with_vfs(response, src_format)
4029.2.1 by Robert Collins
Support streaming push to stacked branches.
1820
        if response[0][0] == 'missing-basis':
1821
            tokens, missing_keys = bencode.bdecode_as_tuple(response[0][1])
4032.3.7 by Robert Collins
Move write locking and write group responsibilities into the Sink objects themselves, allowing complete avoidance of unnecessary calls when the sink is a RemoteSink.
1822
            resume_tokens = tokens
4257.3.3 by Andrew Bennetts
missing_keys from sink.insert_stream should be a set, not a tuple.
1823
            return resume_tokens, set(missing_keys)
4029.2.1 by Robert Collins
Support streaming push to stacked branches.
1824
        else:
4145.1.2 by Robert Collins
Add a refresh_data method on Repository allowing cleaner handling of insertions into RemoteRepository objects with _real_repository instances.
1825
            self.target_repo.refresh_data()
4032.3.7 by Robert Collins
Move write locking and write group responsibilities into the Sink objects themselves, allowing complete avoidance of unnecessary calls when the sink is a RemoteSink.
1826
            return [], set()
4032.1.1 by John Arbash Meinel
Merge the removal of all trailing whitespace, and resolve conflicts.
1827
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1828
    def _resume_stream_with_vfs(self, response, src_format):
1829
        """Resume sending a stream via VFS, first resending the record and
1830
        substream that couldn't be sent via an insert_stream verb.
1831
        """
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1832
        if response[0][0] == 'missing-basis':
1833
            tokens, missing_keys = bencode.bdecode_as_tuple(response[0][1])
1834
            # Ignore missing_keys, we haven't finished inserting yet
1835
        else:
1836
            tokens = []
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1837
        def resume_substream():
4476.3.54 by Andrew Bennetts
Update fallback-to-VFS in remote.py for the inventory-deltas substream.
1838
            # Yield the substream that was interrupted.
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1839
            for record in self._last_substream:
1840
                yield record
1841
            self._last_substream = None
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1842
        def resume_stream():
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1843
            # Finish sending the interrupted substream
4476.3.54 by Andrew Bennetts
Update fallback-to-VFS in remote.py for the inventory-deltas substream.
1844
            yield ('inventory-deltas', resume_substream())
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1845
            # Then simply continue sending the rest of the stream.
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1846
            for substream_kind, substream in self._last_stream:
1847
                yield substream_kind, substream
1848
        return self._insert_real(resume_stream(), src_format, tokens)
1849
1850
    def _stop_stream_if_inventory_delta(self, stream):
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1851
        """Normally this just lets the original stream pass-through unchanged.
1852
4476.3.54 by Andrew Bennetts
Update fallback-to-VFS in remote.py for the inventory-deltas substream.
1853
        However if any 'inventory-deltas' substream occurs it will stop
1854
        streaming, and store the interrupted substream and stream in
1855
        self._last_substream and self._last_stream so that the stream can be
1856
        resumed by _resume_stream_with_vfs.
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1857
        """
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1858
                    
1859
        stream_iter = iter(stream)
1860
        for substream_kind, substream in stream_iter:
4476.3.54 by Andrew Bennetts
Update fallback-to-VFS in remote.py for the inventory-deltas substream.
1861
            if substream_kind == 'inventory-deltas':
1862
                self._last_substream = substream
1863
                self._last_stream = stream_iter
1864
                return
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1865
            else:
1866
                yield substream_kind, substream
1867
            
4022.1.1 by Robert Collins
Refactoring of fetch to have a sender and sink component enabling splitting the logic over a network stream. (Robert Collins, Andrew Bennetts)
1868
4060.1.3 by Robert Collins
Implement the separate source component for fetch - repository.StreamSource.
1869
class RemoteStreamSource(repository.StreamSource):
1870
    """Stream data from a remote server."""
1871
4060.1.4 by Robert Collins
Streaming fetch from remote servers.
1872
    def get_stream(self, search):
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1873
        if (self.from_repository._fallback_repositories and
1874
            self.to_format._fetch_order == 'topological'):
1875
            return self._real_stream(self.from_repository, search)
4577.1.1 by Robert Collins
Fix fetching from smart servers where there is a chain of stacked repositories rather than a single stacking point. (Robert Collins, bug #406597)
1876
        sources = []
1877
        seen = set()
1878
        repos = [self.from_repository]
1879
        while repos:
1880
            repo = repos.pop(0)
1881
            if repo in seen:
1882
                continue
1883
            seen.add(repo)
1884
            repos.extend(repo._fallback_repositories)
1885
            sources.append(repo)
1886
        return self.missing_parents_chain(search, sources)
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1887
4476.3.16 by Andrew Bennetts
Only make inv deltas against bases we've already sent, and other tweaks.
1888
    def get_stream_for_missing_keys(self, missing_keys):
1889
        self.from_repository._ensure_real()
1890
        real_repo = self.from_repository._real_repository
1891
        real_source = real_repo._get_source(self.to_format)
1892
        return real_source.get_stream_for_missing_keys(missing_keys)
1893
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1894
    def _real_stream(self, repo, search):
1895
        """Get a stream for search from repo.
1896
        
1897
        This never called RemoteStreamSource.get_stream, and is a heler
1898
        for RemoteStreamSource._get_stream to allow getting a stream 
1899
        reliably whether fallback back because of old servers or trying
1900
        to stream from a non-RemoteRepository (which the stacked support
1901
        code will do).
1902
        """
1903
        source = repo._get_source(self.to_format)
1904
        if isinstance(source, RemoteStreamSource):
4600.1.1 by Robert Collins
Fix fetching from 2a branches over bzr-v2.
1905
            repo._ensure_real()
1906
            source = repo._real_repository._get_source(self.to_format)
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1907
        return source.get_stream(search)
1908
1909
    def _get_stream(self, repo, search):
1910
        """Core worker to get a stream from repo for search.
1911
1912
        This is used by both get_stream and the stacking support logic. It
1913
        deliberately gets a stream for repo which does not need to be
1914
        self.from_repository. In the event that repo is not Remote, or
1915
        cannot do a smart stream, a fallback is made to the generic
1916
        repository._get_stream() interface, via self._real_stream.
1917
1918
        In the event of stacking, streams from _get_stream will not
1919
        contain all the data for search - this is normal (see get_stream).
1920
1921
        :param repo: A repository.
1922
        :param search: A search.
1923
        """
1924
        # Fallbacks may be non-smart
1925
        if not isinstance(repo, RemoteRepository):
1926
            return self._real_stream(repo, search)
4060.1.4 by Robert Collins
Streaming fetch from remote servers.
1927
        client = repo._client
1928
        medium = client._medium
1929
        path = repo.bzrdir._path_for_remote_call(client)
4476.3.29 by Andrew Bennetts
Add Repository.get_stream_1.18 verb.
1930
        search_bytes = repo._serialise_search_result(search)
1931
        args = (path, self.to_format.network_name())
1932
        candidate_verbs = [
4476.3.82 by Andrew Bennetts
Mention another bug fix in NEWS, and update verb name, comments, and NEWS additions for landing on 1.19 rather than 1.18.
1933
            ('Repository.get_stream_1.19', (1, 19)),
4476.3.29 by Andrew Bennetts
Add Repository.get_stream_1.18 verb.
1934
            ('Repository.get_stream', (1, 13))]
1935
        found_verb = False
1936
        for verb, version in candidate_verbs:
1937
            if medium._is_remote_before(version):
1938
                continue
1939
            try:
1940
                response = repo._call_with_body_bytes_expecting_body(
1941
                    verb, args, search_bytes)
1942
            except errors.UnknownSmartMethod:
1943
                medium._remember_remote_is_before(version)
1944
            else:
1945
                response_tuple, response_handler = response
1946
                found_verb = True
1947
                break
1948
        if not found_verb:
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1949
            return self._real_stream(repo, search)
4060.1.4 by Robert Collins
Streaming fetch from remote servers.
1950
        if response_tuple[0] != 'ok':
1951
            raise errors.UnexpectedSmartServerResponse(response_tuple)
1952
        byte_stream = response_handler.read_streamed_body()
1953
        src_format, stream = smart_repo._byte_stream_to_stream(byte_stream)
1954
        if src_format.network_name() != repo._format.network_name():
1955
            raise AssertionError(
1956
                "Mismatched RemoteRepository and stream src %r, %r" % (
1957
                src_format.network_name(), repo._format.network_name()))
1958
        return stream
1959
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1960
    def missing_parents_chain(self, search, sources):
1961
        """Chain multiple streams together to handle stacking.
1962
1963
        :param search: The overall search to satisfy with streams.
1964
        :param sources: A list of Repository objects to query.
1965
        """
4634.61.2 by Andrew Bennetts
Variable rename suggested by Robert.
1966
        self.from_serialiser = self.from_repository._format._serializer
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1967
        self.seen_revs = set()
1968
        self.referenced_revs = set()
1969
        # If there are heads in the search, or the key count is > 0, we are not
1970
        # done.
1971
        while not search.is_empty() and len(sources) > 1:
1972
            source = sources.pop(0)
1973
            stream = self._get_stream(source, search)
1974
            for kind, substream in stream:
1975
                if kind != 'revisions':
1976
                    yield kind, substream
1977
                else:
1978
                    yield kind, self.missing_parents_rev_handler(substream)
1979
            search = search.refine(self.seen_revs, self.referenced_revs)
1980
            self.seen_revs = set()
1981
            self.referenced_revs = set()
1982
        if not search.is_empty():
1983
            for kind, stream in self._get_stream(sources[0], search):
1984
                yield kind, stream
1985
1986
    def missing_parents_rev_handler(self, substream):
1987
        for content in substream:
1988
            revision_bytes = content.get_bytes_as('fulltext')
4634.61.2 by Andrew Bennetts
Variable rename suggested by Robert.
1989
            revision = self.from_serialiser.read_revision_from_string(
1990
                revision_bytes)
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1991
            self.seen_revs.add(content.key[-1])
1992
            self.referenced_revs.update(revision.parent_ids)
1993
            yield content
1994
4060.1.3 by Robert Collins
Implement the separate source component for fetch - repository.StreamSource.
1995
2018.5.127 by Andrew Bennetts
Fix most of the lockable_files tests for RemoteBranchLockableFiles.
1996
class RemoteBranchLockableFiles(LockableFiles):
2018.5.59 by Robert Collins
Get BranchConfig working somewhat on RemoteBranches (Robert Collins, Vincent Ladeuil).
1997
    """A 'LockableFiles' implementation that talks to a smart server.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
1998
2018.5.59 by Robert Collins
Get BranchConfig working somewhat on RemoteBranches (Robert Collins, Vincent Ladeuil).
1999
    This is not a public interface class.
2000
    """
2001
2002
    def __init__(self, bzrdir, _client):
2003
        self.bzrdir = bzrdir
2004
        self._client = _client
2018.5.135 by Andrew Bennetts
Prevent remote branch clients from determining the 'right' mode for control files, because we don't want clients setting the mode anyway.
2005
        self._need_find_modes = True
2018.5.133 by Andrew Bennetts
All TestLockableFiles_RemoteLockDir tests passing.
2006
        LockableFiles.__init__(
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
2007
            self, bzrdir.get_branch_transport(None),
2018.5.133 by Andrew Bennetts
All TestLockableFiles_RemoteLockDir tests passing.
2008
            'lock', lockdir.LockDir)
2018.5.59 by Robert Collins
Get BranchConfig working somewhat on RemoteBranches (Robert Collins, Vincent Ladeuil).
2009
2018.5.135 by Andrew Bennetts
Prevent remote branch clients from determining the 'right' mode for control files, because we don't want clients setting the mode anyway.
2010
    def _find_modes(self):
2011
        # RemoteBranches don't let the client set the mode of control files.
2012
        self._dir_mode = None
2013
        self._file_mode = None
2014
2018.5.59 by Robert Collins
Get BranchConfig working somewhat on RemoteBranches (Robert Collins, Vincent Ladeuil).
2015
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
2016
class RemoteBranchFormat(branch.BranchFormat):
2017
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2018
    def __init__(self, network_name=None):
3834.5.2 by John Arbash Meinel
Track down the various BranchFormats that weren't setting the branch format as part of the _matchingbzrdir format.
2019
        super(RemoteBranchFormat, self).__init__()
2020
        self._matchingbzrdir = RemoteBzrDirFormat()
2021
        self._matchingbzrdir.set_branch_format(self)
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2022
        self._custom_format = None
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2023
        self._network_name = network_name
3834.5.2 by John Arbash Meinel
Track down the various BranchFormats that weren't setting the branch format as part of the _matchingbzrdir format.
2024
2018.5.124 by Robert Collins
Fix test_format_initialize_find_open by delegating Branch formt lookup to the BzrDir, where it should have stayed from the start.
2025
    def __eq__(self, other):
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
2026
        return (isinstance(other, RemoteBranchFormat) and
2018.5.124 by Robert Collins
Fix test_format_initialize_find_open by delegating Branch formt lookup to the BzrDir, where it should have stayed from the start.
2027
            self.__dict__ == other.__dict__)
2028
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2029
    def _ensure_real(self):
2030
        if self._custom_format is None:
2031
            self._custom_format = branch.network_format_registry.get(
2032
                self._network_name)
2033
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
2034
    def get_format_description(self):
4792.1.1 by Andrew Bennetts
Show real branch/repo format description in 'info -v' over HPSS.
2035
        self._ensure_real()
2036
        return 'Remote: ' + self._custom_format.get_format_description()
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
2037
4032.3.1 by Robert Collins
Add a BranchFormat.network_name() method as preparation for creating branches via RPC calls.
2038
    def network_name(self):
2039
        return self._network_name
2040
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2041
    def open(self, a_bzrdir, name=None, ignore_fallbacks=False):
2042
        return a_bzrdir.open_branch(name=name, 
2043
            ignore_fallbacks=ignore_fallbacks)
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
2044
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2045
    def _vfs_initialize(self, a_bzrdir, name):
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2046
        # Initialisation when using a local bzrdir object, or a non-vfs init
2047
        # method is not available on the server.
2048
        # self._custom_format is always set - the start of initialize ensures
2049
        # that.
2050
        if isinstance(a_bzrdir, RemoteBzrDir):
2051
            a_bzrdir._ensure_real()
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2052
            result = self._custom_format.initialize(a_bzrdir._real_bzrdir,
2053
                name)
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2054
        else:
2055
            # We assume the bzrdir is parameterised; it may not be.
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2056
            result = self._custom_format.initialize(a_bzrdir, name)
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2057
        if (isinstance(a_bzrdir, RemoteBzrDir) and
2058
            not isinstance(result, RemoteBranch)):
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2059
            result = RemoteBranch(a_bzrdir, a_bzrdir.find_repository(), result,
2060
                                  name=name)
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2061
        return result
2062
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2063
    def initialize(self, a_bzrdir, name=None):
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2064
        # 1) get the network name to use.
2065
        if self._custom_format:
2066
            network_name = self._custom_format.network_name()
2067
        else:
2068
            # Select the current bzrlib default and ask for that.
2069
            reference_bzrdir_format = bzrdir.format_registry.get('default')()
2070
            reference_format = reference_bzrdir_format.get_branch_format()
2071
            self._custom_format = reference_format
2072
            network_name = reference_format.network_name()
2073
        # Being asked to create on a non RemoteBzrDir:
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2074
        if not isinstance(a_bzrdir, RemoteBzrDir):
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2075
            return self._vfs_initialize(a_bzrdir, name=name)
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2076
        medium = a_bzrdir._client._medium
2077
        if medium._is_remote_before((1, 13)):
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2078
            return self._vfs_initialize(a_bzrdir, name=name)
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2079
        # Creating on a remote bzr dir.
2080
        # 2) try direct creation via RPC
2081
        path = a_bzrdir._path_for_remote_call(a_bzrdir._client)
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2082
        if name is not None:
5051.3.11 by Jelmer Vernooij
add XXX for creating remote colocated branches.
2083
            # XXX JRV20100304: Support creating colocated branches
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2084
            raise errors.NoColocatedBranchSupport(self)
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2085
        verb = 'BzrDir.create_branch'
2086
        try:
2087
            response = a_bzrdir._call(verb, path, network_name)
2088
        except errors.UnknownSmartMethod:
2089
            # Fallback - use vfs methods
4094.1.1 by Andrew Bennetts
Add some medium._remember_is_before((1, 13)) calls.
2090
            medium._remember_remote_is_before((1, 13))
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2091
            return self._vfs_initialize(a_bzrdir, name=name)
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2092
        if response[0] != 'ok':
2093
            raise errors.UnexpectedSmartServerResponse(response)
2094
        # Turn the response into a RemoteRepository object.
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2095
        format = RemoteBranchFormat(network_name=response[1])
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2096
        repo_format = response_tuple_to_repo_format(response[3:])
2097
        if response[2] == '':
2098
            repo_bzrdir = a_bzrdir
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2099
        else:
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2100
            repo_bzrdir = RemoteBzrDir(
2101
                a_bzrdir.root_transport.clone(response[2]), a_bzrdir._format,
2102
                a_bzrdir._client)
2103
        remote_repo = RemoteRepository(repo_bzrdir, repo_format)
2104
        remote_branch = RemoteBranch(a_bzrdir, remote_repo,
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2105
            format=format, setup_stacking=False, name=name)
4044.1.4 by Robert Collins
Remove a wasted round trip determining the revno and revid of a newly created branch.
2106
        # XXX: We know this is a new branch, so it must have revno 0, revid
2107
        # NULL_REVISION. Creating the branch locked would make this be unable
2108
        # to be wrong; here its simply very unlikely to be wrong. RBC 20090225
2109
        remote_branch._last_revision_info_cache = 0, NULL_REVISION
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2110
        return remote_branch
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
2111
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2112
    def make_tags(self, branch):
2113
        self._ensure_real()
2114
        return self._custom_format.make_tags(branch)
2115
2696.3.6 by Martin Pool
Mark RemoteBranch as (possibly) supporting tags
2116
    def supports_tags(self):
2117
        # Remote branches might support tags, but we won't know until we
2118
        # access the real remote branch.
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2119
        self._ensure_real()
2120
        return self._custom_format.supports_tags()
2696.3.6 by Martin Pool
Mark RemoteBranch as (possibly) supporting tags
2121
4103.2.2 by Andrew Bennetts
Fix RemoteBranchFormat.supports_stacking()
2122
    def supports_stacking(self):
2123
        self._ensure_real()
2124
        return self._custom_format.supports_stacking()
2125
4301.3.3 by Andrew Bennetts
Move check onto base Branch class, and add a supports_set_append_revisions_only method to BranchFormat, as suggested by Robert.
2126
    def supports_set_append_revisions_only(self):
2127
        self._ensure_real()
2128
        return self._custom_format.supports_set_append_revisions_only()
2129
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
2130
4731.1.2 by Andrew Bennetts
Refactor to reduce duplication.
2131
class RemoteBranch(branch.Branch, _RpcHelper, lock._RelockDebugMixin):
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
2132
    """Branch stored on a server accessed by HPSS RPC.
2133
2134
    At the moment most operations are mapped down to simple file operations.
2135
    """
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
2136
2018.5.51 by Wouter van Heyst
Test and implement RemoteBranch.last_revision_info()
2137
    def __init__(self, remote_bzrdir, remote_repository, real_branch=None,
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2138
        _client=None, format=None, setup_stacking=True, name=None):
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
2139
        """Create a RemoteBranch instance.
2140
2141
        :param real_branch: An optional local implementation of the branch
2142
            format, usually accessing the data via the VFS.
2018.5.51 by Wouter van Heyst
Test and implement RemoteBranch.last_revision_info()
2143
        :param _client: Private parameter for testing.
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2144
        :param format: A RemoteBranchFormat object, None to create one
2145
            automatically. If supplied it should have a network_name already
2146
            supplied.
2147
        :param setup_stacking: If True make an RPC call to determine the
2148
            stacked (or not) status of the branch. If False assume the branch
2149
            is not stacked.
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2150
        :param name: Colocated branch name
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
2151
        """
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
2152
        # We intentionally don't call the parent class's __init__, because it
2153
        # will try to assign to self.tags, which is a property in this subclass.
2154
        # And the parent's __init__ doesn't do much anyway.
1752.2.64 by Andrew Bennetts
Improve how RemoteBzrDir.open_branch works to handle references and not double-open repositories.
2155
        self.bzrdir = remote_bzrdir
2018.5.51 by Wouter van Heyst
Test and implement RemoteBranch.last_revision_info()
2156
        if _client is not None:
2157
            self._client = _client
2158
        else:
3313.2.1 by Andrew Bennetts
Change _SmartClient's API to accept a medium and a base, rather than a _SharedConnection.
2159
            self._client = remote_bzrdir._client
1752.2.64 by Andrew Bennetts
Improve how RemoteBzrDir.open_branch works to handle references and not double-open repositories.
2160
        self.repository = remote_repository
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
2161
        if real_branch is not None:
2162
            self._real_branch = real_branch
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2163
            # Give the remote repository the matching real repo.
2018.5.97 by Andrew Bennetts
Fix more tests.
2164
            real_repo = self._real_branch.repository
2165
            if isinstance(real_repo, RemoteRepository):
2166
                real_repo._ensure_real()
2167
                real_repo = real_repo._real_repository
2168
            self.repository._set_real_repository(real_repo)
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2169
            # Give the branch the remote repository to let fast-pathing happen.
2170
            self._real_branch.repository = self.repository
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
2171
        else:
2172
            self._real_branch = None
4031.3.3 by Matt Nordhoff
Review tweaks from Ben Finney
2173
        # Fill out expected attributes of branch for bzrlib API users.
4419.2.2 by Andrew Bennetts
Read lock branch_from in cmd_pull, avoids refetching last_revision_info and so reduces test_pull acceptance ratchet.
2174
        self._clear_cached_state()
2018.5.55 by Robert Collins
Give RemoteBranch a base url in line with the Branch protocol.
2175
        self.base = self.bzrdir.root_transport.base
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2176
        self._name = name
2018.5.169 by Andrew Bennetts
Add a _server_formats flag to BzrDir.open_from_transport and BzrDirFormat.find_format, make RemoteBranch.control_files into a property.
2177
        self._control_files = None
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2178
        self._lock_mode = None
2179
        self._lock_token = None
2892.2.1 by Andrew Bennetts
Add Branch.set_last_revision_info smart method, and make the RemoteBranch client use it.
2180
        self._repo_lock_token = None
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2181
        self._lock_count = 0
2182
        self._leave_lock = False
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2183
        # Setup a format: note that we cannot call _ensure_real until all the
2184
        # attributes above are set: This code cannot be moved higher up in this
2185
        # function.
2186
        if format is None:
2187
            self._format = RemoteBranchFormat()
2188
            if real_branch is not None:
2189
                self._format._network_name = \
2190
                    self._real_branch._format.network_name()
4032.3.1 by Robert Collins
Add a BranchFormat.network_name() method as preparation for creating branches via RPC calls.
2191
        else:
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2192
            self._format = format
4600.2.1 by Robert Collins
Make RemoteBranch honour ignore_fallbacks when _ensure_real is triggered.
2193
        # when we do _ensure_real we may need to pass ignore_fallbacks to the
2194
        # branch.open_branch method.
2195
        self._real_ignore_fallbacks = not setup_stacking
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2196
        if not self._format._network_name:
2197
            # Did not get from open_branchV2 - old server.
2198
            self._ensure_real()
2199
            self._format._network_name = \
2200
                self._real_branch._format.network_name()
2201
        self.tags = self._format.make_tags(self)
3681.1.2 by Robert Collins
Adjust for trunk.
2202
        # The base class init is not called, so we duplicate this:
3681.1.1 by Robert Collins
Create a new hook Branch.open. (Robert Collins)
2203
        hooks = branch.Branch.hooks['open']
2204
        for hook in hooks:
2205
            hook(self)
4419.1.3 by Andrew Bennetts
Quick fix by using self._ensure_real.
2206
        self._is_stacked = False
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2207
        if setup_stacking:
2208
            self._setup_stacking()
3691.2.1 by Martin Pool
RemoteBranch must configure stacking into the repository
2209
2210
    def _setup_stacking(self):
2211
        # configure stacking into the remote repository, by reading it from
3691.2.3 by Martin Pool
Factor out RemoteBranch._remote_path() and disable RemoteBranch stacking
2212
        # the vfs branch.
3691.2.1 by Martin Pool
RemoteBranch must configure stacking into the repository
2213
        try:
2214
            fallback_url = self.get_stacked_on_url()
2215
        except (errors.NotStacked, errors.UnstackableBranchFormat,
2216
            errors.UnstackableRepositoryFormat), e:
3691.2.7 by Martin Pool
FakeClient can know what calls to expect
2217
            return
4419.1.3 by Andrew Bennetts
Quick fix by using self._ensure_real.
2218
        self._is_stacked = True
4379.2.2 by John Arbash Meinel
Change the Repository.add_fallback_repository() contract slightly.
2219
        self._activate_fallback_location(fallback_url)
1752.2.64 by Andrew Bennetts
Improve how RemoteBzrDir.open_branch works to handle references and not double-open repositories.
2220
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
2221
    def _get_config(self):
2222
        return RemoteBranchConfig(self)
2223
3407.2.17 by Martin Pool
better name: _get_real_transport
2224
    def _get_real_transport(self):
3407.2.16 by Martin Pool
Remove RemoteBranch reliance on control_files._transport
2225
        # if we try vfs access, return the real branch's vfs transport
2226
        self._ensure_real()
2227
        return self._real_branch._transport
2228
3407.2.17 by Martin Pool
better name: _get_real_transport
2229
    _transport = property(_get_real_transport)
3407.2.16 by Martin Pool
Remove RemoteBranch reliance on control_files._transport
2230
2477.1.1 by Martin Pool
Add RemoteBranch repr
2231
    def __str__(self):
2232
        return "%s(%s)" % (self.__class__.__name__, self.base)
2233
2234
    __repr__ = __str__
2235
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
2236
    def _ensure_real(self):
2237
        """Ensure that there is a _real_branch set.
2238
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
2239
        Used before calls to self._real_branch.
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
2240
        """
3407.2.16 by Martin Pool
Remove RemoteBranch reliance on control_files._transport
2241
        if self._real_branch is None:
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
2242
            if not vfs.vfs_enabled():
2243
                raise AssertionError('smart server vfs must be enabled '
2244
                    'to use vfs implementation')
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
2245
            self.bzrdir._ensure_real()
4600.2.1 by Robert Collins
Make RemoteBranch honour ignore_fallbacks when _ensure_real is triggered.
2246
            self._real_branch = self.bzrdir._real_bzrdir.open_branch(
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2247
                ignore_fallbacks=self._real_ignore_fallbacks, name=self._name)
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
2248
            if self.repository._real_repository is None:
2249
                # Give the remote repository the matching real repo.
2250
                real_repo = self._real_branch.repository
2251
                if isinstance(real_repo, RemoteRepository):
2252
                    real_repo._ensure_real()
2253
                    real_repo = real_repo._real_repository
2254
                self.repository._set_real_repository(real_repo)
2255
            # Give the real branch the remote repository to let fast-pathing
2256
            # happen.
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
2257
            self._real_branch.repository = self.repository
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
2258
            if self._lock_mode == 'r':
2259
                self._real_branch.lock_read()
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
2260
            elif self._lock_mode == 'w':
2261
                self._real_branch.lock_write(token=self._lock_token)
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
2262
3533.3.1 by Andrew Bennetts
Remove duplication of error translation in bzrlib/remote.py.
2263
    def _translate_error(self, err, **context):
2264
        self.repository._translate_error(err, branch=self, **context)
2265
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2266
    def _clear_cached_state(self):
2267
        super(RemoteBranch, self)._clear_cached_state()
3441.5.5 by Andrew Bennetts
Some small tweaks and comments.
2268
        if self._real_branch is not None:
2269
            self._real_branch._clear_cached_state()
3441.5.29 by Andrew Bennetts
More review tweaks: whitespace nits in test_smart, add (and use) ._clear_cached_state_of_remote_branch_only method in bzrlib/remote.py.
2270
2271
    def _clear_cached_state_of_remote_branch_only(self):
2272
        """Like _clear_cached_state, but doesn't clear the cache of
2273
        self._real_branch.
2274
2275
        This is useful when falling back to calling a method of
2276
        self._real_branch that changes state.  In that case the underlying
2277
        branch changes, so we need to invalidate this RemoteBranch's cache of
2278
        it.  However, there's no need to invalidate the _real_branch's cache
2279
        too, in fact doing so might harm performance.
2280
        """
2281
        super(RemoteBranch, self)._clear_cached_state()
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
2282
2018.5.169 by Andrew Bennetts
Add a _server_formats flag to BzrDir.open_from_transport and BzrDirFormat.find_format, make RemoteBranch.control_files into a property.
2283
    @property
2284
    def control_files(self):
2285
        # Defer actually creating RemoteBranchLockableFiles until its needed,
2286
        # because it triggers an _ensure_real that we otherwise might not need.
2287
        if self._control_files is None:
2288
            self._control_files = RemoteBranchLockableFiles(
2289
                self.bzrdir, self._client)
2290
        return self._control_files
2291
2018.5.166 by Andrew Bennetts
Small changes in response to Aaron's review.
2292
    def _get_checkout_format(self):
2293
        self._ensure_real()
2294
        return self._real_branch._get_checkout_format()
2295
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
2296
    def get_physical_lock_status(self):
2297
        """See Branch.get_physical_lock_status()."""
2298
        # should be an API call to the server, as branches must be lockable.
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
2299
        self._ensure_real()
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
2300
        return self._real_branch.get_physical_lock_status()
2301
3537.3.1 by Martin Pool
Rename branch.get_stacked_on to get_stacked_on_url
2302
    def get_stacked_on_url(self):
3221.11.2 by Robert Collins
Create basic stackable branch facility.
2303
        """Get the URL this branch is stacked against.
2304
2305
        :raises NotStacked: If the branch is not stacked.
2306
        :raises UnstackableBranchFormat: If the branch does not support
2307
            stacking.
2308
        :raises UnstackableRepositoryFormat: If the repository does not support
2309
            stacking.
2310
        """
3691.2.12 by Martin Pool
Add test for coping without Branch.get_stacked_on_url
2311
        try:
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2312
            # there may not be a repository yet, so we can't use
2313
            # self._translate_error, so we can't use self._call either.
3691.2.12 by Martin Pool
Add test for coping without Branch.get_stacked_on_url
2314
            response = self._client.call('Branch.get_stacked_on_url',
2315
                self._remote_path())
2316
        except errors.ErrorFromSmartServer, err:
2317
            # there may not be a repository yet, so we can't call through
2318
            # its _translate_error
2319
            _translate_error(err, branch=self)
2320
        except errors.UnknownSmartMethod, err:
2321
            self._ensure_real()
2322
            return self._real_branch.get_stacked_on_url()
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2323
        if response[0] != 'ok':
2324
            raise errors.UnexpectedSmartServerResponse(response)
2325
        return response[1]
3221.11.2 by Robert Collins
Create basic stackable branch facility.
2326
4419.1.3 by Andrew Bennetts
Quick fix by using self._ensure_real.
2327
    def set_stacked_on_url(self, url):
2328
        branch.Branch.set_stacked_on_url(self, url)
2329
        if not url:
2330
            self._is_stacked = False
2331
        else:
2332
            self._is_stacked = True
2333
        
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2334
    def _vfs_get_tags_bytes(self):
2335
        self._ensure_real()
2336
        return self._real_branch._get_tags_bytes()
2337
2338
    def _get_tags_bytes(self):
2339
        medium = self._client._medium
2340
        if medium._is_remote_before((1, 13)):
2341
            return self._vfs_get_tags_bytes()
2342
        try:
2343
            response = self._call('Branch.get_tags_bytes', self._remote_path())
2344
        except errors.UnknownSmartMethod:
4094.1.1 by Andrew Bennetts
Add some medium._remember_is_before((1, 13)) calls.
2345
            medium._remember_remote_is_before((1, 13))
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2346
            return self._vfs_get_tags_bytes()
2347
        return response[0]
2348
4556.2.10 by Andrew Bennetts
Fix trivial bug in _vfs_set_tags_bytes.
2349
    def _vfs_set_tags_bytes(self, bytes):
4556.2.1 by Andrew Bennetts
Add Branch.set_tags_bytes RPC, with HPSS call count acceptance test. Also fixes serialisation of LockDir, and uses external_url() in LockDir's repr and contention message.
2350
        self._ensure_real()
4556.2.10 by Andrew Bennetts
Fix trivial bug in _vfs_set_tags_bytes.
2351
        return self._real_branch._set_tags_bytes(bytes)
4556.2.1 by Andrew Bennetts
Add Branch.set_tags_bytes RPC, with HPSS call count acceptance test. Also fixes serialisation of LockDir, and uses external_url() in LockDir's repr and contention message.
2352
2353
    def _set_tags_bytes(self, bytes):
2354
        medium = self._client._medium
2355
        if medium._is_remote_before((1, 18)):
2356
            self._vfs_set_tags_bytes(bytes)
4634.36.1 by Andrew Bennetts
Fix trivial bug in RemoteBranch._set_tags_bytes, and add some unit tests for it.
2357
            return
4556.2.1 by Andrew Bennetts
Add Branch.set_tags_bytes RPC, with HPSS call count acceptance test. Also fixes serialisation of LockDir, and uses external_url() in LockDir's repr and contention message.
2358
        try:
2359
            args = (
2360
                self._remote_path(), self._lock_token, self._repo_lock_token)
2361
            response = self._call_with_body_bytes(
2362
                'Branch.set_tags_bytes', args, bytes)
2363
        except errors.UnknownSmartMethod:
2364
            medium._remember_remote_is_before((1, 18))
2365
            self._vfs_set_tags_bytes(bytes)
2366
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
2367
    def lock_read(self):
3692.1.1 by Andrew Bennetts
Make RemoteBranch.lock_write lock the repository too.
2368
        self.repository.lock_read()
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
2369
        if not self._lock_mode:
4731.1.2 by Andrew Bennetts
Refactor to reduce duplication.
2370
            self._note_lock('r')
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
2371
            self._lock_mode = 'r'
2372
            self._lock_count = 1
2373
            if self._real_branch is not None:
2374
                self._real_branch.lock_read()
2375
        else:
2376
            self._lock_count += 1
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
2377
2018.5.142 by Andrew Bennetts
Change Branch.lock_token to only accept and receive the branch lock token (rather than the branch and repo lock tokens).
2378
    def _remote_lock_write(self, token):
2379
        if token is None:
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2380
            branch_token = repo_token = ''
2381
        else:
2018.5.142 by Andrew Bennetts
Change Branch.lock_token to only accept and receive the branch lock token (rather than the branch and repo lock tokens).
2382
            branch_token = token
2383
            repo_token = self.repository.lock_write()
2384
            self.repository.unlock()
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2385
        err_context = {'token': token}
2386
        response = self._call(
2387
            'Branch.lock_write', self._remote_path(), branch_token,
2388
            repo_token or '', **err_context)
3245.4.24 by Andrew Bennetts
Consistently raise errors from the server as ErrorFromSmartServer exceptions.
2389
        if response[0] != 'ok':
2555.1.1 by Martin Pool
Remove use of 'assert False' to raise an exception unconditionally
2390
            raise errors.UnexpectedSmartServerResponse(response)
3245.4.24 by Andrew Bennetts
Consistently raise errors from the server as ErrorFromSmartServer exceptions.
2391
        ok, branch_token, repo_token = response
2392
        return branch_token, repo_token
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
2393
2018.5.142 by Andrew Bennetts
Change Branch.lock_token to only accept and receive the branch lock token (rather than the branch and repo lock tokens).
2394
    def lock_write(self, token=None):
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2395
        if not self._lock_mode:
4731.1.2 by Andrew Bennetts
Refactor to reduce duplication.
2396
            self._note_lock('w')
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
2397
            # Lock the branch and repo in one remote call.
2018.5.142 by Andrew Bennetts
Change Branch.lock_token to only accept and receive the branch lock token (rather than the branch and repo lock tokens).
2398
            remote_tokens = self._remote_lock_write(token)
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2399
            self._lock_token, self._repo_lock_token = remote_tokens
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
2400
            if not self._lock_token:
2401
                raise SmartProtocolError('Remote server did not return a token!')
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
2402
            # Tell the self.repository object that it is locked.
3692.1.2 by Andrew Bennetts
Fix regression introduced by fix, and add a test for that regression.
2403
            self.repository.lock_write(
2404
                self._repo_lock_token, _skip_rpc=True)
3692.1.1 by Andrew Bennetts
Make RemoteBranch.lock_write lock the repository too.
2405
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2406
            if self._real_branch is not None:
3692.1.5 by Andrew Bennetts
Fix bug revealed by removing _ensure_real call from RemoteBranch.lock_write.
2407
                self._real_branch.lock_write(token=self._lock_token)
2018.5.142 by Andrew Bennetts
Change Branch.lock_token to only accept and receive the branch lock token (rather than the branch and repo lock tokens).
2408
            if token is not None:
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2409
                self._leave_lock = True
2410
            else:
2411
                self._leave_lock = False
2412
            self._lock_mode = 'w'
2413
            self._lock_count = 1
2414
        elif self._lock_mode == 'r':
2415
            raise errors.ReadOnlyTransaction
2416
        else:
2018.5.142 by Andrew Bennetts
Change Branch.lock_token to only accept and receive the branch lock token (rather than the branch and repo lock tokens).
2417
            if token is not None:
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
2418
                # A token was given to lock_write, and we're relocking, so
2419
                # check that the given token actually matches the one we
2420
                # already have.
2018.5.142 by Andrew Bennetts
Change Branch.lock_token to only accept and receive the branch lock token (rather than the branch and repo lock tokens).
2421
                if token != self._lock_token:
2422
                    raise errors.TokenMismatch(token, self._lock_token)
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2423
            self._lock_count += 1
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
2424
            # Re-lock the repository too.
3692.1.2 by Andrew Bennetts
Fix regression introduced by fix, and add a test for that regression.
2425
            self.repository.lock_write(self._repo_lock_token)
3015.2.9 by Robert Collins
Handle repositories that do not allow remote locking, like pack repositories, in the client side remote server proxy objects.
2426
        return self._lock_token or None
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2427
2428
    def _unlock(self, branch_token, repo_token):
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2429
        err_context = {'token': str((branch_token, repo_token))}
2430
        response = self._call(
2431
            'Branch.unlock', self._remote_path(), branch_token,
2432
            repo_token or '', **err_context)
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2433
        if response == ('ok',):
2434
            return
3245.4.24 by Andrew Bennetts
Consistently raise errors from the server as ErrorFromSmartServer exceptions.
2435
        raise errors.UnexpectedSmartServerResponse(response)
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
2436
4634.85.9 by Andrew Bennetts
Add some experimental decorators: @only_raises(..) and @cleanup_method.
2437
    @only_raises(errors.LockNotHeld, errors.LockBroken)
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
2438
    def unlock(self):
3692.1.1 by Andrew Bennetts
Make RemoteBranch.lock_write lock the repository too.
2439
        try:
2440
            self._lock_count -= 1
2441
            if not self._lock_count:
2442
                self._clear_cached_state()
2443
                mode = self._lock_mode
2444
                self._lock_mode = None
2445
                if self._real_branch is not None:
2446
                    if (not self._leave_lock and mode == 'w' and
2447
                        self._repo_lock_token):
2448
                        # If this RemoteBranch will remove the physical lock
2449
                        # for the repository, make sure the _real_branch
2450
                        # doesn't do it first.  (Because the _real_branch's
2451
                        # repository is set to be the RemoteRepository.)
2452
                        self._real_branch.repository.leave_lock_in_place()
2453
                    self._real_branch.unlock()
2454
                if mode != 'w':
2455
                    # Only write-locked branched need to make a remote method
4031.3.1 by Frank Aspell
Fixing various typos
2456
                    # call to perform the unlock.
3692.1.1 by Andrew Bennetts
Make RemoteBranch.lock_write lock the repository too.
2457
                    return
2458
                if not self._lock_token:
2459
                    raise AssertionError('Locked, but no token!')
2460
                branch_token = self._lock_token
2461
                repo_token = self._repo_lock_token
2462
                self._lock_token = None
2463
                self._repo_lock_token = None
2464
                if not self._leave_lock:
2465
                    self._unlock(branch_token, repo_token)
2466
        finally:
2467
            self.repository.unlock()
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
2468
2469
    def break_lock(self):
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
2470
        self._ensure_real()
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
2471
        return self._real_branch.break_lock()
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
2472
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2473
    def leave_lock_in_place(self):
3015.2.9 by Robert Collins
Handle repositories that do not allow remote locking, like pack repositories, in the client side remote server proxy objects.
2474
        if not self._lock_token:
2475
            raise NotImplementedError(self.leave_lock_in_place)
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2476
        self._leave_lock = True
2477
2478
    def dont_leave_lock_in_place(self):
3015.2.9 by Robert Collins
Handle repositories that do not allow remote locking, like pack repositories, in the client side remote server proxy objects.
2479
        if not self._lock_token:
3015.2.15 by Robert Collins
Review feedback.
2480
            raise NotImplementedError(self.dont_leave_lock_in_place)
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2481
        self._leave_lock = False
2482
4634.69.1 by Andrew Bennetts
Apply @needs_read_lock to RemoteBranch.get_rev_id.
2483
    @needs_read_lock
4419.2.4 by Andrew Bennetts
Add Repository.get_rev_id_for_revno RPC, removes VFS calls from 'pull -r 123' case.
2484
    def get_rev_id(self, revno, history=None):
4419.2.17 by Andrew Bennetts
Fix test failures in test_lookup_revision_id_by_dotted.
2485
        if revno == 0:
2486
            return _mod_revision.NULL_REVISION
4419.2.4 by Andrew Bennetts
Add Repository.get_rev_id_for_revno RPC, removes VFS calls from 'pull -r 123' case.
2487
        last_revision_info = self.last_revision_info()
4419.2.15 by Andrew Bennetts
Simplify RemoteBranch.get_rev_id a little; get_rev_id_for_revno handles stacking for us.
2488
        ok, result = self.repository.get_rev_id_for_revno(
2489
            revno, last_revision_info)
2490
        if ok:
2491
            return result
2492
        missing_parent = result[1]
4419.2.17 by Andrew Bennetts
Fix test failures in test_lookup_revision_id_by_dotted.
2493
        # Either the revision named by the server is missing, or its parent
2494
        # is.  Call get_parent_map to determine which, so that we report a
2495
        # useful error.
2496
        parent_map = self.repository.get_parent_map([missing_parent])
2497
        if missing_parent in parent_map:
2498
            missing_parent = parent_map[missing_parent]
4419.2.4 by Andrew Bennetts
Add Repository.get_rev_id_for_revno RPC, removes VFS calls from 'pull -r 123' case.
2499
        raise errors.RevisionNotPresent(missing_parent, self.repository)
2500
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2501
    def _last_revision_info(self):
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2502
        response = self._call('Branch.last_revision_info', self._remote_path())
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
2503
        if response[0] != 'ok':
2504
            raise SmartProtocolError('unexpected response code %s' % (response,))
2018.5.51 by Wouter van Heyst
Test and implement RemoteBranch.last_revision_info()
2505
        revno = int(response[1])
2018.5.83 by Andrew Bennetts
Fix some test failures caused by the switch from unicode to UTF-8-encoded strs for revision IDs.
2506
        last_revision = response[2]
2018.5.51 by Wouter van Heyst
Test and implement RemoteBranch.last_revision_info()
2507
        return (revno, last_revision)
2508
2018.5.105 by Andrew Bennetts
Implement revision_history caching for RemoteBranch.
2509
    def _gen_revision_history(self):
2510
        """See Branch._gen_revision_history()."""
4419.1.3 by Andrew Bennetts
Quick fix by using self._ensure_real.
2511
        if self._is_stacked:
2512
            self._ensure_real()
2513
            return self._real_branch._gen_revision_history()
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2514
        response_tuple, response_handler = self._call_expecting_body(
3691.2.3 by Martin Pool
Factor out RemoteBranch._remote_path() and disable RemoteBranch stacking
2515
            'Branch.revision_history', self._remote_path())
3245.4.58 by Andrew Bennetts
Unpack call_expecting_body's return value into variables, to avoid lots of ugly subscripting.
2516
        if response_tuple[0] != 'ok':
3452.2.2 by Andrew Bennetts
Experimental PackRepository.{check_references,autopack} RPCs.
2517
            raise errors.UnexpectedSmartServerResponse(response_tuple)
3245.4.58 by Andrew Bennetts
Unpack call_expecting_body's return value into variables, to avoid lots of ugly subscripting.
2518
        result = response_handler.read_body_bytes().split('\x00')
2018.5.38 by Robert Collins
Implement RemoteBranch.revision_history().
2519
        if result == ['']:
2520
            return []
2521
        return result
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
2522
3691.2.3 by Martin Pool
Factor out RemoteBranch._remote_path() and disable RemoteBranch stacking
2523
    def _remote_path(self):
2524
        return self.bzrdir._path_for_remote_call(self._client)
2525
3441.5.18 by Andrew Bennetts
Fix some test failures.
2526
    def _set_last_revision_descendant(self, revision_id, other_branch,
3441.5.28 by Andrew Bennetts
Another review tweak: rename do_not_overwrite_descendant to allow_overwrite_descendant.
2527
            allow_diverged=False, allow_overwrite_descendant=False):
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2528
        # This performs additional work to meet the hook contract; while its
2529
        # undesirable, we have to synthesise the revno to call the hook, and
2530
        # not calling the hook is worse as it means changes can't be prevented.
2531
        # Having calculated this though, we can't just call into
2532
        # set_last_revision_info as a simple call, because there is a set_rh
2533
        # hook that some folk may still be using.
2534
        old_revno, old_revid = self.last_revision_info()
2535
        history = self._lefthand_history(revision_id)
2536
        self._run_pre_change_branch_tip_hooks(len(history), revision_id)
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2537
        err_context = {'other_branch': other_branch}
2538
        response = self._call('Branch.set_last_revision_ex',
2539
            self._remote_path(), self._lock_token, self._repo_lock_token,
2540
            revision_id, int(allow_diverged), int(allow_overwrite_descendant),
2541
            **err_context)
3441.5.6 by Andrew Bennetts
Greatly simplify RemoteBranch.update_revisions. Still needs more tests.
2542
        self._clear_cached_state()
3441.5.18 by Andrew Bennetts
Fix some test failures.
2543
        if len(response) != 3 and response[0] != 'ok':
3441.5.6 by Andrew Bennetts
Greatly simplify RemoteBranch.update_revisions. Still needs more tests.
2544
            raise errors.UnexpectedSmartServerResponse(response)
3441.5.18 by Andrew Bennetts
Fix some test failures.
2545
        new_revno, new_revision_id = response[1:]
2546
        self._last_revision_info_cache = new_revno, new_revision_id
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2547
        self._run_post_change_branch_tip_hooks(old_revno, old_revid)
3692.1.5 by Andrew Bennetts
Fix bug revealed by removing _ensure_real call from RemoteBranch.lock_write.
2548
        if self._real_branch is not None:
2549
            cache = new_revno, new_revision_id
2550
            self._real_branch._last_revision_info_cache = cache
3441.5.6 by Andrew Bennetts
Greatly simplify RemoteBranch.update_revisions. Still needs more tests.
2551
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2552
    def _set_last_revision(self, revision_id):
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2553
        old_revno, old_revid = self.last_revision_info()
2554
        # This performs additional work to meet the hook contract; while its
2555
        # undesirable, we have to synthesise the revno to call the hook, and
2556
        # not calling the hook is worse as it means changes can't be prevented.
2557
        # Having calculated this though, we can't just call into
2558
        # set_last_revision_info as a simple call, because there is a set_rh
2559
        # hook that some folk may still be using.
2560
        history = self._lefthand_history(revision_id)
2561
        self._run_pre_change_branch_tip_hooks(len(history), revision_id)
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2562
        self._clear_cached_state()
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2563
        response = self._call('Branch.set_last_revision',
2564
            self._remote_path(), self._lock_token, self._repo_lock_token,
2565
            revision_id)
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2566
        if response != ('ok',):
2567
            raise errors.UnexpectedSmartServerResponse(response)
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2568
        self._run_post_change_branch_tip_hooks(old_revno, old_revid)
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2569
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
2570
    @needs_write_lock
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
2571
    def set_revision_history(self, rev_history):
2018.12.3 by Andrew Bennetts
Add a Branch.set_last_revision smart method, and make RemoteBranch.set_revision_history use it.
2572
        # Send just the tip revision of the history; the server will generate
2573
        # the full history from that.  If the revision doesn't exist in this
2574
        # branch, NoSuchRevision will be raised.
2575
        if rev_history == []:
2018.5.170 by Andrew Bennetts
Use 'null:' instead of '' to mean NULL_REVISION on the wire.
2576
            rev_id = 'null:'
2018.12.3 by Andrew Bennetts
Add a Branch.set_last_revision smart method, and make RemoteBranch.set_revision_history use it.
2577
        else:
2578
            rev_id = rev_history[-1]
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2579
        self._set_last_revision(rev_id)
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2580
        for hook in branch.Branch.hooks['set_rh']:
2581
            hook(self, rev_history)
2018.5.105 by Andrew Bennetts
Implement revision_history caching for RemoteBranch.
2582
        self._cache_revision_history(rev_history)
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
2583
4078.2.1 by Robert Collins
Add a Branch.get_parent remote call for RemoteBranch.
2584
    def _get_parent_location(self):
2585
        medium = self._client._medium
2586
        if medium._is_remote_before((1, 13)):
2587
            return self._vfs_get_parent_location()
2588
        try:
4083.1.4 by Andrew Bennetts
Fix trivial bug in get_parent RPC.
2589
            response = self._call('Branch.get_parent', self._remote_path())
4078.2.1 by Robert Collins
Add a Branch.get_parent remote call for RemoteBranch.
2590
        except errors.UnknownSmartMethod:
4094.1.1 by Andrew Bennetts
Add some medium._remember_is_before((1, 13)) calls.
2591
            medium._remember_remote_is_before((1, 13))
4078.2.1 by Robert Collins
Add a Branch.get_parent remote call for RemoteBranch.
2592
            return self._vfs_get_parent_location()
4083.1.4 by Andrew Bennetts
Fix trivial bug in get_parent RPC.
2593
        if len(response) != 1:
4083.1.6 by Andrew Bennetts
Fix trivial bug in my trivial bug fix :)
2594
            raise errors.UnexpectedSmartServerResponse(response)
4083.1.4 by Andrew Bennetts
Fix trivial bug in get_parent RPC.
2595
        parent_location = response[0]
2596
        if parent_location == '':
4078.2.1 by Robert Collins
Add a Branch.get_parent remote call for RemoteBranch.
2597
            return None
4083.1.4 by Andrew Bennetts
Fix trivial bug in get_parent RPC.
2598
        return parent_location
4078.2.1 by Robert Collins
Add a Branch.get_parent remote call for RemoteBranch.
2599
2600
    def _vfs_get_parent_location(self):
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2601
        self._ensure_real()
4083.1.5 by Andrew Bennetts
Fix trivial bug in get_parent RPC.
2602
        return self._real_branch._get_parent_location()
4032.1.1 by John Arbash Meinel
Merge the removal of all trailing whitespace, and resolve conflicts.
2603
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2604
    def _set_parent_location(self, url):
4288.1.7 by Robert Collins
Add new remote server verb Branch.set_parent_location, dropping roundtrips further on push operations.
2605
        medium = self._client._medium
2606
        if medium._is_remote_before((1, 15)):
2607
            return self._vfs_set_parent_location(url)
2608
        try:
2609
            call_url = url or ''
2610
            if type(call_url) is not str:
2611
                raise AssertionError('url must be a str or None (%s)' % url)
2612
            response = self._call('Branch.set_parent_location',
2613
                self._remote_path(), self._lock_token, self._repo_lock_token,
2614
                call_url)
2615
        except errors.UnknownSmartMethod:
2616
            medium._remember_remote_is_before((1, 15))
2617
            return self._vfs_set_parent_location(url)
2618
        if response != ():
2619
            raise errors.UnexpectedSmartServerResponse(response)
2620
2621
    def _vfs_set_parent_location(self, url):
4288.1.4 by Robert Collins
Remove the explicit set_parent method on RemoteBranch in favour of inheriting from Branch.
2622
        self._ensure_real()
2623
        return self._real_branch._set_parent_location(url)
4032.1.1 by John Arbash Meinel
Merge the removal of all trailing whitespace, and resolve conflicts.
2624
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
2625
    @needs_write_lock
2477.1.2 by Martin Pool
Rename push/pull back to 'run_hooks' (jameinel)
2626
    def pull(self, source, overwrite=False, stop_revision=None,
2477.1.9 by Martin Pool
Review cleanups from John, mostly docs
2627
             **kwargs):
3441.5.29 by Andrew Bennetts
More review tweaks: whitespace nits in test_smart, add (and use) ._clear_cached_state_of_remote_branch_only method in bzrlib/remote.py.
2628
        self._clear_cached_state_of_remote_branch_only()
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
2629
        self._ensure_real()
3482.1.1 by John Arbash Meinel
Fix bug #238149, RemoteBranch.pull needs to return the _real_branch's pull result.
2630
        return self._real_branch.pull(
2477.1.2 by Martin Pool
Rename push/pull back to 'run_hooks' (jameinel)
2631
            source, overwrite=overwrite, stop_revision=stop_revision,
3489.2.4 by Andrew Bennetts
Fix all tests broken by fixing make_branch_and_tree.
2632
            _override_hook_target=self, **kwargs)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
2633
2018.14.3 by Andrew Bennetts
Make a couple more branch_implementations tests pass.
2634
    @needs_read_lock
2635
    def push(self, target, overwrite=False, stop_revision=None):
2636
        self._ensure_real()
2018.5.97 by Andrew Bennetts
Fix more tests.
2637
        return self._real_branch.push(
2477.1.5 by Martin Pool
More cleanups of Branch.push to get the right behaviour with RemoteBranches
2638
            target, overwrite=overwrite, stop_revision=stop_revision,
2639
            _override_hook_source_branch=self)
2018.14.3 by Andrew Bennetts
Make a couple more branch_implementations tests pass.
2640
2641
    def is_locked(self):
2642
        return self._lock_count >= 1
2643
3634.2.1 by John Arbash Meinel
Thunk over to the real branch's revision_id_to_revno.
2644
    @needs_read_lock
2645
    def revision_id_to_revno(self, revision_id):
2646
        self._ensure_real()
2647
        return self._real_branch.revision_id_to_revno(revision_id)
2648
2892.2.1 by Andrew Bennetts
Add Branch.set_last_revision_info smart method, and make the RemoteBranch client use it.
2649
    @needs_write_lock
2018.5.83 by Andrew Bennetts
Fix some test failures caused by the switch from unicode to UTF-8-encoded strs for revision IDs.
2650
    def set_last_revision_info(self, revno, revision_id):
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2651
        # XXX: These should be returned by the set_last_revision_info verb
2652
        old_revno, old_revid = self.last_revision_info()
2653
        self._run_pre_change_branch_tip_hooks(revno, revision_id)
2892.2.1 by Andrew Bennetts
Add Branch.set_last_revision_info smart method, and make the RemoteBranch client use it.
2654
        revision_id = ensure_null(revision_id)
3297.4.2 by Andrew Bennetts
Add backwards compatibility for servers older than 1.4.
2655
        try:
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2656
            response = self._call('Branch.set_last_revision_info',
2657
                self._remote_path(), self._lock_token, self._repo_lock_token,
2658
                str(revno), revision_id)
3297.4.2 by Andrew Bennetts
Add backwards compatibility for servers older than 1.4.
2659
        except errors.UnknownSmartMethod:
2660
            self._ensure_real()
3441.5.29 by Andrew Bennetts
More review tweaks: whitespace nits in test_smart, add (and use) ._clear_cached_state_of_remote_branch_only method in bzrlib/remote.py.
2661
            self._clear_cached_state_of_remote_branch_only()
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2662
            self._real_branch.set_last_revision_info(revno, revision_id)
2663
            self._last_revision_info_cache = revno, revision_id
2664
            return
2892.2.1 by Andrew Bennetts
Add Branch.set_last_revision_info smart method, and make the RemoteBranch client use it.
2665
        if response == ('ok',):
2666
            self._clear_cached_state()
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2667
            self._last_revision_info_cache = revno, revision_id
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2668
            self._run_post_change_branch_tip_hooks(old_revno, old_revid)
3441.5.29 by Andrew Bennetts
More review tweaks: whitespace nits in test_smart, add (and use) ._clear_cached_state_of_remote_branch_only method in bzrlib/remote.py.
2669
            # Update the _real_branch's cache too.
2670
            if self._real_branch is not None:
2671
                cache = self._last_revision_info_cache
2672
                self._real_branch._last_revision_info_cache = cache
2892.2.1 by Andrew Bennetts
Add Branch.set_last_revision_info smart method, and make the RemoteBranch client use it.
2673
        else:
2674
            raise errors.UnexpectedSmartServerResponse(response)
2018.5.83 by Andrew Bennetts
Fix some test failures caused by the switch from unicode to UTF-8-encoded strs for revision IDs.
2675
3441.5.27 by Andrew Bennetts
Tweaks suggested by John's review: rename _check_if_descendant_or_diverged, move caching last_revision_info into base Branch, better use of lock decorators.
2676
    @needs_write_lock
2018.5.95 by Andrew Bennetts
Add a Transport.is_readonly remote call, let {Branch,Repository}.lock_write remote call return UnlockableTransport, and miscellaneous test fixes.
2677
    def generate_revision_history(self, revision_id, last_rev=None,
2678
                                  other_branch=None):
3441.5.6 by Andrew Bennetts
Greatly simplify RemoteBranch.update_revisions. Still needs more tests.
2679
        medium = self._client._medium
3441.5.23 by Andrew Bennetts
Fix test failures.
2680
        if not medium._is_remote_before((1, 6)):
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2681
            # Use a smart method for 1.6 and above servers
3441.5.6 by Andrew Bennetts
Greatly simplify RemoteBranch.update_revisions. Still needs more tests.
2682
            try:
3441.5.18 by Andrew Bennetts
Fix some test failures.
2683
                self._set_last_revision_descendant(revision_id, other_branch,
3441.5.28 by Andrew Bennetts
Another review tweak: rename do_not_overwrite_descendant to allow_overwrite_descendant.
2684
                    allow_diverged=True, allow_overwrite_descendant=True)
3441.5.6 by Andrew Bennetts
Greatly simplify RemoteBranch.update_revisions. Still needs more tests.
2685
                return
3441.5.18 by Andrew Bennetts
Fix some test failures.
2686
            except errors.UnknownSmartMethod:
3441.5.23 by Andrew Bennetts
Fix test failures.
2687
                medium._remember_remote_is_before((1, 6))
3441.5.29 by Andrew Bennetts
More review tweaks: whitespace nits in test_smart, add (and use) ._clear_cached_state_of_remote_branch_only method in bzrlib/remote.py.
2688
        self._clear_cached_state_of_remote_branch_only()
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2689
        self.set_revision_history(self._lefthand_history(revision_id,
2690
            last_rev=last_rev,other_branch=other_branch))
2018.5.96 by Andrew Bennetts
Merge from bzr.dev, resolving the worst of the semantic conflicts, but there's
2691
2018.5.97 by Andrew Bennetts
Fix more tests.
2692
    def set_push_location(self, location):
2693
        self._ensure_real()
2694
        return self._real_branch.set_push_location(location)
2695
2018.18.25 by Martin Pool
Repository.tarball fixes for python2.4
2696
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
2697
class RemoteConfig(object):
2698
    """A Config that reads and writes from smart verbs.
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
2699
2700
    It is a low-level object that considers config data to be name/value pairs
2701
    that may be associated with a section. Assigning meaning to the these
2702
    values is done at higher levels like bzrlib.config.TreeConfig.
2703
    """
2704
2705
    def get_option(self, name, section=None, default=None):
2706
        """Return the value associated with a named option.
2707
2708
        :param name: The name of the value
2709
        :param section: The section the option is in (if any)
2710
        :param default: The value to return if the value is not set
2711
        :return: The value or default value
2712
        """
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
2713
        try:
2714
            configobj = self._get_configobj()
2715
            if section is None:
2716
                section_obj = configobj
2717
            else:
2718
                try:
2719
                    section_obj = configobj[section]
2720
                except KeyError:
2721
                    return default
2722
            return section_obj.get(name, default)
2723
        except errors.UnknownSmartMethod:
2724
            return self._vfs_get_option(name, section, default)
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
2725
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
2726
    def _response_to_configobj(self, response):
4288.1.2 by Robert Collins
Create a server verb for doing BzrDir.get_config()
2727
        if len(response[0]) and response[0][0] != 'ok':
4288.1.4 by Robert Collins
Remove the explicit set_parent method on RemoteBranch in favour of inheriting from Branch.
2728
            raise errors.UnexpectedSmartServerResponse(response)
4241.5.1 by Matt Nordhoff
Fix Branch.get_config_file smart verb on multi-line config files. (Bug #354075)
2729
        lines = response[1].read_body_bytes().splitlines()
2730
        return config.ConfigObj(lines, encoding='utf-8')
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
2731
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
2732
2733
class RemoteBranchConfig(RemoteConfig):
2734
    """A RemoteConfig for Branches."""
2735
2736
    def __init__(self, branch):
2737
        self._branch = branch
2738
2739
    def _get_configobj(self):
2740
        path = self._branch._remote_path()
2741
        response = self._branch._client.call_expecting_body(
2742
            'Branch.get_config_file', path)
2743
        return self._response_to_configobj(response)
2744
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
2745
    def set_option(self, value, name, section=None):
2746
        """Set the value associated with a named option.
2747
2748
        :param value: The value to set
2749
        :param name: The name of the value to set
2750
        :param section: The section the option is in (if any)
2751
        """
4226.2.1 by Robert Collins
Set branch config options via a smart method.
2752
        medium = self._branch._client._medium
2753
        if medium._is_remote_before((1, 14)):
2754
            return self._vfs_set_option(value, name, section)
2755
        try:
2756
            path = self._branch._remote_path()
2757
            response = self._branch._client.call('Branch.set_config_option',
2758
                path, self._branch._lock_token, self._branch._repo_lock_token,
4226.2.2 by Robert Collins
Fix setting config options to support unicode values and don't attempt to reset repositories _fallback_repositories as the simple approach fails to work.
2759
                value.encode('utf8'), name, section or '')
4226.2.1 by Robert Collins
Set branch config options via a smart method.
2760
        except errors.UnknownSmartMethod:
2761
            medium._remember_remote_is_before((1, 14))
2762
            return self._vfs_set_option(value, name, section)
2763
        if response != ():
2764
            raise errors.UnexpectedSmartServerResponse(response)
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
2765
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
2766
    def _real_object(self):
2767
        self._branch._ensure_real()
2768
        return self._branch._real_branch
2769
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
2770
    def _vfs_set_option(self, value, name, section=None):
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
2771
        return self._real_object()._get_config().set_option(
2772
            value, name, section)
2773
2774
2775
class RemoteBzrDirConfig(RemoteConfig):
2776
    """A RemoteConfig for BzrDirs."""
2777
2778
    def __init__(self, bzrdir):
2779
        self._bzrdir = bzrdir
2780
2781
    def _get_configobj(self):
4288.1.4 by Robert Collins
Remove the explicit set_parent method on RemoteBranch in favour of inheriting from Branch.
2782
        medium = self._bzrdir._client._medium
2783
        verb = 'BzrDir.get_config_file'
2784
        if medium._is_remote_before((1, 15)):
2785
            raise errors.UnknownSmartMethod(verb)
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
2786
        path = self._bzrdir._path_for_remote_call(self._bzrdir._client)
2787
        response = self._bzrdir._call_expecting_body(
4288.1.4 by Robert Collins
Remove the explicit set_parent method on RemoteBranch in favour of inheriting from Branch.
2788
            verb, path)
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
2789
        return self._response_to_configobj(response)
2790
2791
    def _vfs_get_option(self, name, section, default):
2792
        return self._real_object()._get_config().get_option(
2793
            name, section, default)
2794
2795
    def set_option(self, value, name, section=None):
2796
        """Set the value associated with a named option.
2797
2798
        :param value: The value to set
2799
        :param name: The name of the value to set
2800
        :param section: The section the option is in (if any)
2801
        """
2802
        return self._real_object()._get_config().set_option(
2803
            value, name, section)
2804
2805
    def _real_object(self):
2806
        self._bzrdir._ensure_real()
2807
        return self._bzrdir._real_bzrdir
2808
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
2809
2810
2018.18.25 by Martin Pool
Repository.tarball fixes for python2.4
2811
def _extract_tar(tar, to_dir):
2812
    """Extract all the contents of a tarfile object.
2813
2814
    A replacement for extractall, which is not present in python2.4
2815
    """
2816
    for tarinfo in tar:
2817
        tar.extract(tarinfo, to_dir)
3533.3.1 by Andrew Bennetts
Remove duplication of error translation in bzrlib/remote.py.
2818
2819
2820
def _translate_error(err, **context):
2821
    """Translate an ErrorFromSmartServer into a more useful error.
2822
2823
    Possible context keys:
2824
      - branch
2825
      - repository
2826
      - bzrdir
2827
      - token
2828
      - other_branch
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2829
      - path
3690.1.1 by Andrew Bennetts
Unexpected error responses from a smart server no longer cause the client to traceback.
2830
2831
    If the error from the server doesn't match a known pattern, then
3690.1.2 by Andrew Bennetts
Rename UntranslateableErrorFromSmartServer -> UnknownErrorFromSmartServer.
2832
    UnknownErrorFromSmartServer is raised.
3533.3.1 by Andrew Bennetts
Remove duplication of error translation in bzrlib/remote.py.
2833
    """
2834
    def find(name):
3533.3.4 by Andrew Bennetts
Add tests for _translate_error's robustness.
2835
        try:
2836
            return context[name]
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2837
        except KeyError, key_err:
2838
            mutter('Missing key %r in context %r', key_err.args[0], context)
3533.3.4 by Andrew Bennetts
Add tests for _translate_error's robustness.
2839
            raise err
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2840
    def get_path():
3779.3.3 by Andrew Bennetts
Add a docstring.
2841
        """Get the path from the context if present, otherwise use first error
2842
        arg.
2843
        """
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2844
        try:
2845
            return context['path']
2846
        except KeyError, key_err:
2847
            try:
2848
                return err.error_args[0]
2849
            except IndexError, idx_err:
2850
                mutter(
2851
                    'Missing key %r in context %r', key_err.args[0], context)
2852
                raise err
2853
4650.2.1 by Robert Collins
Deserialise IncompatibleRepositories errors in the client, generating
2854
    if err.error_verb == 'IncompatibleRepositories':
2855
        raise errors.IncompatibleRepositories(err.error_args[0],
2856
            err.error_args[1], err.error_args[2])
2857
    elif err.error_verb == 'NoSuchRevision':
3533.3.1 by Andrew Bennetts
Remove duplication of error translation in bzrlib/remote.py.
2858
        raise NoSuchRevision(find('branch'), err.error_args[0])
2859
    elif err.error_verb == 'nosuchrevision':
2860
        raise NoSuchRevision(find('repository'), err.error_args[0])
4734.4.3 by Brian de Alwis
Add support for the HPSS to do further probing when a the provided
2861
    elif err.error_verb == 'nobranch':
4734.4.8 by Andrew Bennetts
Fix HPSS tests; pass 'location is a repository' message via smart server when possible (adds BzrDir.open_branchV3 verb).
2862
        if len(err.error_args) >= 1:
2863
            extra = err.error_args[0]
2864
        else:
2865
            extra = None
4734.4.3 by Brian de Alwis
Add support for the HPSS to do further probing when a the provided
2866
        raise errors.NotBranchError(path=find('bzrdir').root_transport.base,
4734.4.8 by Andrew Bennetts
Fix HPSS tests; pass 'location is a repository' message via smart server when possible (adds BzrDir.open_branchV3 verb).
2867
            detail=extra)
3533.3.1 by Andrew Bennetts
Remove duplication of error translation in bzrlib/remote.py.
2868
    elif err.error_verb == 'norepository':
2869
        raise errors.NoRepositoryPresent(find('bzrdir'))
2870
    elif err.error_verb == 'LockContention':
2871
        raise errors.LockContention('(remote lock)')
2872
    elif err.error_verb == 'UnlockableTransport':
3533.3.4 by Andrew Bennetts
Add tests for _translate_error's robustness.
2873
        raise errors.UnlockableTransport(find('bzrdir').root_transport)
3533.3.1 by Andrew Bennetts
Remove duplication of error translation in bzrlib/remote.py.
2874
    elif err.error_verb == 'LockFailed':
2875
        raise errors.LockFailed(err.error_args[0], err.error_args[1])
2876
    elif err.error_verb == 'TokenMismatch':
2877
        raise errors.TokenMismatch(find('token'), '(remote token)')
2878
    elif err.error_verb == 'Diverged':
2879
        raise errors.DivergedBranches(find('branch'), find('other_branch'))
3577.1.1 by Andrew Bennetts
Cherry-pick TipChangeRejected changes from pre-branch-tip-changed-hook loom.
2880
    elif err.error_verb == 'TipChangeRejected':
2881
        raise errors.TipChangeRejected(err.error_args[0].decode('utf8'))
3691.2.6 by Martin Pool
Disable RemoteBranch stacking, but get get_stacked_on_url working, and passing back exceptions
2882
    elif err.error_verb == 'UnstackableBranchFormat':
2883
        raise errors.UnstackableBranchFormat(*err.error_args)
2884
    elif err.error_verb == 'UnstackableRepositoryFormat':
2885
        raise errors.UnstackableRepositoryFormat(*err.error_args)
2886
    elif err.error_verb == 'NotStacked':
2887
        raise errors.NotStacked(branch=find('branch'))
3779.3.1 by Andrew Bennetts
Move encoding/decoding logic of PermissionDenied and ReadError so that it happens for all RPCs.
2888
    elif err.error_verb == 'PermissionDenied':
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2889
        path = get_path()
3779.3.1 by Andrew Bennetts
Move encoding/decoding logic of PermissionDenied and ReadError so that it happens for all RPCs.
2890
        if len(err.error_args) >= 2:
2891
            extra = err.error_args[1]
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2892
        else:
2893
            extra = None
3779.3.1 by Andrew Bennetts
Move encoding/decoding logic of PermissionDenied and ReadError so that it happens for all RPCs.
2894
        raise errors.PermissionDenied(path, extra=extra)
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2895
    elif err.error_verb == 'ReadError':
2896
        path = get_path()
2897
        raise errors.ReadError(path)
2898
    elif err.error_verb == 'NoSuchFile':
2899
        path = get_path()
2900
        raise errors.NoSuchFile(path)
2901
    elif err.error_verb == 'FileExists':
2902
        raise errors.FileExists(err.error_args[0])
2903
    elif err.error_verb == 'DirectoryNotEmpty':
2904
        raise errors.DirectoryNotEmpty(err.error_args[0])
2905
    elif err.error_verb == 'ShortReadvError':
2906
        args = err.error_args
2907
        raise errors.ShortReadvError(
2908
            args[0], int(args[1]), int(args[2]), int(args[3]))
2909
    elif err.error_verb in ('UnicodeEncodeError', 'UnicodeDecodeError'):
2910
        encoding = str(err.error_args[0]) # encoding must always be a string
2911
        val = err.error_args[1]
2912
        start = int(err.error_args[2])
2913
        end = int(err.error_args[3])
2914
        reason = str(err.error_args[4]) # reason must always be a string
2915
        if val.startswith('u:'):
2916
            val = val[2:].decode('utf-8')
2917
        elif val.startswith('s:'):
2918
            val = val[2:].decode('base64')
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2919
        if err.error_verb == 'UnicodeDecodeError':
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2920
            raise UnicodeDecodeError(encoding, val, start, end, reason)
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2921
        elif err.error_verb == 'UnicodeEncodeError':
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2922
            raise UnicodeEncodeError(encoding, val, start, end, reason)
2923
    elif err.error_verb == 'ReadOnlyError':
2924
        raise errors.TransportNotPossible('readonly transport')
3690.1.2 by Andrew Bennetts
Rename UntranslateableErrorFromSmartServer -> UnknownErrorFromSmartServer.
2925
    raise errors.UnknownErrorFromSmartServer(err)