/brz/remove-bazaar

To get this branch, use:
bzr branch http://gegoxaren.bato24.eu/bzr/brz/remove-bazaar
4988.10.3 by John Arbash Meinel
Merge bzr.dev 5007, resolve conflict, update NEWS
1
# Copyright (C) 2006-2010 Canonical Ltd
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
2
#
3
# This program is free software; you can redistribute it and/or modify
4
# it under the terms of the GNU General Public License as published by
5
# the Free Software Foundation; either version 2 of the License, or
6
# (at your option) any later version.
7
#
8
# This program is distributed in the hope that it will be useful,
9
# but WITHOUT ANY WARRANTY; without even the implied warranty of
10
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
11
# GNU General Public License for more details.
12
#
13
# You should have received a copy of the GNU General Public License
14
# along with this program; if not, write to the Free Software
4183.7.1 by Sabin Iacob
update FSF mailing address
15
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
16
3211.5.2 by Robert Collins
Change RemoteRepository.get_parent_map to use bz2 not gzip for compression.
17
import bz2
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
18
2490.2.5 by Aaron Bentley
Use GraphWalker.unique_ancestor to determine merge base
19
from bzrlib import (
2694.5.4 by Jelmer Vernooij
Move bzrlib.util.bencode to bzrlib._bencode_py.
20
    bencode,
2490.2.5 by Aaron Bentley
Use GraphWalker.unique_ancestor to determine merge base
21
    branch,
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
22
    bzrdir,
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
23
    config,
3192.1.1 by Andrew Bennetts
Add some -Dhpss debugging to get_parent_map.
24
    debug,
2490.2.5 by Aaron Bentley
Use GraphWalker.unique_ancestor to determine merge base
25
    errors,
3172.5.1 by Robert Collins
Create a RemoteRepository get_graph implementation and delegate get_parents_map to the real repository.
26
    graph,
4509.3.28 by Martin Pool
Add missing import
27
    lock,
2490.2.5 by Aaron Bentley
Use GraphWalker.unique_ancestor to determine merge base
28
    lockdir,
29
    repository,
2948.3.1 by John Arbash Meinel
Fix bug #158333, make sure that Repository.fetch(self) is properly a no-op for all Repository implementations.
30
    revision,
4307.2.3 by Robert Collins
Change RemoteRepository.has_revision to use get_parent_map to leverage the caching.
31
    revision as _mod_revision,
4913.4.2 by Jelmer Vernooij
Add Repository.get_known_graph_ancestry.
32
    static_tuple,
3228.4.11 by John Arbash Meinel
Deprecations abound.
33
    symbol_versioning,
2490.2.5 by Aaron Bentley
Use GraphWalker.unique_ancestor to determine merge base
34
)
2535.4.27 by Andrew Bennetts
Remove some unused imports.
35
from bzrlib.branch import BranchReferenceFormat
2018.5.174 by Andrew Bennetts
Various nits discovered by pyflakes.
36
from bzrlib.bzrdir import BzrDir, RemoteBzrDirFormat
4634.85.9 by Andrew Bennetts
Add some experimental decorators: @only_raises(..) and @cleanup_method.
37
from bzrlib.decorators import needs_read_lock, needs_write_lock, only_raises
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
38
from bzrlib.errors import (
39
    NoSuchRevision,
40
    SmartProtocolError,
41
    )
2018.5.127 by Andrew Bennetts
Fix most of the lockable_files tests for RemoteBranchLockableFiles.
42
from bzrlib.lockable_files import LockableFiles
4060.1.4 by Robert Collins
Streaming fetch from remote servers.
43
from bzrlib.smart import client, vfs, repository as smart_repo
3297.4.1 by Andrew Bennetts
Merge 'Add Branch.set_last_revision_info smart method'.
44
from bzrlib.revision import ensure_null, NULL_REVISION
3441.5.5 by Andrew Bennetts
Some small tweaks and comments.
45
from bzrlib.trace import mutter, note, warning
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
46
3445.1.5 by John Arbash Meinel
allow passing a 'graph' object into Branch.update_revisions.
47
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
48
class _RpcHelper(object):
49
    """Mixin class that helps with issuing RPCs."""
50
51
    def _call(self, method, *args, **err_context):
52
        try:
53
            return self._client.call(method, *args)
54
        except errors.ErrorFromSmartServer, err:
55
            self._translate_error(err, **err_context)
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
56
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
57
    def _call_expecting_body(self, method, *args, **err_context):
58
        try:
59
            return self._client.call_expecting_body(method, *args)
60
        except errors.ErrorFromSmartServer, err:
61
            self._translate_error(err, **err_context)
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
62
4556.2.1 by Andrew Bennetts
Add Branch.set_tags_bytes RPC, with HPSS call count acceptance test. Also fixes serialisation of LockDir, and uses external_url() in LockDir's repr and contention message.
63
    def _call_with_body_bytes(self, method, args, body_bytes, **err_context):
64
        try:
65
            return self._client.call_with_body_bytes(method, args, body_bytes)
66
        except errors.ErrorFromSmartServer, err:
67
            self._translate_error(err, **err_context)
68
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
69
    def _call_with_body_bytes_expecting_body(self, method, args, body_bytes,
70
                                             **err_context):
71
        try:
72
            return self._client.call_with_body_bytes_expecting_body(
73
                method, args, body_bytes)
74
        except errors.ErrorFromSmartServer, err:
75
            self._translate_error(err, **err_context)
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
76
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
77
78
def response_tuple_to_repo_format(response):
79
    """Convert a response tuple describing a repository format to a format."""
80
    format = RemoteRepositoryFormat()
4118.1.1 by Andrew Bennetts
Fix performance regression (many small round-trips) when pushing to a remote pack, and tidy the tests.
81
    format._rich_root_data = (response[0] == 'yes')
82
    format._supports_tree_reference = (response[1] == 'yes')
83
    format._supports_external_lookups = (response[2] == 'yes')
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
84
    format._network_name = response[3]
85
    return format
86
87
2018.5.25 by Andrew Bennetts
Make sure RemoteBzrDirFormat is always registered (John Arbash Meinel, Robert Collins, Andrew Bennetts).
88
# Note: RemoteBzrDirFormat is in bzrdir.py
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
89
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
90
class RemoteBzrDir(BzrDir, _RpcHelper):
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
91
    """Control directory on a remote server, accessed via bzr:// or similar."""
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
92
4634.47.5 by Andrew Bennetts
Add tests, and fix BzrDirMeta1.has_workingtree which was failing if the local transport is decorated with a ChrootTransport or similar.
93
    def __init__(self, transport, format, _client=None, _force_probe=False):
2018.5.51 by Wouter van Heyst
Test and implement RemoteBranch.last_revision_info()
94
        """Construct a RemoteBzrDir.
95
96
        :param _client: Private parameter for testing. Disables probing and the
97
            use of a real bzrdir.
98
        """
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
99
        BzrDir.__init__(self, transport, format)
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
100
        # this object holds a delegated bzrdir that uses file-level operations
101
        # to talk to the other side
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
102
        self._real_bzrdir = None
4634.47.3 by Andrew Bennetts
Add a BzrDir.open_2.1 verb that indicates if there is a workingtree present. Removes the last 2 VFS calls from incremental pushes.
103
        self._has_working_tree = None
4044.1.3 by Robert Collins
Create a one-shot cache of the result of RemoteBzrDir.create_branch, eliminating 3 round trips for nonstacked branches and 5 for stacked.
104
        # 1-shot cache for the call pattern 'create_branch; open_branch' - see
105
        # create_branch for details.
106
        self._next_open_branch_result = None
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
107
108
        if _client is None:
3313.2.3 by Andrew Bennetts
Deprecate Transport.get_shared_medium.
109
            medium = transport.get_smart_medium()
3431.3.2 by Andrew Bennetts
Remove 'base' from _SmartClient entirely, now that the medium has it.
110
            self._client = client._SmartClient(medium)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
111
        else:
112
            self._client = _client
4634.47.5 by Andrew Bennetts
Add tests, and fix BzrDirMeta1.has_workingtree which was failing if the local transport is decorated with a ChrootTransport or similar.
113
            if not _force_probe:
114
                return
115
116
        self._probe_bzrdir()
117
4964.2.1 by Martin Pool
Add RemoteBzrDir repr
118
    def __repr__(self):
119
        return '%s(%r)' % (self.__class__.__name__, self._client)
120
4634.47.5 by Andrew Bennetts
Add tests, and fix BzrDirMeta1.has_workingtree which was failing if the local transport is decorated with a ChrootTransport or similar.
121
    def _probe_bzrdir(self):
122
        medium = self._client._medium
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
123
        path = self._path_for_remote_call(self._client)
4634.47.3 by Andrew Bennetts
Add a BzrDir.open_2.1 verb that indicates if there is a workingtree present. Removes the last 2 VFS calls from incremental pushes.
124
        if medium._is_remote_before((2, 1)):
125
            self._rpc_open(path)
126
            return
127
        try:
128
            self._rpc_open_2_1(path)
129
            return
130
        except errors.UnknownSmartMethod:
131
            medium._remember_remote_is_before((2, 1))
132
            self._rpc_open(path)
133
134
    def _rpc_open_2_1(self, path):
135
        response = self._call('BzrDir.open_2.1', path)
136
        if response == ('no',):
137
            raise errors.NotBranchError(path=self.root_transport.base)
138
        elif response[0] == 'yes':
139
            if response[1] == 'yes':
140
                self._has_working_tree = True
141
            elif response[1] == 'no':
142
                self._has_working_tree = False
143
            else:
144
                raise errors.UnexpectedSmartServerResponse(response)
145
        else:
146
            raise errors.UnexpectedSmartServerResponse(response)
147
148
    def _rpc_open(self, path):
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
149
        response = self._call('BzrDir.open', path)
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
150
        if response not in [('yes',), ('no',)]:
151
            raise errors.UnexpectedSmartServerResponse(response)
2018.5.26 by Andrew Bennetts
Extract a simple SmartClient class from RemoteTransport, and a hack to avoid VFS operations when probing for a bzrdir over a smart transport.
152
        if response == ('no',):
4634.47.3 by Andrew Bennetts
Add a BzrDir.open_2.1 verb that indicates if there is a workingtree present. Removes the last 2 VFS calls from incremental pushes.
153
            raise errors.NotBranchError(path=self.root_transport.base)
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
154
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
155
    def _ensure_real(self):
156
        """Ensure that there is a _real_bzrdir set.
157
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
158
        Used before calls to self._real_bzrdir.
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
159
        """
160
        if not self._real_bzrdir:
4789.2.1 by Robert Collins
Trigger hpssvfs backtrace dumps with RemoteBzrDir._ensure_real.
161
            if 'hpssvfs' in debug.debug_flags:
162
                import traceback
163
                warning('VFS BzrDir access triggered\n%s',
164
                    ''.join(traceback.format_stack()))
2018.5.169 by Andrew Bennetts
Add a _server_formats flag to BzrDir.open_from_transport and BzrDirFormat.find_format, make RemoteBranch.control_files into a property.
165
            self._real_bzrdir = BzrDir.open_from_transport(
166
                self.root_transport, _server_formats=False)
4070.2.1 by Robert Collins
Add a BzrDirFormat.network_name.
167
            self._format._network_name = \
168
                self._real_bzrdir._format.network_name()
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
169
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
170
    def _translate_error(self, err, **context):
171
        _translate_error(err, bzrdir=self, **context)
172
4044.1.3 by Robert Collins
Create a one-shot cache of the result of RemoteBzrDir.create_branch, eliminating 3 round trips for nonstacked branches and 5 for stacked.
173
    def break_lock(self):
174
        # Prevent aliasing problems in the next_open_branch_result cache.
175
        # See create_branch for rationale.
176
        self._next_open_branch_result = None
177
        return BzrDir.break_lock(self)
178
4070.2.3 by Robert Collins
Get BzrDir.cloning_metadir working.
179
    def _vfs_cloning_metadir(self, require_stacking=False):
3242.3.28 by Aaron Bentley
Use repository acquisition policy for sprouting
180
        self._ensure_real()
4070.2.3 by Robert Collins
Get BzrDir.cloning_metadir working.
181
        return self._real_bzrdir.cloning_metadir(
182
            require_stacking=require_stacking)
183
184
    def cloning_metadir(self, require_stacking=False):
185
        medium = self._client._medium
186
        if medium._is_remote_before((1, 13)):
187
            return self._vfs_cloning_metadir(require_stacking=require_stacking)
188
        verb = 'BzrDir.cloning_metadir'
189
        if require_stacking:
190
            stacking = 'True'
191
        else:
192
            stacking = 'False'
193
        path = self._path_for_remote_call(self._client)
194
        try:
195
            response = self._call(verb, path, stacking)
196
        except errors.UnknownSmartMethod:
4094.1.1 by Andrew Bennetts
Add some medium._remember_is_before((1, 13)) calls.
197
            medium._remember_remote_is_before((1, 13))
4070.2.3 by Robert Collins
Get BzrDir.cloning_metadir working.
198
            return self._vfs_cloning_metadir(require_stacking=require_stacking)
4160.2.9 by Andrew Bennetts
Fix BzrDir.cloning_metadir RPC to fail on branch references, and make
199
        except errors.UnknownErrorFromSmartServer, err:
200
            if err.error_tuple != ('BranchReference',):
201
                raise
202
            # We need to resolve the branch reference to determine the
203
            # cloning_metadir.  This causes unnecessary RPCs to open the
204
            # referenced branch (and bzrdir, etc) but only when the caller
205
            # didn't already resolve the branch reference.
206
            referenced_branch = self.open_branch()
207
            return referenced_branch.bzrdir.cloning_metadir()
4070.2.3 by Robert Collins
Get BzrDir.cloning_metadir working.
208
        if len(response) != 3:
209
            raise errors.UnexpectedSmartServerResponse(response)
4070.7.4 by Andrew Bennetts
Deal with branch references better in BzrDir.cloning_metadir RPC (changes protocol).
210
        control_name, repo_name, branch_info = response
211
        if len(branch_info) != 2:
212
            raise errors.UnexpectedSmartServerResponse(response)
213
        branch_ref, branch_name = branch_info
4075.2.1 by Robert Collins
Audit and make sure we are registering network_name's as factories, not instances.
214
        format = bzrdir.network_format_registry.get(control_name)
4070.2.3 by Robert Collins
Get BzrDir.cloning_metadir working.
215
        if repo_name:
216
            format.repository_format = repository.network_format_registry.get(
217
                repo_name)
4084.2.2 by Robert Collins
Review feedback.
218
        if branch_ref == 'ref':
4070.7.4 by Andrew Bennetts
Deal with branch references better in BzrDir.cloning_metadir RPC (changes protocol).
219
            # XXX: we need possible_transports here to avoid reopening the
4070.7.5 by Andrew Bennetts
Tweak comment.
220
            # connection to the referenced location
4070.7.4 by Andrew Bennetts
Deal with branch references better in BzrDir.cloning_metadir RPC (changes protocol).
221
            ref_bzrdir = BzrDir.open(branch_name)
222
            branch_format = ref_bzrdir.cloning_metadir().get_branch_format()
223
            format.set_branch_format(branch_format)
4084.2.2 by Robert Collins
Review feedback.
224
        elif branch_ref == 'branch':
4070.7.4 by Andrew Bennetts
Deal with branch references better in BzrDir.cloning_metadir RPC (changes protocol).
225
            if branch_name:
226
                format.set_branch_format(
227
                    branch.network_format_registry.get(branch_name))
228
        else:
229
            raise errors.UnexpectedSmartServerResponse(response)
4070.2.3 by Robert Collins
Get BzrDir.cloning_metadir working.
230
        return format
3242.3.28 by Aaron Bentley
Use repository acquisition policy for sprouting
231
1752.2.39 by Martin Pool
[broken] implement upgrade apis on remote bzrdirs
232
    def create_repository(self, shared=False):
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
233
        # as per meta1 formats - just delegate to the format object which may
234
        # be parameterised.
235
        result = self._format.repository_format.initialize(self, shared)
236
        if not isinstance(result, RemoteRepository):
237
            return self.open_repository()
238
        else:
239
            return result
1752.2.50 by Andrew Bennetts
Implement RemoteBzrDir.create_{branch,workingtree}
240
2796.2.19 by Aaron Bentley
Support reconfigure --lightweight-checkout
241
    def destroy_repository(self):
242
        """See BzrDir.destroy_repository"""
243
        self._ensure_real()
244
        self._real_bzrdir.destroy_repository()
245
5051.3.3 by Jelmer Vernooij
Add tests for colo branches.
246
    def create_branch(self, name=None):
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
247
        # as per meta1 formats - just delegate to the format object which may
248
        # be parameterised.
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
249
        real_branch = self._format.get_branch_format().initialize(self,
250
            name=name)
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
251
        if not isinstance(real_branch, RemoteBranch):
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
252
            result = RemoteBranch(self, self.find_repository(), real_branch,
253
                                  name=name)
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
254
        else:
4044.1.3 by Robert Collins
Create a one-shot cache of the result of RemoteBzrDir.create_branch, eliminating 3 round trips for nonstacked branches and 5 for stacked.
255
            result = real_branch
256
        # BzrDir.clone_on_transport() uses the result of create_branch but does
257
        # not return it to its callers; we save approximately 8% of our round
258
        # trips by handing the branch we created back to the first caller to
259
        # open_branch rather than probing anew. Long term we need a API in
260
        # bzrdir that doesn't discard result objects (like result_branch).
261
        # RBC 20090225
262
        self._next_open_branch_result = result
263
        return result
1752.2.50 by Andrew Bennetts
Implement RemoteBzrDir.create_{branch,workingtree}
264
5051.3.3 by Jelmer Vernooij
Add tests for colo branches.
265
    def destroy_branch(self, name=None):
2796.2.16 by Aaron Bentley
Documentation updates from review
266
        """See BzrDir.destroy_branch"""
2796.2.6 by Aaron Bentley
Implement destroy_branch
267
        self._ensure_real()
5051.3.3 by Jelmer Vernooij
Add tests for colo branches.
268
        self._real_bzrdir.destroy_branch(name=name)
4044.1.3 by Robert Collins
Create a one-shot cache of the result of RemoteBzrDir.create_branch, eliminating 3 round trips for nonstacked branches and 5 for stacked.
269
        self._next_open_branch_result = None
2796.2.6 by Aaron Bentley
Implement destroy_branch
270
2955.5.3 by Vincent Ladeuil
Fix second unwanted connection by providing the right branch to create_checkout.
271
    def create_workingtree(self, revision_id=None, from_branch=None):
2018.5.174 by Andrew Bennetts
Various nits discovered by pyflakes.
272
        raise errors.NotLocalUrl(self.transport.base)
1752.2.39 by Martin Pool
[broken] implement upgrade apis on remote bzrdirs
273
2018.5.124 by Robert Collins
Fix test_format_initialize_find_open by delegating Branch formt lookup to the BzrDir, where it should have stayed from the start.
274
    def find_branch_format(self):
275
        """Find the branch 'format' for this bzrdir.
276
277
        This might be a synthetic object for e.g. RemoteBranch and SVN.
278
        """
279
        b = self.open_branch()
280
        return b._format
281
2018.5.132 by Robert Collins
Make all BzrDir implementation tests pass on RemoteBzrDir - fix some things, and remove the incomplete_with_basis tests as cruft.
282
    def get_branch_reference(self):
283
        """See BzrDir.get_branch_reference()."""
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
284
        response = self._get_branch_reference()
285
        if response[0] == 'ref':
286
            return response[1]
287
        else:
288
            return None
289
290
    def _get_branch_reference(self):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
291
        path = self._path_for_remote_call(self._client)
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
292
        medium = self._client._medium
4734.4.8 by Andrew Bennetts
Fix HPSS tests; pass 'location is a repository' message via smart server when possible (adds BzrDir.open_branchV3 verb).
293
        candidate_calls = [
294
            ('BzrDir.open_branchV3', (2, 1)),
295
            ('BzrDir.open_branchV2', (1, 13)),
296
            ('BzrDir.open_branch', None),
297
            ]
298
        for verb, required_version in candidate_calls:
299
            if required_version and medium._is_remote_before(required_version):
300
                continue
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
301
            try:
4734.4.8 by Andrew Bennetts
Fix HPSS tests; pass 'location is a repository' message via smart server when possible (adds BzrDir.open_branchV3 verb).
302
                response = self._call(verb, path)
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
303
            except errors.UnknownSmartMethod:
4734.4.8 by Andrew Bennetts
Fix HPSS tests; pass 'location is a repository' message via smart server when possible (adds BzrDir.open_branchV3 verb).
304
                if required_version is None:
305
                    raise
306
                medium._remember_remote_is_before(required_version)
307
            else:
308
                break
309
        if verb == 'BzrDir.open_branch':
310
            if response[0] != 'ok':
311
                raise errors.UnexpectedSmartServerResponse(response)
312
            if response[1] != '':
313
                return ('ref', response[1])
314
            else:
315
                return ('branch', '')
316
        if response[0] not in ('ref', 'branch'):
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
317
            raise errors.UnexpectedSmartServerResponse(response)
4734.4.8 by Andrew Bennetts
Fix HPSS tests; pass 'location is a repository' message via smart server when possible (adds BzrDir.open_branchV3 verb).
318
        return response
2018.5.132 by Robert Collins
Make all BzrDir implementation tests pass on RemoteBzrDir - fix some things, and remove the incomplete_with_basis tests as cruft.
319
3211.4.1 by Robert Collins
* ``RemoteBzrDir._get_tree_branch`` no longer triggers ``_ensure_real``,
320
    def _get_tree_branch(self):
321
        """See BzrDir._get_tree_branch()."""
322
        return None, self.open_branch()
323
5051.3.4 by Jelmer Vernooij
Support name to BzrDir.open_branch.
324
    def open_branch(self, name=None, unsupported=False,
325
                    ignore_fallbacks=False):
326
        if unsupported:
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
327
            raise NotImplementedError('unsupported flag support not implemented yet.')
4044.1.3 by Robert Collins
Create a one-shot cache of the result of RemoteBzrDir.create_branch, eliminating 3 round trips for nonstacked branches and 5 for stacked.
328
        if self._next_open_branch_result is not None:
329
            # See create_branch for details.
330
            result = self._next_open_branch_result
331
            self._next_open_branch_result = None
332
            return result
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
333
        response = self._get_branch_reference()
334
        if response[0] == 'ref':
2018.5.132 by Robert Collins
Make all BzrDir implementation tests pass on RemoteBzrDir - fix some things, and remove the incomplete_with_basis tests as cruft.
335
            # a branch reference, use the existing BranchReference logic.
336
            format = BranchReferenceFormat()
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
337
            return format.open(self, name=name, _found=True,
338
                location=response[1], ignore_fallbacks=ignore_fallbacks)
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
339
        branch_format_name = response[1]
340
        if not branch_format_name:
341
            branch_format_name = None
342
        format = RemoteBranchFormat(network_name=branch_format_name)
4160.2.6 by Andrew Bennetts
Add ignore_fallbacks flag.
343
        return RemoteBranch(self, self.find_repository(), format=format,
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
344
            setup_stacking=not ignore_fallbacks, name=name)
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
345
4053.1.1 by Robert Collins
New version of the BzrDir.find_repository verb supporting _network_name to support removing more _ensure_real calls.
346
    def _open_repo_v1(self, path):
347
        verb = 'BzrDir.find_repository'
348
        response = self._call(verb, path)
349
        if response[0] != 'ok':
350
            raise errors.UnexpectedSmartServerResponse(response)
351
        # servers that only support the v1 method don't support external
352
        # references either.
353
        self._ensure_real()
354
        repo = self._real_bzrdir.open_repository()
355
        response = response + ('no', repo._format.network_name())
356
        return response, repo
357
358
    def _open_repo_v2(self, path):
359
        verb = 'BzrDir.find_repositoryV2'
360
        response = self._call(verb, path)
361
        if response[0] != 'ok':
362
            raise errors.UnexpectedSmartServerResponse(response)
363
        self._ensure_real()
364
        repo = self._real_bzrdir.open_repository()
365
        response = response + (repo._format.network_name(),)
366
        return response, repo
367
368
    def _open_repo_v3(self, path):
369
        verb = 'BzrDir.find_repositoryV3'
4053.1.2 by Robert Collins
Actually make this branch work.
370
        medium = self._client._medium
371
        if medium._is_remote_before((1, 13)):
372
            raise errors.UnknownSmartMethod(verb)
4094.1.1 by Andrew Bennetts
Add some medium._remember_is_before((1, 13)) calls.
373
        try:
374
            response = self._call(verb, path)
375
        except errors.UnknownSmartMethod:
376
            medium._remember_remote_is_before((1, 13))
377
            raise
4053.1.1 by Robert Collins
New version of the BzrDir.find_repository verb supporting _network_name to support removing more _ensure_real calls.
378
        if response[0] != 'ok':
379
            raise errors.UnexpectedSmartServerResponse(response)
380
        return response, None
381
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
382
    def open_repository(self):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
383
        path = self._path_for_remote_call(self._client)
4053.1.1 by Robert Collins
New version of the BzrDir.find_repository verb supporting _network_name to support removing more _ensure_real calls.
384
        response = None
385
        for probe in [self._open_repo_v3, self._open_repo_v2,
386
            self._open_repo_v1]:
387
            try:
388
                response, real_repo = probe(path)
389
                break
390
            except errors.UnknownSmartMethod:
391
                pass
392
        if response is None:
393
            raise errors.UnknownSmartMethod('BzrDir.find_repository{3,2,}')
3245.4.24 by Andrew Bennetts
Consistently raise errors from the server as ErrorFromSmartServer exceptions.
394
        if response[0] != 'ok':
395
            raise errors.UnexpectedSmartServerResponse(response)
4053.1.1 by Robert Collins
New version of the BzrDir.find_repository verb supporting _network_name to support removing more _ensure_real calls.
396
        if len(response) != 6:
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
397
            raise SmartProtocolError('incorrect response length %s' % (response,))
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
398
        if response[1] == '':
4053.1.1 by Robert Collins
New version of the BzrDir.find_repository verb supporting _network_name to support removing more _ensure_real calls.
399
            # repo is at this dir.
400
            format = response_tuple_to_repo_format(response[2:])
3221.15.10 by Robert Collins
Add test that we can stack on a smart server from Jonathan Lange.
401
            # Used to support creating a real format instance when needed.
402
            format._creating_bzrdir = self
3990.5.1 by Andrew Bennetts
Add network_name() to RepositoryFormat.
403
            remote_repo = RemoteRepository(self, format)
404
            format._creating_repo = remote_repo
4053.1.1 by Robert Collins
New version of the BzrDir.find_repository verb supporting _network_name to support removing more _ensure_real calls.
405
            if real_repo is not None:
406
                remote_repo._set_real_repository(real_repo)
3990.5.1 by Andrew Bennetts
Add network_name() to RepositoryFormat.
407
            return remote_repo
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
408
        else:
409
            raise errors.NoRepositoryPresent(self)
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
410
4634.47.5 by Andrew Bennetts
Add tests, and fix BzrDirMeta1.has_workingtree which was failing if the local transport is decorated with a ChrootTransport or similar.
411
    def has_workingtree(self):
4634.47.3 by Andrew Bennetts
Add a BzrDir.open_2.1 verb that indicates if there is a workingtree present. Removes the last 2 VFS calls from incremental pushes.
412
        if self._has_working_tree is None:
413
            self._ensure_real()
414
            self._has_working_tree = self._real_bzrdir.has_workingtree()
4634.47.5 by Andrew Bennetts
Add tests, and fix BzrDirMeta1.has_workingtree which was failing if the local transport is decorated with a ChrootTransport or similar.
415
        return self._has_working_tree
416
417
    def open_workingtree(self, recommend_upgrade=True):
418
        if self.has_workingtree():
2445.1.1 by Andrew Bennetts
Make RemoteBzrDir.open_workingtree raise NoWorkingTree rather than NotLocalUrl
419
            raise errors.NotLocalUrl(self.root_transport)
420
        else:
421
            raise errors.NoWorkingTree(self.root_transport.base)
1752.2.50 by Andrew Bennetts
Implement RemoteBzrDir.create_{branch,workingtree}
422
2018.5.42 by Robert Collins
Various hopefully improvements, but wsgi is broken, handing over to spiv :).
423
    def _path_for_remote_call(self, client):
2018.6.1 by Robert Collins
Implement a BzrDir.open_branch smart server method for opening a branch without VFS.
424
        """Return the path to be used for this bzrdir in a remote call."""
2018.5.42 by Robert Collins
Various hopefully improvements, but wsgi is broken, handing over to spiv :).
425
        return client.remote_path_from_transport(self.root_transport)
2018.6.1 by Robert Collins
Implement a BzrDir.open_branch smart server method for opening a branch without VFS.
426
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
427
    def get_branch_transport(self, branch_format, name=None):
2018.5.162 by Andrew Bennetts
Add some missing _ensure_real calls, and a missing import.
428
        self._ensure_real()
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
429
        return self._real_bzrdir.get_branch_transport(branch_format, name=name)
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
430
1752.2.43 by Andrew Bennetts
Fix get_{branch,repository,workingtree}_transport.
431
    def get_repository_transport(self, repository_format):
2018.5.162 by Andrew Bennetts
Add some missing _ensure_real calls, and a missing import.
432
        self._ensure_real()
1752.2.43 by Andrew Bennetts
Fix get_{branch,repository,workingtree}_transport.
433
        return self._real_bzrdir.get_repository_transport(repository_format)
434
435
    def get_workingtree_transport(self, workingtree_format):
2018.5.162 by Andrew Bennetts
Add some missing _ensure_real calls, and a missing import.
436
        self._ensure_real()
1752.2.43 by Andrew Bennetts
Fix get_{branch,repository,workingtree}_transport.
437
        return self._real_bzrdir.get_workingtree_transport(workingtree_format)
438
1752.2.39 by Martin Pool
[broken] implement upgrade apis on remote bzrdirs
439
    def can_convert_format(self):
440
        """Upgrading of remote bzrdirs is not supported yet."""
441
        return False
442
443
    def needs_format_conversion(self, format=None):
444
        """Upgrading of remote bzrdirs is not supported yet."""
3943.2.5 by Martin Pool
deprecate needs_format_conversion(format=None)
445
        if format is None:
446
            symbol_versioning.warn(symbol_versioning.deprecated_in((1, 13, 0))
447
                % 'needs_format_conversion(format=None)')
1752.2.39 by Martin Pool
[broken] implement upgrade apis on remote bzrdirs
448
        return False
449
3242.3.37 by Aaron Bentley
Updates from reviews
450
    def clone(self, url, revision_id=None, force_new_repo=False,
451
              preserve_stacking=False):
2018.5.94 by Andrew Bennetts
Various small changes in aid of making tests pass (including deleting one invalid test).
452
        self._ensure_real()
453
        return self._real_bzrdir.clone(url, revision_id=revision_id,
3242.3.37 by Aaron Bentley
Updates from reviews
454
            force_new_repo=force_new_repo, preserve_stacking=preserve_stacking)
2018.5.94 by Andrew Bennetts
Various small changes in aid of making tests pass (including deleting one invalid test).
455
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
456
    def _get_config(self):
457
        return RemoteBzrDirConfig(self)
3567.1.3 by Michael Hudson
fix problem
458
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
459
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
460
class RemoteRepositoryFormat(repository.RepositoryFormat):
2018.5.159 by Andrew Bennetts
Rename SmartClient to _SmartClient.
461
    """Format for repositories accessed over a _SmartClient.
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
462
1752.2.50 by Andrew Bennetts
Implement RemoteBzrDir.create_{branch,workingtree}
463
    Instances of this repository are represented by RemoteRepository
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
464
    instances.
2018.5.118 by Robert Collins
Fix RemoteRepositoryFormat to have appropriate rich_root_data and support_tree_reference.
465
3128.1.3 by Vincent Ladeuil
Since we are there s/parameteris.*/parameteriz&/.
466
    The RemoteRepositoryFormat is parameterized during construction
2018.5.118 by Robert Collins
Fix RemoteRepositoryFormat to have appropriate rich_root_data and support_tree_reference.
467
    to reflect the capabilities of the real, remote format. Specifically
2018.5.138 by Robert Collins
Merge bzr.dev.
468
    the attributes rich_root_data and supports_tree_reference are set
2018.5.118 by Robert Collins
Fix RemoteRepositoryFormat to have appropriate rich_root_data and support_tree_reference.
469
    on a per instance basis, and are not set (and should not be) at
470
    the class level.
3990.5.3 by Robert Collins
Docs and polish on RepositoryFormat.network_name.
471
4032.1.1 by John Arbash Meinel
Merge the removal of all trailing whitespace, and resolve conflicts.
472
    :ivar _custom_format: If set, a specific concrete repository format that
3990.5.3 by Robert Collins
Docs and polish on RepositoryFormat.network_name.
473
        will be used when initializing a repository with this
474
        RemoteRepositoryFormat.
475
    :ivar _creating_repo: If set, the repository object that this
476
        RemoteRepositoryFormat was created for: it can be called into
3990.5.4 by Robert Collins
Review feedback.
477
        to obtain data like the network name.
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
478
    """
479
3543.1.2 by Michael Hudson
the two character fix
480
    _matchingbzrdir = RemoteBzrDirFormat()
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
481
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
482
    def __init__(self):
483
        repository.RepositoryFormat.__init__(self)
484
        self._custom_format = None
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
485
        self._network_name = None
4017.3.3 by Robert Collins
Review feedback - make RemoteRepository.initialize use helpers, and version-lock the new method to not attempt the method on older servers.
486
        self._creating_bzrdir = None
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
487
        self._supports_chks = None
4118.1.1 by Andrew Bennetts
Fix performance regression (many small round-trips) when pushing to a remote pack, and tidy the tests.
488
        self._supports_external_lookups = None
489
        self._supports_tree_reference = None
490
        self._rich_root_data = None
491
4608.1.2 by Martin Pool
Add RemoteRepositoryFormat repr
492
    def __repr__(self):
493
        return "%s(_network_name=%r)" % (self.__class__.__name__,
494
            self._network_name)
495
4118.1.1 by Andrew Bennetts
Fix performance regression (many small round-trips) when pushing to a remote pack, and tidy the tests.
496
    @property
4183.5.1 by Robert Collins
Add RepositoryFormat.fast_deltas to signal fast delta creation.
497
    def fast_deltas(self):
498
        self._ensure_real()
499
        return self._custom_format.fast_deltas
500
501
    @property
4118.1.1 by Andrew Bennetts
Fix performance regression (many small round-trips) when pushing to a remote pack, and tidy the tests.
502
    def rich_root_data(self):
503
        if self._rich_root_data is None:
504
            self._ensure_real()
505
            self._rich_root_data = self._custom_format.rich_root_data
506
        return self._rich_root_data
507
508
    @property
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
509
    def supports_chks(self):
510
        if self._supports_chks is None:
511
            self._ensure_real()
512
            self._supports_chks = self._custom_format.supports_chks
513
        return self._supports_chks
514
515
    @property
4118.1.1 by Andrew Bennetts
Fix performance regression (many small round-trips) when pushing to a remote pack, and tidy the tests.
516
    def supports_external_lookups(self):
517
        if self._supports_external_lookups is None:
518
            self._ensure_real()
519
            self._supports_external_lookups = \
4104.4.2 by Robert Collins
Fix test_source for 1.13 landing.
520
                self._custom_format.supports_external_lookups
4118.1.1 by Andrew Bennetts
Fix performance regression (many small round-trips) when pushing to a remote pack, and tidy the tests.
521
        return self._supports_external_lookups
522
523
    @property
524
    def supports_tree_reference(self):
525
        if self._supports_tree_reference is None:
526
            self._ensure_real()
527
            self._supports_tree_reference = \
528
                self._custom_format.supports_tree_reference
529
        return self._supports_tree_reference
4017.3.3 by Robert Collins
Review feedback - make RemoteRepository.initialize use helpers, and version-lock the new method to not attempt the method on older servers.
530
531
    def _vfs_initialize(self, a_bzrdir, shared):
532
        """Helper for common code in initialize."""
533
        if self._custom_format:
534
            # Custom format requested
535
            result = self._custom_format.initialize(a_bzrdir, shared=shared)
536
        elif self._creating_bzrdir is not None:
537
            # Use the format that the repository we were created to back
538
            # has.
539
            prior_repo = self._creating_bzrdir.open_repository()
540
            prior_repo._ensure_real()
541
            result = prior_repo._real_repository._format.initialize(
542
                a_bzrdir, shared=shared)
543
        else:
544
            # assume that a_bzr is a RemoteBzrDir but the smart server didn't
545
            # support remote initialization.
546
            # We delegate to a real object at this point (as RemoteBzrDir
547
            # delegate to the repository format which would lead to infinite
548
            # recursion if we just called a_bzrdir.create_repository.
549
            a_bzrdir._ensure_real()
550
            result = a_bzrdir._real_bzrdir.create_repository(shared=shared)
551
        if not isinstance(result, RemoteRepository):
552
            return self.open(a_bzrdir)
553
        else:
554
            return result
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
555
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
556
    def initialize(self, a_bzrdir, shared=False):
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
557
        # Being asked to create on a non RemoteBzrDir:
558
        if not isinstance(a_bzrdir, RemoteBzrDir):
4017.3.3 by Robert Collins
Review feedback - make RemoteRepository.initialize use helpers, and version-lock the new method to not attempt the method on older servers.
559
            return self._vfs_initialize(a_bzrdir, shared)
560
        medium = a_bzrdir._client._medium
561
        if medium._is_remote_before((1, 13)):
562
            return self._vfs_initialize(a_bzrdir, shared)
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
563
        # Creating on a remote bzr dir.
564
        # 1) get the network name to use.
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
565
        if self._custom_format:
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
566
            network_name = self._custom_format.network_name()
4294.2.5 by Robert Collins
Reasonable unit test coverage for initialize_on_transport_ex.
567
        elif self._network_name:
568
            network_name = self._network_name
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
569
        else:
570
            # Select the current bzrlib default and ask for that.
571
            reference_bzrdir_format = bzrdir.format_registry.get('default')()
572
            reference_format = reference_bzrdir_format.repository_format
573
            network_name = reference_format.network_name()
574
        # 2) try direct creation via RPC
575
        path = a_bzrdir._path_for_remote_call(a_bzrdir._client)
576
        verb = 'BzrDir.create_repository'
577
        if shared:
578
            shared_str = 'True'
579
        else:
580
            shared_str = 'False'
581
        try:
582
            response = a_bzrdir._call(verb, path, network_name, shared_str)
583
        except errors.UnknownSmartMethod:
4017.3.3 by Robert Collins
Review feedback - make RemoteRepository.initialize use helpers, and version-lock the new method to not attempt the method on older servers.
584
            # Fallback - use vfs methods
4094.1.1 by Andrew Bennetts
Add some medium._remember_is_before((1, 13)) calls.
585
            medium._remember_remote_is_before((1, 13))
4017.3.3 by Robert Collins
Review feedback - make RemoteRepository.initialize use helpers, and version-lock the new method to not attempt the method on older servers.
586
            return self._vfs_initialize(a_bzrdir, shared)
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
587
        else:
588
            # Turn the response into a RemoteRepository object.
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
589
            format = response_tuple_to_repo_format(response[1:])
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
590
            # Used to support creating a real format instance when needed.
591
            format._creating_bzrdir = a_bzrdir
592
            remote_repo = RemoteRepository(a_bzrdir, format)
593
            format._creating_repo = remote_repo
594
            return remote_repo
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
595
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
596
    def open(self, a_bzrdir):
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
597
        if not isinstance(a_bzrdir, RemoteBzrDir):
598
            raise AssertionError('%r is not a RemoteBzrDir' % (a_bzrdir,))
1752.2.72 by Andrew Bennetts
Make Remote* classes in remote.py more consistent and remove some dead code.
599
        return a_bzrdir.open_repository()
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
600
4053.1.4 by Robert Collins
Move the fetch control attributes from Repository to RepositoryFormat.
601
    def _ensure_real(self):
602
        if self._custom_format is None:
603
            self._custom_format = repository.network_format_registry.get(
604
                self._network_name)
605
606
    @property
607
    def _fetch_order(self):
608
        self._ensure_real()
609
        return self._custom_format._fetch_order
610
611
    @property
612
    def _fetch_uses_deltas(self):
613
        self._ensure_real()
614
        return self._custom_format._fetch_uses_deltas
615
616
    @property
617
    def _fetch_reconcile(self):
618
        self._ensure_real()
619
        return self._custom_format._fetch_reconcile
620
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
621
    def get_format_description(self):
4792.1.1 by Andrew Bennetts
Show real branch/repo format description in 'info -v' over HPSS.
622
        self._ensure_real()
623
        return 'Remote: ' + self._custom_format.get_format_description()
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
624
625
    def __eq__(self, other):
4088.3.1 by Benjamin Peterson
compare types with 'is' not ==
626
        return self.__class__ is other.__class__
1752.2.87 by Andrew Bennetts
Make tests pass.
627
3990.5.1 by Andrew Bennetts
Add network_name() to RepositoryFormat.
628
    def network_name(self):
4017.3.2 by Robert Collins
Reduce the number of round trips required to create a repository over the network.
629
        if self._network_name:
630
            return self._network_name
3990.5.1 by Andrew Bennetts
Add network_name() to RepositoryFormat.
631
        self._creating_repo._ensure_real()
632
        return self._creating_repo._real_repository._format.network_name()
633
4022.1.1 by Robert Collins
Refactoring of fetch to have a sender and sink component enabling splitting the logic over a network stream. (Robert Collins, Andrew Bennetts)
634
    @property
4431.3.7 by Jonathan Lange
Cherrypick bzr.dev 4470, resolving conflicts.
635
    def pack_compresses(self):
636
        self._ensure_real()
637
        return self._custom_format.pack_compresses
638
639
    @property
4022.1.1 by Robert Collins
Refactoring of fetch to have a sender and sink component enabling splitting the logic over a network stream. (Robert Collins, Andrew Bennetts)
640
    def _serializer(self):
4053.1.4 by Robert Collins
Move the fetch control attributes from Repository to RepositoryFormat.
641
        self._ensure_real()
642
        return self._custom_format._serializer
4022.1.1 by Robert Collins
Refactoring of fetch to have a sender and sink component enabling splitting the logic over a network stream. (Robert Collins, Andrew Bennetts)
643
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
644
5158.6.4 by Martin Pool
Repository implements ControlComponent too
645
class RemoteRepository(_RpcHelper, lock._RelockDebugMixin,
646
    bzrdir.ControlComponent):
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
647
    """Repository accessed over rpc.
648
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
649
    For the moment most operations are performed using local transport-backed
650
    Repository objects.
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
651
    """
652
2018.5.118 by Robert Collins
Fix RemoteRepositoryFormat to have appropriate rich_root_data and support_tree_reference.
653
    def __init__(self, remote_bzrdir, format, real_repository=None, _client=None):
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
654
        """Create a RemoteRepository instance.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
655
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
656
        :param remote_bzrdir: The bzrdir hosting this repository.
2018.5.118 by Robert Collins
Fix RemoteRepositoryFormat to have appropriate rich_root_data and support_tree_reference.
657
        :param format: The RemoteFormat object to use.
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
658
        :param real_repository: If not None, a local implementation of the
659
            repository logic for the repository, usually accessing the data
660
            via the VFS.
2018.5.57 by Robert Collins
Implement RemoteRepository.is_shared (Robert Collins, Vincent Ladeuil).
661
        :param _client: Private testing parameter - override the smart client
662
            to be used by the repository.
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
663
        """
664
        if real_repository:
2018.5.36 by Andrew Bennetts
Fix typo, and clean up some ununsed import warnings from pyflakes at the same time.
665
            self._real_repository = real_repository
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
666
        else:
667
            self._real_repository = None
1752.2.50 by Andrew Bennetts
Implement RemoteBzrDir.create_{branch,workingtree}
668
        self.bzrdir = remote_bzrdir
2018.5.57 by Robert Collins
Implement RemoteRepository.is_shared (Robert Collins, Vincent Ladeuil).
669
        if _client is None:
3313.2.1 by Andrew Bennetts
Change _SmartClient's API to accept a medium and a base, rather than a _SharedConnection.
670
            self._client = remote_bzrdir._client
2018.5.57 by Robert Collins
Implement RemoteRepository.is_shared (Robert Collins, Vincent Ladeuil).
671
        else:
672
            self._client = _client
2018.5.118 by Robert Collins
Fix RemoteRepositoryFormat to have appropriate rich_root_data and support_tree_reference.
673
        self._format = format
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
674
        self._lock_mode = None
675
        self._lock_token = None
676
        self._lock_count = 0
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
677
        self._leave_lock = False
4307.2.4 by Robert Collins
Enable caching of negative revision lookups in RemoteRepository write locks when no _real_repository has been constructed.
678
        # Cache of revision parents; misses are cached during read locks, and
679
        # write locks when no _real_repository has been set.
3835.1.12 by Aaron Bentley
Unify CachingExtraParentsProvider and CachingParentsProvider.
680
        self._unstacked_provider = graph.CachingParentsProvider(
3896.1.1 by Andrew Bennetts
Remove broken debugging cruft, and some unused imports.
681
            get_parent_map=self._get_parent_map_rpc)
3835.1.12 by Aaron Bentley
Unify CachingExtraParentsProvider and CachingParentsProvider.
682
        self._unstacked_provider.disable_cache()
2951.1.10 by Robert Collins
Peer review feedback with Ian.
683
        # For tests:
684
        # These depend on the actual remote format, so force them off for
685
        # maximum compatibility. XXX: In future these should depend on the
686
        # remote repository instance, but this is irrelevant until we perform
687
        # reconcile via an RPC call.
2951.1.5 by Robert Collins
Some work towards including the correct changes for TREE_ROOT in check parameterised tests.
688
        self._reconcile_does_inventory_gc = False
689
        self._reconcile_fixes_text_parents = False
2951.1.3 by Robert Collins
Partial support for native reconcile with packs.
690
        self._reconcile_backsup_inventory = False
2592.4.5 by Martin Pool
Add Repository.base on all repositories.
691
        self.base = self.bzrdir.transport.base
3221.12.1 by Robert Collins
Backport development1 format (stackable packs) to before-shallow-branches.
692
        # Additional places to query for data.
693
        self._fallback_repositories = []
2592.4.5 by Martin Pool
Add Repository.base on all repositories.
694
5158.6.4 by Martin Pool
Repository implements ControlComponent too
695
    @property
696
    def user_transport(self):
697
        return self.bzrdir.user_transport
698
699
    @property
700
    def control_transport(self):
701
        # XXX: Normally you shouldn't directly get at the remote repository
702
        # transport, but I'm not sure it's worth making this method
703
        # optional -- mbp 2010-04-21
704
        return self.bzrdir.get_repository_transport(None)
705
        
2592.4.5 by Martin Pool
Add Repository.base on all repositories.
706
    def __str__(self):
707
        return "%s(%s)" % (self.__class__.__name__, self.base)
708
709
    __repr__ = __str__
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
710
3825.4.1 by Andrew Bennetts
Add suppress_errors to abort_write_group.
711
    def abort_write_group(self, suppress_errors=False):
2617.6.7 by Robert Collins
More review feedback.
712
        """Complete a write group on the decorated repository.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
713
4031.3.3 by Matt Nordhoff
Review tweaks from Ben Finney
714
        Smart methods perform operations in a single step so this API
2617.6.6 by Robert Collins
Some review feedback.
715
        is not really applicable except as a compatibility thunk
2617.6.2 by Robert Collins
Add abort_write_group and wire write_groups into fetch and commit.
716
        for older plugins that don't use e.g. the CommitBuilder
717
        facility.
3825.4.6 by Andrew Bennetts
Document the suppress_errors flag in the docstring.
718
719
        :param suppress_errors: see Repository.abort_write_group.
2617.6.2 by Robert Collins
Add abort_write_group and wire write_groups into fetch and commit.
720
        """
721
        self._ensure_real()
3825.4.1 by Andrew Bennetts
Add suppress_errors to abort_write_group.
722
        return self._real_repository.abort_write_group(
723
            suppress_errors=suppress_errors)
2617.6.2 by Robert Collins
Add abort_write_group and wire write_groups into fetch and commit.
724
4253.1.1 by Robert Collins
Add chk_bytes property to RemoteRepository
725
    @property
726
    def chk_bytes(self):
727
        """Decorate the real repository for now.
728
729
        In the long term a full blown network facility is needed to avoid
730
        creating a real repository object locally.
731
        """
732
        self._ensure_real()
733
        return self._real_repository.chk_bytes
734
2617.6.2 by Robert Collins
Add abort_write_group and wire write_groups into fetch and commit.
735
    def commit_write_group(self):
2617.6.7 by Robert Collins
More review feedback.
736
        """Complete a write group on the decorated repository.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
737
4031.3.3 by Matt Nordhoff
Review tweaks from Ben Finney
738
        Smart methods perform operations in a single step so this API
2617.6.6 by Robert Collins
Some review feedback.
739
        is not really applicable except as a compatibility thunk
2617.6.2 by Robert Collins
Add abort_write_group and wire write_groups into fetch and commit.
740
        for older plugins that don't use e.g. the CommitBuilder
741
        facility.
742
        """
743
        self._ensure_real()
744
        return self._real_repository.commit_write_group()
2617.6.1 by Robert Collins
* New method on Repository - ``start_write_group``, ``end_write_group``
745
4002.1.1 by Andrew Bennetts
Implement suspend_write_group/resume_write_group.
746
    def resume_write_group(self, tokens):
747
        self._ensure_real()
748
        return self._real_repository.resume_write_group(tokens)
749
750
    def suspend_write_group(self):
751
        self._ensure_real()
752
        return self._real_repository.suspend_write_group()
753
4343.3.29 by John Arbash Meinel
Add 'check_for_missing_texts' flag to get_missing_parent_inv..
754
    def get_missing_parent_inventories(self, check_for_missing_texts=True):
4257.4.6 by Andrew Bennetts
Make get_missing_parent_inventories work for all repo formats (it's a no-op for unstackable formats).
755
        self._ensure_real()
4343.3.29 by John Arbash Meinel
Add 'check_for_missing_texts' flag to get_missing_parent_inv..
756
        return self._real_repository.get_missing_parent_inventories(
757
            check_for_missing_texts=check_for_missing_texts)
4257.4.6 by Andrew Bennetts
Make get_missing_parent_inventories work for all repo formats (it's a no-op for unstackable formats).
758
4419.2.9 by Andrew Bennetts
Add per_repository_reference test for get_rev_id_for_revno, fix the bugs it revealed.
759
    def _get_rev_id_for_revno_vfs(self, revno, known_pair):
760
        self._ensure_real()
761
        return self._real_repository.get_rev_id_for_revno(
762
            revno, known_pair)
763
4419.2.5 by Andrew Bennetts
Add Repository.get_rev_id_for_revno, and use it both as the _ensure_real fallback and as the server-side implementation.
764
    def get_rev_id_for_revno(self, revno, known_pair):
765
        """See Repository.get_rev_id_for_revno."""
766
        path = self.bzrdir._path_for_remote_call(self._client)
767
        try:
4476.3.33 by Andrew Bennetts
Revert some accidental s/17/18/ in remote.py.
768
            if self._client._medium._is_remote_before((1, 17)):
4419.2.9 by Andrew Bennetts
Add per_repository_reference test for get_rev_id_for_revno, fix the bugs it revealed.
769
                return self._get_rev_id_for_revno_vfs(revno, known_pair)
4419.2.5 by Andrew Bennetts
Add Repository.get_rev_id_for_revno, and use it both as the _ensure_real fallback and as the server-side implementation.
770
            response = self._call(
771
                'Repository.get_rev_id_for_revno', path, revno, known_pair)
772
        except errors.UnknownSmartMethod:
4476.3.33 by Andrew Bennetts
Revert some accidental s/17/18/ in remote.py.
773
            self._client._medium._remember_remote_is_before((1, 17))
4419.2.9 by Andrew Bennetts
Add per_repository_reference test for get_rev_id_for_revno, fix the bugs it revealed.
774
            return self._get_rev_id_for_revno_vfs(revno, known_pair)
4419.2.5 by Andrew Bennetts
Add Repository.get_rev_id_for_revno, and use it both as the _ensure_real fallback and as the server-side implementation.
775
        if response[0] == 'ok':
776
            return True, response[1]
777
        elif response[0] == 'history-incomplete':
4419.2.8 by Andrew Bennetts
Add unit test for RemoteRepository.get_rev_id_for_revno using fallbacks if it gets a history-incomplete response.
778
            known_pair = response[1:3]
779
            for fallback in self._fallback_repositories:
780
                found, result = fallback.get_rev_id_for_revno(revno, known_pair)
781
                if found:
782
                    return True, result
783
                else:
784
                    known_pair = result
785
            # Not found in any fallbacks
786
            return False, known_pair
4419.2.5 by Andrew Bennetts
Add Repository.get_rev_id_for_revno, and use it both as the _ensure_real fallback and as the server-side implementation.
787
        else:
788
            raise errors.UnexpectedSmartServerResponse(response)
789
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
790
    def _ensure_real(self):
791
        """Ensure that there is a _real_repository set.
792
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
793
        Used before calls to self._real_repository.
4145.1.1 by Robert Collins
Explicitly prevent fetching while the target repository is in a write group.
794
795
        Note that _ensure_real causes many roundtrips to the server which are
796
        not desirable, and prevents the use of smart one-roundtrip RPC's to
797
        perform complex operations (such as accessing parent data, streaming
798
        revisions etc). Adding calls to _ensure_real should only be done when
799
        bringing up new functionality, adding fallbacks for smart methods that
800
        require a fallback path, and never to replace an existing smart method
801
        invocation. If in doubt chat to the bzr network team.
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
802
        """
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
803
        if self._real_repository is None:
4509.2.2 by Martin Pool
Use only -Dhpssvfs for tracebacks, and document -Dhpssdetail
804
            if 'hpssvfs' in debug.debug_flags:
4347.1.1 by Robert Collins
Show a traceback when VFS operations are started on a smart server hosted repository.
805
                import traceback
806
                warning('VFS Repository access triggered\n%s',
807
                    ''.join(traceback.format_stack()))
4307.2.4 by Robert Collins
Enable caching of negative revision lookups in RemoteRepository write locks when no _real_repository has been constructed.
808
            self._unstacked_provider.missing_keys.clear()
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
809
            self.bzrdir._ensure_real()
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
810
            self._set_real_repository(
811
                self.bzrdir._real_bzrdir.open_repository())
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
812
3533.3.1 by Andrew Bennetts
Remove duplication of error translation in bzrlib/remote.py.
813
    def _translate_error(self, err, **context):
814
        self.bzrdir._translate_error(err, repository=self, **context)
815
2988.1.2 by Robert Collins
New Repository API find_text_key_references for use by reconcile and check.
816
    def find_text_key_references(self):
817
        """Find the text key references within the repository.
818
819
        :return: a dictionary mapping (file_id, revision_id) tuples to altered file-ids to an iterable of
820
        revision_ids. Each altered file-ids has the exact revision_ids that
821
        altered it listed explicitly.
822
        :return: A dictionary mapping text keys ((fileid, revision_id) tuples)
823
            to whether they were referred to by the inventory of the
824
            revision_id that they contain. The inventory texts from all present
825
            revision ids are assessed to generate this report.
826
        """
827
        self._ensure_real()
828
        return self._real_repository.find_text_key_references()
829
2988.1.3 by Robert Collins
Add a new repositoy method _generate_text_key_index for use by reconcile/check.
830
    def _generate_text_key_index(self):
831
        """Generate a new text key index for the repository.
832
833
        This is an expensive function that will take considerable time to run.
834
835
        :return: A dict mapping (file_id, revision_id) tuples to a list of
836
            parents, also (file_id, revision_id) tuples.
837
        """
838
        self._ensure_real()
839
        return self._real_repository._generate_text_key_index()
840
3287.6.4 by Robert Collins
Fix up deprecation warnings for get_revision_graph.
841
    def _get_revision_graph(self, revision_id):
842
        """Private method for using with old (< 1.2) servers to fallback."""
2018.5.67 by Wouter van Heyst
Implement RemoteRepository.get_revision_graph (Wouter van Heyst, Robert Collins)
843
        if revision_id is None:
844
            revision_id = ''
2948.3.1 by John Arbash Meinel
Fix bug #158333, make sure that Repository.fetch(self) is properly a no-op for all Repository implementations.
845
        elif revision.is_null(revision_id):
2018.5.67 by Wouter van Heyst
Implement RemoteRepository.get_revision_graph (Wouter van Heyst, Robert Collins)
846
            return {}
847
848
        path = self.bzrdir._path_for_remote_call(self._client)
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
849
        response = self._call_expecting_body(
850
            'Repository.get_revision_graph', path, revision_id)
3245.4.58 by Andrew Bennetts
Unpack call_expecting_body's return value into variables, to avoid lots of ugly subscripting.
851
        response_tuple, response_handler = response
852
        if response_tuple[0] != 'ok':
853
            raise errors.UnexpectedSmartServerResponse(response_tuple)
854
        coded = response_handler.read_body_bytes()
3245.4.24 by Andrew Bennetts
Consistently raise errors from the server as ErrorFromSmartServer exceptions.
855
        if coded == '':
856
            # no revisions in this repository!
857
            return {}
858
        lines = coded.split('\n')
859
        revision_graph = {}
860
        for line in lines:
861
            d = tuple(line.split())
862
            revision_graph[d[0]] = d[1:]
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
863
3245.4.24 by Andrew Bennetts
Consistently raise errors from the server as ErrorFromSmartServer exceptions.
864
        return revision_graph
2018.5.67 by Wouter van Heyst
Implement RemoteRepository.get_revision_graph (Wouter van Heyst, Robert Collins)
865
4022.1.1 by Robert Collins
Refactoring of fetch to have a sender and sink component enabling splitting the logic over a network stream. (Robert Collins, Andrew Bennetts)
866
    def _get_sink(self):
867
        """See Repository._get_sink()."""
4022.1.6 by Robert Collins
Cherrypick and polish the RemoteSink for streaming push.
868
        return RemoteStreamSink(self)
4022.1.1 by Robert Collins
Refactoring of fetch to have a sender and sink component enabling splitting the logic over a network stream. (Robert Collins, Andrew Bennetts)
869
4060.1.3 by Robert Collins
Implement the separate source component for fetch - repository.StreamSource.
870
    def _get_source(self, to_format):
871
        """Return a source for streaming from this repository."""
872
        return RemoteStreamSource(self, to_format)
873
4307.2.3 by Robert Collins
Change RemoteRepository.has_revision to use get_parent_map to leverage the caching.
874
    @needs_read_lock
2018.5.40 by Robert Collins
Implement a remote Repository.has_revision method.
875
    def has_revision(self, revision_id):
4307.2.3 by Robert Collins
Change RemoteRepository.has_revision to use get_parent_map to leverage the caching.
876
        """True if this repository has a copy of the revision."""
877
        # Copy of bzrlib.repository.Repository.has_revision
878
        return revision_id in self.has_revisions((revision_id,))
2018.5.40 by Robert Collins
Implement a remote Repository.has_revision method.
879
4307.2.3 by Robert Collins
Change RemoteRepository.has_revision to use get_parent_map to leverage the caching.
880
    @needs_read_lock
3172.3.1 by Robert Collins
Repository has a new method ``has_revisions`` which signals the presence
881
    def has_revisions(self, revision_ids):
4307.2.3 by Robert Collins
Change RemoteRepository.has_revision to use get_parent_map to leverage the caching.
882
        """Probe to find out the presence of multiple revisions.
883
884
        :param revision_ids: An iterable of revision_ids.
885
        :return: A set of the revision_ids that were present.
886
        """
887
        # Copy of bzrlib.repository.Repository.has_revisions
888
        parent_map = self.get_parent_map(revision_ids)
889
        result = set(parent_map)
890
        if _mod_revision.NULL_REVISION in revision_ids:
891
            result.add(_mod_revision.NULL_REVISION)
3172.3.1 by Robert Collins
Repository has a new method ``has_revisions`` which signals the presence
892
        return result
893
4509.3.37 by Martin Pool
Remove RepositoryBase; make _has_same_location private
894
    def _has_same_fallbacks(self, other_repo):
895
        """Returns true if the repositories have the same fallbacks."""
896
        # XXX: copied from Repository; it should be unified into a base class
897
        # <https://bugs.edge.launchpad.net/bzr/+bug/401622>
898
        my_fb = self._fallback_repositories
899
        other_fb = other_repo._fallback_repositories
900
        if len(my_fb) != len(other_fb):
901
            return False
902
        for f, g in zip(my_fb, other_fb):
903
            if not f.has_same_location(g):
904
                return False
905
        return True
906
2617.6.9 by Robert Collins
Merge bzr.dev.
907
    def has_same_location(self, other):
4509.3.23 by Martin Pool
Comment on has_same_location variation
908
        # TODO: Move to RepositoryBase and unify with the regular Repository
909
        # one; unfortunately the tests rely on slightly different behaviour at
910
        # present -- mbp 20090710
4088.3.1 by Benjamin Peterson
compare types with 'is' not ==
911
        return (self.__class__ is other.__class__ and
2592.3.162 by Robert Collins
Remove some arbitrary differences from bzr.dev.
912
                self.bzrdir.transport.base == other.bzrdir.transport.base)
3835.1.1 by Aaron Bentley
Stack get_parent_map on fallback repos
913
2490.2.21 by Aaron Bentley
Rename graph to deprecated_graph
914
    def get_graph(self, other_repository=None):
915
        """Return the graph for this repository format"""
3835.1.17 by Aaron Bentley
Fix stacking bug
916
        parents_provider = self._make_parents_provider(other_repository)
3441.5.24 by Andrew Bennetts
Remove RemoteGraph experiment.
917
        return graph.Graph(parents_provider)
2490.2.5 by Aaron Bentley
Use GraphWalker.unique_ancestor to determine merge base
918
4913.4.2 by Jelmer Vernooij
Add Repository.get_known_graph_ancestry.
919
    @needs_read_lock
920
    def get_known_graph_ancestry(self, revision_ids):
921
        """Return the known graph for a set of revision ids and their ancestors.
922
        """
923
        st = static_tuple.StaticTuple
924
        revision_keys = [st(r_id).intern() for r_id in revision_ids]
925
        known_graph = self.revisions.get_known_graph_ancestry(revision_keys)
926
        return graph.GraphThunkIdsToKeys(known_graph)
927
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
928
    def gather_stats(self, revid=None, committers=None):
2018.5.62 by Robert Collins
Stub out RemoteRepository.gather_stats while its implemented in parallel.
929
        """See Repository.gather_stats()."""
2018.10.3 by v.ladeuil+lp at free
more tests for gather_stats
930
        path = self.bzrdir._path_for_remote_call(self._client)
2948.3.4 by John Arbash Meinel
Repository.gather_stats() validly can get None for the revid.
931
        # revid can be None to indicate no revisions, not just NULL_REVISION
932
        if revid is None or revision.is_null(revid):
2018.10.3 by v.ladeuil+lp at free
more tests for gather_stats
933
            fmt_revid = ''
934
        else:
2018.5.83 by Andrew Bennetts
Fix some test failures caused by the switch from unicode to UTF-8-encoded strs for revision IDs.
935
            fmt_revid = revid
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
936
        if committers is None or not committers:
2018.10.3 by v.ladeuil+lp at free
more tests for gather_stats
937
            fmt_committers = 'no'
938
        else:
939
            fmt_committers = 'yes'
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
940
        response_tuple, response_handler = self._call_expecting_body(
2018.5.153 by Andrew Bennetts
Rename call2 to call_expecting_body, and other small changes prompted by review.
941
            'Repository.gather_stats', path, fmt_revid, fmt_committers)
3245.4.58 by Andrew Bennetts
Unpack call_expecting_body's return value into variables, to avoid lots of ugly subscripting.
942
        if response_tuple[0] != 'ok':
943
            raise errors.UnexpectedSmartServerResponse(response_tuple)
2018.10.3 by v.ladeuil+lp at free
more tests for gather_stats
944
3245.4.58 by Andrew Bennetts
Unpack call_expecting_body's return value into variables, to avoid lots of ugly subscripting.
945
        body = response_handler.read_body_bytes()
2018.10.3 by v.ladeuil+lp at free
more tests for gather_stats
946
        result = {}
947
        for line in body.split('\n'):
948
            if not line:
949
                continue
950
            key, val_text = line.split(':')
951
            if key in ('revisions', 'size', 'committers'):
952
                result[key] = int(val_text)
953
            elif key in ('firstrev', 'latestrev'):
954
                values = val_text.split(' ')[1:]
955
                result[key] = (float(values[0]), long(values[1]))
956
957
        return result
2018.5.62 by Robert Collins
Stub out RemoteRepository.gather_stats while its implemented in parallel.
958
3140.1.2 by Aaron Bentley
Add ability to find branches inside repositories
959
    def find_branches(self, using=False):
960
        """See Repository.find_branches()."""
961
        # should be an API call to the server.
962
        self._ensure_real()
963
        return self._real_repository.find_branches(using=using)
964
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
965
    def get_physical_lock_status(self):
966
        """See Repository.get_physical_lock_status()."""
3015.2.10 by Robert Collins
Fix regression due to other pack related fixes in tests with packs not-default.
967
        # should be an API call to the server.
968
        self._ensure_real()
969
        return self._real_repository.get_physical_lock_status()
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
970
2617.6.1 by Robert Collins
* New method on Repository - ``start_write_group``, ``end_write_group``
971
    def is_in_write_group(self):
972
        """Return True if there is an open write group.
973
974
        write groups are only applicable locally for the smart server..
975
        """
976
        if self._real_repository:
977
            return self._real_repository.is_in_write_group()
978
979
    def is_locked(self):
980
        return self._lock_count >= 1
981
2018.5.57 by Robert Collins
Implement RemoteRepository.is_shared (Robert Collins, Vincent Ladeuil).
982
    def is_shared(self):
983
        """See Repository.is_shared()."""
984
        path = self.bzrdir._path_for_remote_call(self._client)
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
985
        response = self._call('Repository.is_shared', path)
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
986
        if response[0] not in ('yes', 'no'):
987
            raise SmartProtocolError('unexpected response code %s' % (response,))
2018.5.57 by Robert Collins
Implement RemoteRepository.is_shared (Robert Collins, Vincent Ladeuil).
988
        return response[0] == 'yes'
989
2904.1.1 by Robert Collins
* New method ``bzrlib.repository.Repository.is_write_locked`` useful for
990
    def is_write_locked(self):
991
        return self._lock_mode == 'w'
992
4840.2.7 by Vincent Ladeuil
Move the _warn_if_deprecated call from repo.__init__ to
993
    def _warn_if_deprecated(self, branch=None):
994
        # If we have a real repository, the check will be done there, if we
995
        # don't the check will be done remotely.
996
        pass
997
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
998
    def lock_read(self):
999
        # wrong eventually - want a local lock cache context
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1000
        if not self._lock_mode:
4731.1.2 by Andrew Bennetts
Refactor to reduce duplication.
1001
            self._note_lock('r')
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1002
            self._lock_mode = 'r'
1003
            self._lock_count = 1
4190.1.1 by Robert Collins
Negatively cache misses during read-locks in RemoteRepository.
1004
            self._unstacked_provider.enable_cache(cache_misses=True)
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1005
            if self._real_repository is not None:
1006
                self._real_repository.lock_read()
4379.2.1 by John Arbash Meinel
Change the fallback repository code to only lock/unlock on transition.
1007
            for repo in self._fallback_repositories:
1008
                repo.lock_read()
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1009
        else:
1010
            self._lock_count += 1
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1011
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1012
    def _remote_lock_write(self, token):
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1013
        path = self.bzrdir._path_for_remote_call(self._client)
1014
        if token is None:
1015
            token = ''
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
1016
        err_context = {'token': token}
1017
        response = self._call('Repository.lock_write', path, token,
1018
                              **err_context)
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1019
        if response[0] == 'ok':
1020
            ok, token = response
1021
            return token
1022
        else:
2555.1.1 by Martin Pool
Remove use of 'assert False' to raise an exception unconditionally
1023
            raise errors.UnexpectedSmartServerResponse(response)
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
1024
3692.1.1 by Andrew Bennetts
Make RemoteBranch.lock_write lock the repository too.
1025
    def lock_write(self, token=None, _skip_rpc=False):
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1026
        if not self._lock_mode:
4731.1.2 by Andrew Bennetts
Refactor to reduce duplication.
1027
            self._note_lock('w')
3692.1.1 by Andrew Bennetts
Make RemoteBranch.lock_write lock the repository too.
1028
            if _skip_rpc:
1029
                if self._lock_token is not None:
1030
                    if token != self._lock_token:
3695.1.1 by Andrew Bennetts
Remove some unused imports and fix a couple of trivially broken raise statements.
1031
                        raise errors.TokenMismatch(token, self._lock_token)
3692.1.1 by Andrew Bennetts
Make RemoteBranch.lock_write lock the repository too.
1032
                self._lock_token = token
1033
            else:
1034
                self._lock_token = self._remote_lock_write(token)
3015.2.9 by Robert Collins
Handle repositories that do not allow remote locking, like pack repositories, in the client side remote server proxy objects.
1035
            # if self._lock_token is None, then this is something like packs or
1036
            # svn where we don't get to lock the repo, or a weave style repository
1037
            # where we cannot lock it over the wire and attempts to do so will
1038
            # fail.
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1039
            if self._real_repository is not None:
1040
                self._real_repository.lock_write(token=self._lock_token)
1041
            if token is not None:
1042
                self._leave_lock = True
1043
            else:
1044
                self._leave_lock = False
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1045
            self._lock_mode = 'w'
1046
            self._lock_count = 1
4307.2.4 by Robert Collins
Enable caching of negative revision lookups in RemoteRepository write locks when no _real_repository has been constructed.
1047
            cache_misses = self._real_repository is None
1048
            self._unstacked_provider.enable_cache(cache_misses=cache_misses)
4379.2.1 by John Arbash Meinel
Change the fallback repository code to only lock/unlock on transition.
1049
            for repo in self._fallback_repositories:
1050
                # Writes don't affect fallback repos
1051
                repo.lock_read()
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1052
        elif self._lock_mode == 'r':
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1053
            raise errors.ReadOnlyError(self)
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1054
        else:
1055
            self._lock_count += 1
3015.2.9 by Robert Collins
Handle repositories that do not allow remote locking, like pack repositories, in the client side remote server proxy objects.
1056
        return self._lock_token or None
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1057
1058
    def leave_lock_in_place(self):
3015.2.9 by Robert Collins
Handle repositories that do not allow remote locking, like pack repositories, in the client side remote server proxy objects.
1059
        if not self._lock_token:
1060
            raise NotImplementedError(self.leave_lock_in_place)
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1061
        self._leave_lock = True
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1062
1063
    def dont_leave_lock_in_place(self):
3015.2.9 by Robert Collins
Handle repositories that do not allow remote locking, like pack repositories, in the client side remote server proxy objects.
1064
        if not self._lock_token:
3015.2.15 by Robert Collins
Review feedback.
1065
            raise NotImplementedError(self.dont_leave_lock_in_place)
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1066
        self._leave_lock = False
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1067
1068
    def _set_real_repository(self, repository):
1069
        """Set the _real_repository for this repository.
1070
1071
        :param repository: The repository to fallback to for non-hpss
1072
            implemented operations.
1073
        """
4053.1.5 by Robert Collins
Review feedback on RemoteRepository._set_real_revision.
1074
        if self._real_repository is not None:
1075
            # Replacing an already set real repository.
1076
            # We cannot do this [currently] if the repository is locked -
1077
            # synchronised state might be lost.
1078
            if self.is_locked():
1079
                raise AssertionError('_real_repository is already set')
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
1080
        if isinstance(repository, RemoteRepository):
1081
            raise AssertionError()
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1082
        self._real_repository = repository
4226.2.5 by Robert Collins
Fix handling of fallback repositories some more.
1083
        # three code paths happen here:
1084
        # 1) old servers, RemoteBranch.open() calls _ensure_real before setting
1085
        # up stacking. In this case self._fallback_repositories is [], and the
1086
        # real repo is already setup. Preserve the real repo and
1087
        # RemoteRepository.add_fallback_repository will avoid adding
1088
        # duplicates.
1089
        # 2) new servers, RemoteBranch.open() sets up stacking, and when
1090
        # ensure_real is triggered from a branch, the real repository to
1091
        # set already has a matching list with separate instances, but
1092
        # as they are also RemoteRepositories we don't worry about making the
1093
        # lists be identical.
1094
        # 3) new servers, RemoteRepository.ensure_real is triggered before
1095
        # RemoteBranch.ensure real, in this case we get a repo with no fallbacks
1096
        # and need to populate it.
1097
        if (self._fallback_repositories and
1098
            len(self._real_repository._fallback_repositories) !=
4226.2.2 by Robert Collins
Fix setting config options to support unicode values and don't attempt to reset repositories _fallback_repositories as the simple approach fails to work.
1099
            len(self._fallback_repositories)):
1100
            if len(self._real_repository._fallback_repositories):
1101
                raise AssertionError(
1102
                    "cannot cleanly remove existing _fallback_repositories")
3691.2.12 by Martin Pool
Add test for coping without Branch.get_stacked_on_url
1103
        for fb in self._fallback_repositories:
1104
            self._real_repository.add_fallback_repository(fb)
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1105
        if self._lock_mode == 'w':
1106
            # if we are already locked, the real repository must be able to
1107
            # acquire the lock with our token.
1108
            self._real_repository.lock_write(self._lock_token)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1109
        elif self._lock_mode == 'r':
1110
            self._real_repository.lock_read()
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
1111
2617.6.1 by Robert Collins
* New method on Repository - ``start_write_group``, ``end_write_group``
1112
    def start_write_group(self):
1113
        """Start a write group on the decorated repository.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
1114
4031.3.3 by Matt Nordhoff
Review tweaks from Ben Finney
1115
        Smart methods perform operations in a single step so this API
2617.6.6 by Robert Collins
Some review feedback.
1116
        is not really applicable except as a compatibility thunk
2617.6.1 by Robert Collins
* New method on Repository - ``start_write_group``, ``end_write_group``
1117
        for older plugins that don't use e.g. the CommitBuilder
1118
        facility.
1119
        """
1120
        self._ensure_real()
1121
        return self._real_repository.start_write_group()
1122
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1123
    def _unlock(self, token):
1124
        path = self.bzrdir._path_for_remote_call(self._client)
3015.2.9 by Robert Collins
Handle repositories that do not allow remote locking, like pack repositories, in the client side remote server proxy objects.
1125
        if not token:
1126
            # with no token the remote repository is not persistently locked.
1127
            return
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
1128
        err_context = {'token': token}
1129
        response = self._call('Repository.unlock', path, token,
1130
                              **err_context)
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1131
        if response == ('ok',):
1132
            return
1133
        else:
2555.1.1 by Martin Pool
Remove use of 'assert False' to raise an exception unconditionally
1134
            raise errors.UnexpectedSmartServerResponse(response)
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1135
4634.85.9 by Andrew Bennetts
Add some experimental decorators: @only_raises(..) and @cleanup_method.
1136
    @only_raises(errors.LockNotHeld, errors.LockBroken)
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
1137
    def unlock(self):
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
1138
        if not self._lock_count:
4509.3.25 by Martin Pool
Add an option for unlock errors to be non-fatal
1139
            return lock.cant_unlock_not_held(self)
2018.5.75 by Andrew Bennetts
Add Repository.{dont_,}leave_lock_in_place.
1140
        self._lock_count -= 1
2592.3.244 by Martin Pool
unlock while in a write group now aborts the write group, unlocks, and errors.
1141
        if self._lock_count > 0:
1142
            return
3835.1.8 by Aaron Bentley
Make UnstackedParentsProvider manage the cache
1143
        self._unstacked_provider.disable_cache()
2592.3.244 by Martin Pool
unlock while in a write group now aborts the write group, unlocks, and errors.
1144
        old_mode = self._lock_mode
1145
        self._lock_mode = None
1146
        try:
1147
            # The real repository is responsible at present for raising an
1148
            # exception if it's in an unfinished write group.  However, it
1149
            # normally will *not* actually remove the lock from disk - that's
1150
            # done by the server on receiving the Repository.unlock call.
1151
            # This is just to let the _real_repository stay up to date.
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1152
            if self._real_repository is not None:
1153
                self._real_repository.unlock()
2592.3.244 by Martin Pool
unlock while in a write group now aborts the write group, unlocks, and errors.
1154
        finally:
1155
            # The rpc-level lock should be released even if there was a
1156
            # problem releasing the vfs-based lock.
1157
            if old_mode == 'w':
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
1158
                # Only write-locked repositories need to make a remote method
4031.3.1 by Frank Aspell
Fixing various typos
1159
                # call to perform the unlock.
2592.3.244 by Martin Pool
unlock while in a write group now aborts the write group, unlocks, and errors.
1160
                old_token = self._lock_token
1161
                self._lock_token = None
1162
                if not self._leave_lock:
1163
                    self._unlock(old_token)
4379.2.1 by John Arbash Meinel
Change the fallback repository code to only lock/unlock on transition.
1164
        # Fallbacks are always 'lock_read()' so we don't pay attention to
1165
        # self._leave_lock
1166
        for repo in self._fallback_repositories:
1167
            repo.unlock()
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
1168
1169
    def break_lock(self):
2018.5.78 by Andrew Bennetts
Implement RemoteRepository.lock_write/unlock to expect and send tokens over the
1170
        # should hand off to the network
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
1171
        self._ensure_real()
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
1172
        return self._real_repository.break_lock()
1173
2018.18.8 by Ian Clatworthy
Tarball proxy code & tests
1174
    def _get_tarball(self, compression):
2814.10.2 by Andrew Bennetts
Make the fallback a little tidier.
1175
        """Return a TemporaryFile containing a repository tarball.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
1176
2814.10.2 by Andrew Bennetts
Make the fallback a little tidier.
1177
        Returns None if the server does not support sending tarballs.
1178
        """
2018.18.25 by Martin Pool
Repository.tarball fixes for python2.4
1179
        import tempfile
2018.18.8 by Ian Clatworthy
Tarball proxy code & tests
1180
        path = self.bzrdir._path_for_remote_call(self._client)
3297.3.3 by Andrew Bennetts
SmartClientRequestProtocol*.read_response_tuple can now raise UnknownSmartMethod. Callers no longer need to do their own ad hoc unknown smart method error detection.
1181
        try:
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
1182
            response, protocol = self._call_expecting_body(
3297.3.3 by Andrew Bennetts
SmartClientRequestProtocol*.read_response_tuple can now raise UnknownSmartMethod. Callers no longer need to do their own ad hoc unknown smart method error detection.
1183
                'Repository.tarball', path, compression)
1184
        except errors.UnknownSmartMethod:
1185
            protocol.cancel_read_body()
1186
            return None
2018.18.8 by Ian Clatworthy
Tarball proxy code & tests
1187
        if response[0] == 'ok':
1188
            # Extract the tarball and return it
2018.18.25 by Martin Pool
Repository.tarball fixes for python2.4
1189
            t = tempfile.NamedTemporaryFile()
1190
            # TODO: rpc layer should read directly into it...
1191
            t.write(protocol.read_body_bytes())
1192
            t.seek(0)
1193
            return t
2814.10.1 by Andrew Bennetts
Cope gracefully if the server doesn't support the Repository.tarball smart request.
1194
        raise errors.UnexpectedSmartServerResponse(response)
2018.18.8 by Ian Clatworthy
Tarball proxy code & tests
1195
2440.1.1 by Martin Pool
Add new Repository.sprout,
1196
    def sprout(self, to_bzrdir, revision_id=None):
1197
        # TODO: Option to control what format is created?
3047.1.1 by Andrew Bennetts
Fix for bug 164626, add test that Repository.sprout preserves format.
1198
        self._ensure_real()
3047.1.4 by Andrew Bennetts
Simplify RemoteRepository.sprout thanks to review comments.
1199
        dest_repo = self._real_repository._format.initialize(to_bzrdir,
1200
                                                             shared=False)
2535.3.17 by Andrew Bennetts
[broken] Closer to a working Repository.fetch_revisions smart request.
1201
        dest_repo.fetch(self, revision_id=revision_id)
1202
        return dest_repo
2440.1.1 by Martin Pool
Add new Repository.sprout,
1203
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1204
    ### These methods are just thin shims to the VFS object for now.
1205
1206
    def revision_tree(self, revision_id):
1207
        self._ensure_real()
1208
        return self._real_repository.revision_tree(revision_id)
1209
2520.4.113 by Aaron Bentley
Avoid peeking at Repository._serializer
1210
    def get_serializer_format(self):
1211
        self._ensure_real()
1212
        return self._real_repository.get_serializer_format()
1213
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1214
    def get_commit_builder(self, branch, parents, config, timestamp=None,
1215
                           timezone=None, committer=None, revprops=None,
1216
                           revision_id=None):
1217
        # FIXME: It ought to be possible to call this without immediately
1218
        # triggering _ensure_real.  For now it's the easiest thing to do.
1219
        self._ensure_real()
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
1220
        real_repo = self._real_repository
1221
        builder = real_repo.get_commit_builder(branch, parents,
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1222
                config, timestamp=timestamp, timezone=timezone,
1223
                committer=committer, revprops=revprops, revision_id=revision_id)
1224
        return builder
1225
3221.12.1 by Robert Collins
Backport development1 format (stackable packs) to before-shallow-branches.
1226
    def add_fallback_repository(self, repository):
1227
        """Add a repository to use for looking up data not held locally.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
1228
3221.12.1 by Robert Collins
Backport development1 format (stackable packs) to before-shallow-branches.
1229
        :param repository: A repository.
1230
        """
4118.1.1 by Andrew Bennetts
Fix performance regression (many small round-trips) when pushing to a remote pack, and tidy the tests.
1231
        if not self._format.supports_external_lookups:
1232
            raise errors.UnstackableRepositoryFormat(
1233
                self._format.network_name(), self.base)
3221.12.1 by Robert Collins
Backport development1 format (stackable packs) to before-shallow-branches.
1234
        # We need to accumulate additional repositories here, to pass them in
1235
        # on various RPC's.
4035.2.3 by Robert Collins
Fix trailing whitespace.
1236
        #
4379.2.2 by John Arbash Meinel
Change the Repository.add_fallback_repository() contract slightly.
1237
        if self.is_locked():
1238
            # We will call fallback.unlock() when we transition to the unlocked
1239
            # state, so always add a lock here. If a caller passes us a locked
1240
            # repository, they are responsible for unlocking it later.
1241
            repository.lock_read()
3221.12.1 by Robert Collins
Backport development1 format (stackable packs) to before-shallow-branches.
1242
        self._fallback_repositories.append(repository)
4035.2.2 by Robert Collins
Minor tweaks to fix failing tests.
1243
        # If self._real_repository was parameterised already (e.g. because a
1244
        # _real_branch had its get_stacked_on_url method called), then the
1245
        # repository to be added may already be in the _real_repositories list.
4035.2.1 by Andrew Bennetts
Fix unnecessary get_parent_map calls after insert_stream during push.
1246
        if self._real_repository is not None:
5158.6.7 by Martin Pool
More conversions to using user_url
1247
            fallback_locations = [repo.user_url for repo in
4226.2.5 by Robert Collins
Fix handling of fallback repositories some more.
1248
                self._real_repository._fallback_repositories]
5158.6.7 by Martin Pool
More conversions to using user_url
1249
            if repository.user_url not in fallback_locations:
4035.2.2 by Robert Collins
Minor tweaks to fix failing tests.
1250
                self._real_repository.add_fallback_repository(repository)
3221.12.1 by Robert Collins
Backport development1 format (stackable packs) to before-shallow-branches.
1251
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1252
    def add_inventory(self, revid, inv, parents):
1253
        self._ensure_real()
1254
        return self._real_repository.add_inventory(revid, inv, parents)
1255
3879.2.2 by John Arbash Meinel
Rename add_inventory_delta to add_inventory_by_delta.
1256
    def add_inventory_by_delta(self, basis_revision_id, delta, new_revision_id,
5076.1.1 by Jelmer Vernooij
Allow additional arguments to RemoteRepository.add_inventory_by_delta().
1257
            parents, basis_inv=None, propagate_caches=False):
3775.2.1 by Robert Collins
Create bzrlib.repository.Repository.add_inventory_delta for adding inventories via deltas.
1258
        self._ensure_real()
3879.2.2 by John Arbash Meinel
Rename add_inventory_delta to add_inventory_by_delta.
1259
        return self._real_repository.add_inventory_by_delta(basis_revision_id,
5076.1.2 by Jelmer Vernooij
pass the keyword args as keyword args, per Rob's review.
1260
            delta, new_revision_id, parents, basis_inv=basis_inv,
1261
            propagate_caches=propagate_caches)
3775.2.1 by Robert Collins
Create bzrlib.repository.Repository.add_inventory_delta for adding inventories via deltas.
1262
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1263
    def add_revision(self, rev_id, rev, inv=None, config=None):
1264
        self._ensure_real()
1265
        return self._real_repository.add_revision(
1266
            rev_id, rev, inv=inv, config=config)
1267
1268
    @needs_read_lock
1269
    def get_inventory(self, revision_id):
1270
        self._ensure_real()
1271
        return self._real_repository.get_inventory(revision_id)
1272
4476.3.86 by Andrew Bennetts
Fix bug in declaration of RemoteRepository.iter_inventories that was causing intermittent failures in test_iter_inventories_is_ordered.
1273
    def iter_inventories(self, revision_ids, ordering=None):
3169.2.1 by Robert Collins
New method ``iter_inventories`` on Repository for access to many
1274
        self._ensure_real()
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1275
        return self._real_repository.iter_inventories(revision_ids, ordering)
3169.2.1 by Robert Collins
New method ``iter_inventories`` on Repository for access to many
1276
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1277
    @needs_read_lock
1278
    def get_revision(self, revision_id):
1279
        self._ensure_real()
1280
        return self._real_repository.get_revision(revision_id)
1281
1282
    def get_transaction(self):
1283
        self._ensure_real()
1284
        return self._real_repository.get_transaction()
1285
1286
    @needs_read_lock
2018.5.138 by Robert Collins
Merge bzr.dev.
1287
    def clone(self, a_bzrdir, revision_id=None):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1288
        self._ensure_real()
2018.5.138 by Robert Collins
Merge bzr.dev.
1289
        return self._real_repository.clone(a_bzrdir, revision_id=revision_id)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1290
1291
    def make_working_trees(self):
3349.1.1 by Aaron Bentley
Enable setting and getting make_working_trees for all repositories
1292
        """See Repository.make_working_trees"""
1293
        self._ensure_real()
1294
        return self._real_repository.make_working_trees()
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1295
4145.1.2 by Robert Collins
Add a refresh_data method on Repository allowing cleaner handling of insertions into RemoteRepository objects with _real_repository instances.
1296
    def refresh_data(self):
1297
        """Re-read any data needed to to synchronise with disk.
1298
1299
        This method is intended to be called after another repository instance
1300
        (such as one used by a smart server) has inserted data into the
1301
        repository. It may not be called during a write group, but may be
1302
        called at any other time.
1303
        """
1304
        if self.is_in_write_group():
1305
            raise errors.InternalBzrError(
1306
                "May not refresh_data while in a write group.")
1307
        if self._real_repository is not None:
1308
            self._real_repository.refresh_data()
1309
3184.1.9 by Robert Collins
* ``Repository.get_data_stream`` is now deprecated in favour of
1310
    def revision_ids_to_search_result(self, result_set):
1311
        """Convert a set of revision ids to a graph SearchResult."""
1312
        result_parents = set()
1313
        for parents in self.get_graph().get_parent_map(
1314
            result_set).itervalues():
1315
            result_parents.update(parents)
1316
        included_keys = result_set.intersection(result_parents)
1317
        start_keys = result_set.difference(included_keys)
1318
        exclude_keys = result_parents.difference(result_set)
1319
        result = graph.SearchResult(start_keys, exclude_keys,
1320
            len(result_set), result_set)
1321
        return result
1322
1323
    @needs_read_lock
1324
    def search_missing_revision_ids(self, other, revision_id=None, find_ghosts=True):
1325
        """Return the revision ids that other has that this does not.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
1326
3184.1.9 by Robert Collins
* ``Repository.get_data_stream`` is now deprecated in favour of
1327
        These are returned in topological order.
1328
1329
        revision_id: only return revision ids included by revision_id.
1330
        """
1331
        return repository.InterRepository.get(
1332
            other, self).search_missing_revision_ids(revision_id, find_ghosts)
1333
4070.9.2 by Andrew Bennetts
Rough prototype of allowing a SearchResult to be passed to fetch, and using that to improve network conversations.
1334
    def fetch(self, source, revision_id=None, pb=None, find_ghosts=False,
1335
            fetch_spec=None):
4145.1.1 by Robert Collins
Explicitly prevent fetching while the target repository is in a write group.
1336
        # No base implementation to use as RemoteRepository is not a subclass
1337
        # of Repository; so this is a copy of Repository.fetch().
4070.9.2 by Andrew Bennetts
Rough prototype of allowing a SearchResult to be passed to fetch, and using that to improve network conversations.
1338
        if fetch_spec is not None and revision_id is not None:
1339
            raise AssertionError(
1340
                "fetch_spec and revision_id are mutually exclusive.")
4145.1.1 by Robert Collins
Explicitly prevent fetching while the target repository is in a write group.
1341
        if self.is_in_write_group():
4145.1.3 by Robert Collins
NEWS conflicts.
1342
            raise errors.InternalBzrError(
1343
                "May not fetch while in a write group.")
4145.1.1 by Robert Collins
Explicitly prevent fetching while the target repository is in a write group.
1344
        # fast path same-url fetch operations
4509.3.19 by Martin Pool
RemoteRepository.fetch is not a no-op if there's different stacking
1345
        if (self.has_same_location(source)
1346
            and fetch_spec is None
4509.3.37 by Martin Pool
Remove RepositoryBase; make _has_same_location private
1347
            and self._has_same_fallbacks(source)):
2881.4.1 by Robert Collins
Move responsibility for detecting same-repo fetching from the
1348
            # check that last_revision is in 'from' and then return a
1349
            # no-operation.
1350
            if (revision_id is not None and
4145.1.5 by Robert Collins
More fixes from grabbing the Repository implementation of fetch for RemoteRepository.
1351
                not revision.is_null(revision_id)):
2881.4.1 by Robert Collins
Move responsibility for detecting same-repo fetching from the
1352
                self.get_revision(revision_id)
2592.4.5 by Martin Pool
Add Repository.base on all repositories.
1353
            return 0, []
4145.1.1 by Robert Collins
Explicitly prevent fetching while the target repository is in a write group.
1354
        # if there is no specific appropriate InterRepository, this will get
1355
        # the InterRepository base class, which raises an
1356
        # IncompatibleRepositories when asked to fetch.
4145.1.2 by Robert Collins
Add a refresh_data method on Repository allowing cleaner handling of insertions into RemoteRepository objects with _real_repository instances.
1357
        inter = repository.InterRepository.get(source, self)
4145.1.1 by Robert Collins
Explicitly prevent fetching while the target repository is in a write group.
1358
        return inter.fetch(revision_id=revision_id, pb=pb,
1359
            find_ghosts=find_ghosts, fetch_spec=fetch_spec)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1360
2520.4.54 by Aaron Bentley
Hang a create_bundle method off repository
1361
    def create_bundle(self, target, base, fileobj, format=None):
1362
        self._ensure_real()
1363
        self._real_repository.create_bundle(target, base, fileobj, format)
1364
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1365
    @needs_read_lock
2530.1.1 by Aaron Bentley
Make topological sorting optional for get_ancestry
1366
    def get_ancestry(self, revision_id, topo_sorted=True):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1367
        self._ensure_real()
2530.1.1 by Aaron Bentley
Make topological sorting optional for get_ancestry
1368
        return self._real_repository.get_ancestry(revision_id, topo_sorted)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1369
1370
    def fileids_altered_by_revision_ids(self, revision_ids):
1371
        self._ensure_real()
1372
        return self._real_repository.fileids_altered_by_revision_ids(revision_ids)
1373
3036.1.3 by Robert Collins
Privatise VersionedFileChecker.
1374
    def _get_versioned_file_checker(self, revisions, revision_versions_cache):
2745.6.1 by Aaron Bentley
Initial checking of knit graphs
1375
        self._ensure_real()
3036.1.3 by Robert Collins
Privatise VersionedFileChecker.
1376
        return self._real_repository._get_versioned_file_checker(
2745.6.50 by Andrew Bennetts
Remove find_bad_ancestors; it's not needed anymore.
1377
            revisions, revision_versions_cache)
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
1378
2708.1.7 by Aaron Bentley
Rename extract_files_bytes to iter_files_bytes
1379
    def iter_files_bytes(self, desired_files):
2708.1.9 by Aaron Bentley
Clean-up docs and imports
1380
        """See Repository.iter_file_bytes.
2708.1.3 by Aaron Bentley
Implement extract_files_bytes on Repository
1381
        """
1382
        self._ensure_real()
2708.1.7 by Aaron Bentley
Rename extract_files_bytes to iter_files_bytes
1383
        return self._real_repository.iter_files_bytes(desired_files)
2708.1.3 by Aaron Bentley
Implement extract_files_bytes on Repository
1384
3835.1.1 by Aaron Bentley
Stack get_parent_map on fallback repos
1385
    def get_parent_map(self, revision_ids):
3835.1.6 by Aaron Bentley
Reduce inefficiency when doing make_parents_provider frequently
1386
        """See bzrlib.Graph.get_parent_map()."""
3835.1.5 by Aaron Bentley
Fix make_parents_provider
1387
        return self._make_parents_provider().get_parent_map(revision_ids)
3835.1.1 by Aaron Bentley
Stack get_parent_map on fallback repos
1388
1389
    def _get_parent_map_rpc(self, keys):
3172.5.6 by Robert Collins
Create new smart server verb Repository.get_parent_map.
1390
        """Helper for get_parent_map that performs the RPC."""
3313.2.1 by Andrew Bennetts
Change _SmartClient's API to accept a medium and a base, rather than a _SharedConnection.
1391
        medium = self._client._medium
3453.4.10 by Andrew Bennetts
Change _is_remote_at_least to _is_remote_before.
1392
        if medium._is_remote_before((1, 2)):
3213.1.1 by Andrew Bennetts
Recover (by reconnecting) if the server turns out not to understand the new requests in 1.2 that send bodies.
1393
            # We already found out that the server can't understand
3213.1.3 by Andrew Bennetts
Fix typo in comment.
1394
            # Repository.get_parent_map requests, so just fetch the whole
3213.1.1 by Andrew Bennetts
Recover (by reconnecting) if the server turns out not to understand the new requests in 1.2 that send bodies.
1395
            # graph.
3948.3.7 by Martin Pool
Updated tests for RemoteRepository.get_parent_map on old servers.
1396
            #
1397
            # Note that this reads the whole graph, when only some keys are
1398
            # wanted.  On this old server there's no way (?) to get them all
1399
            # in one go, and the user probably will have seen a warning about
1400
            # the server being old anyhow.
1401
            rg = self._get_revision_graph(None)
4031.3.3 by Matt Nordhoff
Review tweaks from Ben Finney
1402
            # There is an API discrepancy between get_parent_map and
3389.1.1 by John Arbash Meinel
Fix bug #214894. Fix RemoteRepository.get_parent_map() when server is <v1.2
1403
            # get_revision_graph. Specifically, a "key:()" pair in
1404
            # get_revision_graph just means a node has no parents. For
1405
            # "get_parent_map" it means the node is a ghost. So fix up the
1406
            # graph to correct this.
1407
            #   https://bugs.launchpad.net/bzr/+bug/214894
1408
            # There is one other "bug" which is that ghosts in
1409
            # get_revision_graph() are not returned at all. But we won't worry
1410
            # about that for now.
1411
            for node_id, parent_ids in rg.iteritems():
1412
                if parent_ids == ():
1413
                    rg[node_id] = (NULL_REVISION,)
1414
            rg[NULL_REVISION] = ()
1415
            return rg
3213.1.1 by Andrew Bennetts
Recover (by reconnecting) if the server turns out not to understand the new requests in 1.2 that send bodies.
1416
3172.5.6 by Robert Collins
Create new smart server verb Repository.get_parent_map.
1417
        keys = set(keys)
3373.5.2 by John Arbash Meinel
Add repository_implementation tests for get_parent_map
1418
        if None in keys:
1419
            raise ValueError('get_parent_map(None) is not valid')
3172.5.6 by Robert Collins
Create new smart server verb Repository.get_parent_map.
1420
        if NULL_REVISION in keys:
1421
            keys.discard(NULL_REVISION)
1422
            found_parents = {NULL_REVISION:()}
1423
            if not keys:
1424
                return found_parents
1425
        else:
1426
            found_parents = {}
3211.5.1 by Robert Collins
Change the smart server get_parents method to take a graph search to exclude already recieved parents from. This prevents history shortcuts causing huge numbers of duplicates.
1427
        # TODO(Needs analysis): We could assume that the keys being requested
1428
        # from get_parent_map are in a breadth first search, so typically they
1429
        # will all be depth N from some common parent, and we don't have to
1430
        # have the server iterate from the root parent, but rather from the
1431
        # keys we're searching; and just tell the server the keyspace we
1432
        # already have; but this may be more traffic again.
1433
1434
        # Transform self._parents_map into a search request recipe.
1435
        # TODO: Manage this incrementally to avoid covering the same path
1436
        # repeatedly. (The server will have to on each request, but the less
1437
        # work done the better).
4190.1.3 by Robert Collins
Allow optional inclusion of ghost data in server get_parent_map calls.
1438
        #
1439
        # Negative caching notes:
1440
        # new server sends missing when a request including the revid
1441
        # 'include-missing:' is present in the request.
1442
        # missing keys are serialised as missing:X, and we then call
1443
        # provider.note_missing(X) for-all X
3835.1.8 by Aaron Bentley
Make UnstackedParentsProvider manage the cache
1444
        parents_map = self._unstacked_provider.get_cached_map()
3213.1.8 by Andrew Bennetts
Merge from bzr.dev.
1445
        if parents_map is None:
1446
            # Repository is not locked, so there's no cache.
1447
            parents_map = {}
4190.1.3 by Robert Collins
Allow optional inclusion of ghost data in server get_parent_map calls.
1448
        # start_set is all the keys in the cache
3213.1.8 by Andrew Bennetts
Merge from bzr.dev.
1449
        start_set = set(parents_map)
4190.1.3 by Robert Collins
Allow optional inclusion of ghost data in server get_parent_map calls.
1450
        # result set is all the references to keys in the cache
3211.5.1 by Robert Collins
Change the smart server get_parents method to take a graph search to exclude already recieved parents from. This prevents history shortcuts causing huge numbers of duplicates.
1451
        result_parents = set()
3213.1.8 by Andrew Bennetts
Merge from bzr.dev.
1452
        for parents in parents_map.itervalues():
3211.5.1 by Robert Collins
Change the smart server get_parents method to take a graph search to exclude already recieved parents from. This prevents history shortcuts causing huge numbers of duplicates.
1453
            result_parents.update(parents)
1454
        stop_keys = result_parents.difference(start_set)
4190.1.4 by Robert Collins
Cache ghosts when we can get them from a RemoteRepository in get_parent_map.
1455
        # We don't need to send ghosts back to the server as a position to
1456
        # stop either.
1457
        stop_keys.difference_update(self._unstacked_provider.missing_keys)
4214.2.5 by Andrew Bennetts
Fix the bug.
1458
        key_count = len(parents_map)
1459
        if (NULL_REVISION in result_parents
1460
            and NULL_REVISION in self._unstacked_provider.missing_keys):
1461
            # If we pruned NULL_REVISION from the stop_keys because it's also
1462
            # in our cache of "missing" keys we need to increment our key count
1463
            # by 1, because the reconsitituted SearchResult on the server will
1464
            # still consider NULL_REVISION to be an included key.
1465
            key_count += 1
3211.5.1 by Robert Collins
Change the smart server get_parents method to take a graph search to exclude already recieved parents from. This prevents history shortcuts causing huge numbers of duplicates.
1466
        included_keys = start_set.intersection(result_parents)
1467
        start_set.difference_update(included_keys)
4214.2.5 by Andrew Bennetts
Fix the bug.
1468
        recipe = ('manual', start_set, stop_keys, key_count)
3842.3.20 by Andrew Bennetts
Re-revert changes from another thread that accidentally got reinstated here.
1469
        body = self._serialise_search_recipe(recipe)
3172.5.6 by Robert Collins
Create new smart server verb Repository.get_parent_map.
1470
        path = self.bzrdir._path_for_remote_call(self._client)
1471
        for key in keys:
3360.2.8 by Martin Pool
Change assertion to a plain raise
1472
            if type(key) is not str:
1473
                raise ValueError(
1474
                    "key %r not a plain string" % (key,))
3172.5.8 by Robert Collins
Review feedback.
1475
        verb = 'Repository.get_parent_map'
4190.1.4 by Robert Collins
Cache ghosts when we can get them from a RemoteRepository in get_parent_map.
1476
        args = (path, 'include-missing:') + tuple(keys)
3297.3.3 by Andrew Bennetts
SmartClientRequestProtocol*.read_response_tuple can now raise UnknownSmartMethod. Callers no longer need to do their own ad hoc unknown smart method error detection.
1477
        try:
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
1478
            response = self._call_with_body_bytes_expecting_body(
3842.3.20 by Andrew Bennetts
Re-revert changes from another thread that accidentally got reinstated here.
1479
                verb, args, body)
3297.3.3 by Andrew Bennetts
SmartClientRequestProtocol*.read_response_tuple can now raise UnknownSmartMethod. Callers no longer need to do their own ad hoc unknown smart method error detection.
1480
        except errors.UnknownSmartMethod:
3213.1.2 by Andrew Bennetts
Add test for reconnection if get_parent_map is unknown by the server.
1481
            # Server does not support this method, so get the whole graph.
3213.1.1 by Andrew Bennetts
Recover (by reconnecting) if the server turns out not to understand the new requests in 1.2 that send bodies.
1482
            # Worse, we have to force a disconnection, because the server now
1483
            # doesn't realise it has a body on the wire to consume, so the
1484
            # only way to recover is to abandon the connection.
3213.1.6 by Andrew Bennetts
Emit warnings when forcing a reconnect.
1485
            warning(
1486
                'Server is too old for fast get_parent_map, reconnecting.  '
1487
                '(Upgrade the server to Bazaar 1.2 to avoid this)')
3213.1.1 by Andrew Bennetts
Recover (by reconnecting) if the server turns out not to understand the new requests in 1.2 that send bodies.
1488
            medium.disconnect()
1489
            # To avoid having to disconnect repeatedly, we keep track of the
1490
            # fact the server doesn't understand remote methods added in 1.2.
3453.4.9 by Andrew Bennetts
Rename _remote_is_not to _remember_remote_is_before.
1491
            medium._remember_remote_is_before((1, 2))
3948.3.7 by Martin Pool
Updated tests for RemoteRepository.get_parent_map on old servers.
1492
            # Recurse just once and we should use the fallback code.
1493
            return self._get_parent_map_rpc(keys)
3245.4.58 by Andrew Bennetts
Unpack call_expecting_body's return value into variables, to avoid lots of ugly subscripting.
1494
        response_tuple, response_handler = response
1495
        if response_tuple[0] not in ['ok']:
1496
            response_handler.cancel_read_body()
1497
            raise errors.UnexpectedSmartServerResponse(response_tuple)
1498
        if response_tuple[0] == 'ok':
1499
            coded = bz2.decompress(response_handler.read_body_bytes())
3172.5.6 by Robert Collins
Create new smart server verb Repository.get_parent_map.
1500
            if coded == '':
1501
                # no revisions found
1502
                return {}
1503
            lines = coded.split('\n')
1504
            revision_graph = {}
1505
            for line in lines:
1506
                d = tuple(line.split())
1507
                if len(d) > 1:
1508
                    revision_graph[d[0]] = d[1:]
1509
                else:
4190.1.4 by Robert Collins
Cache ghosts when we can get them from a RemoteRepository in get_parent_map.
1510
                    # No parents:
1511
                    if d[0].startswith('missing:'):
1512
                        revid = d[0][8:]
1513
                        self._unstacked_provider.note_missing_key(revid)
1514
                    else:
1515
                        # no parents - so give the Graph result
1516
                        # (NULL_REVISION,).
1517
                        revision_graph[d[0]] = (NULL_REVISION,)
3172.5.6 by Robert Collins
Create new smart server verb Repository.get_parent_map.
1518
            return revision_graph
1519
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1520
    @needs_read_lock
1521
    def get_signature_text(self, revision_id):
1522
        self._ensure_real()
1523
        return self._real_repository.get_signature_text(revision_id)
1524
1525
    @needs_read_lock
4988.5.1 by Jelmer Vernooij
Rename Repository.get_inventory_xml -> Repository._get_inventory_xml.
1526
    def _get_inventory_xml(self, revision_id):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1527
        self._ensure_real()
4988.5.1 by Jelmer Vernooij
Rename Repository.get_inventory_xml -> Repository._get_inventory_xml.
1528
        return self._real_repository._get_inventory_xml(revision_id)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1529
1530
    def reconcile(self, other=None, thorough=False):
1531
        self._ensure_real()
1532
        return self._real_repository.reconcile(other=other, thorough=thorough)
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
1533
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1534
    def all_revision_ids(self):
1535
        self._ensure_real()
1536
        return self._real_repository.all_revision_ids()
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
1537
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1538
    @needs_read_lock
4137.3.2 by Ian Clatworthy
Repository.get_deltas_for_revisions() now supports file-id filtering
1539
    def get_deltas_for_revisions(self, revisions, specific_fileids=None):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1540
        self._ensure_real()
4137.3.2 by Ian Clatworthy
Repository.get_deltas_for_revisions() now supports file-id filtering
1541
        return self._real_repository.get_deltas_for_revisions(revisions,
1542
            specific_fileids=specific_fileids)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1543
1544
    @needs_read_lock
4137.3.2 by Ian Clatworthy
Repository.get_deltas_for_revisions() now supports file-id filtering
1545
    def get_revision_delta(self, revision_id, specific_fileids=None):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1546
        self._ensure_real()
4137.3.2 by Ian Clatworthy
Repository.get_deltas_for_revisions() now supports file-id filtering
1547
        return self._real_repository.get_revision_delta(revision_id,
1548
            specific_fileids=specific_fileids)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1549
1550
    @needs_read_lock
1551
    def revision_trees(self, revision_ids):
1552
        self._ensure_real()
1553
        return self._real_repository.revision_trees(revision_ids)
1554
1555
    @needs_read_lock
1556
    def get_revision_reconcile(self, revision_id):
1557
        self._ensure_real()
1558
        return self._real_repository.get_revision_reconcile(revision_id)
1559
1560
    @needs_read_lock
4332.3.35 by Robert Collins
Fix failing tests.
1561
    def check(self, revision_ids=None, callback_refs=None, check_repo=True):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1562
        self._ensure_real()
4332.3.11 by Robert Collins
Move tree and back callbacks into the repository check core.
1563
        return self._real_repository.check(revision_ids=revision_ids,
4332.3.35 by Robert Collins
Fix failing tests.
1564
            callback_refs=callback_refs, check_repo=check_repo)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1565
2018.5.138 by Robert Collins
Merge bzr.dev.
1566
    def copy_content_into(self, destination, revision_id=None):
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1567
        self._ensure_real()
1568
        return self._real_repository.copy_content_into(
2018.5.138 by Robert Collins
Merge bzr.dev.
1569
            destination, revision_id=revision_id)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1570
2814.10.2 by Andrew Bennetts
Make the fallback a little tidier.
1571
    def _copy_repository_tarball(self, to_bzrdir, revision_id=None):
2018.18.10 by Martin Pool
copy_content_into from Remote repositories by using temporary directories on both ends.
1572
        # get a tarball of the remote repository, and copy from that into the
1573
        # destination
1574
        from bzrlib import osutils
2018.18.9 by Martin Pool
remote Repository.tarball builds a temporary directory and tars that
1575
        import tarfile
2018.18.20 by Martin Pool
Route branch operations through remote copy_content_into
1576
        # TODO: Maybe a progress bar while streaming the tarball?
1577
        note("Copying repository content as tarball...")
2018.18.25 by Martin Pool
Repository.tarball fixes for python2.4
1578
        tar_file = self._get_tarball('bz2')
2814.10.2 by Andrew Bennetts
Make the fallback a little tidier.
1579
        if tar_file is None:
1580
            return None
1581
        destination = to_bzrdir.create_repository()
2018.18.10 by Martin Pool
copy_content_into from Remote repositories by using temporary directories on both ends.
1582
        try:
2018.18.25 by Martin Pool
Repository.tarball fixes for python2.4
1583
            tar = tarfile.open('repository', fileobj=tar_file,
1584
                mode='r|bz2')
3638.3.2 by Vincent Ladeuil
Fix all calls to tempfile.mkdtemp to osutils.mkdtemp.
1585
            tmpdir = osutils.mkdtemp()
2018.18.25 by Martin Pool
Repository.tarball fixes for python2.4
1586
            try:
1587
                _extract_tar(tar, tmpdir)
1588
                tmp_bzrdir = BzrDir.open(tmpdir)
1589
                tmp_repo = tmp_bzrdir.open_repository()
1590
                tmp_repo.copy_content_into(destination, revision_id)
1591
            finally:
1592
                osutils.rmtree(tmpdir)
2018.18.10 by Martin Pool
copy_content_into from Remote repositories by using temporary directories on both ends.
1593
        finally:
2018.18.25 by Martin Pool
Repository.tarball fixes for python2.4
1594
            tar_file.close()
2814.10.2 by Andrew Bennetts
Make the fallback a little tidier.
1595
        return destination
2018.18.23 by Martin Pool
review cleanups
1596
        # TODO: Suggestion from john: using external tar is much faster than
1597
        # python's tarfile library, but it may not work on windows.
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1598
3842.3.20 by Andrew Bennetts
Re-revert changes from another thread that accidentally got reinstated here.
1599
    @property
1600
    def inventories(self):
1601
        """Decorate the real repository for now.
1602
1603
        In the long term a full blown network facility is needed to
1604
        avoid creating a real repository object locally.
1605
        """
1606
        self._ensure_real()
1607
        return self._real_repository.inventories
1608
2604.2.1 by Robert Collins
(robertc) Introduce a pack command.
1609
    @needs_write_lock
4431.3.7 by Jonathan Lange
Cherrypick bzr.dev 4470, resolving conflicts.
1610
    def pack(self, hint=None):
2604.2.1 by Robert Collins
(robertc) Introduce a pack command.
1611
        """Compress the data within the repository.
1612
1613
        This is not currently implemented within the smart server.
1614
        """
1615
        self._ensure_real()
4431.3.7 by Jonathan Lange
Cherrypick bzr.dev 4470, resolving conflicts.
1616
        return self._real_repository.pack(hint=hint)
2604.2.1 by Robert Collins
(robertc) Introduce a pack command.
1617
3842.3.20 by Andrew Bennetts
Re-revert changes from another thread that accidentally got reinstated here.
1618
    @property
1619
    def revisions(self):
1620
        """Decorate the real repository for now.
1621
1622
        In the short term this should become a real object to intercept graph
1623
        lookups.
1624
1625
        In the long term a full blown network facility is needed.
1626
        """
1627
        self._ensure_real()
1628
        return self._real_repository.revisions
1629
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1630
    def set_make_working_trees(self, new_value):
4017.3.4 by Robert Collins
Create a verb for Repository.set_make_working_trees.
1631
        if new_value:
1632
            new_value_str = "True"
1633
        else:
1634
            new_value_str = "False"
1635
        path = self.bzrdir._path_for_remote_call(self._client)
1636
        try:
1637
            response = self._call(
1638
                'Repository.set_make_working_trees', path, new_value_str)
1639
        except errors.UnknownSmartMethod:
1640
            self._ensure_real()
1641
            self._real_repository.set_make_working_trees(new_value)
1642
        else:
1643
            if response[0] != 'ok':
1644
                raise errors.UnexpectedSmartServerResponse(response)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1645
3842.3.20 by Andrew Bennetts
Re-revert changes from another thread that accidentally got reinstated here.
1646
    @property
1647
    def signatures(self):
1648
        """Decorate the real repository for now.
1649
1650
        In the long term a full blown network facility is needed to avoid
1651
        creating a real repository object locally.
1652
        """
1653
        self._ensure_real()
1654
        return self._real_repository.signatures
1655
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1656
    @needs_write_lock
1657
    def sign_revision(self, revision_id, gpg_strategy):
1658
        self._ensure_real()
1659
        return self._real_repository.sign_revision(revision_id, gpg_strategy)
1660
3842.3.20 by Andrew Bennetts
Re-revert changes from another thread that accidentally got reinstated here.
1661
    @property
1662
    def texts(self):
1663
        """Decorate the real repository for now.
1664
1665
        In the long term a full blown network facility is needed to avoid
1666
        creating a real repository object locally.
1667
        """
1668
        self._ensure_real()
1669
        return self._real_repository.texts
1670
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1671
    @needs_read_lock
1672
    def get_revisions(self, revision_ids):
1673
        self._ensure_real()
1674
        return self._real_repository.get_revisions(revision_ids)
1675
1676
    def supports_rich_root(self):
4053.1.4 by Robert Collins
Move the fetch control attributes from Repository to RepositoryFormat.
1677
        return self._format.rich_root_data
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
1678
2018.5.83 by Andrew Bennetts
Fix some test failures caused by the switch from unicode to UTF-8-encoded strs for revision IDs.
1679
    def iter_reverse_revision_history(self, revision_id):
1680
        self._ensure_real()
1681
        return self._real_repository.iter_reverse_revision_history(revision_id)
1682
2018.5.96 by Andrew Bennetts
Merge from bzr.dev, resolving the worst of the semantic conflicts, but there's
1683
    @property
1684
    def _serializer(self):
4053.1.4 by Robert Collins
Move the fetch control attributes from Repository to RepositoryFormat.
1685
        return self._format._serializer
2018.5.96 by Andrew Bennetts
Merge from bzr.dev, resolving the worst of the semantic conflicts, but there's
1686
2018.5.97 by Andrew Bennetts
Fix more tests.
1687
    def store_revision_signature(self, gpg_strategy, plaintext, revision_id):
1688
        self._ensure_real()
1689
        return self._real_repository.store_revision_signature(
1690
            gpg_strategy, plaintext, revision_id)
1691
2996.2.8 by Aaron Bentley
Fix add_signature discrepancies
1692
    def add_signature_text(self, revision_id, signature):
2996.2.3 by Aaron Bentley
Add tests for install_revisions and add_signature
1693
        self._ensure_real()
2996.2.8 by Aaron Bentley
Fix add_signature discrepancies
1694
        return self._real_repository.add_signature_text(revision_id, signature)
2996.2.3 by Aaron Bentley
Add tests for install_revisions and add_signature
1695
2018.5.97 by Andrew Bennetts
Fix more tests.
1696
    def has_signature_for_revision_id(self, revision_id):
1697
        self._ensure_real()
1698
        return self._real_repository.has_signature_for_revision_id(revision_id)
1699
2535.3.45 by Andrew Bennetts
Add item_keys_introduced_by to RemoteRepository.
1700
    def item_keys_introduced_by(self, revision_ids, _files_pb=None):
1701
        self._ensure_real()
1702
        return self._real_repository.item_keys_introduced_by(revision_ids,
1703
            _files_pb=_files_pb)
1704
2819.2.4 by Andrew Bennetts
Add a 'revision_graph_can_have_wrong_parents' method to repository.
1705
    def revision_graph_can_have_wrong_parents(self):
1706
        # The answer depends on the remote repo format.
1707
        self._ensure_real()
1708
        return self._real_repository.revision_graph_can_have_wrong_parents()
1709
4332.3.34 by Robert Collins
Delete obsolete pack tuned find_inconsistent_revisions as the generic code path permits the same optimisation.
1710
    def _find_inconsistent_revision_parents(self, revisions_iterator=None):
2819.2.5 by Andrew Bennetts
Make reconcile abort gracefully if the revision index has bad parents.
1711
        self._ensure_real()
4332.3.34 by Robert Collins
Delete obsolete pack tuned find_inconsistent_revisions as the generic code path permits the same optimisation.
1712
        return self._real_repository._find_inconsistent_revision_parents(
1713
            revisions_iterator)
2819.2.5 by Andrew Bennetts
Make reconcile abort gracefully if the revision index has bad parents.
1714
1715
    def _check_for_inconsistent_revision_parents(self):
1716
        self._ensure_real()
1717
        return self._real_repository._check_for_inconsistent_revision_parents()
1718
3835.1.17 by Aaron Bentley
Fix stacking bug
1719
    def _make_parents_provider(self, other=None):
3835.1.8 by Aaron Bentley
Make UnstackedParentsProvider manage the cache
1720
        providers = [self._unstacked_provider]
3835.1.17 by Aaron Bentley
Fix stacking bug
1721
        if other is not None:
1722
            providers.insert(0, other)
3835.1.7 by Aaron Bentley
Updates from review
1723
        providers.extend(r._make_parents_provider() for r in
1724
                         self._fallback_repositories)
4379.3.3 by Gary van der Merwe
Rename and add doc string for StackedParentsProvider.
1725
        return graph.StackedParentsProvider(providers)
3172.5.1 by Robert Collins
Create a RemoteRepository get_graph implementation and delegate get_parents_map to the real repository.
1726
3842.3.20 by Andrew Bennetts
Re-revert changes from another thread that accidentally got reinstated here.
1727
    def _serialise_search_recipe(self, recipe):
1728
        """Serialise a graph search recipe.
1729
1730
        :param recipe: A search recipe (start, stop, count).
1731
        :return: Serialised bytes.
1732
        """
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1733
        start_keys = ' '.join(recipe[1])
1734
        stop_keys = ' '.join(recipe[2])
1735
        count = str(recipe[3])
3842.3.20 by Andrew Bennetts
Re-revert changes from another thread that accidentally got reinstated here.
1736
        return '\n'.join((start_keys, stop_keys, count))
1737
4070.9.5 by Andrew Bennetts
Better wire protocol: don't shoehorn MiniSearchResult serialisation into previous serialisation format.
1738
    def _serialise_search_result(self, search_result):
4070.9.14 by Andrew Bennetts
Tweaks requested by Robert's review.
1739
        if isinstance(search_result, graph.PendingAncestryResult):
1740
            parts = ['ancestry-of']
1741
            parts.extend(search_result.heads)
4070.9.5 by Andrew Bennetts
Better wire protocol: don't shoehorn MiniSearchResult serialisation into previous serialisation format.
1742
        else:
1743
            recipe = search_result.get_recipe()
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1744
            parts = [recipe[0], self._serialise_search_recipe(recipe)]
4070.9.5 by Andrew Bennetts
Better wire protocol: don't shoehorn MiniSearchResult serialisation into previous serialisation format.
1745
        return '\n'.join(parts)
1746
3842.3.2 by Andrew Bennetts
Revert the RemoteVersionedFiles.get_parent_map implementation, leaving just the skeleton of RemoteVersionedFiles.
1747
    def autopack(self):
1748
        path = self.bzrdir._path_for_remote_call(self._client)
1749
        try:
1750
            response = self._call('PackRepository.autopack', path)
1751
        except errors.UnknownSmartMethod:
1752
            self._ensure_real()
1753
            self._real_repository._pack_collection.autopack()
1754
            return
4145.1.2 by Robert Collins
Add a refresh_data method on Repository allowing cleaner handling of insertions into RemoteRepository objects with _real_repository instances.
1755
        self.refresh_data()
3842.3.2 by Andrew Bennetts
Revert the RemoteVersionedFiles.get_parent_map implementation, leaving just the skeleton of RemoteVersionedFiles.
1756
        if response[0] != 'ok':
1757
            raise errors.UnexpectedSmartServerResponse(response)
1758
1759
4022.1.6 by Robert Collins
Cherrypick and polish the RemoteSink for streaming push.
1760
class RemoteStreamSink(repository.StreamSink):
1761
4032.3.7 by Robert Collins
Move write locking and write group responsibilities into the Sink objects themselves, allowing complete avoidance of unnecessary calls when the sink is a RemoteSink.
1762
    def _insert_real(self, stream, src_format, resume_tokens):
4022.1.6 by Robert Collins
Cherrypick and polish the RemoteSink for streaming push.
1763
        self.target_repo._ensure_real()
1764
        sink = self.target_repo._real_repository._get_sink()
4032.3.7 by Robert Collins
Move write locking and write group responsibilities into the Sink objects themselves, allowing complete avoidance of unnecessary calls when the sink is a RemoteSink.
1765
        result = sink.insert_stream(stream, src_format, resume_tokens)
4029.2.1 by Robert Collins
Support streaming push to stacked branches.
1766
        if not result:
1767
            self.target_repo.autopack()
1768
        return result
4022.1.6 by Robert Collins
Cherrypick and polish the RemoteSink for streaming push.
1769
4032.3.7 by Robert Collins
Move write locking and write group responsibilities into the Sink objects themselves, allowing complete avoidance of unnecessary calls when the sink is a RemoteSink.
1770
    def insert_stream(self, stream, src_format, resume_tokens):
4144.3.2 by Andrew Bennetts
Use Repository.insert_stream_locked if there is a lock_token for the remote repo.
1771
        target = self.target_repo
4307.2.4 by Robert Collins
Enable caching of negative revision lookups in RemoteRepository write locks when no _real_repository has been constructed.
1772
        target._unstacked_provider.missing_keys.clear()
4476.3.82 by Andrew Bennetts
Mention another bug fix in NEWS, and update verb name, comments, and NEWS additions for landing on 1.19 rather than 1.18.
1773
        candidate_calls = [('Repository.insert_stream_1.19', (1, 19))]
4144.3.2 by Andrew Bennetts
Use Repository.insert_stream_locked if there is a lock_token for the remote repo.
1774
        if target._lock_token:
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1775
            candidate_calls.append(('Repository.insert_stream_locked', (1, 14)))
1776
            lock_args = (target._lock_token or '',)
4144.3.2 by Andrew Bennetts
Use Repository.insert_stream_locked if there is a lock_token for the remote repo.
1777
        else:
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1778
            candidate_calls.append(('Repository.insert_stream', (1, 13)))
1779
            lock_args = ()
4144.3.2 by Andrew Bennetts
Use Repository.insert_stream_locked if there is a lock_token for the remote repo.
1780
        client = target._client
4022.1.9 by Robert Collins
Fix critical issue in bzr.dev - pushing to an old bzr:// server fails because the stream being consumed before the fallback code occurs, which makes it fail to do the fetch. (Robert Collins, Andrew Bennetts, #332314)
1781
        medium = client._medium
4144.3.2 by Andrew Bennetts
Use Repository.insert_stream_locked if there is a lock_token for the remote repo.
1782
        path = target.bzrdir._path_for_remote_call(client)
4476.3.70 by Andrew Bennetts
Review tweaks.
1783
        # Probe for the verb to use with an empty stream before sending the
1784
        # real stream to it.  We do this both to avoid the risk of sending a
1785
        # large request that is then rejected, and because we don't want to
1786
        # implement a way to buffer, rewind, or restart the stream.
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1787
        found_verb = False
1788
        for verb, required_version in candidate_calls:
1789
            if medium._is_remote_before(required_version):
1790
                continue
4476.3.25 by Andrew Bennetts
Reinstate don't-reprobe-with-resume-tokens optimisation, fixing acceptance test.
1791
            if resume_tokens:
1792
                # We've already done the probing (and set _is_remote_before) on
1793
                # a previous insert.
1794
                found_verb = True
1795
                break
4060.1.4 by Robert Collins
Streaming fetch from remote servers.
1796
            byte_stream = smart_repo._stream_to_byte_stream([], src_format)
4029.2.1 by Robert Collins
Support streaming push to stacked branches.
1797
            try:
1798
                response = client.call_with_body_stream(
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1799
                    (verb, path, '') + lock_args, byte_stream)
4029.2.1 by Robert Collins
Support streaming push to stacked branches.
1800
            except errors.UnknownSmartMethod:
4144.3.2 by Andrew Bennetts
Use Repository.insert_stream_locked if there is a lock_token for the remote repo.
1801
                medium._remember_remote_is_before(required_version)
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1802
            else:
1803
                found_verb = True
1804
                break
1805
        if not found_verb:
1806
            # Have to use VFS.
1807
            return self._insert_real(stream, src_format, resume_tokens)
1808
        self._last_inv_record = None
1809
        self._last_substream = None
4476.3.82 by Andrew Bennetts
Mention another bug fix in NEWS, and update verb name, comments, and NEWS additions for landing on 1.19 rather than 1.18.
1810
        if required_version < (1, 19):
4476.3.35 by Andrew Bennetts
Expand comment.
1811
            # Remote side doesn't support inventory deltas.  Wrap the stream to
1812
            # make sure we don't send any.  If the stream contains inventory
1813
            # deltas we'll interrupt the smart insert_stream request and
1814
            # fallback to VFS.
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1815
            stream = self._stop_stream_if_inventory_delta(stream)
4060.1.4 by Robert Collins
Streaming fetch from remote servers.
1816
        byte_stream = smart_repo._stream_to_byte_stream(
1817
            stream, src_format)
4032.3.7 by Robert Collins
Move write locking and write group responsibilities into the Sink objects themselves, allowing complete avoidance of unnecessary calls when the sink is a RemoteSink.
1818
        resume_tokens = ' '.join(resume_tokens)
4022.1.9 by Robert Collins
Fix critical issue in bzr.dev - pushing to an old bzr:// server fails because the stream being consumed before the fallback code occurs, which makes it fail to do the fetch. (Robert Collins, Andrew Bennetts, #332314)
1819
        response = client.call_with_body_stream(
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1820
            (verb, path, resume_tokens) + lock_args, byte_stream)
4029.2.1 by Robert Collins
Support streaming push to stacked branches.
1821
        if response[0][0] not in ('ok', 'missing-basis'):
4022.1.9 by Robert Collins
Fix critical issue in bzr.dev - pushing to an old bzr:// server fails because the stream being consumed before the fallback code occurs, which makes it fail to do the fetch. (Robert Collins, Andrew Bennetts, #332314)
1822
            raise errors.UnexpectedSmartServerResponse(response)
4476.3.54 by Andrew Bennetts
Update fallback-to-VFS in remote.py for the inventory-deltas substream.
1823
        if self._last_substream is not None:
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1824
            # The stream included an inventory-delta record, but the remote
1825
            # side isn't new enough to support them.  So we need to send the
1826
            # rest of the stream via VFS.
4634.35.15 by Andrew Bennetts
Fix 'fallback to vfs if RPC does not support inventory-deltas' case to refresh the vfs pack names before starting the fallback.
1827
            self.target_repo.refresh_data()
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1828
            return self._resume_stream_with_vfs(response, src_format)
4029.2.1 by Robert Collins
Support streaming push to stacked branches.
1829
        if response[0][0] == 'missing-basis':
1830
            tokens, missing_keys = bencode.bdecode_as_tuple(response[0][1])
4032.3.7 by Robert Collins
Move write locking and write group responsibilities into the Sink objects themselves, allowing complete avoidance of unnecessary calls when the sink is a RemoteSink.
1831
            resume_tokens = tokens
4257.3.3 by Andrew Bennetts
missing_keys from sink.insert_stream should be a set, not a tuple.
1832
            return resume_tokens, set(missing_keys)
4029.2.1 by Robert Collins
Support streaming push to stacked branches.
1833
        else:
4145.1.2 by Robert Collins
Add a refresh_data method on Repository allowing cleaner handling of insertions into RemoteRepository objects with _real_repository instances.
1834
            self.target_repo.refresh_data()
4032.3.7 by Robert Collins
Move write locking and write group responsibilities into the Sink objects themselves, allowing complete avoidance of unnecessary calls when the sink is a RemoteSink.
1835
            return [], set()
4032.1.1 by John Arbash Meinel
Merge the removal of all trailing whitespace, and resolve conflicts.
1836
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1837
    def _resume_stream_with_vfs(self, response, src_format):
1838
        """Resume sending a stream via VFS, first resending the record and
1839
        substream that couldn't be sent via an insert_stream verb.
1840
        """
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1841
        if response[0][0] == 'missing-basis':
1842
            tokens, missing_keys = bencode.bdecode_as_tuple(response[0][1])
1843
            # Ignore missing_keys, we haven't finished inserting yet
1844
        else:
1845
            tokens = []
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1846
        def resume_substream():
4476.3.54 by Andrew Bennetts
Update fallback-to-VFS in remote.py for the inventory-deltas substream.
1847
            # Yield the substream that was interrupted.
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1848
            for record in self._last_substream:
1849
                yield record
1850
            self._last_substream = None
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1851
        def resume_stream():
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1852
            # Finish sending the interrupted substream
4476.3.54 by Andrew Bennetts
Update fallback-to-VFS in remote.py for the inventory-deltas substream.
1853
            yield ('inventory-deltas', resume_substream())
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1854
            # Then simply continue sending the rest of the stream.
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1855
            for substream_kind, substream in self._last_stream:
1856
                yield substream_kind, substream
1857
        return self._insert_real(resume_stream(), src_format, tokens)
1858
1859
    def _stop_stream_if_inventory_delta(self, stream):
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1860
        """Normally this just lets the original stream pass-through unchanged.
1861
4476.3.54 by Andrew Bennetts
Update fallback-to-VFS in remote.py for the inventory-deltas substream.
1862
        However if any 'inventory-deltas' substream occurs it will stop
1863
        streaming, and store the interrupted substream and stream in
1864
        self._last_substream and self._last_stream so that the stream can be
1865
        resumed by _resume_stream_with_vfs.
4476.3.21 by Andrew Bennetts
Clarify some code and comments, and s/1.17/1.18/ in a few places.
1866
        """
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1867
                    
1868
        stream_iter = iter(stream)
1869
        for substream_kind, substream in stream_iter:
4476.3.54 by Andrew Bennetts
Update fallback-to-VFS in remote.py for the inventory-deltas substream.
1870
            if substream_kind == 'inventory-deltas':
1871
                self._last_substream = substream
1872
                self._last_stream = stream_iter
1873
                return
4476.3.15 by Andrew Bennetts
Partially working fallback for pre-1.17 servers.
1874
            else:
1875
                yield substream_kind, substream
1876
            
4022.1.1 by Robert Collins
Refactoring of fetch to have a sender and sink component enabling splitting the logic over a network stream. (Robert Collins, Andrew Bennetts)
1877
4060.1.3 by Robert Collins
Implement the separate source component for fetch - repository.StreamSource.
1878
class RemoteStreamSource(repository.StreamSource):
1879
    """Stream data from a remote server."""
1880
4060.1.4 by Robert Collins
Streaming fetch from remote servers.
1881
    def get_stream(self, search):
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1882
        if (self.from_repository._fallback_repositories and
1883
            self.to_format._fetch_order == 'topological'):
1884
            return self._real_stream(self.from_repository, search)
4577.1.1 by Robert Collins
Fix fetching from smart servers where there is a chain of stacked repositories rather than a single stacking point. (Robert Collins, bug #406597)
1885
        sources = []
1886
        seen = set()
1887
        repos = [self.from_repository]
1888
        while repos:
1889
            repo = repos.pop(0)
1890
            if repo in seen:
1891
                continue
1892
            seen.add(repo)
1893
            repos.extend(repo._fallback_repositories)
1894
            sources.append(repo)
1895
        return self.missing_parents_chain(search, sources)
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1896
4476.3.16 by Andrew Bennetts
Only make inv deltas against bases we've already sent, and other tweaks.
1897
    def get_stream_for_missing_keys(self, missing_keys):
1898
        self.from_repository._ensure_real()
1899
        real_repo = self.from_repository._real_repository
1900
        real_source = real_repo._get_source(self.to_format)
1901
        return real_source.get_stream_for_missing_keys(missing_keys)
1902
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1903
    def _real_stream(self, repo, search):
1904
        """Get a stream for search from repo.
1905
        
1906
        This never called RemoteStreamSource.get_stream, and is a heler
1907
        for RemoteStreamSource._get_stream to allow getting a stream 
1908
        reliably whether fallback back because of old servers or trying
1909
        to stream from a non-RemoteRepository (which the stacked support
1910
        code will do).
1911
        """
1912
        source = repo._get_source(self.to_format)
1913
        if isinstance(source, RemoteStreamSource):
4600.1.1 by Robert Collins
Fix fetching from 2a branches over bzr-v2.
1914
            repo._ensure_real()
1915
            source = repo._real_repository._get_source(self.to_format)
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1916
        return source.get_stream(search)
1917
1918
    def _get_stream(self, repo, search):
1919
        """Core worker to get a stream from repo for search.
1920
1921
        This is used by both get_stream and the stacking support logic. It
1922
        deliberately gets a stream for repo which does not need to be
1923
        self.from_repository. In the event that repo is not Remote, or
1924
        cannot do a smart stream, a fallback is made to the generic
1925
        repository._get_stream() interface, via self._real_stream.
1926
1927
        In the event of stacking, streams from _get_stream will not
1928
        contain all the data for search - this is normal (see get_stream).
1929
1930
        :param repo: A repository.
1931
        :param search: A search.
1932
        """
1933
        # Fallbacks may be non-smart
1934
        if not isinstance(repo, RemoteRepository):
1935
            return self._real_stream(repo, search)
4060.1.4 by Robert Collins
Streaming fetch from remote servers.
1936
        client = repo._client
1937
        medium = client._medium
1938
        path = repo.bzrdir._path_for_remote_call(client)
4476.3.29 by Andrew Bennetts
Add Repository.get_stream_1.18 verb.
1939
        search_bytes = repo._serialise_search_result(search)
1940
        args = (path, self.to_format.network_name())
1941
        candidate_verbs = [
4476.3.82 by Andrew Bennetts
Mention another bug fix in NEWS, and update verb name, comments, and NEWS additions for landing on 1.19 rather than 1.18.
1942
            ('Repository.get_stream_1.19', (1, 19)),
4476.3.29 by Andrew Bennetts
Add Repository.get_stream_1.18 verb.
1943
            ('Repository.get_stream', (1, 13))]
1944
        found_verb = False
1945
        for verb, version in candidate_verbs:
1946
            if medium._is_remote_before(version):
1947
                continue
1948
            try:
1949
                response = repo._call_with_body_bytes_expecting_body(
1950
                    verb, args, search_bytes)
1951
            except errors.UnknownSmartMethod:
1952
                medium._remember_remote_is_before(version)
1953
            else:
1954
                response_tuple, response_handler = response
1955
                found_verb = True
1956
                break
1957
        if not found_verb:
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1958
            return self._real_stream(repo, search)
4060.1.4 by Robert Collins
Streaming fetch from remote servers.
1959
        if response_tuple[0] != 'ok':
1960
            raise errors.UnexpectedSmartServerResponse(response_tuple)
1961
        byte_stream = response_handler.read_streamed_body()
1962
        src_format, stream = smart_repo._byte_stream_to_stream(byte_stream)
1963
        if src_format.network_name() != repo._format.network_name():
1964
            raise AssertionError(
1965
                "Mismatched RemoteRepository and stream src %r, %r" % (
1966
                src_format.network_name(), repo._format.network_name()))
1967
        return stream
1968
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1969
    def missing_parents_chain(self, search, sources):
1970
        """Chain multiple streams together to handle stacking.
1971
1972
        :param search: The overall search to satisfy with streams.
1973
        :param sources: A list of Repository objects to query.
1974
        """
4634.61.2 by Andrew Bennetts
Variable rename suggested by Robert.
1975
        self.from_serialiser = self.from_repository._format._serializer
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
1976
        self.seen_revs = set()
1977
        self.referenced_revs = set()
1978
        # If there are heads in the search, or the key count is > 0, we are not
1979
        # done.
1980
        while not search.is_empty() and len(sources) > 1:
1981
            source = sources.pop(0)
1982
            stream = self._get_stream(source, search)
1983
            for kind, substream in stream:
1984
                if kind != 'revisions':
1985
                    yield kind, substream
1986
                else:
1987
                    yield kind, self.missing_parents_rev_handler(substream)
1988
            search = search.refine(self.seen_revs, self.referenced_revs)
1989
            self.seen_revs = set()
1990
            self.referenced_revs = set()
1991
        if not search.is_empty():
1992
            for kind, stream in self._get_stream(sources[0], search):
1993
                yield kind, stream
1994
1995
    def missing_parents_rev_handler(self, substream):
1996
        for content in substream:
1997
            revision_bytes = content.get_bytes_as('fulltext')
4634.61.2 by Andrew Bennetts
Variable rename suggested by Robert.
1998
            revision = self.from_serialiser.read_revision_from_string(
1999
                revision_bytes)
4152.1.2 by Robert Collins
Add streaming from a stacked branch when the sort order is compatible with doing so.
2000
            self.seen_revs.add(content.key[-1])
2001
            self.referenced_revs.update(revision.parent_ids)
2002
            yield content
2003
4060.1.3 by Robert Collins
Implement the separate source component for fetch - repository.StreamSource.
2004
2018.5.127 by Andrew Bennetts
Fix most of the lockable_files tests for RemoteBranchLockableFiles.
2005
class RemoteBranchLockableFiles(LockableFiles):
2018.5.59 by Robert Collins
Get BranchConfig working somewhat on RemoteBranches (Robert Collins, Vincent Ladeuil).
2006
    """A 'LockableFiles' implementation that talks to a smart server.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
2007
2018.5.59 by Robert Collins
Get BranchConfig working somewhat on RemoteBranches (Robert Collins, Vincent Ladeuil).
2008
    This is not a public interface class.
2009
    """
2010
2011
    def __init__(self, bzrdir, _client):
2012
        self.bzrdir = bzrdir
2013
        self._client = _client
2018.5.135 by Andrew Bennetts
Prevent remote branch clients from determining the 'right' mode for control files, because we don't want clients setting the mode anyway.
2014
        self._need_find_modes = True
2018.5.133 by Andrew Bennetts
All TestLockableFiles_RemoteLockDir tests passing.
2015
        LockableFiles.__init__(
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
2016
            self, bzrdir.get_branch_transport(None),
2018.5.133 by Andrew Bennetts
All TestLockableFiles_RemoteLockDir tests passing.
2017
            'lock', lockdir.LockDir)
2018.5.59 by Robert Collins
Get BranchConfig working somewhat on RemoteBranches (Robert Collins, Vincent Ladeuil).
2018
2018.5.135 by Andrew Bennetts
Prevent remote branch clients from determining the 'right' mode for control files, because we don't want clients setting the mode anyway.
2019
    def _find_modes(self):
2020
        # RemoteBranches don't let the client set the mode of control files.
2021
        self._dir_mode = None
2022
        self._file_mode = None
2023
2018.5.59 by Robert Collins
Get BranchConfig working somewhat on RemoteBranches (Robert Collins, Vincent Ladeuil).
2024
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
2025
class RemoteBranchFormat(branch.BranchFormat):
2026
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2027
    def __init__(self, network_name=None):
3834.5.2 by John Arbash Meinel
Track down the various BranchFormats that weren't setting the branch format as part of the _matchingbzrdir format.
2028
        super(RemoteBranchFormat, self).__init__()
2029
        self._matchingbzrdir = RemoteBzrDirFormat()
2030
        self._matchingbzrdir.set_branch_format(self)
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2031
        self._custom_format = None
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2032
        self._network_name = network_name
3834.5.2 by John Arbash Meinel
Track down the various BranchFormats that weren't setting the branch format as part of the _matchingbzrdir format.
2033
2018.5.124 by Robert Collins
Fix test_format_initialize_find_open by delegating Branch formt lookup to the BzrDir, where it should have stayed from the start.
2034
    def __eq__(self, other):
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
2035
        return (isinstance(other, RemoteBranchFormat) and
2018.5.124 by Robert Collins
Fix test_format_initialize_find_open by delegating Branch formt lookup to the BzrDir, where it should have stayed from the start.
2036
            self.__dict__ == other.__dict__)
2037
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2038
    def _ensure_real(self):
2039
        if self._custom_format is None:
2040
            self._custom_format = branch.network_format_registry.get(
2041
                self._network_name)
2042
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
2043
    def get_format_description(self):
4792.1.1 by Andrew Bennetts
Show real branch/repo format description in 'info -v' over HPSS.
2044
        self._ensure_real()
2045
        return 'Remote: ' + self._custom_format.get_format_description()
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
2046
4032.3.1 by Robert Collins
Add a BranchFormat.network_name() method as preparation for creating branches via RPC calls.
2047
    def network_name(self):
2048
        return self._network_name
2049
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2050
    def open(self, a_bzrdir, name=None, ignore_fallbacks=False):
2051
        return a_bzrdir.open_branch(name=name, 
2052
            ignore_fallbacks=ignore_fallbacks)
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
2053
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2054
    def _vfs_initialize(self, a_bzrdir, name):
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2055
        # Initialisation when using a local bzrdir object, or a non-vfs init
2056
        # method is not available on the server.
2057
        # self._custom_format is always set - the start of initialize ensures
2058
        # that.
2059
        if isinstance(a_bzrdir, RemoteBzrDir):
2060
            a_bzrdir._ensure_real()
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2061
            result = self._custom_format.initialize(a_bzrdir._real_bzrdir,
2062
                name)
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2063
        else:
2064
            # We assume the bzrdir is parameterised; it may not be.
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2065
            result = self._custom_format.initialize(a_bzrdir, name)
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2066
        if (isinstance(a_bzrdir, RemoteBzrDir) and
2067
            not isinstance(result, RemoteBranch)):
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2068
            result = RemoteBranch(a_bzrdir, a_bzrdir.find_repository(), result,
2069
                                  name=name)
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2070
        return result
2071
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2072
    def initialize(self, a_bzrdir, name=None):
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2073
        # 1) get the network name to use.
2074
        if self._custom_format:
2075
            network_name = self._custom_format.network_name()
2076
        else:
2077
            # Select the current bzrlib default and ask for that.
2078
            reference_bzrdir_format = bzrdir.format_registry.get('default')()
2079
            reference_format = reference_bzrdir_format.get_branch_format()
2080
            self._custom_format = reference_format
2081
            network_name = reference_format.network_name()
2082
        # Being asked to create on a non RemoteBzrDir:
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2083
        if not isinstance(a_bzrdir, RemoteBzrDir):
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2084
            return self._vfs_initialize(a_bzrdir, name=name)
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2085
        medium = a_bzrdir._client._medium
2086
        if medium._is_remote_before((1, 13)):
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2087
            return self._vfs_initialize(a_bzrdir, name=name)
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2088
        # Creating on a remote bzr dir.
2089
        # 2) try direct creation via RPC
2090
        path = a_bzrdir._path_for_remote_call(a_bzrdir._client)
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2091
        if name is not None:
5051.3.11 by Jelmer Vernooij
add XXX for creating remote colocated branches.
2092
            # XXX JRV20100304: Support creating colocated branches
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2093
            raise errors.NoColocatedBranchSupport(self)
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2094
        verb = 'BzrDir.create_branch'
2095
        try:
2096
            response = a_bzrdir._call(verb, path, network_name)
2097
        except errors.UnknownSmartMethod:
2098
            # Fallback - use vfs methods
4094.1.1 by Andrew Bennetts
Add some medium._remember_is_before((1, 13)) calls.
2099
            medium._remember_remote_is_before((1, 13))
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2100
            return self._vfs_initialize(a_bzrdir, name=name)
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2101
        if response[0] != 'ok':
2102
            raise errors.UnexpectedSmartServerResponse(response)
2103
        # Turn the response into a RemoteRepository object.
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2104
        format = RemoteBranchFormat(network_name=response[1])
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2105
        repo_format = response_tuple_to_repo_format(response[3:])
2106
        if response[2] == '':
2107
            repo_bzrdir = a_bzrdir
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2108
        else:
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2109
            repo_bzrdir = RemoteBzrDir(
2110
                a_bzrdir.root_transport.clone(response[2]), a_bzrdir._format,
2111
                a_bzrdir._client)
2112
        remote_repo = RemoteRepository(repo_bzrdir, repo_format)
2113
        remote_branch = RemoteBranch(a_bzrdir, remote_repo,
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2114
            format=format, setup_stacking=False, name=name)
4044.1.4 by Robert Collins
Remove a wasted round trip determining the revno and revid of a newly created branch.
2115
        # XXX: We know this is a new branch, so it must have revno 0, revid
2116
        # NULL_REVISION. Creating the branch locked would make this be unable
2117
        # to be wrong; here its simply very unlikely to be wrong. RBC 20090225
2118
        remote_branch._last_revision_info_cache = 0, NULL_REVISION
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2119
        return remote_branch
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
2120
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2121
    def make_tags(self, branch):
2122
        self._ensure_real()
2123
        return self._custom_format.make_tags(branch)
2124
2696.3.6 by Martin Pool
Mark RemoteBranch as (possibly) supporting tags
2125
    def supports_tags(self):
2126
        # Remote branches might support tags, but we won't know until we
2127
        # access the real remote branch.
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2128
        self._ensure_real()
2129
        return self._custom_format.supports_tags()
2696.3.6 by Martin Pool
Mark RemoteBranch as (possibly) supporting tags
2130
4103.2.2 by Andrew Bennetts
Fix RemoteBranchFormat.supports_stacking()
2131
    def supports_stacking(self):
2132
        self._ensure_real()
2133
        return self._custom_format.supports_stacking()
2134
4301.3.3 by Andrew Bennetts
Move check onto base Branch class, and add a supports_set_append_revisions_only method to BranchFormat, as suggested by Robert.
2135
    def supports_set_append_revisions_only(self):
2136
        self._ensure_real()
2137
        return self._custom_format.supports_set_append_revisions_only()
2138
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
2139
4731.1.2 by Andrew Bennetts
Refactor to reduce duplication.
2140
class RemoteBranch(branch.Branch, _RpcHelper, lock._RelockDebugMixin):
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
2141
    """Branch stored on a server accessed by HPSS RPC.
2142
2143
    At the moment most operations are mapped down to simple file operations.
2144
    """
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
2145
2018.5.51 by Wouter van Heyst
Test and implement RemoteBranch.last_revision_info()
2146
    def __init__(self, remote_bzrdir, remote_repository, real_branch=None,
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2147
        _client=None, format=None, setup_stacking=True, name=None):
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
2148
        """Create a RemoteBranch instance.
2149
2150
        :param real_branch: An optional local implementation of the branch
2151
            format, usually accessing the data via the VFS.
2018.5.51 by Wouter van Heyst
Test and implement RemoteBranch.last_revision_info()
2152
        :param _client: Private parameter for testing.
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2153
        :param format: A RemoteBranchFormat object, None to create one
2154
            automatically. If supplied it should have a network_name already
2155
            supplied.
2156
        :param setup_stacking: If True make an RPC call to determine the
2157
            stacked (or not) status of the branch. If False assume the branch
2158
            is not stacked.
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2159
        :param name: Colocated branch name
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
2160
        """
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
2161
        # We intentionally don't call the parent class's __init__, because it
2162
        # will try to assign to self.tags, which is a property in this subclass.
2163
        # And the parent's __init__ doesn't do much anyway.
1752.2.64 by Andrew Bennetts
Improve how RemoteBzrDir.open_branch works to handle references and not double-open repositories.
2164
        self.bzrdir = remote_bzrdir
2018.5.51 by Wouter van Heyst
Test and implement RemoteBranch.last_revision_info()
2165
        if _client is not None:
2166
            self._client = _client
2167
        else:
3313.2.1 by Andrew Bennetts
Change _SmartClient's API to accept a medium and a base, rather than a _SharedConnection.
2168
            self._client = remote_bzrdir._client
1752.2.64 by Andrew Bennetts
Improve how RemoteBzrDir.open_branch works to handle references and not double-open repositories.
2169
        self.repository = remote_repository
2018.5.34 by Robert Collins
Get test_remote.BasicRemoteObjectTests.test_open_remote_branch passing by implementing a remote method BzrDir.find_repository.
2170
        if real_branch is not None:
2171
            self._real_branch = real_branch
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2172
            # Give the remote repository the matching real repo.
2018.5.97 by Andrew Bennetts
Fix more tests.
2173
            real_repo = self._real_branch.repository
2174
            if isinstance(real_repo, RemoteRepository):
2175
                real_repo._ensure_real()
2176
                real_repo = real_repo._real_repository
2177
            self.repository._set_real_repository(real_repo)
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2178
            # Give the branch the remote repository to let fast-pathing happen.
2179
            self._real_branch.repository = self.repository
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
2180
        else:
2181
            self._real_branch = None
4031.3.3 by Matt Nordhoff
Review tweaks from Ben Finney
2182
        # Fill out expected attributes of branch for bzrlib API users.
4419.2.2 by Andrew Bennetts
Read lock branch_from in cmd_pull, avoids refetching last_revision_info and so reduces test_pull acceptance ratchet.
2183
        self._clear_cached_state()
5158.6.9 by Martin Pool
Simplify various code to use user_url
2184
        # TODO: deprecate self.base in favor of user_url
2185
        self.base = self.bzrdir.user_url
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2186
        self._name = name
2018.5.169 by Andrew Bennetts
Add a _server_formats flag to BzrDir.open_from_transport and BzrDirFormat.find_format, make RemoteBranch.control_files into a property.
2187
        self._control_files = None
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2188
        self._lock_mode = None
2189
        self._lock_token = None
2892.2.1 by Andrew Bennetts
Add Branch.set_last_revision_info smart method, and make the RemoteBranch client use it.
2190
        self._repo_lock_token = None
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2191
        self._lock_count = 0
2192
        self._leave_lock = False
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2193
        # Setup a format: note that we cannot call _ensure_real until all the
2194
        # attributes above are set: This code cannot be moved higher up in this
2195
        # function.
2196
        if format is None:
2197
            self._format = RemoteBranchFormat()
2198
            if real_branch is not None:
2199
                self._format._network_name = \
2200
                    self._real_branch._format.network_name()
4032.3.1 by Robert Collins
Add a BranchFormat.network_name() method as preparation for creating branches via RPC calls.
2201
        else:
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2202
            self._format = format
4600.2.1 by Robert Collins
Make RemoteBranch honour ignore_fallbacks when _ensure_real is triggered.
2203
        # when we do _ensure_real we may need to pass ignore_fallbacks to the
2204
        # branch.open_branch method.
2205
        self._real_ignore_fallbacks = not setup_stacking
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2206
        if not self._format._network_name:
2207
            # Did not get from open_branchV2 - old server.
2208
            self._ensure_real()
2209
            self._format._network_name = \
2210
                self._real_branch._format.network_name()
2211
        self.tags = self._format.make_tags(self)
3681.1.2 by Robert Collins
Adjust for trunk.
2212
        # The base class init is not called, so we duplicate this:
3681.1.1 by Robert Collins
Create a new hook Branch.open. (Robert Collins)
2213
        hooks = branch.Branch.hooks['open']
2214
        for hook in hooks:
2215
            hook(self)
4419.1.3 by Andrew Bennetts
Quick fix by using self._ensure_real.
2216
        self._is_stacked = False
4032.3.2 by Robert Collins
Create and use a RPC call to create branches on bzr servers rather than using VFS calls.
2217
        if setup_stacking:
2218
            self._setup_stacking()
3691.2.1 by Martin Pool
RemoteBranch must configure stacking into the repository
2219
2220
    def _setup_stacking(self):
2221
        # configure stacking into the remote repository, by reading it from
3691.2.3 by Martin Pool
Factor out RemoteBranch._remote_path() and disable RemoteBranch stacking
2222
        # the vfs branch.
3691.2.1 by Martin Pool
RemoteBranch must configure stacking into the repository
2223
        try:
2224
            fallback_url = self.get_stacked_on_url()
2225
        except (errors.NotStacked, errors.UnstackableBranchFormat,
2226
            errors.UnstackableRepositoryFormat), e:
3691.2.7 by Martin Pool
FakeClient can know what calls to expect
2227
            return
4419.1.3 by Andrew Bennetts
Quick fix by using self._ensure_real.
2228
        self._is_stacked = True
4379.2.2 by John Arbash Meinel
Change the Repository.add_fallback_repository() contract slightly.
2229
        self._activate_fallback_location(fallback_url)
1752.2.64 by Andrew Bennetts
Improve how RemoteBzrDir.open_branch works to handle references and not double-open repositories.
2230
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
2231
    def _get_config(self):
2232
        return RemoteBranchConfig(self)
2233
3407.2.17 by Martin Pool
better name: _get_real_transport
2234
    def _get_real_transport(self):
3407.2.16 by Martin Pool
Remove RemoteBranch reliance on control_files._transport
2235
        # if we try vfs access, return the real branch's vfs transport
2236
        self._ensure_real()
2237
        return self._real_branch._transport
2238
3407.2.17 by Martin Pool
better name: _get_real_transport
2239
    _transport = property(_get_real_transport)
3407.2.16 by Martin Pool
Remove RemoteBranch reliance on control_files._transport
2240
2477.1.1 by Martin Pool
Add RemoteBranch repr
2241
    def __str__(self):
2242
        return "%s(%s)" % (self.__class__.__name__, self.base)
2243
2244
    __repr__ = __str__
2245
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
2246
    def _ensure_real(self):
2247
        """Ensure that there is a _real_branch set.
2248
2018.5.163 by Andrew Bennetts
Deal with various review comments from Robert.
2249
        Used before calls to self._real_branch.
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
2250
        """
3407.2.16 by Martin Pool
Remove RemoteBranch reliance on control_files._transport
2251
        if self._real_branch is None:
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
2252
            if not vfs.vfs_enabled():
2253
                raise AssertionError('smart server vfs must be enabled '
2254
                    'to use vfs implementation')
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
2255
            self.bzrdir._ensure_real()
4600.2.1 by Robert Collins
Make RemoteBranch honour ignore_fallbacks when _ensure_real is triggered.
2256
            self._real_branch = self.bzrdir._real_bzrdir.open_branch(
5051.3.10 by Jelmer Vernooij
Pass colocated branch name around in more places.
2257
                ignore_fallbacks=self._real_ignore_fallbacks, name=self._name)
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
2258
            if self.repository._real_repository is None:
2259
                # Give the remote repository the matching real repo.
2260
                real_repo = self._real_branch.repository
2261
                if isinstance(real_repo, RemoteRepository):
2262
                    real_repo._ensure_real()
2263
                    real_repo = real_repo._real_repository
2264
                self.repository._set_real_repository(real_repo)
2265
            # Give the real branch the remote repository to let fast-pathing
2266
            # happen.
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
2267
            self._real_branch.repository = self.repository
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
2268
            if self._lock_mode == 'r':
2269
                self._real_branch.lock_read()
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
2270
            elif self._lock_mode == 'w':
2271
                self._real_branch.lock_write(token=self._lock_token)
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
2272
3533.3.1 by Andrew Bennetts
Remove duplication of error translation in bzrlib/remote.py.
2273
    def _translate_error(self, err, **context):
2274
        self.repository._translate_error(err, branch=self, **context)
2275
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2276
    def _clear_cached_state(self):
2277
        super(RemoteBranch, self)._clear_cached_state()
3441.5.5 by Andrew Bennetts
Some small tweaks and comments.
2278
        if self._real_branch is not None:
2279
            self._real_branch._clear_cached_state()
3441.5.29 by Andrew Bennetts
More review tweaks: whitespace nits in test_smart, add (and use) ._clear_cached_state_of_remote_branch_only method in bzrlib/remote.py.
2280
2281
    def _clear_cached_state_of_remote_branch_only(self):
2282
        """Like _clear_cached_state, but doesn't clear the cache of
2283
        self._real_branch.
2284
2285
        This is useful when falling back to calling a method of
2286
        self._real_branch that changes state.  In that case the underlying
2287
        branch changes, so we need to invalidate this RemoteBranch's cache of
2288
        it.  However, there's no need to invalidate the _real_branch's cache
2289
        too, in fact doing so might harm performance.
2290
        """
2291
        super(RemoteBranch, self)._clear_cached_state()
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
2292
2018.5.169 by Andrew Bennetts
Add a _server_formats flag to BzrDir.open_from_transport and BzrDirFormat.find_format, make RemoteBranch.control_files into a property.
2293
    @property
2294
    def control_files(self):
2295
        # Defer actually creating RemoteBranchLockableFiles until its needed,
2296
        # because it triggers an _ensure_real that we otherwise might not need.
2297
        if self._control_files is None:
2298
            self._control_files = RemoteBranchLockableFiles(
2299
                self.bzrdir, self._client)
2300
        return self._control_files
2301
2018.5.166 by Andrew Bennetts
Small changes in response to Aaron's review.
2302
    def _get_checkout_format(self):
2303
        self._ensure_real()
2304
        return self._real_branch._get_checkout_format()
2305
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
2306
    def get_physical_lock_status(self):
2307
        """See Branch.get_physical_lock_status()."""
2308
        # should be an API call to the server, as branches must be lockable.
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
2309
        self._ensure_real()
2018.5.60 by Robert Collins
More missing methods from RemoteBranch and RemoteRepository to let 'info' get further.
2310
        return self._real_branch.get_physical_lock_status()
2311
3537.3.1 by Martin Pool
Rename branch.get_stacked_on to get_stacked_on_url
2312
    def get_stacked_on_url(self):
3221.11.2 by Robert Collins
Create basic stackable branch facility.
2313
        """Get the URL this branch is stacked against.
2314
2315
        :raises NotStacked: If the branch is not stacked.
2316
        :raises UnstackableBranchFormat: If the branch does not support
2317
            stacking.
2318
        :raises UnstackableRepositoryFormat: If the repository does not support
2319
            stacking.
2320
        """
3691.2.12 by Martin Pool
Add test for coping without Branch.get_stacked_on_url
2321
        try:
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2322
            # there may not be a repository yet, so we can't use
2323
            # self._translate_error, so we can't use self._call either.
3691.2.12 by Martin Pool
Add test for coping without Branch.get_stacked_on_url
2324
            response = self._client.call('Branch.get_stacked_on_url',
2325
                self._remote_path())
2326
        except errors.ErrorFromSmartServer, err:
2327
            # there may not be a repository yet, so we can't call through
2328
            # its _translate_error
2329
            _translate_error(err, branch=self)
2330
        except errors.UnknownSmartMethod, err:
2331
            self._ensure_real()
2332
            return self._real_branch.get_stacked_on_url()
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2333
        if response[0] != 'ok':
2334
            raise errors.UnexpectedSmartServerResponse(response)
2335
        return response[1]
3221.11.2 by Robert Collins
Create basic stackable branch facility.
2336
4419.1.3 by Andrew Bennetts
Quick fix by using self._ensure_real.
2337
    def set_stacked_on_url(self, url):
2338
        branch.Branch.set_stacked_on_url(self, url)
2339
        if not url:
2340
            self._is_stacked = False
2341
        else:
2342
            self._is_stacked = True
2343
        
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2344
    def _vfs_get_tags_bytes(self):
2345
        self._ensure_real()
2346
        return self._real_branch._get_tags_bytes()
2347
2348
    def _get_tags_bytes(self):
2349
        medium = self._client._medium
2350
        if medium._is_remote_before((1, 13)):
2351
            return self._vfs_get_tags_bytes()
2352
        try:
2353
            response = self._call('Branch.get_tags_bytes', self._remote_path())
2354
        except errors.UnknownSmartMethod:
4094.1.1 by Andrew Bennetts
Add some medium._remember_is_before((1, 13)) calls.
2355
            medium._remember_remote_is_before((1, 13))
4084.2.1 by Robert Collins
Make accessing a branch.tags.get_tag_dict use a smart[er] method rather than VFS calls and real objects.
2356
            return self._vfs_get_tags_bytes()
2357
        return response[0]
2358
4556.2.10 by Andrew Bennetts
Fix trivial bug in _vfs_set_tags_bytes.
2359
    def _vfs_set_tags_bytes(self, bytes):
4556.2.1 by Andrew Bennetts
Add Branch.set_tags_bytes RPC, with HPSS call count acceptance test. Also fixes serialisation of LockDir, and uses external_url() in LockDir's repr and contention message.
2360
        self._ensure_real()
4556.2.10 by Andrew Bennetts
Fix trivial bug in _vfs_set_tags_bytes.
2361
        return self._real_branch._set_tags_bytes(bytes)
4556.2.1 by Andrew Bennetts
Add Branch.set_tags_bytes RPC, with HPSS call count acceptance test. Also fixes serialisation of LockDir, and uses external_url() in LockDir's repr and contention message.
2362
2363
    def _set_tags_bytes(self, bytes):
2364
        medium = self._client._medium
2365
        if medium._is_remote_before((1, 18)):
2366
            self._vfs_set_tags_bytes(bytes)
4634.36.1 by Andrew Bennetts
Fix trivial bug in RemoteBranch._set_tags_bytes, and add some unit tests for it.
2367
            return
4556.2.1 by Andrew Bennetts
Add Branch.set_tags_bytes RPC, with HPSS call count acceptance test. Also fixes serialisation of LockDir, and uses external_url() in LockDir's repr and contention message.
2368
        try:
2369
            args = (
2370
                self._remote_path(), self._lock_token, self._repo_lock_token)
2371
            response = self._call_with_body_bytes(
2372
                'Branch.set_tags_bytes', args, bytes)
2373
        except errors.UnknownSmartMethod:
2374
            medium._remember_remote_is_before((1, 18))
2375
            self._vfs_set_tags_bytes(bytes)
2376
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
2377
    def lock_read(self):
3692.1.1 by Andrew Bennetts
Make RemoteBranch.lock_write lock the repository too.
2378
        self.repository.lock_read()
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
2379
        if not self._lock_mode:
4731.1.2 by Andrew Bennetts
Refactor to reduce duplication.
2380
            self._note_lock('r')
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
2381
            self._lock_mode = 'r'
2382
            self._lock_count = 1
2383
            if self._real_branch is not None:
2384
                self._real_branch.lock_read()
2385
        else:
2386
            self._lock_count += 1
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
2387
2018.5.142 by Andrew Bennetts
Change Branch.lock_token to only accept and receive the branch lock token (rather than the branch and repo lock tokens).
2388
    def _remote_lock_write(self, token):
2389
        if token is None:
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2390
            branch_token = repo_token = ''
2391
        else:
2018.5.142 by Andrew Bennetts
Change Branch.lock_token to only accept and receive the branch lock token (rather than the branch and repo lock tokens).
2392
            branch_token = token
2393
            repo_token = self.repository.lock_write()
2394
            self.repository.unlock()
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2395
        err_context = {'token': token}
2396
        response = self._call(
2397
            'Branch.lock_write', self._remote_path(), branch_token,
2398
            repo_token or '', **err_context)
3245.4.24 by Andrew Bennetts
Consistently raise errors from the server as ErrorFromSmartServer exceptions.
2399
        if response[0] != 'ok':
2555.1.1 by Martin Pool
Remove use of 'assert False' to raise an exception unconditionally
2400
            raise errors.UnexpectedSmartServerResponse(response)
3245.4.24 by Andrew Bennetts
Consistently raise errors from the server as ErrorFromSmartServer exceptions.
2401
        ok, branch_token, repo_token = response
2402
        return branch_token, repo_token
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
2403
2018.5.142 by Andrew Bennetts
Change Branch.lock_token to only accept and receive the branch lock token (rather than the branch and repo lock tokens).
2404
    def lock_write(self, token=None):
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2405
        if not self._lock_mode:
4731.1.2 by Andrew Bennetts
Refactor to reduce duplication.
2406
            self._note_lock('w')
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
2407
            # Lock the branch and repo in one remote call.
2018.5.142 by Andrew Bennetts
Change Branch.lock_token to only accept and receive the branch lock token (rather than the branch and repo lock tokens).
2408
            remote_tokens = self._remote_lock_write(token)
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2409
            self._lock_token, self._repo_lock_token = remote_tokens
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
2410
            if not self._lock_token:
2411
                raise SmartProtocolError('Remote server did not return a token!')
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
2412
            # Tell the self.repository object that it is locked.
3692.1.2 by Andrew Bennetts
Fix regression introduced by fix, and add a test for that regression.
2413
            self.repository.lock_write(
2414
                self._repo_lock_token, _skip_rpc=True)
3692.1.1 by Andrew Bennetts
Make RemoteBranch.lock_write lock the repository too.
2415
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2416
            if self._real_branch is not None:
3692.1.5 by Andrew Bennetts
Fix bug revealed by removing _ensure_real call from RemoteBranch.lock_write.
2417
                self._real_branch.lock_write(token=self._lock_token)
2018.5.142 by Andrew Bennetts
Change Branch.lock_token to only accept and receive the branch lock token (rather than the branch and repo lock tokens).
2418
            if token is not None:
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2419
                self._leave_lock = True
2420
            else:
2421
                self._leave_lock = False
2422
            self._lock_mode = 'w'
2423
            self._lock_count = 1
2424
        elif self._lock_mode == 'r':
2425
            raise errors.ReadOnlyTransaction
2426
        else:
2018.5.142 by Andrew Bennetts
Change Branch.lock_token to only accept and receive the branch lock token (rather than the branch and repo lock tokens).
2427
            if token is not None:
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
2428
                # A token was given to lock_write, and we're relocking, so
2429
                # check that the given token actually matches the one we
2430
                # already have.
2018.5.142 by Andrew Bennetts
Change Branch.lock_token to only accept and receive the branch lock token (rather than the branch and repo lock tokens).
2431
                if token != self._lock_token:
2432
                    raise errors.TokenMismatch(token, self._lock_token)
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2433
            self._lock_count += 1
3692.1.3 by Andrew Bennetts
Delete some cruft (like the _ensure_real call in RemoteBranch.lock_write), improve some comments, and wrap some long lines.
2434
            # Re-lock the repository too.
3692.1.2 by Andrew Bennetts
Fix regression introduced by fix, and add a test for that regression.
2435
            self.repository.lock_write(self._repo_lock_token)
3015.2.9 by Robert Collins
Handle repositories that do not allow remote locking, like pack repositories, in the client side remote server proxy objects.
2436
        return self._lock_token or None
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2437
2438
    def _unlock(self, branch_token, repo_token):
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2439
        err_context = {'token': str((branch_token, repo_token))}
2440
        response = self._call(
2441
            'Branch.unlock', self._remote_path(), branch_token,
2442
            repo_token or '', **err_context)
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2443
        if response == ('ok',):
2444
            return
3245.4.24 by Andrew Bennetts
Consistently raise errors from the server as ErrorFromSmartServer exceptions.
2445
        raise errors.UnexpectedSmartServerResponse(response)
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
2446
4634.85.9 by Andrew Bennetts
Add some experimental decorators: @only_raises(..) and @cleanup_method.
2447
    @only_raises(errors.LockNotHeld, errors.LockBroken)
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
2448
    def unlock(self):
3692.1.1 by Andrew Bennetts
Make RemoteBranch.lock_write lock the repository too.
2449
        try:
2450
            self._lock_count -= 1
2451
            if not self._lock_count:
2452
                self._clear_cached_state()
2453
                mode = self._lock_mode
2454
                self._lock_mode = None
2455
                if self._real_branch is not None:
2456
                    if (not self._leave_lock and mode == 'w' and
2457
                        self._repo_lock_token):
2458
                        # If this RemoteBranch will remove the physical lock
2459
                        # for the repository, make sure the _real_branch
2460
                        # doesn't do it first.  (Because the _real_branch's
2461
                        # repository is set to be the RemoteRepository.)
2462
                        self._real_branch.repository.leave_lock_in_place()
2463
                    self._real_branch.unlock()
2464
                if mode != 'w':
2465
                    # Only write-locked branched need to make a remote method
4031.3.1 by Frank Aspell
Fixing various typos
2466
                    # call to perform the unlock.
3692.1.1 by Andrew Bennetts
Make RemoteBranch.lock_write lock the repository too.
2467
                    return
2468
                if not self._lock_token:
2469
                    raise AssertionError('Locked, but no token!')
2470
                branch_token = self._lock_token
2471
                repo_token = self._repo_lock_token
2472
                self._lock_token = None
2473
                self._repo_lock_token = None
2474
                if not self._leave_lock:
2475
                    self._unlock(branch_token, repo_token)
2476
        finally:
2477
            self.repository.unlock()
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
2478
2479
    def break_lock(self):
2018.5.70 by Robert Collins
Only try to get real repositories when an operation requires them.
2480
        self._ensure_real()
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
2481
        return self._real_branch.break_lock()
1752.2.31 by Martin Pool
[broken] some support for write operations over hpss
2482
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2483
    def leave_lock_in_place(self):
3015.2.9 by Robert Collins
Handle repositories that do not allow remote locking, like pack repositories, in the client side remote server proxy objects.
2484
        if not self._lock_token:
2485
            raise NotImplementedError(self.leave_lock_in_place)
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2486
        self._leave_lock = True
2487
2488
    def dont_leave_lock_in_place(self):
3015.2.9 by Robert Collins
Handle repositories that do not allow remote locking, like pack repositories, in the client side remote server proxy objects.
2489
        if not self._lock_token:
3015.2.15 by Robert Collins
Review feedback.
2490
            raise NotImplementedError(self.dont_leave_lock_in_place)
2018.5.79 by Andrew Bennetts
Implement RemoteBranch.lock_write/unlock as smart operations.
2491
        self._leave_lock = False
2492
4634.69.1 by Andrew Bennetts
Apply @needs_read_lock to RemoteBranch.get_rev_id.
2493
    @needs_read_lock
4419.2.4 by Andrew Bennetts
Add Repository.get_rev_id_for_revno RPC, removes VFS calls from 'pull -r 123' case.
2494
    def get_rev_id(self, revno, history=None):
4419.2.17 by Andrew Bennetts
Fix test failures in test_lookup_revision_id_by_dotted.
2495
        if revno == 0:
2496
            return _mod_revision.NULL_REVISION
4419.2.4 by Andrew Bennetts
Add Repository.get_rev_id_for_revno RPC, removes VFS calls from 'pull -r 123' case.
2497
        last_revision_info = self.last_revision_info()
4419.2.15 by Andrew Bennetts
Simplify RemoteBranch.get_rev_id a little; get_rev_id_for_revno handles stacking for us.
2498
        ok, result = self.repository.get_rev_id_for_revno(
2499
            revno, last_revision_info)
2500
        if ok:
2501
            return result
2502
        missing_parent = result[1]
4419.2.17 by Andrew Bennetts
Fix test failures in test_lookup_revision_id_by_dotted.
2503
        # Either the revision named by the server is missing, or its parent
2504
        # is.  Call get_parent_map to determine which, so that we report a
2505
        # useful error.
2506
        parent_map = self.repository.get_parent_map([missing_parent])
2507
        if missing_parent in parent_map:
2508
            missing_parent = parent_map[missing_parent]
4419.2.4 by Andrew Bennetts
Add Repository.get_rev_id_for_revno RPC, removes VFS calls from 'pull -r 123' case.
2509
        raise errors.RevisionNotPresent(missing_parent, self.repository)
2510
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2511
    def _last_revision_info(self):
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2512
        response = self._call('Branch.last_revision_info', self._remote_path())
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
2513
        if response[0] != 'ok':
2514
            raise SmartProtocolError('unexpected response code %s' % (response,))
2018.5.51 by Wouter van Heyst
Test and implement RemoteBranch.last_revision_info()
2515
        revno = int(response[1])
2018.5.83 by Andrew Bennetts
Fix some test failures caused by the switch from unicode to UTF-8-encoded strs for revision IDs.
2516
        last_revision = response[2]
2018.5.51 by Wouter van Heyst
Test and implement RemoteBranch.last_revision_info()
2517
        return (revno, last_revision)
2518
2018.5.105 by Andrew Bennetts
Implement revision_history caching for RemoteBranch.
2519
    def _gen_revision_history(self):
2520
        """See Branch._gen_revision_history()."""
4419.1.3 by Andrew Bennetts
Quick fix by using self._ensure_real.
2521
        if self._is_stacked:
2522
            self._ensure_real()
2523
            return self._real_branch._gen_revision_history()
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2524
        response_tuple, response_handler = self._call_expecting_body(
3691.2.3 by Martin Pool
Factor out RemoteBranch._remote_path() and disable RemoteBranch stacking
2525
            'Branch.revision_history', self._remote_path())
3245.4.58 by Andrew Bennetts
Unpack call_expecting_body's return value into variables, to avoid lots of ugly subscripting.
2526
        if response_tuple[0] != 'ok':
3452.2.2 by Andrew Bennetts
Experimental PackRepository.{check_references,autopack} RPCs.
2527
            raise errors.UnexpectedSmartServerResponse(response_tuple)
3245.4.58 by Andrew Bennetts
Unpack call_expecting_body's return value into variables, to avoid lots of ugly subscripting.
2528
        result = response_handler.read_body_bytes().split('\x00')
2018.5.38 by Robert Collins
Implement RemoteBranch.revision_history().
2529
        if result == ['']:
2530
            return []
2531
        return result
1752.2.30 by Martin Pool
Start adding a RemoteBzrDir, etc
2532
3691.2.3 by Martin Pool
Factor out RemoteBranch._remote_path() and disable RemoteBranch stacking
2533
    def _remote_path(self):
2534
        return self.bzrdir._path_for_remote_call(self._client)
2535
3441.5.18 by Andrew Bennetts
Fix some test failures.
2536
    def _set_last_revision_descendant(self, revision_id, other_branch,
3441.5.28 by Andrew Bennetts
Another review tweak: rename do_not_overwrite_descendant to allow_overwrite_descendant.
2537
            allow_diverged=False, allow_overwrite_descendant=False):
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2538
        # This performs additional work to meet the hook contract; while its
2539
        # undesirable, we have to synthesise the revno to call the hook, and
2540
        # not calling the hook is worse as it means changes can't be prevented.
2541
        # Having calculated this though, we can't just call into
2542
        # set_last_revision_info as a simple call, because there is a set_rh
2543
        # hook that some folk may still be using.
2544
        old_revno, old_revid = self.last_revision_info()
2545
        history = self._lefthand_history(revision_id)
2546
        self._run_pre_change_branch_tip_hooks(len(history), revision_id)
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2547
        err_context = {'other_branch': other_branch}
2548
        response = self._call('Branch.set_last_revision_ex',
2549
            self._remote_path(), self._lock_token, self._repo_lock_token,
2550
            revision_id, int(allow_diverged), int(allow_overwrite_descendant),
2551
            **err_context)
3441.5.6 by Andrew Bennetts
Greatly simplify RemoteBranch.update_revisions. Still needs more tests.
2552
        self._clear_cached_state()
3441.5.18 by Andrew Bennetts
Fix some test failures.
2553
        if len(response) != 3 and response[0] != 'ok':
3441.5.6 by Andrew Bennetts
Greatly simplify RemoteBranch.update_revisions. Still needs more tests.
2554
            raise errors.UnexpectedSmartServerResponse(response)
3441.5.18 by Andrew Bennetts
Fix some test failures.
2555
        new_revno, new_revision_id = response[1:]
2556
        self._last_revision_info_cache = new_revno, new_revision_id
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2557
        self._run_post_change_branch_tip_hooks(old_revno, old_revid)
3692.1.5 by Andrew Bennetts
Fix bug revealed by removing _ensure_real call from RemoteBranch.lock_write.
2558
        if self._real_branch is not None:
2559
            cache = new_revno, new_revision_id
2560
            self._real_branch._last_revision_info_cache = cache
3441.5.6 by Andrew Bennetts
Greatly simplify RemoteBranch.update_revisions. Still needs more tests.
2561
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2562
    def _set_last_revision(self, revision_id):
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2563
        old_revno, old_revid = self.last_revision_info()
2564
        # This performs additional work to meet the hook contract; while its
2565
        # undesirable, we have to synthesise the revno to call the hook, and
2566
        # not calling the hook is worse as it means changes can't be prevented.
2567
        # Having calculated this though, we can't just call into
2568
        # set_last_revision_info as a simple call, because there is a set_rh
2569
        # hook that some folk may still be using.
2570
        history = self._lefthand_history(revision_id)
2571
        self._run_pre_change_branch_tip_hooks(len(history), revision_id)
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2572
        self._clear_cached_state()
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2573
        response = self._call('Branch.set_last_revision',
2574
            self._remote_path(), self._lock_token, self._repo_lock_token,
2575
            revision_id)
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2576
        if response != ('ok',):
2577
            raise errors.UnexpectedSmartServerResponse(response)
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2578
        self._run_post_change_branch_tip_hooks(old_revno, old_revid)
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2579
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
2580
    @needs_write_lock
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
2581
    def set_revision_history(self, rev_history):
2018.12.3 by Andrew Bennetts
Add a Branch.set_last_revision smart method, and make RemoteBranch.set_revision_history use it.
2582
        # Send just the tip revision of the history; the server will generate
2583
        # the full history from that.  If the revision doesn't exist in this
2584
        # branch, NoSuchRevision will be raised.
2585
        if rev_history == []:
2018.5.170 by Andrew Bennetts
Use 'null:' instead of '' to mean NULL_REVISION on the wire.
2586
            rev_id = 'null:'
2018.12.3 by Andrew Bennetts
Add a Branch.set_last_revision smart method, and make RemoteBranch.set_revision_history use it.
2587
        else:
2588
            rev_id = rev_history[-1]
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2589
        self._set_last_revision(rev_id)
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2590
        for hook in branch.Branch.hooks['set_rh']:
2591
            hook(self, rev_history)
2018.5.105 by Andrew Bennetts
Implement revision_history caching for RemoteBranch.
2592
        self._cache_revision_history(rev_history)
1752.2.52 by Andrew Bennetts
Flesh out more Remote* methods needed to open and initialise remote branches/trees/repositories.
2593
4078.2.1 by Robert Collins
Add a Branch.get_parent remote call for RemoteBranch.
2594
    def _get_parent_location(self):
2595
        medium = self._client._medium
2596
        if medium._is_remote_before((1, 13)):
2597
            return self._vfs_get_parent_location()
2598
        try:
4083.1.4 by Andrew Bennetts
Fix trivial bug in get_parent RPC.
2599
            response = self._call('Branch.get_parent', self._remote_path())
4078.2.1 by Robert Collins
Add a Branch.get_parent remote call for RemoteBranch.
2600
        except errors.UnknownSmartMethod:
4094.1.1 by Andrew Bennetts
Add some medium._remember_is_before((1, 13)) calls.
2601
            medium._remember_remote_is_before((1, 13))
4078.2.1 by Robert Collins
Add a Branch.get_parent remote call for RemoteBranch.
2602
            return self._vfs_get_parent_location()
4083.1.4 by Andrew Bennetts
Fix trivial bug in get_parent RPC.
2603
        if len(response) != 1:
4083.1.6 by Andrew Bennetts
Fix trivial bug in my trivial bug fix :)
2604
            raise errors.UnexpectedSmartServerResponse(response)
4083.1.4 by Andrew Bennetts
Fix trivial bug in get_parent RPC.
2605
        parent_location = response[0]
2606
        if parent_location == '':
4078.2.1 by Robert Collins
Add a Branch.get_parent remote call for RemoteBranch.
2607
            return None
4083.1.4 by Andrew Bennetts
Fix trivial bug in get_parent RPC.
2608
        return parent_location
4078.2.1 by Robert Collins
Add a Branch.get_parent remote call for RemoteBranch.
2609
2610
    def _vfs_get_parent_location(self):
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2611
        self._ensure_real()
4083.1.5 by Andrew Bennetts
Fix trivial bug in get_parent RPC.
2612
        return self._real_branch._get_parent_location()
4032.1.1 by John Arbash Meinel
Merge the removal of all trailing whitespace, and resolve conflicts.
2613
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2614
    def _set_parent_location(self, url):
4288.1.7 by Robert Collins
Add new remote server verb Branch.set_parent_location, dropping roundtrips further on push operations.
2615
        medium = self._client._medium
2616
        if medium._is_remote_before((1, 15)):
2617
            return self._vfs_set_parent_location(url)
2618
        try:
2619
            call_url = url or ''
2620
            if type(call_url) is not str:
2621
                raise AssertionError('url must be a str or None (%s)' % url)
2622
            response = self._call('Branch.set_parent_location',
2623
                self._remote_path(), self._lock_token, self._repo_lock_token,
2624
                call_url)
2625
        except errors.UnknownSmartMethod:
2626
            medium._remember_remote_is_before((1, 15))
2627
            return self._vfs_set_parent_location(url)
2628
        if response != ():
2629
            raise errors.UnexpectedSmartServerResponse(response)
2630
2631
    def _vfs_set_parent_location(self, url):
4288.1.4 by Robert Collins
Remove the explicit set_parent method on RemoteBranch in favour of inheriting from Branch.
2632
        self._ensure_real()
2633
        return self._real_branch._set_parent_location(url)
4032.1.1 by John Arbash Meinel
Merge the removal of all trailing whitespace, and resolve conflicts.
2634
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
2635
    @needs_write_lock
2477.1.2 by Martin Pool
Rename push/pull back to 'run_hooks' (jameinel)
2636
    def pull(self, source, overwrite=False, stop_revision=None,
2477.1.9 by Martin Pool
Review cleanups from John, mostly docs
2637
             **kwargs):
3441.5.29 by Andrew Bennetts
More review tweaks: whitespace nits in test_smart, add (and use) ._clear_cached_state_of_remote_branch_only method in bzrlib/remote.py.
2638
        self._clear_cached_state_of_remote_branch_only()
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
2639
        self._ensure_real()
3482.1.1 by John Arbash Meinel
Fix bug #238149, RemoteBranch.pull needs to return the _real_branch's pull result.
2640
        return self._real_branch.pull(
2477.1.2 by Martin Pool
Rename push/pull back to 'run_hooks' (jameinel)
2641
            source, overwrite=overwrite, stop_revision=stop_revision,
3489.2.4 by Andrew Bennetts
Fix all tests broken by fixing make_branch_and_tree.
2642
            _override_hook_target=self, **kwargs)
2018.14.1 by Andrew Bennetts
Update to current hpss branch? Fix lots of test failures.
2643
2018.14.3 by Andrew Bennetts
Make a couple more branch_implementations tests pass.
2644
    @needs_read_lock
2645
    def push(self, target, overwrite=False, stop_revision=None):
2646
        self._ensure_real()
2018.5.97 by Andrew Bennetts
Fix more tests.
2647
        return self._real_branch.push(
2477.1.5 by Martin Pool
More cleanups of Branch.push to get the right behaviour with RemoteBranches
2648
            target, overwrite=overwrite, stop_revision=stop_revision,
2649
            _override_hook_source_branch=self)
2018.14.3 by Andrew Bennetts
Make a couple more branch_implementations tests pass.
2650
2651
    def is_locked(self):
2652
        return self._lock_count >= 1
2653
3634.2.1 by John Arbash Meinel
Thunk over to the real branch's revision_id_to_revno.
2654
    @needs_read_lock
2655
    def revision_id_to_revno(self, revision_id):
2656
        self._ensure_real()
2657
        return self._real_branch.revision_id_to_revno(revision_id)
2658
2892.2.1 by Andrew Bennetts
Add Branch.set_last_revision_info smart method, and make the RemoteBranch client use it.
2659
    @needs_write_lock
2018.5.83 by Andrew Bennetts
Fix some test failures caused by the switch from unicode to UTF-8-encoded strs for revision IDs.
2660
    def set_last_revision_info(self, revno, revision_id):
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2661
        # XXX: These should be returned by the set_last_revision_info verb
2662
        old_revno, old_revid = self.last_revision_info()
2663
        self._run_pre_change_branch_tip_hooks(revno, revision_id)
2892.2.1 by Andrew Bennetts
Add Branch.set_last_revision_info smart method, and make the RemoteBranch client use it.
2664
        revision_id = ensure_null(revision_id)
3297.4.2 by Andrew Bennetts
Add backwards compatibility for servers older than 1.4.
2665
        try:
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2666
            response = self._call('Branch.set_last_revision_info',
2667
                self._remote_path(), self._lock_token, self._repo_lock_token,
2668
                str(revno), revision_id)
3297.4.2 by Andrew Bennetts
Add backwards compatibility for servers older than 1.4.
2669
        except errors.UnknownSmartMethod:
2670
            self._ensure_real()
3441.5.29 by Andrew Bennetts
More review tweaks: whitespace nits in test_smart, add (and use) ._clear_cached_state_of_remote_branch_only method in bzrlib/remote.py.
2671
            self._clear_cached_state_of_remote_branch_only()
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2672
            self._real_branch.set_last_revision_info(revno, revision_id)
2673
            self._last_revision_info_cache = revno, revision_id
2674
            return
2892.2.1 by Andrew Bennetts
Add Branch.set_last_revision_info smart method, and make the RemoteBranch client use it.
2675
        if response == ('ok',):
2676
            self._clear_cached_state()
3441.5.1 by Andrew Bennetts
Avoid necessarily calling get_parent_map when pushing.
2677
            self._last_revision_info_cache = revno, revision_id
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2678
            self._run_post_change_branch_tip_hooks(old_revno, old_revid)
3441.5.29 by Andrew Bennetts
More review tweaks: whitespace nits in test_smart, add (and use) ._clear_cached_state_of_remote_branch_only method in bzrlib/remote.py.
2679
            # Update the _real_branch's cache too.
2680
            if self._real_branch is not None:
2681
                cache = self._last_revision_info_cache
2682
                self._real_branch._last_revision_info_cache = cache
2892.2.1 by Andrew Bennetts
Add Branch.set_last_revision_info smart method, and make the RemoteBranch client use it.
2683
        else:
2684
            raise errors.UnexpectedSmartServerResponse(response)
2018.5.83 by Andrew Bennetts
Fix some test failures caused by the switch from unicode to UTF-8-encoded strs for revision IDs.
2685
3441.5.27 by Andrew Bennetts
Tweaks suggested by John's review: rename _check_if_descendant_or_diverged, move caching last_revision_info into base Branch, better use of lock decorators.
2686
    @needs_write_lock
2018.5.95 by Andrew Bennetts
Add a Transport.is_readonly remote call, let {Branch,Repository}.lock_write remote call return UnlockableTransport, and miscellaneous test fixes.
2687
    def generate_revision_history(self, revision_id, last_rev=None,
2688
                                  other_branch=None):
3441.5.6 by Andrew Bennetts
Greatly simplify RemoteBranch.update_revisions. Still needs more tests.
2689
        medium = self._client._medium
3441.5.23 by Andrew Bennetts
Fix test failures.
2690
        if not medium._is_remote_before((1, 6)):
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2691
            # Use a smart method for 1.6 and above servers
3441.5.6 by Andrew Bennetts
Greatly simplify RemoteBranch.update_revisions. Still needs more tests.
2692
            try:
3441.5.18 by Andrew Bennetts
Fix some test failures.
2693
                self._set_last_revision_descendant(revision_id, other_branch,
3441.5.28 by Andrew Bennetts
Another review tweak: rename do_not_overwrite_descendant to allow_overwrite_descendant.
2694
                    allow_diverged=True, allow_overwrite_descendant=True)
3441.5.6 by Andrew Bennetts
Greatly simplify RemoteBranch.update_revisions. Still needs more tests.
2695
                return
3441.5.18 by Andrew Bennetts
Fix some test failures.
2696
            except errors.UnknownSmartMethod:
3441.5.23 by Andrew Bennetts
Fix test failures.
2697
                medium._remember_remote_is_before((1, 6))
3441.5.29 by Andrew Bennetts
More review tweaks: whitespace nits in test_smart, add (and use) ._clear_cached_state_of_remote_branch_only method in bzrlib/remote.py.
2698
        self._clear_cached_state_of_remote_branch_only()
4005.2.1 by Robert Collins
Fix RemoteBranch to be used correctly in tests using bzr+ssh, to fire off Branch hooks correctly, and improve the branch_implementations tests to check that making a branch gets the right format under test.
2699
        self.set_revision_history(self._lefthand_history(revision_id,
2700
            last_rev=last_rev,other_branch=other_branch))
2018.5.96 by Andrew Bennetts
Merge from bzr.dev, resolving the worst of the semantic conflicts, but there's
2701
2018.5.97 by Andrew Bennetts
Fix more tests.
2702
    def set_push_location(self, location):
2703
        self._ensure_real()
2704
        return self._real_branch.set_push_location(location)
2705
2018.18.25 by Martin Pool
Repository.tarball fixes for python2.4
2706
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
2707
class RemoteConfig(object):
2708
    """A Config that reads and writes from smart verbs.
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
2709
2710
    It is a low-level object that considers config data to be name/value pairs
2711
    that may be associated with a section. Assigning meaning to the these
2712
    values is done at higher levels like bzrlib.config.TreeConfig.
2713
    """
2714
2715
    def get_option(self, name, section=None, default=None):
2716
        """Return the value associated with a named option.
2717
2718
        :param name: The name of the value
2719
        :param section: The section the option is in (if any)
2720
        :param default: The value to return if the value is not set
2721
        :return: The value or default value
2722
        """
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
2723
        try:
2724
            configobj = self._get_configobj()
2725
            if section is None:
2726
                section_obj = configobj
2727
            else:
2728
                try:
2729
                    section_obj = configobj[section]
2730
                except KeyError:
2731
                    return default
2732
            return section_obj.get(name, default)
2733
        except errors.UnknownSmartMethod:
2734
            return self._vfs_get_option(name, section, default)
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
2735
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
2736
    def _response_to_configobj(self, response):
4288.1.2 by Robert Collins
Create a server verb for doing BzrDir.get_config()
2737
        if len(response[0]) and response[0][0] != 'ok':
4288.1.4 by Robert Collins
Remove the explicit set_parent method on RemoteBranch in favour of inheriting from Branch.
2738
            raise errors.UnexpectedSmartServerResponse(response)
4241.5.1 by Matt Nordhoff
Fix Branch.get_config_file smart verb on multi-line config files. (Bug #354075)
2739
        lines = response[1].read_body_bytes().splitlines()
2740
        return config.ConfigObj(lines, encoding='utf-8')
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
2741
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
2742
2743
class RemoteBranchConfig(RemoteConfig):
2744
    """A RemoteConfig for Branches."""
2745
2746
    def __init__(self, branch):
2747
        self._branch = branch
2748
2749
    def _get_configobj(self):
2750
        path = self._branch._remote_path()
2751
        response = self._branch._client.call_expecting_body(
2752
            'Branch.get_config_file', path)
2753
        return self._response_to_configobj(response)
2754
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
2755
    def set_option(self, value, name, section=None):
2756
        """Set the value associated with a named option.
2757
2758
        :param value: The value to set
2759
        :param name: The name of the value to set
2760
        :param section: The section the option is in (if any)
2761
        """
4226.2.1 by Robert Collins
Set branch config options via a smart method.
2762
        medium = self._branch._client._medium
2763
        if medium._is_remote_before((1, 14)):
2764
            return self._vfs_set_option(value, name, section)
2765
        try:
2766
            path = self._branch._remote_path()
2767
            response = self._branch._client.call('Branch.set_config_option',
2768
                path, self._branch._lock_token, self._branch._repo_lock_token,
4226.2.2 by Robert Collins
Fix setting config options to support unicode values and don't attempt to reset repositories _fallback_repositories as the simple approach fails to work.
2769
                value.encode('utf8'), name, section or '')
4226.2.1 by Robert Collins
Set branch config options via a smart method.
2770
        except errors.UnknownSmartMethod:
2771
            medium._remember_remote_is_before((1, 14))
2772
            return self._vfs_set_option(value, name, section)
2773
        if response != ():
2774
            raise errors.UnexpectedSmartServerResponse(response)
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
2775
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
2776
    def _real_object(self):
2777
        self._branch._ensure_real()
2778
        return self._branch._real_branch
2779
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
2780
    def _vfs_set_option(self, value, name, section=None):
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
2781
        return self._real_object()._get_config().set_option(
2782
            value, name, section)
2783
2784
2785
class RemoteBzrDirConfig(RemoteConfig):
2786
    """A RemoteConfig for BzrDirs."""
2787
2788
    def __init__(self, bzrdir):
2789
        self._bzrdir = bzrdir
2790
2791
    def _get_configobj(self):
4288.1.4 by Robert Collins
Remove the explicit set_parent method on RemoteBranch in favour of inheriting from Branch.
2792
        medium = self._bzrdir._client._medium
2793
        verb = 'BzrDir.get_config_file'
2794
        if medium._is_remote_before((1, 15)):
2795
            raise errors.UnknownSmartMethod(verb)
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
2796
        path = self._bzrdir._path_for_remote_call(self._bzrdir._client)
2797
        response = self._bzrdir._call_expecting_body(
4288.1.4 by Robert Collins
Remove the explicit set_parent method on RemoteBranch in favour of inheriting from Branch.
2798
            verb, path)
4288.1.1 by Robert Collins
Add support for a RemoteBzrDirConfig to support optimising push operations which need to look for default stacking locations.
2799
        return self._response_to_configobj(response)
2800
2801
    def _vfs_get_option(self, name, section, default):
2802
        return self._real_object()._get_config().get_option(
2803
            name, section, default)
2804
2805
    def set_option(self, value, name, section=None):
2806
        """Set the value associated with a named option.
2807
2808
        :param value: The value to set
2809
        :param name: The name of the value to set
2810
        :param section: The section the option is in (if any)
2811
        """
2812
        return self._real_object()._get_config().set_option(
2813
            value, name, section)
2814
2815
    def _real_object(self):
2816
        self._bzrdir._ensure_real()
2817
        return self._bzrdir._real_bzrdir
2818
4226.1.5 by Robert Collins
Reinstate the use of the Branch.get_config_file verb.
2819
2820
2018.18.25 by Martin Pool
Repository.tarball fixes for python2.4
2821
def _extract_tar(tar, to_dir):
2822
    """Extract all the contents of a tarfile object.
2823
2824
    A replacement for extractall, which is not present in python2.4
2825
    """
2826
    for tarinfo in tar:
2827
        tar.extract(tarinfo, to_dir)
3533.3.1 by Andrew Bennetts
Remove duplication of error translation in bzrlib/remote.py.
2828
2829
2830
def _translate_error(err, **context):
2831
    """Translate an ErrorFromSmartServer into a more useful error.
2832
2833
    Possible context keys:
2834
      - branch
2835
      - repository
2836
      - bzrdir
2837
      - token
2838
      - other_branch
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2839
      - path
3690.1.1 by Andrew Bennetts
Unexpected error responses from a smart server no longer cause the client to traceback.
2840
2841
    If the error from the server doesn't match a known pattern, then
3690.1.2 by Andrew Bennetts
Rename UntranslateableErrorFromSmartServer -> UnknownErrorFromSmartServer.
2842
    UnknownErrorFromSmartServer is raised.
3533.3.1 by Andrew Bennetts
Remove duplication of error translation in bzrlib/remote.py.
2843
    """
2844
    def find(name):
3533.3.4 by Andrew Bennetts
Add tests for _translate_error's robustness.
2845
        try:
2846
            return context[name]
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2847
        except KeyError, key_err:
2848
            mutter('Missing key %r in context %r', key_err.args[0], context)
3533.3.4 by Andrew Bennetts
Add tests for _translate_error's robustness.
2849
            raise err
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2850
    def get_path():
3779.3.3 by Andrew Bennetts
Add a docstring.
2851
        """Get the path from the context if present, otherwise use first error
2852
        arg.
2853
        """
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2854
        try:
2855
            return context['path']
2856
        except KeyError, key_err:
2857
            try:
2858
                return err.error_args[0]
2859
            except IndexError, idx_err:
2860
                mutter(
2861
                    'Missing key %r in context %r', key_err.args[0], context)
2862
                raise err
2863
4650.2.1 by Robert Collins
Deserialise IncompatibleRepositories errors in the client, generating
2864
    if err.error_verb == 'IncompatibleRepositories':
2865
        raise errors.IncompatibleRepositories(err.error_args[0],
2866
            err.error_args[1], err.error_args[2])
2867
    elif err.error_verb == 'NoSuchRevision':
3533.3.1 by Andrew Bennetts
Remove duplication of error translation in bzrlib/remote.py.
2868
        raise NoSuchRevision(find('branch'), err.error_args[0])
2869
    elif err.error_verb == 'nosuchrevision':
2870
        raise NoSuchRevision(find('repository'), err.error_args[0])
4734.4.3 by Brian de Alwis
Add support for the HPSS to do further probing when a the provided
2871
    elif err.error_verb == 'nobranch':
4734.4.8 by Andrew Bennetts
Fix HPSS tests; pass 'location is a repository' message via smart server when possible (adds BzrDir.open_branchV3 verb).
2872
        if len(err.error_args) >= 1:
2873
            extra = err.error_args[0]
2874
        else:
2875
            extra = None
4734.4.3 by Brian de Alwis
Add support for the HPSS to do further probing when a the provided
2876
        raise errors.NotBranchError(path=find('bzrdir').root_transport.base,
4734.4.8 by Andrew Bennetts
Fix HPSS tests; pass 'location is a repository' message via smart server when possible (adds BzrDir.open_branchV3 verb).
2877
            detail=extra)
3533.3.1 by Andrew Bennetts
Remove duplication of error translation in bzrlib/remote.py.
2878
    elif err.error_verb == 'norepository':
2879
        raise errors.NoRepositoryPresent(find('bzrdir'))
2880
    elif err.error_verb == 'LockContention':
2881
        raise errors.LockContention('(remote lock)')
2882
    elif err.error_verb == 'UnlockableTransport':
3533.3.4 by Andrew Bennetts
Add tests for _translate_error's robustness.
2883
        raise errors.UnlockableTransport(find('bzrdir').root_transport)
3533.3.1 by Andrew Bennetts
Remove duplication of error translation in bzrlib/remote.py.
2884
    elif err.error_verb == 'LockFailed':
2885
        raise errors.LockFailed(err.error_args[0], err.error_args[1])
2886
    elif err.error_verb == 'TokenMismatch':
2887
        raise errors.TokenMismatch(find('token'), '(remote token)')
2888
    elif err.error_verb == 'Diverged':
2889
        raise errors.DivergedBranches(find('branch'), find('other_branch'))
3577.1.1 by Andrew Bennetts
Cherry-pick TipChangeRejected changes from pre-branch-tip-changed-hook loom.
2890
    elif err.error_verb == 'TipChangeRejected':
2891
        raise errors.TipChangeRejected(err.error_args[0].decode('utf8'))
3691.2.6 by Martin Pool
Disable RemoteBranch stacking, but get get_stacked_on_url working, and passing back exceptions
2892
    elif err.error_verb == 'UnstackableBranchFormat':
2893
        raise errors.UnstackableBranchFormat(*err.error_args)
2894
    elif err.error_verb == 'UnstackableRepositoryFormat':
2895
        raise errors.UnstackableRepositoryFormat(*err.error_args)
2896
    elif err.error_verb == 'NotStacked':
2897
        raise errors.NotStacked(branch=find('branch'))
3779.3.1 by Andrew Bennetts
Move encoding/decoding logic of PermissionDenied and ReadError so that it happens for all RPCs.
2898
    elif err.error_verb == 'PermissionDenied':
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2899
        path = get_path()
3779.3.1 by Andrew Bennetts
Move encoding/decoding logic of PermissionDenied and ReadError so that it happens for all RPCs.
2900
        if len(err.error_args) >= 2:
2901
            extra = err.error_args[1]
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2902
        else:
2903
            extra = None
3779.3.1 by Andrew Bennetts
Move encoding/decoding logic of PermissionDenied and ReadError so that it happens for all RPCs.
2904
        raise errors.PermissionDenied(path, extra=extra)
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2905
    elif err.error_verb == 'ReadError':
2906
        path = get_path()
2907
        raise errors.ReadError(path)
2908
    elif err.error_verb == 'NoSuchFile':
2909
        path = get_path()
2910
        raise errors.NoSuchFile(path)
2911
    elif err.error_verb == 'FileExists':
2912
        raise errors.FileExists(err.error_args[0])
2913
    elif err.error_verb == 'DirectoryNotEmpty':
2914
        raise errors.DirectoryNotEmpty(err.error_args[0])
2915
    elif err.error_verb == 'ShortReadvError':
2916
        args = err.error_args
2917
        raise errors.ShortReadvError(
2918
            args[0], int(args[1]), int(args[2]), int(args[3]))
2919
    elif err.error_verb in ('UnicodeEncodeError', 'UnicodeDecodeError'):
2920
        encoding = str(err.error_args[0]) # encoding must always be a string
2921
        val = err.error_args[1]
2922
        start = int(err.error_args[2])
2923
        end = int(err.error_args[3])
2924
        reason = str(err.error_args[4]) # reason must always be a string
2925
        if val.startswith('u:'):
2926
            val = val[2:].decode('utf-8')
2927
        elif val.startswith('s:'):
2928
            val = val[2:].decode('base64')
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2929
        if err.error_verb == 'UnicodeDecodeError':
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2930
            raise UnicodeDecodeError(encoding, val, start, end, reason)
3786.2.3 by Andrew Bennetts
Remove duplicated 'call & translate errors' code in bzrlib.remote.
2931
        elif err.error_verb == 'UnicodeEncodeError':
3779.3.2 by Andrew Bennetts
Unify error translation done in bzrlib.remote and bzrlib.transport.remote.
2932
            raise UnicodeEncodeError(encoding, val, start, end, reason)
2933
    elif err.error_verb == 'ReadOnlyError':
2934
        raise errors.TransportNotPossible('readonly transport')
3690.1.2 by Andrew Bennetts
Rename UntranslateableErrorFromSmartServer -> UnknownErrorFromSmartServer.
2935
    raise errors.UnknownErrorFromSmartServer(err)