/brz/remove-bazaar

To get this branch, use:
bzr branch http://gegoxaren.bato24.eu/bzr/brz/remove-bazaar
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
1
# Copyright (C) 2005 by Canonical Ltd
2
#
3
# Authors:
4
#   Johan Rydberg <jrydberg@gnu.org>
5
#
6
# This program is free software; you can redistribute it and/or modify
7
# it under the terms of the GNU General Public License as published by
8
# the Free Software Foundation; either version 2 of the License, or
9
# (at your option) any later version.
10
11
# This program is distributed in the hope that it will be useful,
12
# but WITHOUT ANY WARRANTY; without even the implied warranty of
13
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
14
# GNU General Public License for more details.
15
16
# You should have received a copy of the GNU General Public License
17
# along with this program; if not, write to the Free Software
18
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
19
20
# Remaing to do is to figure out if get_graph should return a simple
21
# map, or a graph object of some kind.
22
23
24
"""Versioned text file storage api."""
25
26
1563.2.12 by Robert Collins
Checkpointing: created InterObject to factor out common inter object worker code, added InterVersionedFile and tests to allow making join work between any versionedfile.
27
from copy import deepcopy
28
from unittest import TestSuite
29
30
31
from bzrlib.inter import InterObject
1563.2.11 by Robert Collins
Consolidate reweave and join as we have no separate usage, make reweave tests apply to all versionedfile implementations and deprecate the old reweave apis.
32
from bzrlib.symbol_versioning import *
1563.2.13 by Robert Collins
InterVersionedFile implemented.
33
from bzrlib.transport.memory import MemoryTransport
34
from bzrlib.tsort import topo_sort
1563.2.33 by Robert Collins
Nicer progress updates during conversion to knits.
35
from bzrlib import ui
1563.2.11 by Robert Collins
Consolidate reweave and join as we have no separate usage, make reweave tests apply to all versionedfile implementations and deprecate the old reweave apis.
36
37
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
38
class VersionedFile(object):
39
    """Versioned text file storage.
40
    
41
    A versioned file manages versions of line-based text files,
42
    keeping track of the originating version for each line.
43
44
    To clients the "lines" of the file are represented as a list of
45
    strings. These strings will typically have terminal newline
46
    characters, but this is not required.  In particular files commonly
47
    do not have a newline at the end of the file.
48
49
    Texts are identified by a version-id string.
50
    """
51
1563.2.15 by Robert Collins
remove the weavestore assumptions about the number and nature of files it manages.
52
    def copy_to(self, name, transport):
53
        """Copy this versioned file to name on transport."""
54
        raise NotImplementedError(self.copy_to)
55
    
1563.2.11 by Robert Collins
Consolidate reweave and join as we have no separate usage, make reweave tests apply to all versionedfile implementations and deprecate the old reweave apis.
56
    @deprecated_method(zero_eight)
57
    def names(self):
58
        """Return a list of all the versions in this versioned file.
59
60
        Please use versionedfile.versions() now.
61
        """
62
        return self.versions()
63
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
64
    def versions(self):
65
        """Return a unsorted list of versions."""
66
        raise NotImplementedError(self.versions)
67
68
    def has_version(self, version_id):
69
        """Returns whether version is present."""
70
        raise NotImplementedError(self.has_version)
71
72
    def add_lines(self, version_id, parents, lines):
73
        """Add a single text on top of the versioned file.
74
75
        Must raise RevisionAlreadyPresent if the new version is
76
        already present in file history.
77
78
        Must raise RevisionNotPresent if any of the given parents are
79
        not present in file history."""
1563.2.4 by Robert Collins
First cut at including the knit implementation of versioned_file.
80
        raise NotImplementedError(self.add_lines)
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
81
1563.2.19 by Robert Collins
stub out a check for knits.
82
    def check(self, progress_bar=None):
83
        """Check the versioned file for integrity."""
84
        raise NotImplementedError(self.check)
85
1563.2.7 by Robert Collins
add versioned file clear_cache entry.
86
    def clear_cache(self):
87
        """Remove any data cached in the versioned file object."""
88
1563.2.5 by Robert Collins
Remove unused transaction references from knit.py and the versionedfile interface.
89
    def clone_text(self, new_version_id, old_version_id, parents):
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
90
        """Add an identical text to old_version_id as new_version_id.
91
92
        Must raise RevisionNotPresent if the old version or any of the
93
        parents are not present in file history.
94
95
        Must raise RevisionAlreadyPresent if the new version is
96
        already present in file history."""
97
        raise NotImplementedError(self.clone_text)
98
1563.2.13 by Robert Collins
InterVersionedFile implemented.
99
    def create_empty(self, name, transport, mode=None):
100
        """Create a new versioned file of this exact type.
101
102
        :param name: the file name
103
        :param transport: the transport
104
        :param mode: optional file mode.
105
        """
106
        raise NotImplementedError(self.create_empty)
107
1563.2.15 by Robert Collins
remove the weavestore assumptions about the number and nature of files it manages.
108
    def get_suffixes(self):
109
        """Return the file suffixes associated with this versioned file."""
110
        raise NotImplementedError(self.get_suffixes)
111
    
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
112
    def get_text(self, version_id):
113
        """Return version contents as a text string.
114
115
        Raises RevisionNotPresent if version is not present in
116
        file history.
117
        """
118
        return ''.join(self.get_lines(version_id))
119
    get_string = get_text
120
121
    def get_lines(self, version_id):
122
        """Return version contents as a sequence of lines.
123
124
        Raises RevisionNotPresent if version is not present in
125
        file history.
126
        """
127
        raise NotImplementedError(self.get_lines)
128
129
    def get_ancestry(self, version_ids):
130
        """Return a list of all ancestors of given version(s). This
131
        will not include the null revision.
132
133
        Must raise RevisionNotPresent if any of the given versions are
134
        not present in file history."""
135
        if isinstance(version_ids, basestring):
136
            version_ids = [version_ids]
137
        raise NotImplementedError(self.get_ancestry)
138
        
1563.2.13 by Robert Collins
InterVersionedFile implemented.
139
    def get_graph(self):
140
        """Return a graph for the entire versioned file."""
141
        result = {}
142
        for version in self.versions():
143
            result[version] = self.get_parents(version)
144
        return result
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
145
1563.2.11 by Robert Collins
Consolidate reweave and join as we have no separate usage, make reweave tests apply to all versionedfile implementations and deprecate the old reweave apis.
146
    @deprecated_method(zero_eight)
147
    def parent_names(self, version):
148
        """Return version names for parents of a version.
149
        
150
        See get_parents for the current api.
151
        """
152
        return self.get_parents(version)
153
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
154
    def get_parents(self, version_id):
155
        """Return version names for parents of a version.
156
157
        Must raise RevisionNotPresent if version is not present in
158
        file history.
159
        """
160
        raise NotImplementedError(self.get_parents)
161
162
    def annotate_iter(self, version_id):
163
        """Yield list of (version-id, line) pairs for the specified
164
        version.
165
166
        Must raise RevisionNotPresent if any of the given versions are
167
        not present in file history.
168
        """
169
        raise NotImplementedError(self.annotate_iter)
170
171
    def annotate(self, version_id):
172
        return list(self.annotate_iter(version_id))
173
1563.2.31 by Robert Collins
Convert Knit repositories to use knits.
174
    def join(self, other, pb=None, msg=None, version_ids=None,
175
             ignore_missing=False):
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
176
        """Integrate versions from other into this versioned file.
177
178
        If version_ids is None all versions from other should be
179
        incorporated into this versioned file.
180
181
        Must raise RevisionNotPresent if any of the specified versions
1563.2.31 by Robert Collins
Convert Knit repositories to use knits.
182
        are not present in the other files history unless ignore_missing
183
        is supplied when they are silently skipped.
184
        """
185
        return InterVersionedFile.get(other, self).join(
186
            pb,
187
            msg,
188
            version_ids,
189
            ignore_missing)
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
190
1594.2.6 by Robert Collins
Introduce a api specifically for looking at lines in some versions of the inventory, for fileid_involved.
191
    def iter_lines_added_or_present_in_versions(self, version_ids=None):
192
        """Iterate over the lines in the versioned file from version_ids.
193
194
        This may return lines from other versions, and does not return the
195
        specific version marker at this point. The api may be changed
196
        during development to include the version that the versioned file
197
        thinks is relevant, but given that such hints are just guesses,
198
        its better not to have it if we dont need it.
199
200
        NOTES: Lines are normalised: they will all have \n terminators.
201
               Lines are returned in arbitrary order.
202
        """
203
        raise NotImplementedError(self.iter_lines_added_or_present_in_versions)
204
205
    @deprecated_method(zero_eight)
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
206
    def walk(self, version_ids=None):
207
        """Walk the versioned file as a weave-like structure, for
208
        versions relative to version_ids.  Yields sequence of (lineno,
209
        insert, deletes, text) for each relevant line.
210
211
        Must raise RevisionNotPresent if any of the specified versions
212
        are not present in the file history.
213
214
        :param version_ids: the version_ids to walk with respect to. If not
215
                            supplied the entire weave-like structure is walked.
1594.2.6 by Robert Collins
Introduce a api specifically for looking at lines in some versions of the inventory, for fileid_involved.
216
217
        walk is deprecated in favour of iter_lines_added_or_present_in_versions
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
218
        """
219
        raise NotImplementedError(self.walk)
220
1563.2.11 by Robert Collins
Consolidate reweave and join as we have no separate usage, make reweave tests apply to all versionedfile implementations and deprecate the old reweave apis.
221
    @deprecated_method(zero_eight)
222
    def iter_names(self):
223
        """Walk the names list."""
224
        return iter(self.versions())
225
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
226
    def plan_merge(self, ver_a, ver_b):
227
        """Return pseudo-annotation indicating how the two versions merge.
228
229
        This is computed between versions a and b and their common
230
        base.
231
232
        Weave lines present in none of them are skipped entirely.
233
        """
1563.2.35 by Robert Collins
cleanup deprecation warnings and finish conversion so the inventory is knit based too.
234
        inc_a = set(self.get_ancestry([ver_a]))
235
        inc_b = set(self.get_ancestry([ver_b]))
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
236
        inc_c = inc_a & inc_b
237
1594.2.6 by Robert Collins
Introduce a api specifically for looking at lines in some versions of the inventory, for fileid_involved.
238
        for lineno, insert, deleteset, line in self.walk([ver_a, ver_b]):
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
239
            if deleteset & inc_c:
240
                # killed in parent; can't be in either a or b
241
                # not relevant to our work
242
                yield 'killed-base', line
243
            elif insert in inc_c:
244
                # was inserted in base
245
                killed_a = bool(deleteset & inc_a)
246
                killed_b = bool(deleteset & inc_b)
247
                if killed_a and killed_b:
248
                    yield 'killed-both', line
249
                elif killed_a:
250
                    yield 'killed-a', line
251
                elif killed_b:
252
                    yield 'killed-b', line
253
                else:
254
                    yield 'unchanged', line
255
            elif insert in inc_a:
256
                if deleteset & inc_a:
257
                    yield 'ghost-a', line
258
                else:
259
                    # new in A; not in B
260
                    yield 'new-a', line
261
            elif insert in inc_b:
262
                if deleteset & inc_b:
263
                    yield 'ghost-b', line
264
                else:
265
                    yield 'new-b', line
266
            else:
267
                # not in either revision
268
                yield 'irrelevant', line
269
270
        yield 'unchanged', ''           # terminator
271
1563.2.2 by Robert Collins
merge from bzr.dev
272
    def weave_merge(self, plan, a_marker='<<<<<<< \n', b_marker='>>>>>>> \n'):
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
273
        lines_a = []
274
        lines_b = []
275
        ch_a = ch_b = False
276
        # TODO: Return a structured form of the conflicts (e.g. 2-tuples for
277
        # conflicted regions), rather than just inserting the markers.
278
        # 
279
        # TODO: Show some version information (e.g. author, date) on 
280
        # conflicted regions.
281
        for state, line in plan:
282
            if state == 'unchanged' or state == 'killed-both':
283
                # resync and flush queued conflicts changes if any
284
                if not lines_a and not lines_b:
285
                    pass
286
                elif ch_a and not ch_b:
287
                    # one-sided change:                    
288
                    for l in lines_a: yield l
289
                elif ch_b and not ch_a:
290
                    for l in lines_b: yield l
291
                elif lines_a == lines_b:
292
                    for l in lines_a: yield l
293
                else:
1563.2.2 by Robert Collins
merge from bzr.dev
294
                    yield a_marker
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
295
                    for l in lines_a: yield l
296
                    yield '=======\n'
297
                    for l in lines_b: yield l
1563.2.2 by Robert Collins
merge from bzr.dev
298
                    yield b_marker
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
299
300
                del lines_a[:]
301
                del lines_b[:]
302
                ch_a = ch_b = False
303
                
304
            if state == 'unchanged':
305
                if line:
306
                    yield line
307
            elif state == 'killed-a':
308
                ch_a = True
309
                lines_b.append(line)
310
            elif state == 'killed-b':
311
                ch_b = True
312
                lines_a.append(line)
313
            elif state == 'new-a':
314
                ch_a = True
315
                lines_a.append(line)
316
            elif state == 'new-b':
317
                ch_b = True
318
                lines_b.append(line)
319
            else:
320
                assert state in ('irrelevant', 'ghost-a', 'ghost-b', 'killed-base',
321
                                 'killed-both'), \
322
                       state
1563.2.12 by Robert Collins
Checkpointing: created InterObject to factor out common inter object worker code, added InterVersionedFile and tests to allow making join work between any versionedfile.
323
324
325
class InterVersionedFile(InterObject):
326
    """This class represents operations taking place between two versionedfiles..
327
328
    Its instances have methods like join, and contain
329
    references to the source and target versionedfiles these operations can be 
330
    carried out on.
331
332
    Often we will provide convenience methods on 'versionedfile' which carry out
333
    operations with another versionedfile - they will always forward to
334
    InterVersionedFile.get(other).method_name(parameters).
335
    """
336
337
    _optimisers = set()
338
    """The available optimised InterVersionedFile types."""
339
1563.2.31 by Robert Collins
Convert Knit repositories to use knits.
340
    def join(self, pb=None, msg=None, version_ids=None, ignore_missing=False):
1563.2.13 by Robert Collins
InterVersionedFile implemented.
341
        """Integrate versions from self.source into self.target.
342
343
        If version_ids is None all versions from source should be
344
        incorporated into this versioned file.
345
346
        Must raise RevisionNotPresent if any of the specified versions
1563.2.31 by Robert Collins
Convert Knit repositories to use knits.
347
        are not present in the other files history unless ignore_missing is 
348
        supplied when they are silently skipped.
1563.2.13 by Robert Collins
InterVersionedFile implemented.
349
        """
350
        # the default join: 
351
        # - make a temporary versioned file of type target
352
        # - insert the source content into it one at a time
353
        # - join them
354
        # Make a new target-format versioned file. 
355
        temp_source = self.target.create_empty("temp", MemoryTransport())
356
        graph = self.source.get_graph()
357
        order = topo_sort(graph.items())
1563.2.37 by Robert Collins
Merge in nested progress bars
358
        pb = ui.ui_factory.nested_progress_bar()
359
        try:
360
            for index, version in enumerate(order):
361
                pb.update('Converting versioned data', index, len(order))
362
                temp_source.add_lines(version,
363
                                      self.source.get_parents(version),
364
                                      self.source.get_lines(version))
365
            
366
            # this should hit the native code path for target
367
            return self.target.join(temp_source,
368
                                    pb,
369
                                    msg,
370
                                    version_ids,
371
                                    ignore_missing)
372
        finally:
373
            pb.finished()
1563.2.13 by Robert Collins
InterVersionedFile implemented.
374
1563.2.12 by Robert Collins
Checkpointing: created InterObject to factor out common inter object worker code, added InterVersionedFile and tests to allow making join work between any versionedfile.
375
376
class InterVersionedFileTestProviderAdapter(object):
377
    """A tool to generate a suite testing multiple inter versioned-file classes.
378
379
    This is done by copying the test once for each interversionedfile provider
380
    and injecting the transport_server, transport_readonly_server,
381
    versionedfile_factory and versionedfile_factory_to classes into each copy.
382
    Each copy is also given a new id() to make it easy to identify.
383
    """
384
385
    def __init__(self, transport_server, transport_readonly_server, formats):
386
        self._transport_server = transport_server
387
        self._transport_readonly_server = transport_readonly_server
388
        self._formats = formats
389
    
390
    def adapt(self, test):
391
        result = TestSuite()
392
        for (interversionedfile_class,
393
             versionedfile_factory,
394
             versionedfile_factory_to) in self._formats:
395
            new_test = deepcopy(test)
396
            new_test.transport_server = self._transport_server
397
            new_test.transport_readonly_server = self._transport_readonly_server
398
            new_test.interversionedfile_class = interversionedfile_class
399
            new_test.versionedfile_factory = versionedfile_factory
400
            new_test.versionedfile_factory_to = versionedfile_factory_to
401
            def make_new_test_id():
402
                new_id = "%s(%s)" % (new_test.id(), interversionedfile_class.__name__)
403
                return lambda: new_id
404
            new_test.id = make_new_test_id()
405
            result.addTest(new_test)
406
        return result
407
408
    @staticmethod
409
    def default_test_list():
410
        """Generate the default list of interversionedfile permutations to test."""
411
        from bzrlib.weave import WeaveFile
1563.2.16 by Robert Collins
Change WeaveStore into VersionedFileStore and make its versoined file class parameterisable.
412
        from bzrlib.knit import KnitVersionedFile
1563.2.12 by Robert Collins
Checkpointing: created InterObject to factor out common inter object worker code, added InterVersionedFile and tests to allow making join work between any versionedfile.
413
        result = []
414
        # test the fallback InterVersionedFile from weave to annotated knits
415
        result.append((InterVersionedFile, 
416
                       WeaveFile,
1563.2.16 by Robert Collins
Change WeaveStore into VersionedFileStore and make its versoined file class parameterisable.
417
                       KnitVersionedFile))
1563.2.12 by Robert Collins
Checkpointing: created InterObject to factor out common inter object worker code, added InterVersionedFile and tests to allow making join work between any versionedfile.
418
        for optimiser in InterVersionedFile._optimisers:
419
            result.append((optimiser,
420
                           optimiser._matching_file_factory,
421
                           optimiser._matching_file_factory
422
                           ))
423
        # if there are specific combinations we want to use, we can add them 
424
        # here.
425
        return result