/brz/remove-bazaar

To get this branch, use:
bzr branch http://gegoxaren.bato24.eu/bzr/brz/remove-bazaar
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
1
# Copyright (C) 2005 by Canonical Ltd
2
#
3
# Authors:
4
#   Johan Rydberg <jrydberg@gnu.org>
5
#
6
# This program is free software; you can redistribute it and/or modify
7
# it under the terms of the GNU General Public License as published by
8
# the Free Software Foundation; either version 2 of the License, or
9
# (at your option) any later version.
10
11
# This program is distributed in the hope that it will be useful,
12
# but WITHOUT ANY WARRANTY; without even the implied warranty of
13
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
14
# GNU General Public License for more details.
15
16
# You should have received a copy of the GNU General Public License
17
# along with this program; if not, write to the Free Software
18
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
19
20
# Remaing to do is to figure out if get_graph should return a simple
21
# map, or a graph object of some kind.
22
23
24
"""Versioned text file storage api."""
25
26
1563.2.12 by Robert Collins
Checkpointing: created InterObject to factor out common inter object worker code, added InterVersionedFile and tests to allow making join work between any versionedfile.
27
from copy import deepcopy
28
from unittest import TestSuite
29
30
31
from bzrlib.inter import InterObject
1563.2.11 by Robert Collins
Consolidate reweave and join as we have no separate usage, make reweave tests apply to all versionedfile implementations and deprecate the old reweave apis.
32
from bzrlib.symbol_versioning import *
1563.2.13 by Robert Collins
InterVersionedFile implemented.
33
from bzrlib.transport.memory import MemoryTransport
34
from bzrlib.tsort import topo_sort
1563.2.33 by Robert Collins
Nicer progress updates during conversion to knits.
35
from bzrlib import ui
1563.2.11 by Robert Collins
Consolidate reweave and join as we have no separate usage, make reweave tests apply to all versionedfile implementations and deprecate the old reweave apis.
36
37
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
38
class VersionedFile(object):
39
    """Versioned text file storage.
40
    
41
    A versioned file manages versions of line-based text files,
42
    keeping track of the originating version for each line.
43
44
    To clients the "lines" of the file are represented as a list of
45
    strings. These strings will typically have terminal newline
46
    characters, but this is not required.  In particular files commonly
47
    do not have a newline at the end of the file.
48
49
    Texts are identified by a version-id string.
50
    """
51
1563.2.15 by Robert Collins
remove the weavestore assumptions about the number and nature of files it manages.
52
    def copy_to(self, name, transport):
53
        """Copy this versioned file to name on transport."""
54
        raise NotImplementedError(self.copy_to)
55
    
1563.2.11 by Robert Collins
Consolidate reweave and join as we have no separate usage, make reweave tests apply to all versionedfile implementations and deprecate the old reweave apis.
56
    @deprecated_method(zero_eight)
57
    def names(self):
58
        """Return a list of all the versions in this versioned file.
59
60
        Please use versionedfile.versions() now.
61
        """
62
        return self.versions()
63
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
64
    def versions(self):
65
        """Return a unsorted list of versions."""
66
        raise NotImplementedError(self.versions)
67
1594.2.8 by Robert Collins
add ghost aware apis to knits.
68
    def has_ghost(self, version_id):
69
        """Returns whether version is present as a ghost."""
70
        raise NotImplementedError(self.has_ghost)
71
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
72
    def has_version(self, version_id):
73
        """Returns whether version is present."""
74
        raise NotImplementedError(self.has_version)
75
76
    def add_lines(self, version_id, parents, lines):
77
        """Add a single text on top of the versioned file.
78
79
        Must raise RevisionAlreadyPresent if the new version is
80
        already present in file history.
81
82
        Must raise RevisionNotPresent if any of the given parents are
83
        not present in file history."""
1563.2.4 by Robert Collins
First cut at including the knit implementation of versioned_file.
84
        raise NotImplementedError(self.add_lines)
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
85
1594.2.8 by Robert Collins
add ghost aware apis to knits.
86
    def add_lines_with_ghosts(self, version_id, parents, lines):
87
        """Add lines to the versioned file, allowing ghosts to be present."""
88
        raise NotImplementedError(self.add_lines_with_ghosts)
89
1563.2.19 by Robert Collins
stub out a check for knits.
90
    def check(self, progress_bar=None):
91
        """Check the versioned file for integrity."""
92
        raise NotImplementedError(self.check)
93
1563.2.7 by Robert Collins
add versioned file clear_cache entry.
94
    def clear_cache(self):
95
        """Remove any data cached in the versioned file object."""
96
1563.2.5 by Robert Collins
Remove unused transaction references from knit.py and the versionedfile interface.
97
    def clone_text(self, new_version_id, old_version_id, parents):
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
98
        """Add an identical text to old_version_id as new_version_id.
99
100
        Must raise RevisionNotPresent if the old version or any of the
101
        parents are not present in file history.
102
103
        Must raise RevisionAlreadyPresent if the new version is
104
        already present in file history."""
105
        raise NotImplementedError(self.clone_text)
106
1563.2.13 by Robert Collins
InterVersionedFile implemented.
107
    def create_empty(self, name, transport, mode=None):
108
        """Create a new versioned file of this exact type.
109
110
        :param name: the file name
111
        :param transport: the transport
112
        :param mode: optional file mode.
113
        """
114
        raise NotImplementedError(self.create_empty)
115
1594.2.7 by Robert Collins
Add versionedfile.fix_parents api for correcting data post hoc.
116
    def fix_parents(self, version, new_parents):
117
        """Fix the parents list for version.
118
        
119
        This is done by appending a new version to the index
120
        with identical data except for the parents list.
121
        the parents list must be a superset of the current
122
        list.
123
        """
124
        raise NotImplementedError(self.fix_parents)
125
1563.2.15 by Robert Collins
remove the weavestore assumptions about the number and nature of files it manages.
126
    def get_suffixes(self):
127
        """Return the file suffixes associated with this versioned file."""
128
        raise NotImplementedError(self.get_suffixes)
129
    
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
130
    def get_text(self, version_id):
131
        """Return version contents as a text string.
132
133
        Raises RevisionNotPresent if version is not present in
134
        file history.
135
        """
136
        return ''.join(self.get_lines(version_id))
137
    get_string = get_text
138
139
    def get_lines(self, version_id):
140
        """Return version contents as a sequence of lines.
141
142
        Raises RevisionNotPresent if version is not present in
143
        file history.
144
        """
145
        raise NotImplementedError(self.get_lines)
146
147
    def get_ancestry(self, version_ids):
148
        """Return a list of all ancestors of given version(s). This
149
        will not include the null revision.
150
151
        Must raise RevisionNotPresent if any of the given versions are
152
        not present in file history."""
153
        if isinstance(version_ids, basestring):
154
            version_ids = [version_ids]
155
        raise NotImplementedError(self.get_ancestry)
156
        
1594.2.8 by Robert Collins
add ghost aware apis to knits.
157
    def get_ancestry_with_ghosts(self, version_ids):
158
        """Return a list of all ancestors of given version(s). This
159
        will not include the null revision.
160
161
        Must raise RevisionNotPresent if any of the given versions are
162
        not present in file history.
163
        
164
        Ghosts that are known about will be included in ancestry list,
165
        but are not explicitly marked.
166
        """
167
        raise NotImplementedError(self.get_ancestry_with_ghosts)
168
        
1563.2.13 by Robert Collins
InterVersionedFile implemented.
169
    def get_graph(self):
1594.2.8 by Robert Collins
add ghost aware apis to knits.
170
        """Return a graph for the entire versioned file.
171
        
172
        Ghosts are not listed or referenced in the graph.
173
        """
1563.2.13 by Robert Collins
InterVersionedFile implemented.
174
        result = {}
175
        for version in self.versions():
176
            result[version] = self.get_parents(version)
177
        return result
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
178
1594.2.8 by Robert Collins
add ghost aware apis to knits.
179
    def get_graph_with_ghosts(self):
180
        """Return a graph for the entire versioned file.
181
        
182
        Ghosts are referenced in parents list but are not
183
        explicitly listed.
184
        """
185
        raise NotImplementedError(self.get_graph_with_ghosts)
186
1563.2.11 by Robert Collins
Consolidate reweave and join as we have no separate usage, make reweave tests apply to all versionedfile implementations and deprecate the old reweave apis.
187
    @deprecated_method(zero_eight)
188
    def parent_names(self, version):
189
        """Return version names for parents of a version.
190
        
191
        See get_parents for the current api.
192
        """
193
        return self.get_parents(version)
194
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
195
    def get_parents(self, version_id):
196
        """Return version names for parents of a version.
197
198
        Must raise RevisionNotPresent if version is not present in
199
        file history.
200
        """
201
        raise NotImplementedError(self.get_parents)
202
1594.2.8 by Robert Collins
add ghost aware apis to knits.
203
    def get_parents_with_ghosts(self, version_id):
204
        """Return version names for parents of version_id.
205
206
        Will raise RevisionNotPresent if version_id is not present
207
        in the history.
208
209
        Ghosts that are known about will be included in the parent list,
210
        but are not explicitly marked.
211
        """
212
        raise NotImplementedError(self.get_parents_with_ghosts)
213
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
214
    def annotate_iter(self, version_id):
215
        """Yield list of (version-id, line) pairs for the specified
216
        version.
217
218
        Must raise RevisionNotPresent if any of the given versions are
219
        not present in file history.
220
        """
221
        raise NotImplementedError(self.annotate_iter)
222
223
    def annotate(self, version_id):
224
        return list(self.annotate_iter(version_id))
225
1563.2.31 by Robert Collins
Convert Knit repositories to use knits.
226
    def join(self, other, pb=None, msg=None, version_ids=None,
227
             ignore_missing=False):
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
228
        """Integrate versions from other into this versioned file.
229
230
        If version_ids is None all versions from other should be
231
        incorporated into this versioned file.
232
233
        Must raise RevisionNotPresent if any of the specified versions
1563.2.31 by Robert Collins
Convert Knit repositories to use knits.
234
        are not present in the other files history unless ignore_missing
235
        is supplied when they are silently skipped.
236
        """
237
        return InterVersionedFile.get(other, self).join(
238
            pb,
239
            msg,
240
            version_ids,
241
            ignore_missing)
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
242
1594.2.6 by Robert Collins
Introduce a api specifically for looking at lines in some versions of the inventory, for fileid_involved.
243
    def iter_lines_added_or_present_in_versions(self, version_ids=None):
244
        """Iterate over the lines in the versioned file from version_ids.
245
246
        This may return lines from other versions, and does not return the
247
        specific version marker at this point. The api may be changed
248
        during development to include the version that the versioned file
249
        thinks is relevant, but given that such hints are just guesses,
250
        its better not to have it if we dont need it.
251
252
        NOTES: Lines are normalised: they will all have \n terminators.
253
               Lines are returned in arbitrary order.
254
        """
255
        raise NotImplementedError(self.iter_lines_added_or_present_in_versions)
256
257
    @deprecated_method(zero_eight)
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
258
    def walk(self, version_ids=None):
259
        """Walk the versioned file as a weave-like structure, for
260
        versions relative to version_ids.  Yields sequence of (lineno,
261
        insert, deletes, text) for each relevant line.
262
263
        Must raise RevisionNotPresent if any of the specified versions
264
        are not present in the file history.
265
266
        :param version_ids: the version_ids to walk with respect to. If not
267
                            supplied the entire weave-like structure is walked.
1594.2.6 by Robert Collins
Introduce a api specifically for looking at lines in some versions of the inventory, for fileid_involved.
268
269
        walk is deprecated in favour of iter_lines_added_or_present_in_versions
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
270
        """
271
        raise NotImplementedError(self.walk)
272
1563.2.11 by Robert Collins
Consolidate reweave and join as we have no separate usage, make reweave tests apply to all versionedfile implementations and deprecate the old reweave apis.
273
    @deprecated_method(zero_eight)
274
    def iter_names(self):
275
        """Walk the names list."""
276
        return iter(self.versions())
277
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
278
    def plan_merge(self, ver_a, ver_b):
279
        """Return pseudo-annotation indicating how the two versions merge.
280
281
        This is computed between versions a and b and their common
282
        base.
283
284
        Weave lines present in none of them are skipped entirely.
285
        """
1563.2.35 by Robert Collins
cleanup deprecation warnings and finish conversion so the inventory is knit based too.
286
        inc_a = set(self.get_ancestry([ver_a]))
287
        inc_b = set(self.get_ancestry([ver_b]))
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
288
        inc_c = inc_a & inc_b
289
1594.2.6 by Robert Collins
Introduce a api specifically for looking at lines in some versions of the inventory, for fileid_involved.
290
        for lineno, insert, deleteset, line in self.walk([ver_a, ver_b]):
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
291
            if deleteset & inc_c:
292
                # killed in parent; can't be in either a or b
293
                # not relevant to our work
294
                yield 'killed-base', line
295
            elif insert in inc_c:
296
                # was inserted in base
297
                killed_a = bool(deleteset & inc_a)
298
                killed_b = bool(deleteset & inc_b)
299
                if killed_a and killed_b:
300
                    yield 'killed-both', line
301
                elif killed_a:
302
                    yield 'killed-a', line
303
                elif killed_b:
304
                    yield 'killed-b', line
305
                else:
306
                    yield 'unchanged', line
307
            elif insert in inc_a:
308
                if deleteset & inc_a:
309
                    yield 'ghost-a', line
310
                else:
311
                    # new in A; not in B
312
                    yield 'new-a', line
313
            elif insert in inc_b:
314
                if deleteset & inc_b:
315
                    yield 'ghost-b', line
316
                else:
317
                    yield 'new-b', line
318
            else:
319
                # not in either revision
320
                yield 'irrelevant', line
321
322
        yield 'unchanged', ''           # terminator
323
1563.2.2 by Robert Collins
merge from bzr.dev
324
    def weave_merge(self, plan, a_marker='<<<<<<< \n', b_marker='>>>>>>> \n'):
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
325
        lines_a = []
326
        lines_b = []
327
        ch_a = ch_b = False
328
        # TODO: Return a structured form of the conflicts (e.g. 2-tuples for
329
        # conflicted regions), rather than just inserting the markers.
330
        # 
331
        # TODO: Show some version information (e.g. author, date) on 
332
        # conflicted regions.
333
        for state, line in plan:
334
            if state == 'unchanged' or state == 'killed-both':
335
                # resync and flush queued conflicts changes if any
336
                if not lines_a and not lines_b:
337
                    pass
338
                elif ch_a and not ch_b:
339
                    # one-sided change:                    
340
                    for l in lines_a: yield l
341
                elif ch_b and not ch_a:
342
                    for l in lines_b: yield l
343
                elif lines_a == lines_b:
344
                    for l in lines_a: yield l
345
                else:
1563.2.2 by Robert Collins
merge from bzr.dev
346
                    yield a_marker
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
347
                    for l in lines_a: yield l
348
                    yield '=======\n'
349
                    for l in lines_b: yield l
1563.2.2 by Robert Collins
merge from bzr.dev
350
                    yield b_marker
1563.2.1 by Robert Collins
Merge in a variation of the versionedfile api from versioned-file.
351
352
                del lines_a[:]
353
                del lines_b[:]
354
                ch_a = ch_b = False
355
                
356
            if state == 'unchanged':
357
                if line:
358
                    yield line
359
            elif state == 'killed-a':
360
                ch_a = True
361
                lines_b.append(line)
362
            elif state == 'killed-b':
363
                ch_b = True
364
                lines_a.append(line)
365
            elif state == 'new-a':
366
                ch_a = True
367
                lines_a.append(line)
368
            elif state == 'new-b':
369
                ch_b = True
370
                lines_b.append(line)
371
            else:
372
                assert state in ('irrelevant', 'ghost-a', 'ghost-b', 'killed-base',
373
                                 'killed-both'), \
374
                       state
1563.2.12 by Robert Collins
Checkpointing: created InterObject to factor out common inter object worker code, added InterVersionedFile and tests to allow making join work between any versionedfile.
375
376
377
class InterVersionedFile(InterObject):
378
    """This class represents operations taking place between two versionedfiles..
379
380
    Its instances have methods like join, and contain
381
    references to the source and target versionedfiles these operations can be 
382
    carried out on.
383
384
    Often we will provide convenience methods on 'versionedfile' which carry out
385
    operations with another versionedfile - they will always forward to
386
    InterVersionedFile.get(other).method_name(parameters).
387
    """
388
389
    _optimisers = set()
390
    """The available optimised InterVersionedFile types."""
391
1563.2.31 by Robert Collins
Convert Knit repositories to use knits.
392
    def join(self, pb=None, msg=None, version_ids=None, ignore_missing=False):
1563.2.13 by Robert Collins
InterVersionedFile implemented.
393
        """Integrate versions from self.source into self.target.
394
395
        If version_ids is None all versions from source should be
396
        incorporated into this versioned file.
397
398
        Must raise RevisionNotPresent if any of the specified versions
1563.2.31 by Robert Collins
Convert Knit repositories to use knits.
399
        are not present in the other files history unless ignore_missing is 
400
        supplied when they are silently skipped.
1563.2.13 by Robert Collins
InterVersionedFile implemented.
401
        """
402
        # the default join: 
403
        # - make a temporary versioned file of type target
404
        # - insert the source content into it one at a time
405
        # - join them
406
        # Make a new target-format versioned file. 
407
        temp_source = self.target.create_empty("temp", MemoryTransport())
408
        graph = self.source.get_graph()
409
        order = topo_sort(graph.items())
1563.2.37 by Robert Collins
Merge in nested progress bars
410
        pb = ui.ui_factory.nested_progress_bar()
411
        try:
412
            for index, version in enumerate(order):
413
                pb.update('Converting versioned data', index, len(order))
414
                temp_source.add_lines(version,
415
                                      self.source.get_parents(version),
416
                                      self.source.get_lines(version))
417
            
418
            # this should hit the native code path for target
419
            return self.target.join(temp_source,
420
                                    pb,
421
                                    msg,
422
                                    version_ids,
423
                                    ignore_missing)
424
        finally:
425
            pb.finished()
1563.2.13 by Robert Collins
InterVersionedFile implemented.
426
1563.2.12 by Robert Collins
Checkpointing: created InterObject to factor out common inter object worker code, added InterVersionedFile and tests to allow making join work between any versionedfile.
427
428
class InterVersionedFileTestProviderAdapter(object):
429
    """A tool to generate a suite testing multiple inter versioned-file classes.
430
431
    This is done by copying the test once for each interversionedfile provider
432
    and injecting the transport_server, transport_readonly_server,
433
    versionedfile_factory and versionedfile_factory_to classes into each copy.
434
    Each copy is also given a new id() to make it easy to identify.
435
    """
436
437
    def __init__(self, transport_server, transport_readonly_server, formats):
438
        self._transport_server = transport_server
439
        self._transport_readonly_server = transport_readonly_server
440
        self._formats = formats
441
    
442
    def adapt(self, test):
443
        result = TestSuite()
444
        for (interversionedfile_class,
445
             versionedfile_factory,
446
             versionedfile_factory_to) in self._formats:
447
            new_test = deepcopy(test)
448
            new_test.transport_server = self._transport_server
449
            new_test.transport_readonly_server = self._transport_readonly_server
450
            new_test.interversionedfile_class = interversionedfile_class
451
            new_test.versionedfile_factory = versionedfile_factory
452
            new_test.versionedfile_factory_to = versionedfile_factory_to
453
            def make_new_test_id():
454
                new_id = "%s(%s)" % (new_test.id(), interversionedfile_class.__name__)
455
                return lambda: new_id
456
            new_test.id = make_new_test_id()
457
            result.addTest(new_test)
458
        return result
459
460
    @staticmethod
461
    def default_test_list():
462
        """Generate the default list of interversionedfile permutations to test."""
463
        from bzrlib.weave import WeaveFile
1563.2.16 by Robert Collins
Change WeaveStore into VersionedFileStore and make its versoined file class parameterisable.
464
        from bzrlib.knit import KnitVersionedFile
1563.2.12 by Robert Collins
Checkpointing: created InterObject to factor out common inter object worker code, added InterVersionedFile and tests to allow making join work between any versionedfile.
465
        result = []
466
        # test the fallback InterVersionedFile from weave to annotated knits
467
        result.append((InterVersionedFile, 
468
                       WeaveFile,
1563.2.16 by Robert Collins
Change WeaveStore into VersionedFileStore and make its versoined file class parameterisable.
469
                       KnitVersionedFile))
1563.2.12 by Robert Collins
Checkpointing: created InterObject to factor out common inter object worker code, added InterVersionedFile and tests to allow making join work between any versionedfile.
470
        for optimiser in InterVersionedFile._optimisers:
471
            result.append((optimiser,
472
                           optimiser._matching_file_factory,
473
                           optimiser._matching_file_factory
474
                           ))
475
        # if there are specific combinations we want to use, we can add them 
476
        # here.
477
        return result