/brz/remove-bazaar

To get this branch, use:
bzr branch http://gegoxaren.bato24.eu/bzr/brz/remove-bazaar
3350.6.10 by Martin Pool
VersionedFiles review cleanups
1
# Copyright (C) 2005, 2006, 2008 Canonical Ltd
1773.4.1 by Martin Pool
Add pyflakes makefile target; fix many warnings
2
#
1110 by Martin Pool
- merge aaron's merge improvements:
3
# This program is free software; you can redistribute it and/or modify
4
# it under the terms of the GNU General Public License as published by
5
# the Free Software Foundation; either version 2 of the License, or
6
# (at your option) any later version.
1773.4.1 by Martin Pool
Add pyflakes makefile target; fix many warnings
7
#
1110 by Martin Pool
- merge aaron's merge improvements:
8
# This program is distributed in the hope that it will be useful,
9
# but WITHOUT ANY WARRANTY; without even the implied warranty of
10
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
11
# GNU General Public License for more details.
1773.4.1 by Martin Pool
Add pyflakes makefile target; fix many warnings
12
#
1110 by Martin Pool
- merge aaron's merge improvements:
13
# You should have received a copy of the GNU General Public License
14
# along with this program; if not, write to the Free Software
15
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
16
1545.2.3 by Aaron Bentley
Updated following j-a-meinel's comments
17
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
18
import errno
19
from itertools import chain
1185.1.2 by Martin Pool
- merge various windows and other fixes from Ollie Rutherfurd
20
import os
1773.4.1 by Martin Pool
Add pyflakes makefile target; fix many warnings
21
import warnings
1113 by Martin Pool
- fix is_ancestor import problem in merge
22
1996.3.18 by John Arbash Meinel
Now that mkdtemp and rmtree are lazy, they should not be directly improted.
23
from bzrlib import (
1551.19.17 by Aaron Bentley
Add debugging flag for merges
24
    debug,
2590.2.11 by Aaron Bentley
Aggressively cache trees, use dirstate. re-mplement _add_parent.
25
    errors,
3514.2.14 by John Arbash Meinel
Bring in the code to collapse linear portions of the graph.
26
    graph as _mod_graph,
1996.3.18 by John Arbash Meinel
Now that mkdtemp and rmtree are lazy, they should not be directly improted.
27
    osutils,
1551.15.46 by Aaron Bentley
Move plan merge to tree
28
    patiencediff,
2221.4.15 by Aaron Bentley
Use RegistryOption for merge type
29
    registry,
2598.5.1 by Aaron Bentley
Start eliminating the use of None to indicate null revision
30
    revision as _mod_revision,
3514.4.5 by John Arbash Meinel
Initial work on _entries_lca.
31
    tree as _mod_tree,
3514.2.5 by John Arbash Meinel
Switch to the get_parent_map design I had settled on.
32
    tsort,
1996.3.18 by John Arbash Meinel
Now that mkdtemp and rmtree are lazy, they should not be directly improted.
33
    )
1185.11.5 by John Arbash Meinel
Merged up-to-date against mainline, still broken.
34
from bzrlib.branch import Branch
1666.1.4 by Robert Collins
* 'Metadir' is now the default disk format. This improves behaviour in
35
from bzrlib.conflicts import ConflictList, Conflict
1457.1.8 by Robert Collins
Replace the WorkingTree.revert method algorithm with a call to merge_inner.
36
from bzrlib.errors import (BzrCommandError,
1534.4.28 by Robert Collins
first cut at merge from integration.
37
                           BzrError,
1457.1.8 by Robert Collins
Replace the WorkingTree.revert method algorithm with a call to merge_inner.
38
                           NoCommonAncestor,
39
                           NoCommits,
1534.4.28 by Robert Collins
first cut at merge from integration.
40
                           NoSuchRevision,
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
41
                           NoSuchFile,
1545.2.6 by Aaron Bentley
Removed _merge, renamed MergeConflictHandler to _MergeConflictHandler
42
                           NotBranchError,
1185.33.27 by Martin Pool
[merge] much integrated work from robert and john
43
                           NotVersionedError,
1457.1.8 by Robert Collins
Replace the WorkingTree.revert method algorithm with a call to merge_inner.
44
                           UnrelatedBranches,
1534.10.12 by Aaron Bentley
Merge produces new conflicts
45
                           UnsupportedOperation,
1457.1.8 by Robert Collins
Replace the WorkingTree.revert method algorithm with a call to merge_inner.
46
                           WorkingTreeNotRevision,
1558.15.3 by Aaron Bentley
Handle binary files for diff3 merges
47
                           BinaryFile,
1534.4.28 by Robert Collins
first cut at merge from integration.
48
                           )
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
49
from bzrlib.graph import Graph
1534.7.167 by Aaron Bentley
PEP8 and comment cleanups
50
from bzrlib.merge3 import Merge3
1996.3.18 by John Arbash Meinel
Now that mkdtemp and rmtree are lazy, they should not be directly improted.
51
from bzrlib.osutils import rename, pathjoin
1551.2.32 by Aaron Bentley
Handle progress phases more nicely in merge
52
from progress import DummyProgress, ProgressPhase
3052.1.3 by John Arbash Meinel
deprecate revision.is_ancestor, update the callers and the tests.
53
from bzrlib.revision import (NULL_REVISION, ensure_null)
1558.15.5 by Aaron Bentley
Fixed binary handling in weave merge
54
from bzrlib.textfile import check_text_lines
3200.1.1 by James Westby
Make pull --quiet more quiet. Fixes #185907.
55
from bzrlib.trace import mutter, warning, note, is_quiet
3008.1.9 by Michael Hudson
wanton hacking that lets me write an efficient version of get_diff_as_merged
56
from bzrlib.transform import (TransformPreview, TreeTransform,
57
                              resolve_conflicts, cook_conflicts,
2590.2.8 by Aaron Bentley
Restore conflict handling changes
58
                              conflict_pass, FinalPaths, create_by_entry,
59
                              unique_add, ROOT_PARENT)
1551.15.46 by Aaron Bentley
Move plan merge to tree
60
from bzrlib.versionedfile import PlanWeaveMerge
1773.4.1 by Martin Pool
Add pyflakes makefile target; fix many warnings
61
from bzrlib import ui
1545.2.6 by Aaron Bentley
Removed _merge, renamed MergeConflictHandler to _MergeConflictHandler
62
63
# TODO: Report back as changes are merged in
64
2325.3.1 by John Arbash Meinel
New helper function for merge, which allows us to re-use the existing workingtree, rather than opening it again.
65
1185.35.4 by Aaron Bentley
Implemented remerge
66
def transform_tree(from_tree, to_tree, interesting_ids=None):
67
    merge_inner(from_tree.branch, to_tree, from_tree, ignore_zero=True,
1558.1.3 by Aaron Bentley
Fixed deprecated op use in test suite
68
                interesting_ids=interesting_ids, this_tree=from_tree)
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
69
1457.1.12 by Robert Collins
Update comment to reflect author.
70
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
71
class Merger(object):
2255.2.31 by Robert Collins
Work in progress to make merge_inner work with dirstate trees.
72
    def __init__(self, this_branch, other_tree=None, base_tree=None,
2100.3.31 by Aaron Bentley
Merged bzr.dev (17 tests failing)
73
                 this_tree=None, pb=DummyProgress(), change_reporter=None,
3146.5.3 by Aaron Bentley
Avoid retrieving revision graph twice
74
                 recurse='down', revision_graph=None):
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
75
        object.__init__(self)
76
        self.this_branch = this_branch
2598.5.4 by Aaron Bentley
Restore original Branch.last_revision behavior, fix bits that care
77
        self.this_basis = _mod_revision.ensure_null(
78
            this_branch.last_revision())
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
79
        self.this_rev_id = None
1534.4.26 by Robert Collins
Move working tree initialisation out from Branch.initialize, deprecated Branch.initialize to Branch.create.
80
        self.this_tree = this_tree
1185.12.83 by Aaron Bentley
Preliminary weave merge support
81
        self.this_revision_tree = None
1185.35.5 by Aaron Bentley
Made weave merge succeed if interesting files match history
82
        self.this_basis_tree = None
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
83
        self.other_tree = other_tree
2100.3.29 by Aaron Bentley
Get merge working initially
84
        self.other_branch = None
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
85
        self.base_tree = base_tree
86
        self.ignore_zero = False
87
        self.backup_files = False
88
        self.interesting_ids = None
2590.2.5 by Aaron Bentley
Allow selected files to be specified instead of selected ids
89
        self.interesting_files = None
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
90
        self.show_base = False
1185.24.3 by Aaron Bentley
Integrated reprocessing into the rest of the merge stuff
91
        self.reprocess = False
2255.2.31 by Robert Collins
Work in progress to make merge_inner work with dirstate trees.
92
        self._pb = pb
1551.2.32 by Aaron Bentley
Handle progress phases more nicely in merge
93
        self.pp = None
2100.3.29 by Aaron Bentley
Get merge working initially
94
        self.recurse = recurse
1551.11.9 by Aaron Bentley
Apply change reporting to merge
95
        self.change_reporter = change_reporter
2590.2.11 by Aaron Bentley
Aggressively cache trees, use dirstate. re-mplement _add_parent.
96
        self._cached_trees = {}
3146.5.3 by Aaron Bentley
Avoid retrieving revision graph twice
97
        self._revision_graph = revision_graph
3146.5.1 by Aaron Bentley
Make merge --uncommitted work with merge-type weave
98
        self._base_is_ancestor = None
99
        self._base_is_other_ancestor = None
3514.4.1 by John Arbash Meinel
Update Merger to set a flag when we encounter a criss-cross merge.
100
        self._is_criss_cross = False
3514.4.2 by John Arbash Meinel
Extract the lca trees and the base tree when we find a criss-cross
101
        self._lca_trees = None
3146.5.1 by Aaron Bentley
Make merge --uncommitted work with merge-type weave
102
103
    @property
104
    def revision_graph(self):
105
        if self._revision_graph is None:
3146.5.3 by Aaron Bentley
Avoid retrieving revision graph twice
106
            self._revision_graph = self.this_branch.repository.get_graph()
3146.5.1 by Aaron Bentley
Make merge --uncommitted work with merge-type weave
107
        return self._revision_graph
108
109
    def _set_base_is_ancestor(self, value):
110
        self._base_is_ancestor = value
111
112
    def _get_base_is_ancestor(self):
113
        if self._base_is_ancestor is None:
114
            self._base_is_ancestor = self.revision_graph.is_ancestor(
115
                self.base_rev_id, self.this_basis)
116
        return self._base_is_ancestor
117
118
    base_is_ancestor = property(_get_base_is_ancestor, _set_base_is_ancestor)
119
120
    def _set_base_is_other_ancestor(self, value):
121
        self._base_is_other_ancestor = value
122
123
    def _get_base_is_other_ancestor(self):
124
        if self._base_is_other_ancestor is None:
3249.3.1 by John Arbash Meinel
Implement cherrypick support for Merge3
125
            if self.other_basis is None:
126
                return True
127
            self._base_is_other_ancestor = self.revision_graph.is_ancestor(
3146.5.3 by Aaron Bentley
Avoid retrieving revision graph twice
128
                self.base_rev_id, self.other_basis)
3146.5.1 by Aaron Bentley
Make merge --uncommitted work with merge-type weave
129
        return self._base_is_other_ancestor
130
131
    base_is_other_ancestor = property(_get_base_is_other_ancestor,
132
                                      _set_base_is_other_ancestor)
2590.2.11 by Aaron Bentley
Aggressively cache trees, use dirstate. re-mplement _add_parent.
133
1551.15.67 by Aaron Bentley
Stop using _merge_helper for merging
134
    @staticmethod
135
    def from_uncommitted(tree, other_tree, pb):
1551.15.74 by Aaron Bentley
Textual updates from review
136
        """Return a Merger for uncommitted changes in other_tree.
137
138
        :param tree: The tree to merge into
139
        :param other_tree: The tree to get uncommitted changes from
140
        :param pb: A progress indicator
141
        """
1551.15.67 by Aaron Bentley
Stop using _merge_helper for merging
142
        merger = Merger(tree.branch, other_tree, other_tree.basis_tree(), tree,
143
                        pb)
144
        merger.base_rev_id = merger.base_tree.get_revision_id()
145
        merger.other_rev_id = None
3146.5.1 by Aaron Bentley
Make merge --uncommitted work with merge-type weave
146
        merger.other_basis = merger.base_rev_id
1551.15.67 by Aaron Bentley
Stop using _merge_helper for merging
147
        return merger
148
149
    @classmethod
150
    def from_mergeable(klass, tree, mergeable, pb):
1551.15.74 by Aaron Bentley
Textual updates from review
151
        """Return a Merger for a bundle or merge directive.
152
153
        :param tree: The tree to merge changes into
154
        :param mergeable: A merge directive or bundle
155
        :param pb: A progress indicator
156
        """
1551.15.67 by Aaron Bentley
Stop using _merge_helper for merging
157
        mergeable.install_revisions(tree.branch.repository)
158
        base_revision_id, other_revision_id, verified =\
159
            mergeable.get_merge_request(tree.branch.repository)
3146.5.3 by Aaron Bentley
Avoid retrieving revision graph twice
160
        revision_graph = tree.branch.repository.get_graph()
1551.19.44 by Aaron Bentley
Fix handling of old merge directives with stricter get_parent_map
161
        if base_revision_id is not None:
162
            if (base_revision_id != _mod_revision.NULL_REVISION and
163
                revision_graph.is_ancestor(
164
                base_revision_id, tree.branch.last_revision())):
165
                base_revision_id = None
166
            else:
167
                warning('Performing cherrypick')
1551.15.67 by Aaron Bentley
Stop using _merge_helper for merging
168
        merger = klass.from_revision_ids(pb, tree, other_revision_id,
3146.5.3 by Aaron Bentley
Avoid retrieving revision graph twice
169
                                         base_revision_id, revision_graph=
170
                                         revision_graph)
1551.15.67 by Aaron Bentley
Stop using _merge_helper for merging
171
        return merger, verified
172
173
    @staticmethod
3249.3.1 by John Arbash Meinel
Implement cherrypick support for Merge3
174
    def from_revision_ids(pb, tree, other, base=None, other_branch=None,
3146.5.3 by Aaron Bentley
Avoid retrieving revision graph twice
175
                          base_branch=None, revision_graph=None):
1551.15.74 by Aaron Bentley
Textual updates from review
176
        """Return a Merger for revision-ids.
177
178
        :param tree: The tree to merge changes into
179
        :param other: The revision-id to use as OTHER
180
        :param base: The revision-id to use as BASE.  If not specified, will
181
            be auto-selected.
182
        :param other_branch: A branch containing the other revision-id.  If
3249.3.1 by John Arbash Meinel
Implement cherrypick support for Merge3
183
            not supplied, tree.branch is used.
1551.15.74 by Aaron Bentley
Textual updates from review
184
        :param base_branch: A branch containing the base revision-id.  If
3249.3.1 by John Arbash Meinel
Implement cherrypick support for Merge3
185
            not supplied, other_branch or tree.branch will be used.
186
        :param revision_graph: If you have a revision_graph precomputed, pass
187
            it in, otherwise it will be created for you.
1551.15.74 by Aaron Bentley
Textual updates from review
188
        :param pb: A progress indicator
189
        """
3249.3.1 by John Arbash Meinel
Implement cherrypick support for Merge3
190
        merger = Merger(tree.branch, this_tree=tree, pb=pb,
3146.5.3 by Aaron Bentley
Avoid retrieving revision graph twice
191
                        revision_graph=revision_graph)
1551.15.67 by Aaron Bentley
Stop using _merge_helper for merging
192
        if other_branch is None:
3249.3.1 by John Arbash Meinel
Implement cherrypick support for Merge3
193
            other_branch = tree.branch
1551.15.67 by Aaron Bentley
Stop using _merge_helper for merging
194
        merger.set_other_revision(other, other_branch)
195
        if base is None:
196
            merger.find_base()
197
        else:
198
            if base_branch is None:
199
                base_branch = other_branch
200
            merger.set_base_revision(base, base_branch)
201
        return merger
202
2590.2.11 by Aaron Bentley
Aggressively cache trees, use dirstate. re-mplement _add_parent.
203
    def revision_tree(self, revision_id, branch=None):
204
        if revision_id not in self._cached_trees:
205
            if branch is None:
206
                branch = self.this_branch
207
            try:
208
                tree = self.this_tree.revision_tree(revision_id)
209
            except errors.NoSuchRevisionInTree:
210
                tree = branch.repository.revision_tree(revision_id)
211
            self._cached_trees[revision_id] = tree
212
        return self._cached_trees[revision_id]
213
2485.8.58 by Vincent Ladeuil
merge bzr.dev@1617
214
    def _get_tree(self, treespec, possible_transports=None):
2590.2.11 by Aaron Bentley
Aggressively cache trees, use dirstate. re-mplement _add_parent.
215
        from bzrlib import workingtree
216
        location, revno = treespec
217
        if revno is None:
218
            tree = workingtree.WorkingTree.open_containing(location)[0]
219
            return tree.branch, tree
2485.8.58 by Vincent Ladeuil
merge bzr.dev@1617
220
        branch = Branch.open_containing(location, possible_transports)[0]
2590.2.11 by Aaron Bentley
Aggressively cache trees, use dirstate. re-mplement _add_parent.
221
        if revno == -1:
222
            revision_id = branch.last_revision()
223
        else:
224
            revision_id = branch.get_rev_id(revno)
225
        revision_id = ensure_null(revision_id)
226
        return branch, self.revision_tree(revision_id, branch)
1185.12.83 by Aaron Bentley
Preliminary weave merge support
227
228
    def ensure_revision_trees(self):
229
        if self.this_revision_tree is None:
2590.2.11 by Aaron Bentley
Aggressively cache trees, use dirstate. re-mplement _add_parent.
230
            self.this_basis_tree = self.revision_tree(self.this_basis)
1185.35.5 by Aaron Bentley
Made weave merge succeed if interesting files match history
231
            if self.this_basis == self.this_rev_id:
232
                self.this_revision_tree = self.this_basis_tree
233
1185.12.83 by Aaron Bentley
Preliminary weave merge support
234
        if self.other_rev_id is None:
235
            other_basis_tree = self.revision_tree(self.other_basis)
1852.10.3 by Robert Collins
Remove all uses of compare_trees and replace with Tree.changes_from throughout bzrlib.
236
            changes = other_basis_tree.changes_from(self.other_tree)
1185.12.83 by Aaron Bentley
Preliminary weave merge support
237
            if changes.has_changed():
238
                raise WorkingTreeNotRevision(self.this_tree)
1773.4.1 by Martin Pool
Add pyflakes makefile target; fix many warnings
239
            other_rev_id = self.other_basis
1185.12.83 by Aaron Bentley
Preliminary weave merge support
240
            self.other_tree = other_basis_tree
241
242
    def file_revisions(self, file_id):
243
        self.ensure_revision_trees()
244
        def get_id(tree, file_id):
245
            revision_id = tree.inventory[file_id].revision
246
            return revision_id
1185.35.5 by Aaron Bentley
Made weave merge succeed if interesting files match history
247
        if self.this_rev_id is None:
248
            if self.this_basis_tree.get_file_sha1(file_id) != \
249
                self.this_tree.get_file_sha1(file_id):
250
                raise WorkingTreeNotRevision(self.this_tree)
251
252
        trees = (self.this_basis_tree, self.other_tree)
1185.12.83 by Aaron Bentley
Preliminary weave merge support
253
        return [get_id(tree, file_id) for tree in trees]
254
1185.82.44 by Aaron Bentley
Switch to merge_changeset in test suite
255
    def check_basis(self, check_clean, require_commits=True):
256
        if self.this_basis is None and require_commits is True:
2249.3.1 by John Arbash Meinel
Mention using 'bzr pull' if there are no commits in current branch
257
            raise BzrCommandError("This branch has no commits."
258
                                  " (perhaps you would prefer 'bzr pull')")
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
259
        if check_clean:
260
            self.compare_basis()
261
            if self.this_basis != self.this_rev_id:
2796.2.1 by Aaron Bentley
Begin work on reconfigure command
262
                raise errors.UncommittedChanges(self.this_tree)
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
263
264
    def compare_basis(self):
2590.2.20 by Aaron Bentley
Fix handling of ghost base trees
265
        try:
266
            basis_tree = self.revision_tree(self.this_tree.last_revision())
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
267
        except errors.NoSuchRevision:
2590.2.20 by Aaron Bentley
Fix handling of ghost base trees
268
            basis_tree = self.this_tree.basis_tree()
269
        changes = self.this_tree.changes_from(basis_tree)
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
270
        if not changes.has_changed():
271
            self.this_rev_id = self.this_basis
272
273
    def set_interesting_files(self, file_list):
2590.2.7 by Aaron Bentley
Misc cleanup
274
        self.interesting_files = file_list
1457.1.8 by Robert Collins
Replace the WorkingTree.revert method algorithm with a call to merge_inner.
275
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
276
    def set_pending(self):
2644.1.1 by Wouter van Heyst
Fix bug #127115 by checking for self.other_rev_id being None in Merger.set_pending()
277
        if not self.base_is_ancestor or not self.base_is_other_ancestor or self.other_rev_id is None:
1185.12.77 by Aaron Bentley
Prevented all ancestors from being marked as pending merges
278
            return
2590.2.11 by Aaron Bentley
Aggressively cache trees, use dirstate. re-mplement _add_parent.
279
        self._add_parent()
280
281
    def _add_parent(self):
282
        new_parents = self.this_tree.get_parent_ids() + [self.other_rev_id]
2590.2.20 by Aaron Bentley
Fix handling of ghost base trees
283
        new_parent_trees = []
284
        for revision_id in new_parents:
285
            try:
286
                tree = self.revision_tree(revision_id)
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
287
            except errors.NoSuchRevision:
2590.2.20 by Aaron Bentley
Fix handling of ghost base trees
288
                tree = None
289
            else:
290
                tree.lock_read()
291
            new_parent_trees.append((revision_id, tree))
2590.2.11 by Aaron Bentley
Aggressively cache trees, use dirstate. re-mplement _add_parent.
292
        try:
2590.2.20 by Aaron Bentley
Fix handling of ghost base trees
293
            self.this_tree.set_parent_trees(new_parent_trees,
294
                                            allow_leftmost_as_ghost=True)
2590.2.11 by Aaron Bentley
Aggressively cache trees, use dirstate. re-mplement _add_parent.
295
        finally:
296
            for _revision_id, tree in new_parent_trees:
2590.2.20 by Aaron Bentley
Fix handling of ghost base trees
297
                if tree is not None:
298
                    tree.unlock()
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
299
2485.8.37 by Vincent Ladeuil
Fix merge multiple connections. Test suite *not* passing (sftp
300
    def set_other(self, other_revision, possible_transports=None):
1979.2.1 by Robert Collins
(robertc) adds a convenience method "merge_from_branch" to WorkingTree.
301
        """Set the revision and tree to merge from.
302
303
        This sets the other_tree, other_rev_id, other_basis attributes.
304
305
        :param other_revision: The [path, revision] list to merge from.
306
        """
2485.8.58 by Vincent Ladeuil
merge bzr.dev@1617
307
        self.other_branch, self.other_tree = self._get_tree(other_revision,
308
                                                            possible_transports)
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
309
        if other_revision[1] == -1:
2598.5.4 by Aaron Bentley
Restore original Branch.last_revision behavior, fix bits that care
310
            self.other_rev_id = _mod_revision.ensure_null(
311
                self.other_branch.last_revision())
2598.5.1 by Aaron Bentley
Start eliminating the use of None to indicate null revision
312
            if _mod_revision.is_null(self.other_rev_id):
2100.3.31 by Aaron Bentley
Merged bzr.dev (17 tests failing)
313
                raise NoCommits(self.other_branch)
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
314
            self.other_basis = self.other_rev_id
315
        elif other_revision[1] is not None:
2100.3.31 by Aaron Bentley
Merged bzr.dev (17 tests failing)
316
            self.other_rev_id = self.other_branch.get_rev_id(other_revision[1])
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
317
            self.other_basis = self.other_rev_id
318
        else:
319
            self.other_rev_id = None
2100.3.31 by Aaron Bentley
Merged bzr.dev (17 tests failing)
320
            self.other_basis = self.other_branch.last_revision()
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
321
            if self.other_basis is None:
2100.3.31 by Aaron Bentley
Merged bzr.dev (17 tests failing)
322
                raise NoCommits(self.other_branch)
2590.2.11 by Aaron Bentley
Aggressively cache trees, use dirstate. re-mplement _add_parent.
323
        if self.other_rev_id is not None:
324
            self._cached_trees[self.other_rev_id] = self.other_tree
2590.2.19 by Aaron Bentley
Avoid fetch within a repository
325
        self._maybe_fetch(self.other_branch,self.this_branch, self.other_basis)
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
326
2100.3.29 by Aaron Bentley
Get merge working initially
327
    def set_other_revision(self, revision_id, other_branch):
328
        """Set 'other' based on a branch and revision id
329
330
        :param revision_id: The revision to use for a tree
331
        :param other_branch: The branch containing this tree
332
        """
333
        self.other_rev_id = revision_id
334
        self.other_branch = other_branch
2590.2.19 by Aaron Bentley
Avoid fetch within a repository
335
        self._maybe_fetch(other_branch, self.this_branch, self.other_rev_id)
2100.3.29 by Aaron Bentley
Get merge working initially
336
        self.other_tree = self.revision_tree(revision_id)
337
        self.other_basis = revision_id
338
2520.4.110 by Aaron Bentley
Implement cherrypick support for merge directives
339
    def set_base_revision(self, revision_id, branch):
340
        """Set 'base' based on a branch and revision id
341
342
        :param revision_id: The revision to use for a tree
343
        :param branch: The branch containing this tree
344
        """
345
        self.base_rev_id = revision_id
346
        self.base_branch = branch
2520.4.132 by Aaron Bentley
Merge from bzr.dev
347
        self._maybe_fetch(branch, self.this_branch, revision_id)
2520.4.110 by Aaron Bentley
Implement cherrypick support for merge directives
348
        self.base_tree = self.revision_tree(revision_id)
2520.4.132 by Aaron Bentley
Merge from bzr.dev
349
2590.2.19 by Aaron Bentley
Avoid fetch within a repository
350
    def _maybe_fetch(self, source, target, revision_id):
2665.5.2 by Aaron Bentley
Switch commit and merge to Repository.has_same_location
351
        if not source.repository.has_same_location(target.repository):
2590.2.19 by Aaron Bentley
Avoid fetch within a repository
352
            target.fetch(source, revision_id)
2520.4.110 by Aaron Bentley
Implement cherrypick support for merge directives
353
1185.82.25 by Aaron Bentley
Added changeset-merging functionality
354
    def find_base(self):
2590.2.22 by Aaron Bentley
Remove cruft
355
        revisions = [ensure_null(self.this_basis),
356
                     ensure_null(self.other_basis)]
357
        if NULL_REVISION in revisions:
358
            self.base_rev_id = NULL_REVISION
359
        else:
3514.4.2 by John Arbash Meinel
Extract the lca trees and the base tree when we find a criss-cross
360
            lcas = self.revision_graph.find_lca(revisions[0], revisions[1])
361
            if len(lcas) == 0:
362
                self.base_rev_id = NULL_REVISION
363
            elif len(lcas) == 1:
364
                self.base_rev_id = list(lcas)[0]
365
            else: # len(lcas) > 1
366
                self.base_rev_id = self.revision_graph.find_unique_lca(
367
                                        *lcas)
368
                self._is_criss_cross = True
2590.2.22 by Aaron Bentley
Remove cruft
369
            if self.base_rev_id == NULL_REVISION:
370
                raise UnrelatedBranches()
3514.4.2 by John Arbash Meinel
Extract the lca trees and the base tree when we find a criss-cross
371
            if self._is_criss_cross:
1551.19.11 by Aaron Bentley
Add criss-cross help topic
372
                warning('Warning: criss-cross merge encountered.  See bzr'
373
                        ' help criss-cross.')
3514.4.2 by John Arbash Meinel
Extract the lca trees and the base tree when we find a criss-cross
374
                interesting_revision_ids = [self.base_rev_id]
375
                interesting_revision_ids.extend(lcas)
376
                interesting_trees = dict((t.get_revision_id(), t)
377
                    for t in self.this_branch.repository.revision_trees(
378
                        interesting_revision_ids))
379
                self._cached_trees.update(interesting_trees)
380
                self.base_tree = interesting_trees.pop(self.base_rev_id)
381
                self._lca_trees = interesting_trees
382
            else:
383
                self.base_tree = self.revision_tree(self.base_rev_id)
2590.2.13 by Aaron Bentley
Make find_base implement the base_finding code
384
        self.base_is_ancestor = True
2590.2.18 by Aaron Bentley
Merge is_ancestor fix
385
        self.base_is_other_ancestor = True
1185.82.25 by Aaron Bentley
Added changeset-merging functionality
386
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
387
    def set_base(self, base_revision):
1979.2.1 by Robert Collins
(robertc) adds a convenience method "merge_from_branch" to WorkingTree.
388
        """Set the base revision to use for the merge.
389
390
        :param base_revision: A 2-list containing a path and revision number.
391
        """
1185.12.96 by Aaron Bentley
Merge from mpool
392
        mutter("doing merge() with no base_revision specified")
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
393
        if base_revision == [None, None]:
2590.2.13 by Aaron Bentley
Make find_base implement the base_finding code
394
            self.find_base()
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
395
        else:
2590.2.11 by Aaron Bentley
Aggressively cache trees, use dirstate. re-mplement _add_parent.
396
            base_branch, self.base_tree = self._get_tree(base_revision)
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
397
            if base_revision[1] == -1:
398
                self.base_rev_id = base_branch.last_revision()
399
            elif base_revision[1] is None:
2598.5.3 by Aaron Bentley
Push NULL_REVISION deeper
400
                self.base_rev_id = _mod_revision.NULL_REVISION
493 by Martin Pool
- Merge aaron's merge command
401
            else:
2598.5.3 by Aaron Bentley
Push NULL_REVISION deeper
402
                self.base_rev_id = _mod_revision.ensure_null(
403
                    base_branch.get_rev_id(base_revision[1]))
2590.2.19 by Aaron Bentley
Avoid fetch within a repository
404
            self._maybe_fetch(base_branch, self.this_branch, self.base_rev_id)
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
405
3008.1.8 by Michael Hudson
extract merger creation into a method
406
    def make_merger(self):
2255.2.31 by Robert Collins
Work in progress to make merge_inner work with dirstate trees.
407
        kwargs = {'working_tree':self.this_tree, 'this_tree': self.this_tree,
408
                  'other_tree': self.other_tree,
1551.2.32 by Aaron Bentley
Handle progress phases more nicely in merge
409
                  'interesting_ids': self.interesting_ids,
2590.2.5 by Aaron Bentley
Allow selected files to be specified instead of selected ids
410
                  'interesting_files': self.interesting_files,
3008.1.11 by Michael Hudson
restore the default behaviour of Merge3Merger.__init__().
411
                  'pp': self.pp,
412
                  'do_merge': False}
1534.7.84 by Aaron Bentley
Added reprocess support, support for varying merge types
413
        if self.merge_type.requires_base:
414
            kwargs['base_tree'] = self.base_tree
1534.7.137 by Aaron Bentley
Avoided generating a new tree for every weave merge
415
        if self.merge_type.supports_reprocess:
416
            kwargs['reprocess'] = self.reprocess
417
        elif self.reprocess:
1551.6.14 by Aaron Bentley
Tweaks from merge review
418
            raise BzrError("Conflict reduction is not supported for merge"
419
                                  " type %s." % self.merge_type)
1534.7.137 by Aaron Bentley
Avoided generating a new tree for every weave merge
420
        if self.merge_type.supports_show_base:
421
            kwargs['show_base'] = self.show_base
422
        elif self.show_base:
1534.8.2 by Aaron Bentley
Implemented weave merge
423
            raise BzrError("Showing base is not supported for this"
3008.1.14 by Michael Hudson
trivial commit for testing
424
                           " merge type. %s" % self.merge_type)
3062.2.7 by Aaron Bentley
Prevent reverse cherry-picking with weave
425
        if (not getattr(self.merge_type, 'supports_reverse_cherrypick', True)
426
            and not self.base_is_other_ancestor):
427
            raise errors.CannotReverseCherrypick()
3249.3.1 by John Arbash Meinel
Implement cherrypick support for Merge3
428
        if self.merge_type.supports_cherrypick:
3062.2.9 by Aaron Bentley
Don't use the base if not cherrypicking
429
            kwargs['cherrypick'] = (not self.base_is_ancestor or
430
                                    not self.base_is_other_ancestor)
3514.4.4 by John Arbash Meinel
Test that the lca_trees are passed down to the Merger object when appropriate.
431
        if self._is_criss_cross and getattr(self.merge_type,
432
                                            'supports_lca_trees', False):
433
            kwargs['lca_trees'] = self._lca_trees
3008.1.8 by Michael Hudson
extract merger creation into a method
434
        return self.merge_type(pb=self._pb,
435
                               change_reporter=self.change_reporter,
436
                               **kwargs)
437
438
    def do_merge(self):
2255.2.50 by John Arbash Meinel
merge only needs a lock_tree_write() on the working tree, not a full lock_write()
439
        self.this_tree.lock_tree_write()
2255.2.31 by Robert Collins
Work in progress to make merge_inner work with dirstate trees.
440
        if self.base_tree is not None:
441
            self.base_tree.lock_read()
442
        if self.other_tree is not None:
443
            self.other_tree.lock_read()
444
        try:
3249.3.1 by John Arbash Meinel
Implement cherrypick support for Merge3
445
            merge = self.make_merger()
3008.1.6 by Michael Hudson
chop up Merge3Merger.__init__ into pieces
446
            merge.do_merge()
2255.2.226 by Robert Collins
Get merge_nested finally working: change nested tree iterators to take file_ids, and ensure the right branch is connected to in the merge logic. May not be suitable for shared repositories yet.
447
            if self.recurse == 'down':
3504.2.2 by John Arbash Meinel
Find some code that thought we were dealing in absolute paths
448
                for relpath, file_id in self.this_tree.iter_references():
449
                    sub_tree = self.this_tree.get_nested_tree(file_id, relpath)
2255.2.226 by Robert Collins
Get merge_nested finally working: change nested tree iterators to take file_ids, and ensure the right branch is connected to in the merge logic. May not be suitable for shared repositories yet.
450
                    other_revision = self.other_tree.get_reference_revision(
3504.2.2 by John Arbash Meinel
Find some code that thought we were dealing in absolute paths
451
                        file_id, relpath)
2255.2.226 by Robert Collins
Get merge_nested finally working: change nested tree iterators to take file_ids, and ensure the right branch is connected to in the merge logic. May not be suitable for shared repositories yet.
452
                    if  other_revision == sub_tree.last_revision():
453
                        continue
454
                    sub_merge = Merger(sub_tree.branch, this_tree=sub_tree)
455
                    sub_merge.merge_type = self.merge_type
456
                    other_branch = self.other_branch.reference_parent(file_id, relpath)
457
                    sub_merge.set_other_revision(other_revision, other_branch)
458
                    base_revision = self.base_tree.get_reference_revision(file_id)
459
                    sub_merge.base_tree = \
460
                        sub_tree.branch.repository.revision_tree(base_revision)
3249.3.1 by John Arbash Meinel
Implement cherrypick support for Merge3
461
                    sub_merge.base_rev_id = base_revision
2255.2.226 by Robert Collins
Get merge_nested finally working: change nested tree iterators to take file_ids, and ensure the right branch is connected to in the merge logic. May not be suitable for shared repositories yet.
462
                    sub_merge.do_merge()
463
2255.2.31 by Robert Collins
Work in progress to make merge_inner work with dirstate trees.
464
        finally:
465
            if self.other_tree is not None:
466
                self.other_tree.unlock()
467
            if self.base_tree is not None:
468
                self.base_tree.unlock()
469
            self.this_tree.unlock()
1534.7.151 by Aaron Bentley
Fixed all changes applied successfully
470
        if len(merge.cooked_conflicts) == 0:
3200.1.1 by James Westby
Make pull --quiet more quiet. Fixes #185907.
471
            if not self.ignore_zero and not is_quiet():
1534.7.141 by Aaron Bentley
Added conflict reporting
472
                note("All changes applied successfully.")
473
        else:
474
            note("%d conflicts encountered." % len(merge.cooked_conflicts))
475
1534.7.134 by Aaron Bentley
Hid raw conflicts
476
        return len(merge.cooked_conflicts)
1185.12.76 by Aaron Bentley
Refactored merge and merge_inner to use Merger
477
1545.2.4 by Aaron Bentley
PEP8 fixes
478
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
479
class Merge3Merger(object):
1534.7.167 by Aaron Bentley
PEP8 and comment cleanups
480
    """Three-way merger that uses the merge3 text merger"""
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
481
    requires_base = True
482
    supports_reprocess = True
483
    supports_show_base = True
484
    history_based = False
3249.3.1 by John Arbash Meinel
Implement cherrypick support for Merge3
485
    supports_cherrypick = True
3062.2.7 by Aaron Bentley
Prevent reverse cherry-picking with weave
486
    supports_reverse_cherrypick = True
2590.2.7 by Aaron Bentley
Misc cleanup
487
    winner_idx = {"this": 2, "other": 1, "conflict": 1}
3514.4.4 by John Arbash Meinel
Test that the lca_trees are passed down to the Merger object when appropriate.
488
    supports_lca_trees = True
1534.7.167 by Aaron Bentley
PEP8 and comment cleanups
489
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
490
    def __init__(self, working_tree, this_tree, base_tree, other_tree, 
1558.2.2 by Aaron Bentley
Make remerge honour interesting-ids
491
                 interesting_ids=None, reprocess=False, show_base=False,
2590.2.5 by Aaron Bentley
Allow selected files to be specified instead of selected ids
492
                 pb=DummyProgress(), pp=None, change_reporter=None,
3249.3.1 by John Arbash Meinel
Implement cherrypick support for Merge3
493
                 interesting_files=None, do_merge=True,
3514.4.4 by John Arbash Meinel
Test that the lca_trees are passed down to the Merger object when appropriate.
494
                 cherrypick=False, lca_trees=None):
2590.2.10 by Aaron Bentley
Updates from review
495
        """Initialize the merger object and perform the merge.
496
497
        :param working_tree: The working tree to apply the merge to
498
        :param this_tree: The local tree in the merge operation
499
        :param base_tree: The common tree in the merge operation
500
        :param other_tree: The other other tree to merge changes from
501
        :param interesting_ids: The file_ids of files that should be
502
            participate in the merge.  May not be combined with
503
            interesting_files.
504
        :param: reprocess If True, perform conflict-reduction processing.
505
        :param show_base: If True, show the base revision in text conflicts.
506
            (incompatible with reprocess)
507
        :param pb: A Progress bar
508
        :param pp: A ProgressPhase object
509
        :param change_reporter: An object that should report changes made
510
        :param interesting_files: The tree-relative paths of files that should
511
            participate in the merge.  If these paths refer to directories,
512
            the contents of those directories will also be included.  May not
513
            be combined with interesting_ids.  If neither interesting_files nor
514
            interesting_ids is specified, all files may participate in the
515
            merge.
3514.4.4 by John Arbash Meinel
Test that the lca_trees are passed down to the Merger object when appropriate.
516
        :param lca_trees: Can be set to a dictionary of {revision_id:rev_tree}
517
            if the ancestry was found to include a criss-cross merge.
518
            Otherwise should be None.
2590.2.10 by Aaron Bentley
Updates from review
519
        """
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
520
        object.__init__(self)
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
521
        if interesting_files is not None and interesting_ids is not None:
522
            raise ValueError(
523
                'specify either interesting_ids or interesting_files')
2590.2.4 by Aaron Bentley
Move entry generation to a helper
524
        self.interesting_ids = interesting_ids
2590.2.5 by Aaron Bentley
Allow selected files to be specified instead of selected ids
525
        self.interesting_files = interesting_files
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
526
        self.this_tree = working_tree
527
        self.base_tree = base_tree
528
        self.other_tree = other_tree
529
        self._raw_conflicts = []
530
        self.cooked_conflicts = []
531
        self.reprocess = reprocess
532
        self.show_base = show_base
3514.4.4 by John Arbash Meinel
Test that the lca_trees are passed down to the Merger object when appropriate.
533
        self._lca_trees = lca_trees
1534.9.1 by Aaron Bentley
Added progress bars to merge
534
        self.pb = pb
1551.2.32 by Aaron Bentley
Handle progress phases more nicely in merge
535
        self.pp = pp
1551.11.9 by Aaron Bentley
Apply change reporting to merge
536
        self.change_reporter = change_reporter
3249.3.1 by John Arbash Meinel
Implement cherrypick support for Merge3
537
        self.cherrypick = cherrypick
1551.2.32 by Aaron Bentley
Handle progress phases more nicely in merge
538
        if self.pp is None:
539
            self.pp = ProgressPhase("Merge phase", 3, self.pb)
3008.1.11 by Michael Hudson
restore the default behaviour of Merge3Merger.__init__().
540
        if do_merge:
541
            self.do_merge()
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
542
3008.1.6 by Michael Hudson
chop up Merge3Merger.__init__ into pieces
543
    def do_merge(self):
544
        self.this_tree.lock_tree_write()
545
        self.base_tree.lock_read()
546
        self.other_tree.lock_read()
547
        self.tt = TreeTransform(self.this_tree, self.pb)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
548
        try:
1551.2.32 by Aaron Bentley
Handle progress phases more nicely in merge
549
            self.pp.next_phase()
3008.1.21 by Aaron Bentley
Make compute_transform private, test make_preview_transform
550
            self._compute_transform()
1551.2.32 by Aaron Bentley
Handle progress phases more nicely in merge
551
            self.pp.next_phase()
2590.2.17 by Aaron Bentley
Avoid redundant conflict check
552
            results = self.tt.apply(no_conflicts=True)
1558.4.3 by Aaron Bentley
Merge_modified performance/concurrency fix
553
            self.write_modified(results)
1534.10.12 by Aaron Bentley
Merge produces new conflicts
554
            try:
3008.1.6 by Michael Hudson
chop up Merge3Merger.__init__ into pieces
555
                self.this_tree.add_conflicts(self.cooked_conflicts)
1534.10.12 by Aaron Bentley
Merge produces new conflicts
556
            except UnsupportedOperation:
557
                pass
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
558
        finally:
1711.7.7 by John Arbash Meinel
Don't squelch errors in tt.finalize()
559
            self.tt.finalize()
2255.7.52 by Robert Collins
Lock trees in Merge3Merger correctly.
560
            self.other_tree.unlock()
561
            self.base_tree.unlock()
562
            self.this_tree.unlock()
1551.2.31 by Aaron Bentley
Got merge and revert using nested pbs
563
            self.pb.clear()
1534.7.192 by Aaron Bentley
Record hashes produced by merges
564
3008.1.9 by Michael Hudson
wanton hacking that lets me write an efficient version of get_diff_as_merged
565
    def make_preview_transform(self):
566
        self.base_tree.lock_read()
567
        self.other_tree.lock_read()
568
        self.tt = TransformPreview(self.this_tree)
569
        try:
570
            self.pp.next_phase()
3008.1.21 by Aaron Bentley
Make compute_transform private, test make_preview_transform
571
            self._compute_transform()
3008.1.9 by Michael Hudson
wanton hacking that lets me write an efficient version of get_diff_as_merged
572
            self.pp.next_phase()
573
        finally:
574
            self.other_tree.unlock()
575
            self.base_tree.unlock()
576
            self.pb.clear()
577
        return self.tt
578
3008.1.21 by Aaron Bentley
Make compute_transform private, test make_preview_transform
579
    def _compute_transform(self):
3008.1.6 by Michael Hudson
chop up Merge3Merger.__init__ into pieces
580
        entries = self._entries3()
581
        child_pb = ui.ui_factory.nested_progress_bar()
582
        try:
583
            for num, (file_id, changed, parents3, names3,
584
                      executable3) in enumerate(entries):
585
                child_pb.update('Preparing file merge', num, len(entries))
586
                self._merge_names(file_id, parents3, names3)
587
                if changed:
588
                    file_status = self.merge_contents(file_id)
589
                else:
590
                    file_status = 'unmodified'
591
                self._merge_executable(file_id,
592
                    executable3, file_status)
593
        finally:
594
            child_pb.finished()
595
        self.fix_root()
596
        self.pp.next_phase()
597
        child_pb = ui.ui_factory.nested_progress_bar()
598
        try:
599
            fs_conflicts = resolve_conflicts(self.tt, child_pb,
600
                lambda t, c: conflict_pass(t, c, self.other_tree))
601
        finally:
602
            child_pb.finished()
603
        if self.change_reporter is not None:
604
            from bzrlib import delta
605
            delta.report_changes(
3254.1.1 by Aaron Bentley
Make Tree.iter_changes a public method
606
                self.tt.iter_changes(), self.change_reporter)
3008.1.6 by Michael Hudson
chop up Merge3Merger.__init__ into pieces
607
        self.cook_conflicts(fs_conflicts)
608
        for conflict in self.cooked_conflicts:
609
            warning(conflict)
610
2590.2.4 by Aaron Bentley
Move entry generation to a helper
611
    def _entries3(self):
2590.2.7 by Aaron Bentley
Misc cleanup
612
        """Gather data about files modified between three trees.
613
614
        Return a list of tuples of file_id, changed, parents3, names3,
615
        executable3.  changed is a boolean indicating whether the file contents
616
        or kind were changed.  parents3 is a tuple of parent ids for base,
617
        other and this.  names3 is a tuple of names for base, other and this.
618
        executable3 is a tuple of execute-bit values for base, other and this.
619
        """
2590.2.4 by Aaron Bentley
Move entry generation to a helper
620
        result = []
3254.1.1 by Aaron Bentley
Make Tree.iter_changes a public method
621
        iterator = self.other_tree.iter_changes(self.base_tree,
3514.2.16 by John Arbash Meinel
Review feedback from Ian.
622
                include_unchanged=True, specific_files=self.interesting_files,
2590.2.5 by Aaron Bentley
Allow selected files to be specified instead of selected ids
623
                extra_trees=[self.this_tree])
2590.2.4 by Aaron Bentley
Move entry generation to a helper
624
        for (file_id, paths, changed, versioned, parents, names, kind,
2590.2.5 by Aaron Bentley
Allow selected files to be specified instead of selected ids
625
             executable) in iterator:
2590.2.4 by Aaron Bentley
Move entry generation to a helper
626
            if (self.interesting_ids is not None and
627
                file_id not in self.interesting_ids):
628
                continue
629
            if file_id in self.this_tree.inventory:
630
                entry = self.this_tree.inventory[file_id]
631
                this_name = entry.name
632
                this_parent = entry.parent_id
633
                this_executable = entry.executable
634
            else:
635
                this_name = None
636
                this_parent = None
637
                this_executable = None
638
            parents3 = parents + (this_parent,)
639
            names3 = names + (this_name,)
640
            executable3 = executable + (this_executable,)
641
            result.append((file_id, changed, parents3, names3, executable3))
642
        return result
643
3514.4.5 by John Arbash Meinel
Initial work on _entries_lca.
644
    def _entries_lca(self):
645
        """Gather data about files modified between multiple trees.
646
647
        This compares OTHER versus all LCA trees, and for interesting entries,
648
        it then compares with THIS and BASE.
649
650
        For the multi-valued entries, the format will be (BASE, [lca1, lca2])
651
        :return: [(file_id, changed, parents, names, executable)]
652
            file_id     Simple file_id of the entry
653
            changed     Boolean, True if the kind or contents changed
654
                        else False
655
            parents     ((base, [parent_id, in, lcas]), parent_id_other,
656
                         parent_id_this)
657
            names       ((base, [name, in, lcas]), name_in_other, name_in_this)
658
            executable  ((base, [exec, in, lcas]), exec_in_other, exec_in_this)
659
        """
660
        result = []
661
        walker = _mod_tree.MultiWalker(self.other_tree,
662
                                       self._lca_trees.values())
663
664
        for path, file_id, other_ie, lca_values in walker.iter_all():
665
            # Is this modified at all from any of the other trees?
666
            last_rev = other_ie.revision
667
            for lca_path, ie in lca_values:
668
                if ie.revision != last_rev:
669
                    break
670
            else: # Identical in all trees
671
                continue
672
            base_ie = self.base_tree.inventory[file_id]
673
            this_ie = self.this_tree.inventory[file_id]
674
            result.append((file_id, True,
675
                           ((base_ie.parent_id,
676
                            [ie.parent_id for path, ie in lca_values]),
677
                            other_ie.parent_id, this_ie.parent_id),
678
                           ((base_ie.name,
679
                            [ie.name for path, ie in lca_values]),
680
                            other_ie.name, this_ie.name),
681
                           ((base_ie.executable,
682
                            [ie.executable for path, ie in lca_values]),
683
                            other_ie.executable, this_ie.executable)
684
                          ))
685
        return result
686
687
1731.1.33 by Aaron Bentley
Revert no-special-root changes
688
    def fix_root(self):
689
        try:
690
            self.tt.final_kind(self.tt.root)
691
        except NoSuchFile:
692
            self.tt.cancel_deletion(self.tt.root)
693
        if self.tt.final_file_id(self.tt.root) is None:
694
            self.tt.version_file(self.tt.tree_file_id(self.tt.root), 
695
                                 self.tt.root)
696
        if self.other_tree.inventory.root is None:
697
            return
2946.3.3 by John Arbash Meinel
Prefer tree.get_root_id() as more explicit than tree.path2id('')
698
        other_root_file_id = self.other_tree.get_root_id()
1731.1.33 by Aaron Bentley
Revert no-special-root changes
699
        other_root = self.tt.trans_id_file_id(other_root_file_id)
700
        if other_root == self.tt.root:
701
            return
702
        try:
703
            self.tt.final_kind(other_root)
704
        except NoSuchFile:
705
            return
706
        self.reparent_children(self.other_tree.inventory.root, self.tt.root)
707
        self.tt.cancel_creation(other_root)
708
        self.tt.cancel_versioning(other_root)
709
710
    def reparent_children(self, ie, target):
711
        for thing, child in ie.children.iteritems():
712
            trans_id = self.tt.trans_id_file_id(child.file_id)
713
            self.tt.adjust_path(self.tt.final_name(trans_id), target, trans_id)
714
1534.7.192 by Aaron Bentley
Record hashes produced by merges
715
    def write_modified(self, results):
716
        modified_hashes = {}
717
        for path in results.modified_paths:
718
            file_id = self.this_tree.path2id(self.this_tree.relpath(path))
719
            if file_id is None:
720
                continue
721
            hash = self.this_tree.get_file_sha1(file_id)
722
            if hash is None:
723
                continue
724
            modified_hashes[file_id] = hash
725
        self.this_tree.set_merge_modified(modified_hashes)
726
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
727
    @staticmethod
728
    def parent(entry, file_id):
1534.7.157 by Aaron Bentley
Added more docs
729
        """Determine the parent for a file_id (used as a key method)"""
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
730
        if entry is None:
731
            return None
732
        return entry.parent_id
733
734
    @staticmethod
735
    def name(entry, file_id):
1534.7.157 by Aaron Bentley
Added more docs
736
        """Determine the name for a file_id (used as a key method)"""
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
737
        if entry is None:
738
            return None
739
        return entry.name
740
    
741
    @staticmethod
742
    def contents_sha1(tree, file_id):
1534.7.157 by Aaron Bentley
Added more docs
743
        """Determine the sha1 of the file contents (used as a key method)."""
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
744
        if file_id not in tree:
745
            return None
746
        return tree.get_file_sha1(file_id)
747
748
    @staticmethod
749
    def executable(tree, file_id):
1534.7.157 by Aaron Bentley
Added more docs
750
        """Determine the executability of a file-id (used as a key method)."""
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
751
        if file_id not in tree:
752
            return None
753
        if tree.kind(file_id) != "file":
754
            return False
755
        return tree.is_executable(file_id)
756
757
    @staticmethod
758
    def kind(tree, file_id):
1534.7.157 by Aaron Bentley
Added more docs
759
        """Determine the kind of a file-id (used as a key method)."""
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
760
        if file_id not in tree:
761
            return None
762
        return tree.kind(file_id)
763
764
    @staticmethod
2590.2.8 by Aaron Bentley
Restore conflict handling changes
765
    def _three_way(base, other, this):
766
        #if base == other, either they all agree, or only THIS has changed.
767
        if base == other:
768
            return 'this'
2590.2.10 by Aaron Bentley
Updates from review
769
        elif this not in (base, other):
2590.2.8 by Aaron Bentley
Restore conflict handling changes
770
            return 'conflict'
2590.2.10 by Aaron Bentley
Updates from review
771
        # "Ambiguous clean merge" -- both sides have made the same change.
2590.2.8 by Aaron Bentley
Restore conflict handling changes
772
        elif this == other:
773
            return "this"
2590.2.10 by Aaron Bentley
Updates from review
774
        # this == base: only other has changed.
2590.2.8 by Aaron Bentley
Restore conflict handling changes
775
        else:
776
            return "other"
777
778
    @staticmethod
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
779
    def scalar_three_way(this_tree, base_tree, other_tree, file_id, key):
780
        """Do a three-way test on a scalar.
781
        Return "this", "other" or "conflict", depending whether a value wins.
782
        """
783
        key_base = key(base_tree, file_id)
784
        key_other = key(other_tree, file_id)
785
        #if base == other, either they all agree, or only THIS has changed.
786
        if key_base == key_other:
787
            return "this"
788
        key_this = key(this_tree, file_id)
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
789
        # "Ambiguous clean merge"
790
        if key_this == key_other:
791
            return "this"
792
        elif key_this == key_base:
793
            return "other"
794
        else:
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
795
            return "conflict"
796
797
    def merge_names(self, file_id):
798
        def get_entry(tree):
799
            if file_id in tree.inventory:
800
                return tree.inventory[file_id]
801
            else:
802
                return None
803
        this_entry = get_entry(self.this_tree)
804
        other_entry = get_entry(self.other_tree)
805
        base_entry = get_entry(self.base_tree)
2590.2.1 by Aaron Bentley
Start work on merging names based on iter_changes
806
        entries = (base_entry, other_entry, this_entry)
807
        names = []
808
        parents = []
809
        for entry in entries:
810
            if entry is None:
811
                names.append(None)
812
                parents.append(None)
813
            else:
814
                names.append(entry.name)
815
                parents.append(entry.parent_id)
2590.2.2 by Aaron Bentley
Do most name merging from iter_changes output
816
        return self._merge_names(file_id, parents, names)
2590.2.1 by Aaron Bentley
Start work on merging names based on iter_changes
817
2590.2.2 by Aaron Bentley
Do most name merging from iter_changes output
818
    def _merge_names(self, file_id, parents, names):
2590.2.7 by Aaron Bentley
Misc cleanup
819
        """Perform a merge on file_id names and parents"""
2590.2.1 by Aaron Bentley
Start work on merging names based on iter_changes
820
        base_name, other_name, this_name = names
821
        base_parent, other_parent, this_parent = parents
2590.2.2 by Aaron Bentley
Do most name merging from iter_changes output
822
2590.2.1 by Aaron Bentley
Start work on merging names based on iter_changes
823
        name_winner = self._three_way(*names)
824
825
        parent_id_winner = self._three_way(*parents)
2590.2.2 by Aaron Bentley
Do most name merging from iter_changes output
826
        if this_name is None:
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
827
            if name_winner == "this":
828
                name_winner = "other"
829
            if parent_id_winner == "this":
830
                parent_id_winner = "other"
831
        if name_winner == "this" and parent_id_winner == "this":
832
            return
833
        if name_winner == "conflict":
1534.7.181 by Aaron Bentley
Renamed a bunch of functions
834
            trans_id = self.tt.trans_id_file_id(file_id)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
835
            self._raw_conflicts.append(('name conflict', trans_id, 
2590.2.1 by Aaron Bentley
Start work on merging names based on iter_changes
836
                                        this_name, other_name))
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
837
        if parent_id_winner == "conflict":
1534.7.181 by Aaron Bentley
Renamed a bunch of functions
838
            trans_id = self.tt.trans_id_file_id(file_id)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
839
            self._raw_conflicts.append(('parent conflict', trans_id, 
2590.2.1 by Aaron Bentley
Start work on merging names based on iter_changes
840
                                        this_parent, other_parent))
2590.2.2 by Aaron Bentley
Do most name merging from iter_changes output
841
        if other_name is None:
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
842
            # it doesn't matter whether the result was 'other' or 
843
            # 'conflict'-- if there's no 'other', we leave it alone.
844
            return
845
        # if we get here, name_winner and parent_winner are set to safe values.
1534.7.181 by Aaron Bentley
Renamed a bunch of functions
846
        trans_id = self.tt.trans_id_file_id(file_id)
2590.2.7 by Aaron Bentley
Misc cleanup
847
        parent_id = parents[self.winner_idx[parent_id_winner]]
1731.1.33 by Aaron Bentley
Revert no-special-root changes
848
        if parent_id is not None:
849
            parent_trans_id = self.tt.trans_id_file_id(parent_id)
2590.2.7 by Aaron Bentley
Misc cleanup
850
            self.tt.adjust_path(names[self.winner_idx[name_winner]],
1731.1.33 by Aaron Bentley
Revert no-special-root changes
851
                                parent_trans_id, trans_id)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
852
853
    def merge_contents(self, file_id):
1534.7.157 by Aaron Bentley
Added more docs
854
        """Performa a merge on file_id contents."""
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
855
        def contents_pair(tree):
856
            if file_id not in tree:
857
                return (None, None)
858
            kind = tree.kind(file_id)
859
            if kind == "file":
860
                contents = tree.get_file_sha1(file_id)
861
            elif kind == "symlink":
862
                contents = tree.get_symlink_target(file_id)
863
            else:
864
                contents = None
865
            return kind, contents
1558.15.3 by Aaron Bentley
Handle binary files for diff3 merges
866
867
        def contents_conflict():
868
            trans_id = self.tt.trans_id_file_id(file_id)
869
            name = self.tt.final_name(trans_id)
870
            parent_id = self.tt.final_parent(trans_id)
871
            if file_id in self.this_tree.inventory:
872
                self.tt.unversion_file(trans_id)
1551.10.2 by Aaron Bentley
Handle merge with dangling inventory entries
873
                if file_id in self.this_tree:
874
                    self.tt.delete_contents(trans_id)
1558.15.3 by Aaron Bentley
Handle binary files for diff3 merges
875
            file_group = self._dump_conflicts(name, parent_id, file_id, 
876
                                              set_version=True)
877
            self._raw_conflicts.append(('contents conflict', file_group))
878
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
879
        # See SPOT run.  run, SPOT, run.
880
        # So we're not QUITE repeating ourselves; we do tricky things with
881
        # file kind...
882
        base_pair = contents_pair(self.base_tree)
883
        other_pair = contents_pair(self.other_tree)
884
        if base_pair == other_pair:
1534.7.145 by Aaron Bentley
More fixups after get_trans_id
885
            # OTHER introduced no changes
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
886
            return "unmodified"
887
        this_pair = contents_pair(self.this_tree)
888
        if this_pair == other_pair:
1534.7.145 by Aaron Bentley
More fixups after get_trans_id
889
            # THIS and OTHER introduced the same changes
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
890
            return "unmodified"
891
        else:
1534.7.181 by Aaron Bentley
Renamed a bunch of functions
892
            trans_id = self.tt.trans_id_file_id(file_id)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
893
            if this_pair == base_pair:
1534.7.145 by Aaron Bentley
More fixups after get_trans_id
894
                # only OTHER introduced changes
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
895
                if file_id in self.this_tree:
1534.7.145 by Aaron Bentley
More fixups after get_trans_id
896
                    # Remove any existing contents
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
897
                    self.tt.delete_contents(trans_id)
1534.7.147 by Aaron Bentley
Tweak to check inventory, not tree for file ids
898
                if file_id in self.other_tree:
1534.7.145 by Aaron Bentley
More fixups after get_trans_id
899
                    # OTHER changed the file
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
900
                    create_by_entry(self.tt, 
901
                                    self.other_tree.inventory[file_id], 
902
                                    self.other_tree, trans_id)
1534.7.147 by Aaron Bentley
Tweak to check inventory, not tree for file ids
903
                    if file_id not in self.this_tree.inventory:
1534.7.145 by Aaron Bentley
More fixups after get_trans_id
904
                        self.tt.version_file(file_id, trans_id)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
905
                    return "modified"
1534.7.147 by Aaron Bentley
Tweak to check inventory, not tree for file ids
906
                elif file_id in self.this_tree.inventory:
1534.7.145 by Aaron Bentley
More fixups after get_trans_id
907
                    # OTHER deleted the file
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
908
                    self.tt.unversion_file(trans_id)
909
                    return "deleted"
1534.7.145 by Aaron Bentley
More fixups after get_trans_id
910
            #BOTH THIS and OTHER introduced changes; scalar conflict
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
911
            elif this_pair[0] == "file" and other_pair[0] == "file":
1534.7.145 by Aaron Bentley
More fixups after get_trans_id
912
                # THIS and OTHER are both files, so text merge.  Either
913
                # BASE is a file, or both converted to files, so at least we
914
                # have agreement that output should be a file.
1558.15.3 by Aaron Bentley
Handle binary files for diff3 merges
915
                try:
916
                    self.text_merge(file_id, trans_id)
917
                except BinaryFile:
918
                    return contents_conflict()
1534.7.147 by Aaron Bentley
Tweak to check inventory, not tree for file ids
919
                if file_id not in self.this_tree.inventory:
1534.7.145 by Aaron Bentley
More fixups after get_trans_id
920
                    self.tt.version_file(file_id, trans_id)
1534.7.152 by Aaron Bentley
Fixed overwrites
921
                try:
922
                    self.tt.tree_kind(trans_id)
923
                    self.tt.delete_contents(trans_id)
924
                except NoSuchFile:
925
                    pass
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
926
                return "modified"
927
            else:
1534.7.145 by Aaron Bentley
More fixups after get_trans_id
928
                # Scalar conflict, can't text merge.  Dump conflicts
1558.15.3 by Aaron Bentley
Handle binary files for diff3 merges
929
                return contents_conflict()
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
930
931
    def get_lines(self, tree, file_id):
1534.7.157 by Aaron Bentley
Added more docs
932
        """Return the lines in a file, or an empty list."""
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
933
        if file_id in tree:
934
            return tree.get_file(file_id).readlines()
935
        else:
936
            return []
937
938
    def text_merge(self, file_id, trans_id):
939
        """Perform a three-way text merge on a file_id"""
940
        # it's possible that we got here with base as a different type.
941
        # if so, we just want two-way text conflicts.
942
        if file_id in self.base_tree and \
943
            self.base_tree.kind(file_id) == "file":
944
            base_lines = self.get_lines(self.base_tree, file_id)
945
        else:
946
            base_lines = []
947
        other_lines = self.get_lines(self.other_tree, file_id)
948
        this_lines = self.get_lines(self.this_tree, file_id)
3249.3.1 by John Arbash Meinel
Implement cherrypick support for Merge3
949
        m3 = Merge3(base_lines, this_lines, other_lines,
950
                    is_cherrypick=self.cherrypick)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
951
        start_marker = "!START OF MERGE CONFLICT!" + "I HOPE THIS IS UNIQUE"
952
        if self.show_base is True:
953
            base_marker = '|' * 7
954
        else:
955
            base_marker = None
956
957
        def iter_merge3(retval):
958
            retval["text_conflicts"] = False
959
            for line in m3.merge_lines(name_a = "TREE", 
960
                                       name_b = "MERGE-SOURCE", 
961
                                       name_base = "BASE-REVISION",
962
                                       start_marker=start_marker, 
963
                                       base_marker=base_marker,
964
                                       reprocess=self.reprocess):
965
                if line.startswith(start_marker):
966
                    retval["text_conflicts"] = True
967
                    yield line.replace(start_marker, '<' * 7)
968
                else:
969
                    yield line
970
        retval = {}
971
        merge3_iterator = iter_merge3(retval)
972
        self.tt.create_file(merge3_iterator, trans_id)
973
        if retval["text_conflicts"] is True:
974
            self._raw_conflicts.append(('text conflict', trans_id))
975
            name = self.tt.final_name(trans_id)
976
            parent_id = self.tt.final_parent(trans_id)
977
            file_group = self._dump_conflicts(name, parent_id, file_id, 
978
                                              this_lines, base_lines,
979
                                              other_lines)
980
            file_group.append(trans_id)
981
982
    def _dump_conflicts(self, name, parent_id, file_id, this_lines=None, 
983
                        base_lines=None, other_lines=None, set_version=False,
984
                        no_base=False):
1534.7.157 by Aaron Bentley
Added more docs
985
        """Emit conflict files.
986
        If this_lines, base_lines, or other_lines are omitted, they will be
987
        determined automatically.  If set_version is true, the .OTHER, .THIS
988
        or .BASE (in that order) will be created as versioned files.
989
        """
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
990
        data = [('OTHER', self.other_tree, other_lines), 
991
                ('THIS', self.this_tree, this_lines)]
992
        if not no_base:
993
            data.append(('BASE', self.base_tree, base_lines))
994
        versioned = False
995
        file_group = []
996
        for suffix, tree, lines in data:
997
            if file_id in tree:
998
                trans_id = self._conflict_file(name, parent_id, tree, file_id,
999
                                               suffix, lines)
1000
                file_group.append(trans_id)
1001
                if set_version and not versioned:
1002
                    self.tt.version_file(file_id, trans_id)
1003
                    versioned = True
1004
        return file_group
1005
           
1006
    def _conflict_file(self, name, parent_id, tree, file_id, suffix, 
1007
                       lines=None):
1534.7.157 by Aaron Bentley
Added more docs
1008
        """Emit a single conflict file."""
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1009
        name = name + '.' + suffix
1010
        trans_id = self.tt.create_path(name, parent_id)
1011
        entry = tree.inventory[file_id]
1012
        create_by_entry(self.tt, entry, tree, trans_id, lines)
1013
        return trans_id
1014
1015
    def merge_executable(self, file_id, file_status):
1534.7.157 by Aaron Bentley
Added more docs
1016
        """Perform a merge on the execute bit."""
2590.2.3 by Aaron Bentley
Merge the execute bit based on iter_changes
1017
        executable = [self.executable(t, file_id) for t in (self.base_tree,
1018
                      self.other_tree, self.this_tree)]
1019
        self._merge_executable(file_id, executable, file_status)
1020
1021
    def _merge_executable(self, file_id, executable, file_status):
1022
        """Perform a merge on the execute bit."""
1023
        base_executable, other_executable, this_executable = executable
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1024
        if file_status == "deleted":
1025
            return
2590.2.3 by Aaron Bentley
Merge the execute bit based on iter_changes
1026
        winner = self._three_way(*executable)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1027
        if winner == "conflict":
1028
        # There must be a None in here, if we have a conflict, but we
1029
        # need executability since file status was not deleted.
1534.10.35 by Aaron Bentley
Merge handles contents + executable + deletion conflict
1030
            if self.executable(self.other_tree, file_id) is None:
1534.7.142 by Aaron Bentley
Fixed executability conflicts
1031
                winner = "this"
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1032
            else:
1534.7.142 by Aaron Bentley
Fixed executability conflicts
1033
                winner = "other"
1551.19.30 by Aaron Bentley
Accelerate merge by skipping file existence check when merging execute bit
1034
        if winner == 'this' and file_status != "modified":
1035
            return
1036
        trans_id = self.tt.trans_id_file_id(file_id)
1037
        try:
1038
            if self.tt.final_kind(trans_id) != "file":
1039
                return
1040
        except NoSuchFile:
1041
            return
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1042
        if winner == "this":
1551.19.30 by Aaron Bentley
Accelerate merge by skipping file existence check when merging execute bit
1043
            executability = this_executable
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1044
        else:
1045
            if file_id in self.other_tree:
2590.2.3 by Aaron Bentley
Merge the execute bit based on iter_changes
1046
                executability = other_executable
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1047
            elif file_id in self.this_tree:
2590.2.3 by Aaron Bentley
Merge the execute bit based on iter_changes
1048
                executability = this_executable
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1049
            elif file_id in self.base_tree:
2590.2.3 by Aaron Bentley
Merge the execute bit based on iter_changes
1050
                executability = base_executable
1551.19.30 by Aaron Bentley
Accelerate merge by skipping file existence check when merging execute bit
1051
        if executability is not None:
1052
            trans_id = self.tt.trans_id_file_id(file_id)
1053
            self.tt.set_executability(executability, trans_id)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1054
1534.7.172 by Aaron Bentley
Integrated fs conflicts with merge conflicts.
1055
    def cook_conflicts(self, fs_conflicts):
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1056
        """Convert all conflicts into a form that doesn't depend on trans_id"""
1534.10.19 by Aaron Bentley
Stanza conversion, cooking
1057
        from conflicts import Conflict
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1058
        name_conflicts = {}
1534.7.172 by Aaron Bentley
Integrated fs conflicts with merge conflicts.
1059
        self.cooked_conflicts.extend(cook_conflicts(fs_conflicts, self.tt))
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1060
        fp = FinalPaths(self.tt)
1061
        for conflict in self._raw_conflicts:
1062
            conflict_type = conflict[0]
1063
            if conflict_type in ('name conflict', 'parent conflict'):
1064
                trans_id = conflict[1]
1065
                conflict_args = conflict[2:]
1066
                if trans_id not in name_conflicts:
1067
                    name_conflicts[trans_id] = {}
1068
                unique_add(name_conflicts[trans_id], conflict_type, 
1069
                           conflict_args)
1070
            if conflict_type == 'contents conflict':
1071
                for trans_id in conflict[1]:
1072
                    file_id = self.tt.final_file_id(trans_id)
1073
                    if file_id is not None:
1074
                        break
1075
                path = fp.get_path(trans_id)
1076
                for suffix in ('.BASE', '.THIS', '.OTHER'):
1077
                    if path.endswith(suffix):
1078
                        path = path[:-len(suffix)]
1079
                        break
1534.10.19 by Aaron Bentley
Stanza conversion, cooking
1080
                c = Conflict.factory(conflict_type, path=path, file_id=file_id)
1081
                self.cooked_conflicts.append(c)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1082
            if conflict_type == 'text conflict':
1083
                trans_id = conflict[1]
1084
                path = fp.get_path(trans_id)
1085
                file_id = self.tt.final_file_id(trans_id)
1534.10.19 by Aaron Bentley
Stanza conversion, cooking
1086
                c = Conflict.factory(conflict_type, path=path, file_id=file_id)
1087
                self.cooked_conflicts.append(c)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1088
1089
        for trans_id, conflicts in name_conflicts.iteritems():
1090
            try:
1091
                this_parent, other_parent = conflicts['parent conflict']
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
1092
                if this_parent == other_parent:
1093
                    raise AssertionError()
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1094
            except KeyError:
1095
                this_parent = other_parent = \
1096
                    self.tt.final_file_id(self.tt.final_parent(trans_id))
1097
            try:
1098
                this_name, other_name = conflicts['name conflict']
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
1099
                if this_name == other_name:
1100
                    raise AssertionError()
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1101
            except KeyError:
1102
                this_name = other_name = self.tt.final_name(trans_id)
1103
            other_path = fp.get_path(trans_id)
1551.16.2 by Aaron Bentley
Don't crash on merging renamed deleted files (#110279)
1104
            if this_parent is not None and this_name is not None:
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1105
                this_parent_path = \
1534.7.181 by Aaron Bentley
Renamed a bunch of functions
1106
                    fp.get_path(self.tt.trans_id_file_id(this_parent))
1534.7.166 by Aaron Bentley
Swapped os.path.join for pathjoin everywhere
1107
                this_path = pathjoin(this_parent_path, this_name)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1108
            else:
1109
                this_path = "<deleted>"
1110
            file_id = self.tt.final_file_id(trans_id)
1534.10.20 by Aaron Bentley
Got all tests passing
1111
            c = Conflict.factory('path conflict', path=this_path,
1534.10.19 by Aaron Bentley
Stanza conversion, cooking
1112
                                 conflict_path=other_path, file_id=file_id)
1113
            self.cooked_conflicts.append(c)
1666.1.4 by Robert Collins
* 'Metadir' is now the default disk format. This improves behaviour in
1114
        self.cooked_conflicts.sort(key=Conflict.sort_key)
1534.7.141 by Aaron Bentley
Added conflict reporting
1115
1116
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1117
class WeaveMerger(Merge3Merger):
1534.7.167 by Aaron Bentley
PEP8 and comment cleanups
1118
    """Three-way tree merger, text weave merger."""
1551.6.8 by Aaron Bentley
Implemented reprocess for weave
1119
    supports_reprocess = True
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1120
    supports_show_base = False
3062.2.7 by Aaron Bentley
Prevent reverse cherry-picking with weave
1121
    supports_reverse_cherrypick = False
3062.2.9 by Aaron Bentley
Don't use the base if not cherrypicking
1122
    history_based = True
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1123
1124
    def _merged_lines(self, file_id):
1125
        """Generate the merged lines.
1126
        There is no distinction between lines that are meant to contain <<<<<<<
1127
        and conflicts.
1128
        """
3062.2.9 by Aaron Bentley
Don't use the base if not cherrypicking
1129
        if self.cherrypick:
1130
            base = self.base_tree
1131
        else:
1132
            base = None
3062.2.4 by Aaron Bentley
Start supporting merge-with-base
1133
        plan = self.this_tree.plan_file_merge(file_id, self.other_tree,
3062.2.9 by Aaron Bentley
Don't use the base if not cherrypicking
1134
                                              base=base)
1551.19.17 by Aaron Bentley
Add debugging flag for merges
1135
        if 'merge' in debug.debug_flags:
1136
            plan = list(plan)
1137
            trans_id = self.tt.trans_id_file_id(file_id)
1138
            name = self.tt.final_name(trans_id) + '.plan'
1139
            contents = ('%10s|%s' % l for l in plan)
1140
            self.tt.new_file(name, self.tt.final_parent(trans_id), contents)
1551.15.52 by Aaron Bentley
Tweak from review comments
1141
        textmerge = PlanWeaveMerge(plan, '<<<<<<< TREE\n',
1142
            '>>>>>>> MERGE-SOURCE\n')
1143
        return textmerge.merge_lines(self.reprocess)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1144
1145
    def text_merge(self, file_id, trans_id):
1534.7.157 by Aaron Bentley
Added more docs
1146
        """Perform a (weave) text merge for a given file and file-id.
1147
        If conflicts are encountered, .THIS and .OTHER files will be emitted,
1148
        and a conflict will be noted.
1149
        """
1551.6.12 by Aaron Bentley
Indicate conflicts from merge_lines, insead of guessing
1150
        lines, conflicts = self._merged_lines(file_id)
1558.15.10 by Aaron Bentley
Merge bzr.dev
1151
        lines = list(lines)
1558.15.5 by Aaron Bentley
Fixed binary handling in weave merge
1152
        # Note we're checking whether the OUTPUT is binary in this case, 
1153
        # because we don't want to get into weave merge guts.
1154
        check_text_lines(lines)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1155
        self.tt.create_file(lines, trans_id)
1156
        if conflicts:
1157
            self._raw_conflicts.append(('text conflict', trans_id))
1158
            name = self.tt.final_name(trans_id)
1159
            parent_id = self.tt.final_parent(trans_id)
1160
            file_group = self._dump_conflicts(name, parent_id, file_id, 
1161
                                              no_base=True)
1162
            file_group.append(trans_id)
1163
1164
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1165
class LCAMerger(WeaveMerger):
1166
1167
    def _merged_lines(self, file_id):
1168
        """Generate the merged lines.
1169
        There is no distinction between lines that are meant to contain <<<<<<<
1170
        and conflicts.
1171
        """
1172
        if self.cherrypick:
1173
            base = self.base_tree
1174
        else:
1175
            base = None
1176
        plan = self.this_tree.plan_file_lca_merge(file_id, self.other_tree,
1177
                                                  base=base)
1178
        if 'merge' in debug.debug_flags:
1179
            plan = list(plan)
1180
            trans_id = self.tt.trans_id_file_id(file_id)
1181
            name = self.tt.final_name(trans_id) + '.plan'
1182
            contents = ('%10s|%s' % l for l in plan)
1183
            self.tt.new_file(name, self.tt.final_parent(trans_id), contents)
1184
        textmerge = PlanWeaveMerge(plan, '<<<<<<< TREE\n',
1185
            '>>>>>>> MERGE-SOURCE\n')
1186
        return textmerge.merge_lines(self.reprocess)
1187
1188
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1189
class Diff3Merger(Merge3Merger):
1534.7.167 by Aaron Bentley
PEP8 and comment cleanups
1190
    """Three-way merger using external diff3 for text merging"""
1711.7.20 by John Arbash Meinel
always close files, minor PEP8 cleanup
1191
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1192
    def dump_file(self, temp_dir, name, tree, file_id):
1193
        out_path = pathjoin(temp_dir, name)
1711.7.20 by John Arbash Meinel
always close files, minor PEP8 cleanup
1194
        out_file = open(out_path, "wb")
1195
        try:
1196
            in_file = tree.get_file(file_id)
1197
            for line in in_file:
1198
                out_file.write(line)
1199
        finally:
1200
            out_file.close()
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1201
        return out_path
1202
1203
    def text_merge(self, file_id, trans_id):
1534.7.157 by Aaron Bentley
Added more docs
1204
        """Perform a diff3 merge using a specified file-id and trans-id.
1205
        If conflicts are encountered, .BASE, .THIS. and .OTHER conflict files
1206
        will be dumped, and a will be conflict noted.
1207
        """
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1208
        import bzrlib.patch
1996.3.18 by John Arbash Meinel
Now that mkdtemp and rmtree are lazy, they should not be directly improted.
1209
        temp_dir = osutils.mkdtemp(prefix="bzr-")
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1210
        try:
1534.7.166 by Aaron Bentley
Swapped os.path.join for pathjoin everywhere
1211
            new_file = pathjoin(temp_dir, "new")
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1212
            this = self.dump_file(temp_dir, "this", self.this_tree, file_id)
1213
            base = self.dump_file(temp_dir, "base", self.base_tree, file_id)
1214
            other = self.dump_file(temp_dir, "other", self.other_tree, file_id)
1215
            status = bzrlib.patch.diff3(new_file, this, base, other)
1216
            if status not in (0, 1):
1217
                raise BzrError("Unhandled diff3 exit code")
1711.7.20 by John Arbash Meinel
always close files, minor PEP8 cleanup
1218
            f = open(new_file, 'rb')
1219
            try:
1220
                self.tt.create_file(f, trans_id)
1221
            finally:
1222
                f.close()
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1223
            if status == 1:
1224
                name = self.tt.final_name(trans_id)
1225
                parent_id = self.tt.final_parent(trans_id)
1226
                self._dump_conflicts(name, parent_id, file_id)
1551.8.39 by Aaron Bentley
Fix diff3 conflict-reporting bug
1227
                self._raw_conflicts.append(('text conflict', trans_id))
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1228
        finally:
1996.3.18 by John Arbash Meinel
Now that mkdtemp and rmtree are lazy, they should not be directly improted.
1229
            osutils.rmtree(temp_dir)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1230
1231
1232
def merge_inner(this_branch, other_tree, base_tree, ignore_zero=False,
2255.2.31 by Robert Collins
Work in progress to make merge_inner work with dirstate trees.
1233
                backup_files=False,
1234
                merge_type=Merge3Merger,
1235
                interesting_ids=None,
1236
                show_base=False,
1237
                reprocess=False,
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1238
                other_rev_id=None,
1239
                interesting_files=None,
1534.9.9 by Aaron Bentley
Added progress bar to pull
1240
                this_tree=None,
1551.11.10 by Aaron Bentley
Add change reporting to pull
1241
                pb=DummyProgress(),
1242
                change_reporter=None):
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1243
    """Primary interface for merging. 
1244
1245
        typical use is probably 
1246
        'merge_inner(branch, branch.get_revision_tree(other_revision),
1247
                     branch.get_revision_tree(base_revision))'
1248
        """
1249
    if this_tree is None:
2367.2.1 by Robert Collins
Remove bzrlib 0.8 compatability where it was making the code unclear or messy. (Robert Collins)
1250
        raise BzrError("bzrlib.merge.merge_inner requires a this_tree "
1251
            "parameter as of bzrlib version 0.8.")
2255.2.31 by Robert Collins
Work in progress to make merge_inner work with dirstate trees.
1252
    merger = Merger(this_branch, other_tree, base_tree, this_tree=this_tree,
1551.11.10 by Aaron Bentley
Add change reporting to pull
1253
                    pb=pb, change_reporter=change_reporter)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1254
    merger.backup_files = backup_files
1255
    merger.merge_type = merge_type
1256
    merger.interesting_ids = interesting_ids
1551.2.23 by Aaron Bentley
Got merge_inner's ignore_zero parameter working
1257
    merger.ignore_zero = ignore_zero
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1258
    if interesting_files:
3376.2.4 by Martin Pool
Remove every assert statement from bzrlib!
1259
        if interesting_ids:
1260
            raise ValueError('Only supply interesting_ids'
1261
                             ' or interesting_files')
2590.2.5 by Aaron Bentley
Allow selected files to be specified instead of selected ids
1262
        merger.interesting_files = interesting_files
1979.2.1 by Robert Collins
(robertc) adds a convenience method "merge_from_branch" to WorkingTree.
1263
    merger.show_base = show_base
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1264
    merger.reprocess = reprocess
1265
    merger.other_rev_id = other_rev_id
1266
    merger.other_basis = other_rev_id
3249.3.1 by John Arbash Meinel
Implement cherrypick support for Merge3
1267
    get_revision_id = getattr(base_tree, 'get_revision_id', None)
1268
    if get_revision_id is None:
1269
        get_revision_id = base_tree.last_revision
1270
    merger.set_base_revision(get_revision_id(), this_branch)
1534.7.140 by Aaron Bentley
Moved the merge stuff into merge.py
1271
    return merger.do_merge()
1272
2221.4.15 by Aaron Bentley
Use RegistryOption for merge type
1273
def get_merge_type_registry():
2221.4.17 by Aaron Bentley
PEP8-ness
1274
    """Merge type registry is in bzrlib.option to avoid circular imports.
2221.4.15 by Aaron Bentley
Use RegistryOption for merge type
1275
1276
    This method provides a sanctioned way to retrieve it.
1277
    """
1278
    from bzrlib import option
2221.4.16 by Aaron Bentley
Add tests for get_merge_type_registry
1279
    return option._merge_type_registry
1551.15.46 by Aaron Bentley
Move plan merge to tree
1280
1281
1282
def _plan_annotate_merge(annotated_a, annotated_b, ancestors_a, ancestors_b):
1283
    def status_a(revision, text):
1284
        if revision in ancestors_b:
1285
            return 'killed-b', text
1286
        else:
1287
            return 'new-a', text
1288
1289
    def status_b(revision, text):
1290
        if revision in ancestors_a:
1291
            return 'killed-a', text
1292
        else:
1293
            return 'new-b', text
1294
1295
    plain_a = [t for (a, t) in annotated_a]
1296
    plain_b = [t for (a, t) in annotated_b]
1297
    matcher = patiencediff.PatienceSequenceMatcher(None, plain_a, plain_b)
1298
    blocks = matcher.get_matching_blocks()
1299
    a_cur = 0
1300
    b_cur = 0
1301
    for ai, bi, l in blocks:
1302
        # process all mismatched sections
1303
        # (last mismatched section is handled because blocks always
1304
        # includes a 0-length last block)
1305
        for revision, text in annotated_a[a_cur:ai]:
1306
            yield status_a(revision, text)
1307
        for revision, text in annotated_b[b_cur:bi]:
1308
            yield status_b(revision, text)
1309
        # and now the matched section
1310
        a_cur = ai + l
1311
        b_cur = bi + l
3376.2.8 by Martin Pool
Some review cleanups for assertion removal
1312
        for text_a in plain_a[ai:a_cur]:
1551.15.46 by Aaron Bentley
Move plan merge to tree
1313
            yield "unchanged", text_a
3062.1.9 by Aaron Bentley
Move PlanMerge into merge and _PlanMergeVersionedFile into versionedfile
1314
1315
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1316
class _PlanMergeBase(object):
3062.1.9 by Aaron Bentley
Move PlanMerge into merge and _PlanMergeVersionedFile into versionedfile
1317
3350.6.10 by Martin Pool
VersionedFiles review cleanups
1318
    def __init__(self, a_rev, b_rev, vf, key_prefix):
3062.1.9 by Aaron Bentley
Move PlanMerge into merge and _PlanMergeVersionedFile into versionedfile
1319
        """Contructor.
1320
1321
        :param a_rev: Revision-id of one revision to merge
1322
        :param b_rev: Revision-id of the other revision to merge
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
1323
        :param vf: A VersionedFiles containing both revisions
3350.6.10 by Martin Pool
VersionedFiles review cleanups
1324
        :param key_prefix: A prefix for accessing keys in vf, typically
1325
            (file_id,).
3062.1.9 by Aaron Bentley
Move PlanMerge into merge and _PlanMergeVersionedFile into versionedfile
1326
        """
1327
        self.a_rev = a_rev
1328
        self.b_rev = b_rev
1329
        self.vf = vf
3062.1.12 by Aaron Bentley
Implement simple text cache
1330
        self._last_lines = None
1331
        self._last_lines_revision_id = None
3144.3.7 by Aaron Bentley
Update from review
1332
        self._cached_matching_blocks = {}
3350.6.10 by Martin Pool
VersionedFiles review cleanups
1333
        self._key_prefix = key_prefix
3514.2.2 by John Arbash Meinel
Restore a real weave merge to 'bzr merge --weave'.
1334
        self._precache_tip_lines()
1335
1336
    def _precache_tip_lines(self):
1337
        lines = self.get_lines([self.a_rev, self.b_rev])
1338
        self.lines_a = lines[self.a_rev]
1339
        self.lines_b = lines[self.b_rev]
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
1340
1341
    def get_lines(self, revisions):
1342
        """Get lines for revisions from the backing VersionedFiles.
1343
        
3350.6.10 by Martin Pool
VersionedFiles review cleanups
1344
        :raises RevisionNotPresent: on absent texts.
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
1345
        """
3350.6.10 by Martin Pool
VersionedFiles review cleanups
1346
        keys = [(self._key_prefix + (rev,)) for rev in revisions]
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
1347
        result = {}
1348
        for record in self.vf.get_record_stream(keys, 'unordered', True):
1349
            if record.storage_kind == 'absent':
1350
                raise errors.RevisionNotPresent(record.key, self.vf)
3350.6.10 by Martin Pool
VersionedFiles review cleanups
1351
            result[record.key[-1]] = osutils.split_lines(
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
1352
                record.get_bytes_as('fulltext'))
1353
        return result
3062.1.9 by Aaron Bentley
Move PlanMerge into merge and _PlanMergeVersionedFile into versionedfile
1354
1355
    def plan_merge(self):
1356
        """Generate a 'plan' for merging the two revisions.
1357
1358
        This involves comparing their texts and determining the cause of
1359
        differences.  If text A has a line and text B does not, then either the
1360
        line was added to text A, or it was deleted from B.  Once the causes
1361
        are combined, they are written out in the format described in
1362
        VersionedFile.plan_merge
1363
        """
1364
        blocks = self._get_matching_blocks(self.a_rev, self.b_rev)
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1365
        unique_a, unique_b = self._unique_lines(blocks)
1366
        new_a, killed_b = self._determine_status(self.a_rev, unique_a)
1367
        new_b, killed_a = self._determine_status(self.b_rev, unique_b)
1368
        return self._iter_plan(blocks, new_a, killed_b, new_b, killed_a)
1369
1370
    def _iter_plan(self, blocks, new_a, killed_b, new_b, killed_a):
3062.1.9 by Aaron Bentley
Move PlanMerge into merge and _PlanMergeVersionedFile into versionedfile
1371
        last_i = 0
1372
        last_j = 0
1373
        for i, j, n in blocks:
1374
            for a_index in range(last_i, i):
1375
                if a_index in new_a:
3144.3.2 by Aaron Bentley
Get conflict handling working
1376
                    if a_index in killed_b:
1377
                        yield 'conflicted-a', self.lines_a[a_index]
1378
                    else:
1379
                        yield 'new-a', self.lines_a[a_index]
1380
                else:
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1381
                    yield 'killed-b', self.lines_a[a_index]
3062.1.9 by Aaron Bentley
Move PlanMerge into merge and _PlanMergeVersionedFile into versionedfile
1382
            for b_index in range(last_j, j):
1383
                if b_index in new_b:
3144.3.2 by Aaron Bentley
Get conflict handling working
1384
                    if b_index in killed_a:
3144.3.10 by Aaron Bentley
Use correct index when emitting conflicted-b
1385
                        yield 'conflicted-b', self.lines_b[b_index]
3144.3.2 by Aaron Bentley
Get conflict handling working
1386
                    else:
1387
                        yield 'new-b', self.lines_b[b_index]
1388
                else:
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1389
                    yield 'killed-a', self.lines_b[b_index]
3062.1.9 by Aaron Bentley
Move PlanMerge into merge and _PlanMergeVersionedFile into versionedfile
1390
            # handle common lines
1391
            for a_index in range(i, i+n):
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1392
                yield 'unchanged', self.lines_a[a_index]
3062.1.9 by Aaron Bentley
Move PlanMerge into merge and _PlanMergeVersionedFile into versionedfile
1393
            last_i = i+n
1394
            last_j = j+n
1395
1396
    def _get_matching_blocks(self, left_revision, right_revision):
1397
        """Return a description of which sections of two revisions match.
1398
1399
        See SequenceMatcher.get_matching_blocks
1400
        """
3144.3.7 by Aaron Bentley
Update from review
1401
        cached = self._cached_matching_blocks.get((left_revision,
1402
                                                   right_revision))
1403
        if cached is not None:
1404
            return cached
3062.1.12 by Aaron Bentley
Implement simple text cache
1405
        if self._last_lines_revision_id == left_revision:
1406
            left_lines = self._last_lines
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
1407
            right_lines = self.get_lines([right_revision])[right_revision]
3062.1.12 by Aaron Bentley
Implement simple text cache
1408
        else:
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
1409
            lines = self.get_lines([left_revision, right_revision])
1410
            left_lines = lines[left_revision]
1411
            right_lines = lines[right_revision]
3062.1.12 by Aaron Bentley
Implement simple text cache
1412
        self._last_lines = right_lines
1413
        self._last_lines_revision_id = right_revision
3062.1.9 by Aaron Bentley
Move PlanMerge into merge and _PlanMergeVersionedFile into versionedfile
1414
        matcher = patiencediff.PatienceSequenceMatcher(None, left_lines,
1415
                                                       right_lines)
1416
        return matcher.get_matching_blocks()
1417
1418
    def _unique_lines(self, matching_blocks):
1419
        """Analyse matching_blocks to determine which lines are unique
1420
1421
        :return: a tuple of (unique_left, unique_right), where the values are
1422
            sets of line numbers of unique lines.
1423
        """
1424
        last_i = 0
1425
        last_j = 0
1426
        unique_left = []
1427
        unique_right = []
1428
        for i, j, n in matching_blocks:
1429
            unique_left.extend(range(last_i, i))
1430
            unique_right.extend(range(last_j, j))
1431
            last_i = i + n
1432
            last_j = j + n
1433
        return unique_left, unique_right
1434
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1435
    @staticmethod
1436
    def _subtract_plans(old_plan, new_plan):
3144.3.7 by Aaron Bentley
Update from review
1437
        """Remove changes from new_plan that came from old_plan.
1438
1439
        It is assumed that the difference between the old_plan and new_plan
1440
        is their choice of 'b' text.
1441
1442
        All lines from new_plan that differ from old_plan are emitted
1443
        verbatim.  All lines from new_plan that match old_plan but are
1444
        not about the 'b' revision are emitted verbatim.
1445
1446
        Lines that match and are about the 'b' revision are the lines we
1447
        don't want, so we convert 'killed-b' -> 'unchanged', and 'new-b'
1448
        is skipped entirely.
1449
        """
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1450
        matcher = patiencediff.PatienceSequenceMatcher(None, old_plan,
1451
                                                       new_plan)
1452
        last_j = 0
1453
        for i, j, n in matcher.get_matching_blocks():
1454
            for jj in range(last_j, j):
1455
                yield new_plan[jj]
1456
            for jj in range(j, j+n):
1457
                plan_line = new_plan[jj]
1458
                if plan_line[0] == 'new-b':
1459
                    pass
1460
                elif plan_line[0] == 'killed-b':
1461
                    yield 'unchanged', plan_line[1]
1462
                else:
1463
                    yield plan_line
1464
            last_j = j + n
1465
1466
1467
class _PlanMerge(_PlanMergeBase):
1468
    """Plan an annotate merge using on-the-fly annotation"""
1469
3350.6.10 by Martin Pool
VersionedFiles review cleanups
1470
    def __init__(self, a_rev, b_rev, vf, key_prefix):
3514.2.2 by John Arbash Meinel
Restore a real weave merge to 'bzr merge --weave'.
1471
        super(_PlanMerge, self).__init__(a_rev, b_rev, vf, key_prefix)
1472
        self.a_key = self._key_prefix + (self.a_rev,)
1473
        self.b_key = self._key_prefix + (self.b_rev,)
1474
        self.graph = Graph(self.vf)
3514.2.11 by John Arbash Meinel
Shortcut the case when one revision is in the ancestry of the other.
1475
        heads = self.graph.heads((self.a_key, self.b_key))
1476
        if len(heads) == 1:
1477
            # one side dominates, so we can just return its values, yay for
1478
            # per-file graphs
1479
            # Ideally we would know that before we get this far
1480
            self._head_key = heads.pop()
1481
            if self._head_key == self.a_key:
1482
                other = b_rev
1483
            else:
1484
                other = a_rev
1485
            mutter('found dominating revision for %s\n%s > %s', self.vf,
1486
                   self._head_key[-1], other)
1487
            self._weave = None
1488
        else:
1489
            self._head_key = None
1490
            self._build_weave()
3514.2.2 by John Arbash Meinel
Restore a real weave merge to 'bzr merge --weave'.
1491
1492
    def _precache_tip_lines(self):
1493
        # Turn this into a no-op, because we will do this later
1494
        pass
1495
1496
    def _find_recursive_lcas(self):
1497
        """Find all the ancestors back to a unique lca"""
1498
        cur_ancestors = (self.a_key, self.b_key)
1499
        # graph.find_lca(uncommon, keys) now returns plain NULL_REVISION,
3514.2.5 by John Arbash Meinel
Switch to the get_parent_map design I had settled on.
1500
        # rather than a key tuple. We will just map that directly to no common
1501
        # ancestors.
1502
        parent_map = {}
3514.2.2 by John Arbash Meinel
Restore a real weave merge to 'bzr merge --weave'.
1503
        while True:
1504
            next_lcas = self.graph.find_lca(*cur_ancestors)
3514.2.5 by John Arbash Meinel
Switch to the get_parent_map design I had settled on.
1505
            # Map a plain NULL_REVISION to a simple no-ancestors
1506
            if next_lcas == set([NULL_REVISION]):
1507
                next_lcas = ()
3514.2.9 by John Arbash Meinel
Add some debugging, and work on getting the graph right so we get the weave insertion order correct.
1508
            # Order the lca's based on when they were merged into the tip
1509
            # While the actual merge portion of weave merge uses a set() of
1510
            # active revisions, the order of insertion *does* effect the
1511
            # implicit ordering of the texts.
3514.2.5 by John Arbash Meinel
Switch to the get_parent_map design I had settled on.
1512
            for rev_key in cur_ancestors:
3514.2.9 by John Arbash Meinel
Add some debugging, and work on getting the graph right so we get the weave insertion order correct.
1513
                ordered_parents = tuple(self.graph.find_merge_order(rev_key,
1514
                                                                    next_lcas))
1515
                parent_map[rev_key] = ordered_parents
3514.2.2 by John Arbash Meinel
Restore a real weave merge to 'bzr merge --weave'.
1516
            if len(next_lcas) == 0:
1517
                break
1518
            elif len(next_lcas) == 1:
3514.2.9 by John Arbash Meinel
Add some debugging, and work on getting the graph right so we get the weave insertion order correct.
1519
                parent_map[list(next_lcas)[0]] = ()
3514.2.2 by John Arbash Meinel
Restore a real weave merge to 'bzr merge --weave'.
1520
                break
3514.2.10 by John Arbash Meinel
Handle more edge cases.
1521
            elif len(next_lcas) > 2:
3514.2.7 by John Arbash Meinel
Fix the failing test by implementing the fallback logic.
1522
                # More than 2 lca's, fall back to grabbing all nodes between
1523
                # this and the unique lca.
3514.2.16 by John Arbash Meinel
Review feedback from Ian.
1524
                mutter('More than 2 LCAs, falling back to all nodes for:'
1525
                       ' %s, %s\n=> %s', self.a_key, self.b_key, cur_ancestors)
3514.2.7 by John Arbash Meinel
Fix the failing test by implementing the fallback logic.
1526
                cur_lcas = next_lcas
1527
                while len(cur_lcas) > 1:
1528
                    cur_lcas = self.graph.find_lca(*cur_lcas)
1529
                if len(cur_lcas) == 0:
1530
                    # No common base to find, use the full ancestry
1531
                    unique_lca = None
1532
                else:
3514.2.8 by John Arbash Meinel
The insertion ordering into the weave has an impact on conflicts.
1533
                    unique_lca = list(cur_lcas)[0]
3514.2.14 by John Arbash Meinel
Bring in the code to collapse linear portions of the graph.
1534
                    if unique_lca == NULL_REVISION:
1535
                        # find_lca will return a plain 'NULL_REVISION' rather
1536
                        # than a key tuple when there is no common ancestor, we
1537
                        # prefer to just use None, because it doesn't confuse
1538
                        # _get_interesting_texts()
1539
                        unique_lca = None
3514.2.7 by John Arbash Meinel
Fix the failing test by implementing the fallback logic.
1540
                parent_map.update(self._find_unique_parents(next_lcas,
1541
                                                            unique_lca))
1542
                break
3514.2.2 by John Arbash Meinel
Restore a real weave merge to 'bzr merge --weave'.
1543
            cur_ancestors = next_lcas
3514.2.5 by John Arbash Meinel
Switch to the get_parent_map design I had settled on.
1544
        return parent_map
3514.2.2 by John Arbash Meinel
Restore a real weave merge to 'bzr merge --weave'.
1545
3514.2.7 by John Arbash Meinel
Fix the failing test by implementing the fallback logic.
1546
    def _find_unique_parents(self, tip_keys, base_key):
1547
        """Find ancestors of tip that aren't ancestors of base.
1548
        
1549
        :param tip_keys: Nodes that are interesting
1550
        :param base_key: Cull all ancestors of this node
1551
        :return: The parent map for all revisions between tip_keys and
1552
            base_key. base_key will be included. References to nodes outside of
1553
            the ancestor set will also be removed.
1554
        """
1555
        # TODO: this would be simpler if find_unique_ancestors took a list
1556
        #       instead of a single tip, internally it supports it, but it
1557
        #       isn't a "backwards compatible" api change.
1558
        if base_key is None:
1559
            parent_map = dict(self.graph.iter_ancestry(tip_keys))
3514.2.14 by John Arbash Meinel
Bring in the code to collapse linear portions of the graph.
1560
            # We remove NULL_REVISION because it isn't a proper tuple key, and
1561
            # thus confuses things like _get_interesting_texts, and our logic
1562
            # to add the texts into the memory weave.
1563
            if NULL_REVISION in parent_map:
1564
                parent_map.pop(NULL_REVISION)
3514.2.7 by John Arbash Meinel
Fix the failing test by implementing the fallback logic.
1565
        else:
1566
            interesting = set()
1567
            for tip in tip_keys:
1568
                interesting.update(
1569
                    self.graph.find_unique_ancestors(tip, [base_key]))
1570
            parent_map = self.graph.get_parent_map(interesting)
1571
            parent_map[base_key] = ()
3514.2.12 by John Arbash Meinel
Start refactoring into helper functions
1572
        culled_parent_map, child_map, tails = self._remove_external_references(
1573
            parent_map)
3514.2.13 by John Arbash Meinel
Add the ability to prune extra tails from the parent_map.
1574
        # Remove all the tails but base_key
3514.2.14 by John Arbash Meinel
Bring in the code to collapse linear portions of the graph.
1575
        if base_key is not None:
1576
            tails.remove(base_key)
1577
            self._prune_tails(culled_parent_map, child_map, tails)
1578
        # Now remove all the uninteresting 'linear' regions
3514.2.15 by John Arbash Meinel
Enable collapsing linear regions.
1579
        simple_map = _mod_graph.collapse_linear_regions(culled_parent_map)
1580
        return simple_map
3514.2.12 by John Arbash Meinel
Start refactoring into helper functions
1581
1582
    @staticmethod
1583
    def _remove_external_references(parent_map):
1584
        """Remove references that go outside of the parent map.
1585
1586
        :param parent_map: Something returned from Graph.get_parent_map(keys)
1587
        :return: (filtered_parent_map, child_map, tails)
1588
            filtered_parent_map is parent_map without external references
1589
            child_map is the {parent_key: [child_keys]} mapping
1590
            tails is a list of nodes that do not have any parents in the map
1591
        """
1592
        # TODO: The basic effect of this function seems more generic than
1593
        #       _PlanMerge. But the specific details of building a child_map,
1594
        #       and computing tails seems very specific to _PlanMerge.
1595
        #       Still, should this be in Graph land?
1596
        filtered_parent_map = {}
1597
        child_map = {}
1598
        tails = []
3514.2.7 by John Arbash Meinel
Fix the failing test by implementing the fallback logic.
1599
        for key, parent_keys in parent_map.iteritems():
3514.2.12 by John Arbash Meinel
Start refactoring into helper functions
1600
            culled_parent_keys = [p for p in parent_keys if p in parent_map]
1601
            if not culled_parent_keys:
1602
                tails.append(key)
1603
            for parent_key in culled_parent_keys:
1604
                child_map.setdefault(parent_key, []).append(key)
1605
            # TODO: Do we want to do this, it adds overhead for every node,
1606
            #       just to say that the node has no children
1607
            child_map.setdefault(key, [])
1608
            filtered_parent_map[key] = culled_parent_keys
1609
        return filtered_parent_map, child_map, tails
1610
3514.2.13 by John Arbash Meinel
Add the ability to prune extra tails from the parent_map.
1611
    @staticmethod
1612
    def _prune_tails(parent_map, child_map, tails_to_remove):
1613
        """Remove tails from the parent map.
1614
        
1615
        This will remove the supplied revisions until no more children have 0
1616
        parents.
1617
1618
        :param parent_map: A dict of {child: [parents]}, this dictionary will
1619
            be modified in place.
1620
        :param tails_to_remove: A list of tips that should be removed,
1621
            this list will be consumed
1622
        :param child_map: The reverse dict of parent_map ({parent: [children]})
1623
            this dict will be modified
1624
        :return: None, parent_map will be modified in place.
1625
        """
1626
        while tails_to_remove:
1627
            next = tails_to_remove.pop()
1628
            parent_map.pop(next)
1629
            children = child_map.pop(next)
1630
            for child in children:
1631
                child_parents = parent_map[child]
1632
                child_parents.remove(next)
1633
                if len(child_parents) == 0:
1634
                    tails_to_remove.append(child)
3514.2.7 by John Arbash Meinel
Fix the failing test by implementing the fallback logic.
1635
3514.2.5 by John Arbash Meinel
Switch to the get_parent_map design I had settled on.
1636
    def _get_interesting_texts(self, parent_map):
3514.2.2 by John Arbash Meinel
Restore a real weave merge to 'bzr merge --weave'.
1637
        """Return a dict of texts we are interested in.
1638
1639
        Note that the input is in key tuples, but the output is in plain
1640
        revision ids.
1641
3514.2.5 by John Arbash Meinel
Switch to the get_parent_map design I had settled on.
1642
        :param parent_map: The output from _find_recursive_lcas
3514.2.2 by John Arbash Meinel
Restore a real weave merge to 'bzr merge --weave'.
1643
        :return: A dict of {'revision_id':lines} as returned by
1644
            _PlanMergeBase.get_lines()
1645
        """
3514.2.5 by John Arbash Meinel
Switch to the get_parent_map design I had settled on.
1646
        all_revision_keys = set(parent_map)
1647
        all_revision_keys.add(self.a_key)
1648
        all_revision_keys.add(self.b_key)
3514.2.2 by John Arbash Meinel
Restore a real weave merge to 'bzr merge --weave'.
1649
3514.2.5 by John Arbash Meinel
Switch to the get_parent_map design I had settled on.
1650
        # Everything else is in 'keys' but get_lines is in 'revision_ids'
1651
        all_texts = self.get_lines([k[-1] for k in all_revision_keys])
3514.2.2 by John Arbash Meinel
Restore a real weave merge to 'bzr merge --weave'.
1652
        return all_texts
1653
1654
    def _build_weave(self):
1655
        from bzrlib import weave
1656
        self._weave = weave.Weave(weave_name='in_memory_weave',
1657
                                  allow_reserved=True)
3514.2.5 by John Arbash Meinel
Switch to the get_parent_map design I had settled on.
1658
        parent_map = self._find_recursive_lcas()
1659
1660
        all_texts = self._get_interesting_texts(parent_map)
1661
1662
        # Note: Unfortunately, the order given by topo_sort will effect the
1663
        # ordering resolution in the output. Specifically, if you add A then B,
1664
        # then in the output text A lines will show up before B lines. And, of
1665
        # course, topo_sort doesn't guarantee any real ordering.
3514.2.8 by John Arbash Meinel
The insertion ordering into the weave has an impact on conflicts.
1666
        # So we use merge_sort, and add a fake node on the tip.
1667
        # This ensures that left-hand parents will always be inserted into the
1668
        # weave before right-hand parents.
1669
        tip_key = self._key_prefix + (_mod_revision.CURRENT_REVISION,)
1670
        parent_map[tip_key] = (self.a_key, self.b_key)
1671
1672
        for seq_num, key, depth, eom in reversed(tsort.merge_sort(parent_map,
1673
                                                                  tip_key)):
1674
            if key == tip_key:
1675
                continue
3514.2.9 by John Arbash Meinel
Add some debugging, and work on getting the graph right so we get the weave insertion order correct.
1676
        # for key in tsort.topo_sort(parent_map):
3514.2.5 by John Arbash Meinel
Switch to the get_parent_map design I had settled on.
1677
            parent_keys = parent_map[key]
1678
            revision_id = key[-1]
1679
            parent_ids = [k[-1] for k in parent_keys]
1680
            self._weave.add_lines(revision_id, parent_ids,
1681
                                  all_texts[revision_id])
3514.2.2 by John Arbash Meinel
Restore a real weave merge to 'bzr merge --weave'.
1682
1683
    def plan_merge(self):
1684
        """Generate a 'plan' for merging the two revisions.
1685
1686
        This involves comparing their texts and determining the cause of
1687
        differences.  If text A has a line and text B does not, then either the
1688
        line was added to text A, or it was deleted from B.  Once the causes
1689
        are combined, they are written out in the format described in
1690
        VersionedFile.plan_merge
1691
        """
1692
        if self._head_key is not None: # There was a single head
1693
            if self._head_key == self.a_key:
1694
                plan = 'new-a'
3062.1.9 by Aaron Bentley
Move PlanMerge into merge and _PlanMergeVersionedFile into versionedfile
1695
            else:
3514.2.2 by John Arbash Meinel
Restore a real weave merge to 'bzr merge --weave'.
1696
                if self._head_key != self.b_key:
1697
                    raise AssertionError('There was an invalid head: %s != %s'
1698
                                         % (self.b_key, self._head_key))
1699
                plan = 'new-b'
3514.2.16 by John Arbash Meinel
Review feedback from Ian.
1700
            head_rev = self._head_key[-1]
1701
            lines = self.get_lines([head_rev])[head_rev]
3514.2.2 by John Arbash Meinel
Restore a real weave merge to 'bzr merge --weave'.
1702
            return ((plan, line) for line in lines)
1703
        return self._weave.plan_merge(self.a_rev, self.b_rev)
3062.2.1 by Aaron Bentley
Add support for plan-merge with a base
1704
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1705
1706
class _PlanLCAMerge(_PlanMergeBase):
1707
    """
3144.3.7 by Aaron Bentley
Update from review
1708
    This merge algorithm differs from _PlanMerge in that:
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1709
    1. comparisons are done against LCAs only
1710
    2. cases where a contested line is new versus one LCA but old versus
3144.3.7 by Aaron Bentley
Update from review
1711
       another are marked as conflicts, by emitting the line as conflicted-a
1712
       or conflicted-b.
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1713
1714
    This is faster, and hopefully produces more useful output.
1715
    """
1716
3350.6.10 by Martin Pool
VersionedFiles review cleanups
1717
    def __init__(self, a_rev, b_rev, vf, key_prefix, graph):
1718
        _PlanMergeBase.__init__(self, a_rev, b_rev, vf, key_prefix)
1719
        lcas = graph.find_lca(key_prefix + (a_rev,), key_prefix + (b_rev,))
3350.6.5 by Robert Collins
Update to bzr.dev.
1720
        self.lcas = set()
1721
        for lca in lcas:
1722
            if lca == NULL_REVISION:
1723
                self.lcas.add(lca)
1724
            else:
1725
                self.lcas.add(lca[-1])
3144.3.7 by Aaron Bentley
Update from review
1726
        for lca in self.lcas:
3287.17.1 by John Arbash Meinel
Fix bug #235715 by using the empty list as the text for a base of NULL_REVISION.
1727
            if _mod_revision.is_null(lca):
1728
                lca_lines = []
1729
            else:
3350.6.5 by Robert Collins
Update to bzr.dev.
1730
                lca_lines = self.get_lines([lca])[lca]
3144.3.7 by Aaron Bentley
Update from review
1731
            matcher = patiencediff.PatienceSequenceMatcher(None, self.lines_a,
1732
                                                           lca_lines)
1733
            blocks = list(matcher.get_matching_blocks())
1734
            self._cached_matching_blocks[(a_rev, lca)] = blocks
1735
            matcher = patiencediff.PatienceSequenceMatcher(None, self.lines_b,
1736
                                                           lca_lines)
1737
            blocks = list(matcher.get_matching_blocks())
1738
            self._cached_matching_blocks[(b_rev, lca)] = blocks
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1739
3144.3.7 by Aaron Bentley
Update from review
1740
    def _determine_status(self, revision_id, unique_line_numbers):
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1741
        """Determines the status unique lines versus all lcas.
1742
1743
        Basically, determines why the line is unique to this revision.
1744
1745
        A line may be determined new, killed, or both.
1746
3144.3.7 by Aaron Bentley
Update from review
1747
        If a line is determined new, that means it was not present in at least
1748
        one LCA, and is not present in the other merge revision.
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1749
1750
        If a line is determined killed, that means the line was present in
1751
        at least one LCA.
1752
1753
        If a line is killed and new, this indicates that the two merge
1754
        revisions contain differing conflict resolutions.
3144.3.7 by Aaron Bentley
Update from review
1755
        :param revision_id: The id of the revision in which the lines are
1756
            unique
1757
        :param unique_line_numbers: The line numbers of unique lines.
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1758
        :return a tuple of (new_this, killed_other):
1759
        """
1760
        new = set()
1761
        killed = set()
3144.3.7 by Aaron Bentley
Update from review
1762
        unique_line_numbers = set(unique_line_numbers)
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1763
        for lca in self.lcas:
1764
            blocks = self._get_matching_blocks(revision_id, lca)
1765
            unique_vs_lca, _ignored = self._unique_lines(blocks)
3144.3.7 by Aaron Bentley
Update from review
1766
            new.update(unique_line_numbers.intersection(unique_vs_lca))
1767
            killed.update(unique_line_numbers.difference(unique_vs_lca))
3144.3.1 by Aaron Bentley
Implement LCA merge, with problematic conflict markers
1768
        return new, killed