/brz/remove-bazaar

To get this branch, use:
bzr branch http://gegoxaren.bato24.eu/bzr/brz/remove-bazaar
1608.2.1 by Martin Pool
[merge] Storage filename escaping
1
# Copyright (C) 2005, 2006 Canonical Ltd
1887.1.1 by Adeodato Simó
Do not separate paragraphs in the copyright statement with blank lines,
2
#
1196 by Martin Pool
- [WIP] retrieve historical texts from weaves
3
# This program is free software; you can redistribute it and/or modify
4
# it under the terms of the GNU General Public License as published by
5
# the Free Software Foundation; either version 2 of the License, or
6
# (at your option) any later version.
1887.1.1 by Adeodato Simó
Do not separate paragraphs in the copyright statement with blank lines,
7
#
1196 by Martin Pool
- [WIP] retrieve historical texts from weaves
8
# This program is distributed in the hope that it will be useful,
9
# but WITHOUT ANY WARRANTY; without even the implied warranty of
10
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
11
# GNU General Public License for more details.
1887.1.1 by Adeodato Simó
Do not separate paragraphs in the copyright statement with blank lines,
12
#
1196 by Martin Pool
- [WIP] retrieve historical texts from weaves
13
# You should have received a copy of the GNU General Public License
14
# along with this program; if not, write to the Free Software
4183.7.1 by Sabin Iacob
update FSF mailing address
15
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
1196 by Martin Pool
- [WIP] retrieve historical texts from weaves
16
6379.6.3 by Jelmer Vernooij
Use absolute_import.
17
from __future__ import absolute_import
18
1393.1.19 by Martin Pool
- add WeaveStore.__iter__, __contains__ and copy_multi()
19
# XXX: Some consideration of the problems that might occur if there are
20
# files whose id differs only in case.  That should probably be forbidden.
21
22
1471 by Robert Collins
Bugfix to previous url escaping patch - include weave stores
23
import os
1223 by Martin Pool
- store inventories in weave
24
6686.2.1 by Jelmer Vernooij
Move breezy.store to breezy.plugins.weave_fmt, its only user.
25
from .... import (
2294.1.10 by John Arbash Meinel
Switch all apis over to utf8 file ids. All tests pass
26
    errors,
27
    osutils,
6686.2.1 by Jelmer Vernooij
Move breezy.store to breezy.plugins.weave_fmt, its only user.
28
    ui,
2294.1.10 by John Arbash Meinel
Switch all apis over to utf8 file ids. All tests pass
29
    )
6686.2.1 by Jelmer Vernooij
Move breezy.store to breezy.plugins.weave_fmt, its only user.
30
from . import TransportStore
31
from ....trace import mutter
1393.2.2 by John Arbash Meinel
Updated stores to use Transport
32
33
1563.2.16 by Robert Collins
Change WeaveStore into VersionedFileStore and make its versoined file class parameterisable.
34
class VersionedFileStore(TransportStore):
35
    """Collection of many versioned files in a transport."""
1393.1.19 by Martin Pool
- add WeaveStore.__iter__, __contains__ and copy_multi()
36
1651.1.1 by Martin Pool
[merge][wip] Storage escaping
37
    # TODO: Rather than passing versionedfile_kwargs, perhaps pass in a
38
    # transport factory callable?
1185.58.4 by John Arbash Meinel
Added permission checking to Branch, and propogated that change into the stores.
39
    def __init__(self, transport, prefixed=False, precious=False,
1563.2.16 by Robert Collins
Change WeaveStore into VersionedFileStore and make its versoined file class parameterisable.
40
                 dir_mode=None, file_mode=None,
5582.10.23 by Jelmer Vernooij
Move bzrlib.weavefile.
41
                 versionedfile_class=None,
1651.1.1 by Martin Pool
[merge][wip] Storage escaping
42
                 versionedfile_kwargs={},
1608.2.1 by Martin Pool
[merge] Storage filename escaping
43
                 escaped=False):
44
        super(VersionedFileStore, self).__init__(transport,
1185.58.4 by John Arbash Meinel
Added permission checking to Branch, and propogated that change into the stores.
45
                dir_mode=dir_mode, file_mode=file_mode,
1185.80.1 by John Arbash Meinel
Text store and weave store both allow escaping fileid paths.
46
                prefixed=prefixed, compressed=False, escaped=escaped)
1417.1.10 by Robert Collins
add a cache bound to Transactions, and a precious facility, so that we keep inventory.weave in memory, but can discard weaves for other such files.
47
        self._precious = precious
1563.2.16 by Robert Collins
Change WeaveStore into VersionedFileStore and make its versoined file class parameterisable.
48
        self._versionedfile_class = versionedfile_class
1628.1.4 by Robert Collins
Change knit format to use non-delta, non-annotated revisions and signatures.
49
        self._versionedfile_kwargs = versionedfile_kwargs
3316.2.3 by Robert Collins
Remove manual notification of transaction finishing on versioned files.
50
        # Used for passing get_scope to versioned file constructors;
51
        self.get_scope = None
1196 by Martin Pool
- [WIP] retrieve historical texts from weaves
52
1223 by Martin Pool
- store inventories in weave
53
    def filename(self, file_id):
1393.2.2 by John Arbash Meinel
Updated stores to use Transport
54
        """Return the path relative to the transport root."""
1608.2.1 by Martin Pool
[merge] Storage filename escaping
55
        return self._relpath(file_id)
1429 by Robert Collins
merge in niemeyers prefixed-store patch
56
1393.1.19 by Martin Pool
- add WeaveStore.__iter__, __contains__ and copy_multi()
57
    def __iter__(self):
1563.2.16 by Robert Collins
Change WeaveStore into VersionedFileStore and make its versoined file class parameterisable.
58
        suffixes = self._versionedfile_class.get_suffixes()
1563.2.15 by Robert Collins
remove the weavestore assumptions about the number and nature of files it manages.
59
        ids = set()
1479 by Robert Collins
More quoting at the transport layer bugfixes.
60
        for relpath in self._iter_files_recursive():
1563.2.15 by Robert Collins
remove the weavestore assumptions about the number and nature of files it manages.
61
            for suffix in suffixes:
62
                if relpath.endswith(suffix):
1608.2.1 by Martin Pool
[merge] Storage filename escaping
63
                    # TODO: use standard remove_suffix function
64
                    escaped_id = os.path.basename(relpath[:-len(suffix)])
3350.6.1 by Robert Collins
* New ``versionedfile.KeyMapper`` interface to abstract out the access to
65
                    file_id = self._mapper.unmap(escaped_id)[0]
1608.2.1 by Martin Pool
[merge] Storage filename escaping
66
                    if file_id not in ids:
67
                        ids.add(file_id)
68
                        yield file_id
69
                    break # only one suffix can match
1393.1.19 by Martin Pool
- add WeaveStore.__iter__, __contains__ and copy_multi()
70
2294.1.10 by John Arbash Meinel
Switch all apis over to utf8 file ids. All tests pass
71
    def has_id(self, file_id):
1563.2.16 by Robert Collins
Change WeaveStore into VersionedFileStore and make its versoined file class parameterisable.
72
        suffixes = self._versionedfile_class.get_suffixes()
2294.1.10 by John Arbash Meinel
Switch all apis over to utf8 file ids. All tests pass
73
        filename = self.filename(file_id)
1563.2.15 by Robert Collins
remove the weavestore assumptions about the number and nature of files it manages.
74
        for suffix in suffixes:
75
            if not self._transport.has(filename + suffix):
76
                return False
77
        return True
1223 by Martin Pool
- store inventories in weave
78
1563.2.25 by Robert Collins
Merge in upstream.
79
    def get_empty(self, file_id, transaction):
80
        """Get an empty weave, which implies deleting the existing one first."""
81
        if self.has_id(file_id):
82
            self.delete(file_id, transaction)
83
        return self.get_weave_or_empty(file_id, transaction)
84
85
    def delete(self, file_id, transaction):
86
        """Remove file_id from the store."""
87
        suffixes = self._versionedfile_class.get_suffixes()
88
        filename = self.filename(file_id)
89
        for suffix in suffixes:
90
            self._transport.delete(filename + suffix)
1393.2.2 by John Arbash Meinel
Updated stores to use Transport
91
92
    def _get(self, file_id):
93
        return self._transport.get(self.filename(file_id))
94
95
    def _put(self, file_id, f):
1185.80.6 by John Arbash Meinel
Adding tests to make sure weave stores can retrieve the files they add.
96
        fn = self.filename(file_id)
97
        try:
1955.3.9 by John Arbash Meinel
Find more occurrances of put() and replace with put_file or put_bytes
98
            return self._transport.put_file(fn, f, mode=self._file_mode)
1773.4.1 by Martin Pool
Add pyflakes makefile target; fix many warnings
99
        except errors.NoSuchFile:
1185.80.6 by John Arbash Meinel
Adding tests to make sure weave stores can retrieve the files they add.
100
            if not self._prefixed:
101
                raise
102
            self._transport.mkdir(os.path.dirname(fn), mode=self._dir_mode)
1955.3.9 by John Arbash Meinel
Find more occurrances of put() and replace with put_file or put_bytes
103
            return self._transport.put_file(fn, f, mode=self._file_mode)
1223 by Martin Pool
- store inventories in weave
104
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
105
    def get_weave(self, file_id, transaction, _filename=None):
106
        """Return the VersionedFile for file_id.
107
108
        :param _filename: filename that would be returned from self.filename for
109
        file_id. This is used to reduce duplicate filename calculations when
110
        using 'get_weave_or_empty'. FOR INTERNAL USE ONLY.
111
        """
112
        if _filename is None:
113
            _filename = self.filename(file_id)
1594.2.23 by Robert Collins
Test versioned file storage handling of clean/dirty status for accessed versioned files.
114
        if transaction.writeable():
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
115
            w = self._versionedfile_class(_filename, self._transport, self._file_mode,
3316.2.3 by Robert Collins
Remove manual notification of transaction finishing on versioned files.
116
                get_scope=self.get_scope, **self._versionedfile_kwargs)
1594.2.23 by Robert Collins
Test versioned file storage handling of clean/dirty status for accessed versioned files.
117
        else:
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
118
            w = self._versionedfile_class(_filename,
1594.2.23 by Robert Collins
Test versioned file storage handling of clean/dirty status for accessed versioned files.
119
                                          self._transport,
120
                                          self._file_mode,
121
                                          create=False,
1628.1.4 by Robert Collins
Change knit format to use non-delta, non-annotated revisions and signatures.
122
                                          access_mode='r',
3316.2.3 by Robert Collins
Remove manual notification of transaction finishing on versioned files.
123
                                          get_scope=self.get_scope,
1628.1.4 by Robert Collins
Change knit format to use non-delta, non-annotated revisions and signatures.
124
                                          **self._versionedfile_kwargs)
1363 by Martin Pool
- add quick-and-dirty cache for weaves to speed check command
125
        return w
1262 by Martin Pool
- fetch should also copy ancestry records
126
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
127
    def _make_new_versionedfile(self, file_id, transaction,
128
        known_missing=False, _filename=None):
129
        """Make a new versioned file.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
130
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
131
        :param _filename: filename that would be returned from self.filename for
132
        file_id. This is used to reduce duplicate filename calculations when
133
        using 'get_weave_or_empty'. FOR INTERNAL USE ONLY.
134
        """
135
        if not known_missing and self.has_id(file_id):
1563.2.34 by Robert Collins
Remove the commit and rollback transaction methods as misleading, and implement a WriteTransaction
136
            self.delete(file_id, transaction)
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
137
        if _filename is None:
138
            _filename = self.filename(file_id)
1563.2.14 by Robert Collins
Prepare weave store to delegate copy details to the versioned file.
139
        try:
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
140
            # we try without making the directory first because thats optimising
141
            # for the common case.
142
            weave = self._versionedfile_class(_filename, self._transport, self._file_mode, create=True,
3316.2.3 by Robert Collins
Remove manual notification of transaction finishing on versioned files.
143
                get_scope=self.get_scope, **self._versionedfile_kwargs)
1773.4.1 by Martin Pool
Add pyflakes makefile target; fix many warnings
144
        except errors.NoSuchFile:
1563.2.14 by Robert Collins
Prepare weave store to delegate copy details to the versioned file.
145
            if not self._prefixed:
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
146
                # unexpected error - NoSuchFile is expected to be raised on a
147
                # missing dir only and that only occurs when we are prefixed.
1563.2.14 by Robert Collins
Prepare weave store to delegate copy details to the versioned file.
148
                raise
3350.6.1 by Robert Collins
* New ``versionedfile.KeyMapper`` interface to abstract out the access to
149
            dirname = osutils.dirname(_filename)
150
            self._transport.mkdir(dirname, mode=self._dir_mode)
3316.2.3 by Robert Collins
Remove manual notification of transaction finishing on versioned files.
151
            weave = self._versionedfile_class(_filename, self._transport,
1651.1.1 by Martin Pool
[merge][wip] Storage escaping
152
                                              self._file_mode, create=True,
3316.2.3 by Robert Collins
Remove manual notification of transaction finishing on versioned files.
153
                                              get_scope=self.get_scope,
1628.1.4 by Robert Collins
Change knit format to use non-delta, non-annotated revisions and signatures.
154
                                              **self._versionedfile_kwargs)
1563.2.14 by Robert Collins
Prepare weave store to delegate copy details to the versioned file.
155
        return weave
156
1417.1.8 by Robert Collins
use transactions in the weave store interface, which enables caching for log
157
    def get_weave_or_empty(self, file_id, transaction):
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
158
        """Return a weave, or an empty one if it doesn't exist."""
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
159
        # This is typically used from 'commit' and 'fetch/push/pull' where
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
160
        # we scan across many versioned files once. As such the small overhead
161
        # of calculating the filename before doing a cache lookup is more than
162
        # compensated for by not calculating the filename when making new
163
        # versioned files.
164
        _filename = self.filename(file_id)
1224 by Martin Pool
- new method WeaveStore.get_weave_or_empty
165
        try:
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
166
            return self.get_weave(file_id, transaction, _filename=_filename)
1773.4.1 by Martin Pool
Add pyflakes makefile target; fix many warnings
167
        except errors.NoSuchFile:
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
168
            weave = self._make_new_versionedfile(file_id, transaction,
169
                known_missing=True, _filename=_filename)
170
            return weave
1224 by Martin Pool
- new method WeaveStore.get_weave_or_empty
171
1563.2.10 by Robert Collins
Change weave store to be a versioned store, using WeaveFiles which maintain integrity without needing explicit 'put' operations.
172
    def _put_weave(self, file_id, weave, transaction):
173
        """Preserved here for upgrades-to-weaves to use."""
1563.2.34 by Robert Collins
Remove the commit and rollback transaction methods as misleading, and implement a WriteTransaction
174
        myweave = self._make_new_versionedfile(file_id, transaction)
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
175
        myweave.insert_record_stream(weave.get_record_stream(
176
            [(version,) for version in weave.versions()],
3350.3.20 by Robert Collins
Remove more uses of versionedfile.join() and fix a bug with weave's insert_record_stream.
177
            'topological', False))
1223 by Martin Pool
- store inventories in weave
178
1563.2.31 by Robert Collins
Convert Knit repositories to use knits.
179
    def total_size(self):
180
        count, bytes =  super(VersionedFileStore, self).total_size()
181
        return (count / len(self._versionedfile_class.get_suffixes())), bytes