/brz/remove-bazaar

To get this branch, use:
bzr branch http://gegoxaren.bato24.eu/bzr/brz/remove-bazaar
1608.2.1 by Martin Pool
[merge] Storage filename escaping
1
# Copyright (C) 2005, 2006 Canonical Ltd
1887.1.1 by Adeodato Simó
Do not separate paragraphs in the copyright statement with blank lines,
2
#
1196 by Martin Pool
- [WIP] retrieve historical texts from weaves
3
# This program is free software; you can redistribute it and/or modify
4
# it under the terms of the GNU General Public License as published by
5
# the Free Software Foundation; either version 2 of the License, or
6
# (at your option) any later version.
1887.1.1 by Adeodato Simó
Do not separate paragraphs in the copyright statement with blank lines,
7
#
1196 by Martin Pool
- [WIP] retrieve historical texts from weaves
8
# This program is distributed in the hope that it will be useful,
9
# but WITHOUT ANY WARRANTY; without even the implied warranty of
10
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
11
# GNU General Public License for more details.
1887.1.1 by Adeodato Simó
Do not separate paragraphs in the copyright statement with blank lines,
12
#
1196 by Martin Pool
- [WIP] retrieve historical texts from weaves
13
# You should have received a copy of the GNU General Public License
14
# along with this program; if not, write to the Free Software
4183.7.1 by Sabin Iacob
update FSF mailing address
15
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
1196 by Martin Pool
- [WIP] retrieve historical texts from weaves
16
1393.1.19 by Martin Pool
- add WeaveStore.__iter__, __contains__ and copy_multi()
17
# XXX: Some consideration of the problems that might occur if there are
18
# files whose id differs only in case.  That should probably be forbidden.
19
20
1471 by Robert Collins
Bugfix to previous url escaping patch - include weave stores
21
import os
1223 by Martin Pool
- store inventories in weave
22
6686.2.1 by Jelmer Vernooij
Move breezy.store to breezy.plugins.weave_fmt, its only user.
23
from .... import (
2294.1.10 by John Arbash Meinel
Switch all apis over to utf8 file ids. All tests pass
24
    errors,
25
    osutils,
6686.2.1 by Jelmer Vernooij
Move breezy.store to breezy.plugins.weave_fmt, its only user.
26
    ui,
2294.1.10 by John Arbash Meinel
Switch all apis over to utf8 file ids. All tests pass
27
    )
6686.2.1 by Jelmer Vernooij
Move breezy.store to breezy.plugins.weave_fmt, its only user.
28
from . import TransportStore
29
from ....trace import mutter
1393.2.2 by John Arbash Meinel
Updated stores to use Transport
30
31
1563.2.16 by Robert Collins
Change WeaveStore into VersionedFileStore and make its versoined file class parameterisable.
32
class VersionedFileStore(TransportStore):
33
    """Collection of many versioned files in a transport."""
1393.1.19 by Martin Pool
- add WeaveStore.__iter__, __contains__ and copy_multi()
34
1651.1.1 by Martin Pool
[merge][wip] Storage escaping
35
    # TODO: Rather than passing versionedfile_kwargs, perhaps pass in a
36
    # transport factory callable?
1185.58.4 by John Arbash Meinel
Added permission checking to Branch, and propogated that change into the stores.
37
    def __init__(self, transport, prefixed=False, precious=False,
1563.2.16 by Robert Collins
Change WeaveStore into VersionedFileStore and make its versoined file class parameterisable.
38
                 dir_mode=None, file_mode=None,
5582.10.23 by Jelmer Vernooij
Move bzrlib.weavefile.
39
                 versionedfile_class=None,
1651.1.1 by Martin Pool
[merge][wip] Storage escaping
40
                 versionedfile_kwargs={},
1608.2.1 by Martin Pool
[merge] Storage filename escaping
41
                 escaped=False):
42
        super(VersionedFileStore, self).__init__(transport,
7143.15.2 by Jelmer Vernooij
Run autopep8.
43
                                                 dir_mode=dir_mode, file_mode=file_mode,
44
                                                 prefixed=prefixed, compressed=False, escaped=escaped)
1417.1.10 by Robert Collins
add a cache bound to Transactions, and a precious facility, so that we keep inventory.weave in memory, but can discard weaves for other such files.
45
        self._precious = precious
1563.2.16 by Robert Collins
Change WeaveStore into VersionedFileStore and make its versoined file class parameterisable.
46
        self._versionedfile_class = versionedfile_class
1628.1.4 by Robert Collins
Change knit format to use non-delta, non-annotated revisions and signatures.
47
        self._versionedfile_kwargs = versionedfile_kwargs
3316.2.3 by Robert Collins
Remove manual notification of transaction finishing on versioned files.
48
        # Used for passing get_scope to versioned file constructors;
49
        self.get_scope = None
1196 by Martin Pool
- [WIP] retrieve historical texts from weaves
50
1223 by Martin Pool
- store inventories in weave
51
    def filename(self, file_id):
1393.2.2 by John Arbash Meinel
Updated stores to use Transport
52
        """Return the path relative to the transport root."""
1608.2.1 by Martin Pool
[merge] Storage filename escaping
53
        return self._relpath(file_id)
1429 by Robert Collins
merge in niemeyers prefixed-store patch
54
1393.1.19 by Martin Pool
- add WeaveStore.__iter__, __contains__ and copy_multi()
55
    def __iter__(self):
1563.2.16 by Robert Collins
Change WeaveStore into VersionedFileStore and make its versoined file class parameterisable.
56
        suffixes = self._versionedfile_class.get_suffixes()
1563.2.15 by Robert Collins
remove the weavestore assumptions about the number and nature of files it manages.
57
        ids = set()
1479 by Robert Collins
More quoting at the transport layer bugfixes.
58
        for relpath in self._iter_files_recursive():
1563.2.15 by Robert Collins
remove the weavestore assumptions about the number and nature of files it manages.
59
            for suffix in suffixes:
60
                if relpath.endswith(suffix):
1608.2.1 by Martin Pool
[merge] Storage filename escaping
61
                    # TODO: use standard remove_suffix function
62
                    escaped_id = os.path.basename(relpath[:-len(suffix)])
3350.6.1 by Robert Collins
* New ``versionedfile.KeyMapper`` interface to abstract out the access to
63
                    file_id = self._mapper.unmap(escaped_id)[0]
1608.2.1 by Martin Pool
[merge] Storage filename escaping
64
                    if file_id not in ids:
65
                        ids.add(file_id)
66
                        yield file_id
7143.15.2 by Jelmer Vernooij
Run autopep8.
67
                    break  # only one suffix can match
1393.1.19 by Martin Pool
- add WeaveStore.__iter__, __contains__ and copy_multi()
68
2294.1.10 by John Arbash Meinel
Switch all apis over to utf8 file ids. All tests pass
69
    def has_id(self, file_id):
1563.2.16 by Robert Collins
Change WeaveStore into VersionedFileStore and make its versoined file class parameterisable.
70
        suffixes = self._versionedfile_class.get_suffixes()
2294.1.10 by John Arbash Meinel
Switch all apis over to utf8 file ids. All tests pass
71
        filename = self.filename(file_id)
1563.2.15 by Robert Collins
remove the weavestore assumptions about the number and nature of files it manages.
72
        for suffix in suffixes:
73
            if not self._transport.has(filename + suffix):
74
                return False
75
        return True
1223 by Martin Pool
- store inventories in weave
76
1563.2.25 by Robert Collins
Merge in upstream.
77
    def get_empty(self, file_id, transaction):
78
        """Get an empty weave, which implies deleting the existing one first."""
79
        if self.has_id(file_id):
80
            self.delete(file_id, transaction)
81
        return self.get_weave_or_empty(file_id, transaction)
82
83
    def delete(self, file_id, transaction):
84
        """Remove file_id from the store."""
85
        suffixes = self._versionedfile_class.get_suffixes()
86
        filename = self.filename(file_id)
87
        for suffix in suffixes:
88
            self._transport.delete(filename + suffix)
1393.2.2 by John Arbash Meinel
Updated stores to use Transport
89
90
    def _get(self, file_id):
91
        return self._transport.get(self.filename(file_id))
92
93
    def _put(self, file_id, f):
1185.80.6 by John Arbash Meinel
Adding tests to make sure weave stores can retrieve the files they add.
94
        fn = self.filename(file_id)
95
        try:
1955.3.9 by John Arbash Meinel
Find more occurrances of put() and replace with put_file or put_bytes
96
            return self._transport.put_file(fn, f, mode=self._file_mode)
1773.4.1 by Martin Pool
Add pyflakes makefile target; fix many warnings
97
        except errors.NoSuchFile:
1185.80.6 by John Arbash Meinel
Adding tests to make sure weave stores can retrieve the files they add.
98
            if not self._prefixed:
99
                raise
100
            self._transport.mkdir(os.path.dirname(fn), mode=self._dir_mode)
1955.3.9 by John Arbash Meinel
Find more occurrances of put() and replace with put_file or put_bytes
101
            return self._transport.put_file(fn, f, mode=self._file_mode)
1223 by Martin Pool
- store inventories in weave
102
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
103
    def get_weave(self, file_id, transaction, _filename=None):
104
        """Return the VersionedFile for file_id.
105
106
        :param _filename: filename that would be returned from self.filename for
107
        file_id. This is used to reduce duplicate filename calculations when
108
        using 'get_weave_or_empty'. FOR INTERNAL USE ONLY.
109
        """
110
        if _filename is None:
111
            _filename = self.filename(file_id)
1594.2.23 by Robert Collins
Test versioned file storage handling of clean/dirty status for accessed versioned files.
112
        if transaction.writeable():
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
113
            w = self._versionedfile_class(_filename, self._transport, self._file_mode,
7143.15.2 by Jelmer Vernooij
Run autopep8.
114
                                          get_scope=self.get_scope, **self._versionedfile_kwargs)
1594.2.23 by Robert Collins
Test versioned file storage handling of clean/dirty status for accessed versioned files.
115
        else:
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
116
            w = self._versionedfile_class(_filename,
1594.2.23 by Robert Collins
Test versioned file storage handling of clean/dirty status for accessed versioned files.
117
                                          self._transport,
118
                                          self._file_mode,
119
                                          create=False,
1628.1.4 by Robert Collins
Change knit format to use non-delta, non-annotated revisions and signatures.
120
                                          access_mode='r',
3316.2.3 by Robert Collins
Remove manual notification of transaction finishing on versioned files.
121
                                          get_scope=self.get_scope,
1628.1.4 by Robert Collins
Change knit format to use non-delta, non-annotated revisions and signatures.
122
                                          **self._versionedfile_kwargs)
1363 by Martin Pool
- add quick-and-dirty cache for weaves to speed check command
123
        return w
1262 by Martin Pool
- fetch should also copy ancestry records
124
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
125
    def _make_new_versionedfile(self, file_id, transaction,
7143.15.2 by Jelmer Vernooij
Run autopep8.
126
                                known_missing=False, _filename=None):
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
127
        """Make a new versioned file.
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
128
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
129
        :param _filename: filename that would be returned from self.filename for
130
        file_id. This is used to reduce duplicate filename calculations when
131
        using 'get_weave_or_empty'. FOR INTERNAL USE ONLY.
132
        """
133
        if not known_missing and self.has_id(file_id):
1563.2.34 by Robert Collins
Remove the commit and rollback transaction methods as misleading, and implement a WriteTransaction
134
            self.delete(file_id, transaction)
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
135
        if _filename is None:
136
            _filename = self.filename(file_id)
1563.2.14 by Robert Collins
Prepare weave store to delegate copy details to the versioned file.
137
        try:
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
138
            # we try without making the directory first because thats optimising
139
            # for the common case.
140
            weave = self._versionedfile_class(_filename, self._transport, self._file_mode, create=True,
7143.15.2 by Jelmer Vernooij
Run autopep8.
141
                                              get_scope=self.get_scope, **self._versionedfile_kwargs)
1773.4.1 by Martin Pool
Add pyflakes makefile target; fix many warnings
142
        except errors.NoSuchFile:
1563.2.14 by Robert Collins
Prepare weave store to delegate copy details to the versioned file.
143
            if not self._prefixed:
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
144
                # unexpected error - NoSuchFile is expected to be raised on a
145
                # missing dir only and that only occurs when we are prefixed.
1563.2.14 by Robert Collins
Prepare weave store to delegate copy details to the versioned file.
146
                raise
3350.6.1 by Robert Collins
* New ``versionedfile.KeyMapper`` interface to abstract out the access to
147
            dirname = osutils.dirname(_filename)
148
            self._transport.mkdir(dirname, mode=self._dir_mode)
3316.2.3 by Robert Collins
Remove manual notification of transaction finishing on versioned files.
149
            weave = self._versionedfile_class(_filename, self._transport,
1651.1.1 by Martin Pool
[merge][wip] Storage escaping
150
                                              self._file_mode, create=True,
3316.2.3 by Robert Collins
Remove manual notification of transaction finishing on versioned files.
151
                                              get_scope=self.get_scope,
1628.1.4 by Robert Collins
Change knit format to use non-delta, non-annotated revisions and signatures.
152
                                              **self._versionedfile_kwargs)
1563.2.14 by Robert Collins
Prepare weave store to delegate copy details to the versioned file.
153
        return weave
154
1417.1.8 by Robert Collins
use transactions in the weave store interface, which enables caching for log
155
    def get_weave_or_empty(self, file_id, transaction):
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
156
        """Return a weave, or an empty one if it doesn't exist."""
3943.8.1 by Marius Kruger
remove all trailing whitespace from bzr source
157
        # This is typically used from 'commit' and 'fetch/push/pull' where
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
158
        # we scan across many versioned files once. As such the small overhead
159
        # of calculating the filename before doing a cache lookup is more than
160
        # compensated for by not calculating the filename when making new
161
        # versioned files.
162
        _filename = self.filename(file_id)
1224 by Martin Pool
- new method WeaveStore.get_weave_or_empty
163
        try:
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
164
            return self.get_weave(file_id, transaction, _filename=_filename)
1773.4.1 by Martin Pool
Add pyflakes makefile target; fix many warnings
165
        except errors.NoSuchFile:
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
166
            weave = self._make_new_versionedfile(file_id, transaction,
7143.15.2 by Jelmer Vernooij
Run autopep8.
167
                                                 known_missing=True, _filename=_filename)
1725.2.2 by Robert Collins
reduce file path escaping calls during commit.
168
            return weave
1224 by Martin Pool
- new method WeaveStore.get_weave_or_empty
169
1563.2.10 by Robert Collins
Change weave store to be a versioned store, using WeaveFiles which maintain integrity without needing explicit 'put' operations.
170
    def _put_weave(self, file_id, weave, transaction):
171
        """Preserved here for upgrades-to-weaves to use."""
1563.2.34 by Robert Collins
Remove the commit and rollback transaction methods as misleading, and implement a WriteTransaction
172
        myweave = self._make_new_versionedfile(file_id, transaction)
3350.6.4 by Robert Collins
First cut at pluralised VersionedFiles. Some rather massive API incompatabilities, primarily because of the difficulty of coherence among competing stores.
173
        myweave.insert_record_stream(weave.get_record_stream(
174
            [(version,) for version in weave.versions()],
3350.3.20 by Robert Collins
Remove more uses of versionedfile.join() and fix a bug with weave's insert_record_stream.
175
            'topological', False))
1223 by Martin Pool
- store inventories in weave
176
1563.2.31 by Robert Collins
Convert Knit repositories to use knits.
177
    def total_size(self):
7143.15.2 by Jelmer Vernooij
Run autopep8.
178
        count, bytes = super(VersionedFileStore, self).total_size()
1563.2.31 by Robert Collins
Convert Knit repositories to use knits.
179
        return (count / len(self._versionedfile_class.get_suffixes())), bytes