/brz/remove-bazaar

To get this branch, use:
bzr branch http://gegoxaren.bato24.eu/bzr/brz/remove-bazaar
1553.5.13 by Martin Pool
New Transport.rename that mustn't overwrite
1
# Copyright (C) 2005, 2006 Canonical Ltd
1185.11.19 by John Arbash Meinel
Testing put and append, also testing agaist file-like objects as well as strings.
2
3
# This program is free software; you can redistribute it and/or modify
4
# it under the terms of the GNU General Public License as published by
5
# the Free Software Foundation; either version 2 of the License, or
6
# (at your option) any later version.
7
8
# This program is distributed in the hope that it will be useful,
9
# but WITHOUT ANY WARRANTY; without even the implied warranty of
10
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
11
# GNU General Public License for more details.
12
13
# You should have received a copy of the GNU General Public License
14
# along with this program; if not, write to the Free Software
15
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
1185.16.72 by Martin Pool
[merge] from robert and fix up tests
16
17
"""Transport for the local filesystem.
18
19
This is a fairly thin wrapper on regular file IO."""
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
20
1442.1.42 by Robert Collins
rebuild ScratchBranch on top of ScratchTransport
21
import os
22
import shutil
1608.2.5 by Martin Pool
Add Transport.supports_unix_modebits, so tests can
23
import sys
1442.1.44 by Robert Collins
Many transport related tweaks:
24
from stat import ST_MODE, S_ISDIR, ST_SIZE
1442.1.42 by Robert Collins
rebuild ScratchBranch on top of ScratchTransport
25
import tempfile
1469 by Robert Collins
Change Transport.* to work with URL's.
26
import urllib
1442.1.42 by Robert Collins
rebuild ScratchBranch on top of ScratchTransport
27
28
from bzrlib.trace import mutter
1530.1.3 by Robert Collins
transport implementations now tested consistently.
29
from bzrlib.transport import Transport, Server
1185.31.58 by John Arbash Meinel
Updating for new transport tests so that they pass on win32
30
from bzrlib.osutils import abspath, realpath, normpath, pathjoin, rename
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
31
1442.1.42 by Robert Collins
rebuild ScratchBranch on top of ScratchTransport
32
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
33
class LocalTransport(Transport):
34
    """This is the transport agent for local filesystem access."""
35
36
    def __init__(self, base):
37
        """Set the base path where files will be stored."""
1185.11.1 by John Arbash Meinel
(broken) Transport work is merged in. Tests do not pass yet.
38
        if base.startswith('file://'):
1608.1.1 by Martin Pool
[patch] LocalTransport.list_dir should return url-quoted strings (ddaa)
39
            base = base[len('file://'):]
1092.2.24 by Robert Collins
merge from martins newformat branch - brings in transport abstraction
40
        # realpath is incompatible with symlinks. When we traverse
41
        # up we might be able to normpath stuff. RBC 20051003
1530.1.7 by Robert Collins
merge integration.
42
        base = normpath(abspath(base))
1530.1.3 by Robert Collins
transport implementations now tested consistently.
43
        if base[-1] != '/':
44
            base = base + '/'
45
        super(LocalTransport, self).__init__(base)
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
46
907.1.32 by John Arbash Meinel
Renaming is_remote to should_cache as it is more appropriate.
47
    def should_cache(self):
907.1.22 by John Arbash Meinel
Fixed some encoding issues, added is_remote function for Transport objects.
48
        return False
49
907.1.2 by John Arbash Meinel
Working on making Branch() do all of it's work over a Transport.
50
    def clone(self, offset=None):
51
        """Return a new LocalTransport with root at self.base + offset
52
        Because the local filesystem does not require a connection, 
53
        we can just return a new object.
54
        """
55
        if offset is None:
56
            return LocalTransport(self.base)
57
        else:
58
            return LocalTransport(self.abspath(offset))
59
907.1.8 by John Arbash Meinel
Changed the format for abspath. Updated branch to use a hidden _transport
60
    def abspath(self, relpath):
1636.1.1 by Robert Collins
Fix calling relpath() and abspath() on transports at their root.
61
        """Return the full url to the given relative URL."""
1185.12.70 by Aaron Bentley
Removed b
62
        assert isinstance(relpath, basestring), (type(relpath), relpath)
1636.1.1 by Robert Collins
Fix calling relpath() and abspath() on transports at their root.
63
        result = normpath(pathjoin(self.base, urllib.unquote(relpath)))
64
        #if result[-1] != '/':
65
        #    result += '/'
66
        return result
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
67
907.1.24 by John Arbash Meinel
Remote functionality work.
68
    def relpath(self, abspath):
69
        """Return the local path portion from a given absolute path.
70
        """
1457.1.2 by Robert Collins
move branch._relpath into osutils as relpath
71
        from bzrlib.osutils import relpath
1442.1.64 by Robert Collins
Branch.open_containing now returns a tuple (Branch, relative-path).
72
        if abspath is None:
1185.33.66 by Martin Pool
[patch] use unicode literals for all hardcoded paths (Alexander Belchenko)
73
            abspath = u'.'
1636.1.1 by Robert Collins
Fix calling relpath() and abspath() on transports at their root.
74
        if len(abspath) > 1 and abspath.endswith('/'):
1530.1.3 by Robert Collins
transport implementations now tested consistently.
75
            abspath = abspath[:-1]
1636.1.1 by Robert Collins
Fix calling relpath() and abspath() on transports at their root.
76
        if self.base == '/':
77
            root = '/'
78
        else:
79
            root = self.base[:-1]
80
        return relpath(root, abspath)
907.1.24 by John Arbash Meinel
Remote functionality work.
81
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
82
    def has(self, relpath):
907.1.2 by John Arbash Meinel
Working on making Branch() do all of it's work over a Transport.
83
        return os.access(self.abspath(relpath), os.F_OK)
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
84
907.1.50 by John Arbash Meinel
Removed encode/decode from Transport.put/get, added more exceptions that can be thrown.
85
    def get(self, relpath):
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
86
        """Get the file at the given relative path.
907.1.20 by John Arbash Meinel
Removed Transport.open(), making get + put encode/decode to utf-8
87
88
        :param relpath: The relative path to the file
89
        """
907.1.48 by John Arbash Meinel
Updated LocalTransport by passing it through the transport_test suite, and got it to pass.
90
        try:
907.1.50 by John Arbash Meinel
Removed encode/decode from Transport.put/get, added more exceptions that can be thrown.
91
            path = self.abspath(relpath)
92
            return open(path, 'rb')
1185.31.44 by John Arbash Meinel
Cleaned up Exceptions for all transports.
93
        except (IOError, OSError),e:
94
            self._translate_error(e, path)
907.1.20 by John Arbash Meinel
Removed Transport.open(), making get + put encode/decode to utf-8
95
1185.58.2 by John Arbash Meinel
Added mode to the appropriate transport functions, and tests to make sure they work.
96
    def put(self, relpath, f, mode=None):
907.1.20 by John Arbash Meinel
Removed Transport.open(), making get + put encode/decode to utf-8
97
        """Copy the file-like or string object into the location.
98
99
        :param relpath: Location to put the contents, relative to base.
100
        :param f:       File-like or string object.
101
        """
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
102
        from bzrlib.atomicfile import AtomicFile
103
1185.31.44 by John Arbash Meinel
Cleaned up Exceptions for all transports.
104
        path = relpath
907.1.48 by John Arbash Meinel
Updated LocalTransport by passing it through the transport_test suite, and got it to pass.
105
        try:
907.1.50 by John Arbash Meinel
Removed encode/decode from Transport.put/get, added more exceptions that can be thrown.
106
            path = self.abspath(relpath)
1185.58.2 by John Arbash Meinel
Added mode to the appropriate transport functions, and tests to make sure they work.
107
            fp = AtomicFile(path, 'wb', new_mode=mode)
1185.31.44 by John Arbash Meinel
Cleaned up Exceptions for all transports.
108
        except (IOError, OSError),e:
109
            self._translate_error(e, path)
907.1.2 by John Arbash Meinel
Working on making Branch() do all of it's work over a Transport.
110
        try:
111
            self._pump(f, fp)
112
            fp.commit()
113
        finally:
114
            fp.close()
115
1442.1.44 by Robert Collins
Many transport related tweaks:
116
    def iter_files_recursive(self):
117
        """Iter the relative paths of files in the transports sub-tree."""
1185.33.66 by Martin Pool
[patch] use unicode literals for all hardcoded paths (Alexander Belchenko)
118
        queue = list(self.list_dir(u'.'))
1442.1.44 by Robert Collins
Many transport related tweaks:
119
        while queue:
1608.1.1 by Martin Pool
[patch] LocalTransport.list_dir should return url-quoted strings (ddaa)
120
            relpath = queue.pop(0)
1442.1.44 by Robert Collins
Many transport related tweaks:
121
            st = self.stat(relpath)
122
            if S_ISDIR(st[ST_MODE]):
123
                for i, basename in enumerate(self.list_dir(relpath)):
124
                    queue.insert(i, relpath+'/'+basename)
125
            else:
126
                yield relpath
127
1185.58.2 by John Arbash Meinel
Added mode to the appropriate transport functions, and tests to make sure they work.
128
    def mkdir(self, relpath, mode=None):
907.1.2 by John Arbash Meinel
Working on making Branch() do all of it's work over a Transport.
129
        """Create a directory at the given path."""
1185.31.44 by John Arbash Meinel
Cleaned up Exceptions for all transports.
130
        path = relpath
907.1.48 by John Arbash Meinel
Updated LocalTransport by passing it through the transport_test suite, and got it to pass.
131
        try:
1185.31.44 by John Arbash Meinel
Cleaned up Exceptions for all transports.
132
            path = self.abspath(relpath)
133
            os.mkdir(path)
1185.58.2 by John Arbash Meinel
Added mode to the appropriate transport functions, and tests to make sure they work.
134
            if mode is not None:
135
                os.chmod(path, mode)
1185.31.44 by John Arbash Meinel
Cleaned up Exceptions for all transports.
136
        except (IOError, OSError),e:
137
            self._translate_error(e, path)
907.1.2 by John Arbash Meinel
Working on making Branch() do all of it's work over a Transport.
138
907.1.50 by John Arbash Meinel
Removed encode/decode from Transport.put/get, added more exceptions that can be thrown.
139
    def append(self, relpath, f):
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
140
        """Append the text in the file-like object into the final
141
        location.
142
        """
1530.1.4 by Robert Collins
integrate Memory tests into transport interface tests.
143
        try:
144
            fp = open(self.abspath(relpath), 'ab')
145
        except (IOError, OSError),e:
146
            self._translate_error(e, relpath)
1551.2.38 by abentley
Fixed win32 bug with tell
147
        # win32 workaround (tell on an unwritten file returns 0)
148
        fp.seek(0, 2)
1563.2.3 by Robert Collins
Change the return signature of transport.append and append_multi to return the length of the pre-append content.
149
        result = fp.tell()
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
150
        self._pump(f, fp)
1563.2.3 by Robert Collins
Change the return signature of transport.append and append_multi to return the length of the pre-append content.
151
        return result
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
152
153
    def copy(self, rel_from, rel_to):
154
        """Copy the item at rel_from to the location at rel_to"""
155
        import shutil
907.1.2 by John Arbash Meinel
Working on making Branch() do all of it's work over a Transport.
156
        path_from = self.abspath(rel_from)
157
        path_to = self.abspath(rel_to)
907.1.48 by John Arbash Meinel
Updated LocalTransport by passing it through the transport_test suite, and got it to pass.
158
        try:
159
            shutil.copy(path_from, path_to)
1185.31.44 by John Arbash Meinel
Cleaned up Exceptions for all transports.
160
        except (IOError, OSError),e:
161
            # TODO: What about path_to?
162
            self._translate_error(e, path_from)
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
163
1553.5.13 by Martin Pool
New Transport.rename that mustn't overwrite
164
    def rename(self, rel_from, rel_to):
165
        path_from = self.abspath(rel_from)
166
        try:
167
            # *don't* call bzrlib.osutils.rename, because we want to 
168
            # detect errors on rename
169
            os.rename(path_from, self.abspath(rel_to))
170
        except (IOError, OSError),e:
171
            # TODO: What about path_to?
172
            self._translate_error(e, path_from)
173
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
174
    def move(self, rel_from, rel_to):
175
        """Move the item at rel_from to the location at rel_to"""
907.1.2 by John Arbash Meinel
Working on making Branch() do all of it's work over a Transport.
176
        path_from = self.abspath(rel_from)
177
        path_to = self.abspath(rel_to)
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
178
907.1.48 by John Arbash Meinel
Updated LocalTransport by passing it through the transport_test suite, and got it to pass.
179
        try:
1553.5.13 by Martin Pool
New Transport.rename that mustn't overwrite
180
            # this version will delete the destination if necessary
1185.31.58 by John Arbash Meinel
Updating for new transport tests so that they pass on win32
181
            rename(path_from, path_to)
1185.31.44 by John Arbash Meinel
Cleaned up Exceptions for all transports.
182
        except (IOError, OSError),e:
183
            # TODO: What about path_to?
184
            self._translate_error(e, path_from)
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
185
186
    def delete(self, relpath):
187
        """Delete the item at relpath"""
1185.31.44 by John Arbash Meinel
Cleaned up Exceptions for all transports.
188
        path = relpath
907.1.48 by John Arbash Meinel
Updated LocalTransport by passing it through the transport_test suite, and got it to pass.
189
        try:
1185.31.44 by John Arbash Meinel
Cleaned up Exceptions for all transports.
190
            path = self.abspath(relpath)
191
            os.remove(path)
192
        except (IOError, OSError),e:
193
            # TODO: What about path_to?
194
            self._translate_error(e, path)
907.1.1 by John Arbash Meinel
Reworking the Branch and Store code to support an abstracted filesystem layer.
195
1185.58.2 by John Arbash Meinel
Added mode to the appropriate transport functions, and tests to make sure they work.
196
    def copy_to(self, relpaths, other, mode=None, pb=None):
907.1.28 by John Arbash Meinel
Added pb to function that were missing, implemented a basic double-dispatch copy_to function.
197
        """Copy a set of entries from self into another Transport.
198
199
        :param relpaths: A list/generator of entries to be copied.
200
        """
201
        if isinstance(other, LocalTransport):
202
            # Both from & to are on the local filesystem
203
            # Unfortunately, I can't think of anything faster than just
204
            # copying them across, one by one :(
205
            import shutil
206
207
            total = self._get_total(relpaths)
208
            count = 0
209
            for path in relpaths:
210
                self._update_pb(pb, 'copy-to', count, total)
1185.16.158 by John Arbash Meinel
Added a test that copy_to raises NoSuchFile when a directory is missing (not IOError)
211
                try:
1185.58.2 by John Arbash Meinel
Added mode to the appropriate transport functions, and tests to make sure they work.
212
                    mypath = self.abspath(path)
213
                    otherpath = other.abspath(path)
214
                    shutil.copy(mypath, otherpath)
215
                    if mode is not None:
216
                        os.chmod(otherpath, mode)
1185.31.44 by John Arbash Meinel
Cleaned up Exceptions for all transports.
217
                except (IOError, OSError),e:
218
                    self._translate_error(e, path)
907.1.28 by John Arbash Meinel
Added pb to function that were missing, implemented a basic double-dispatch copy_to function.
219
                count += 1
220
            return count
221
        else:
1185.58.2 by John Arbash Meinel
Added mode to the appropriate transport functions, and tests to make sure they work.
222
            return super(LocalTransport, self).copy_to(relpaths, other, mode=mode, pb=pb)
907.1.28 by John Arbash Meinel
Added pb to function that were missing, implemented a basic double-dispatch copy_to function.
223
1400.1.1 by Robert Collins
implement a basic test for the ui branch command from http servers
224
    def listable(self):
225
        """See Transport.listable."""
226
        return True
907.1.28 by John Arbash Meinel
Added pb to function that were missing, implemented a basic double-dispatch copy_to function.
227
907.1.2 by John Arbash Meinel
Working on making Branch() do all of it's work over a Transport.
228
    def list_dir(self, relpath):
229
        """Return a list of all files at the given location.
230
        WARNING: many transports do not support this, so trying avoid using
231
        it if at all possible.
232
        """
1608.1.1 by Martin Pool
[patch] LocalTransport.list_dir should return url-quoted strings (ddaa)
233
        path = self.abspath(relpath)
907.1.48 by John Arbash Meinel
Updated LocalTransport by passing it through the transport_test suite, and got it to pass.
234
        try:
1608.1.1 by Martin Pool
[patch] LocalTransport.list_dir should return url-quoted strings (ddaa)
235
            return [urllib.quote(entry) for entry in os.listdir(path)]
1607.1.3 by Robert Collins
Apply David Allouches list_dir quoting fix.
236
        except (IOError, OSError), e:
1185.31.44 by John Arbash Meinel
Cleaned up Exceptions for all transports.
237
            self._translate_error(e, path)
907.1.2 by John Arbash Meinel
Working on making Branch() do all of it's work over a Transport.
238
239
    def stat(self, relpath):
240
        """Return the stat information for a file.
241
        """
1185.31.44 by John Arbash Meinel
Cleaned up Exceptions for all transports.
242
        path = relpath
907.1.48 by John Arbash Meinel
Updated LocalTransport by passing it through the transport_test suite, and got it to pass.
243
        try:
1185.31.44 by John Arbash Meinel
Cleaned up Exceptions for all transports.
244
            path = self.abspath(relpath)
245
            return os.stat(path)
246
        except (IOError, OSError),e:
247
            self._translate_error(e, path)
907.1.2 by John Arbash Meinel
Working on making Branch() do all of it's work over a Transport.
248
907.1.24 by John Arbash Meinel
Remote functionality work.
249
    def lock_read(self, relpath):
250
        """Lock the given file for shared (read) access.
251
        :return: A lock object, which should be passed to Transport.unlock()
252
        """
253
        from bzrlib.lock import ReadLock
1185.65.29 by Robert Collins
Implement final review suggestions.
254
        path = relpath
255
        try:
256
            path = self.abspath(relpath)
257
            return ReadLock(path)
258
        except (IOError, OSError), e:
259
            self._translate_error(e, path)
907.1.24 by John Arbash Meinel
Remote functionality work.
260
261
    def lock_write(self, relpath):
262
        """Lock the given file for exclusive (write) access.
263
        WARNING: many transports do not support this, so trying avoid using it
264
265
        :return: A lock object, which should be passed to Transport.unlock()
266
        """
267
        from bzrlib.lock import WriteLock
268
        return WriteLock(self.abspath(relpath))
269
1534.4.15 by Robert Collins
Remove shutil dependency in upgrade - create a delete_tree method for transports.
270
    def rmdir(self, relpath):
271
        """See Transport.rmdir."""
272
        path = relpath
273
        try:
274
            path = self.abspath(relpath)
275
            os.rmdir(path)
276
        except (IOError, OSError),e:
277
            self._translate_error(e, path)
1442.1.41 by Robert Collins
move duplicate scratch logic into a scratch transport
278
1608.2.7 by Martin Pool
Rename supports_unix_modebits to _can_roundtrip_unix_modebits for clarity
279
    def _can_roundtrip_unix_modebits(self):
1608.2.5 by Martin Pool
Add Transport.supports_unix_modebits, so tests can
280
        if sys.platform == 'win32':
281
            # anyone else?
282
            return False
283
        else:
284
            return True
285
286
1442.1.41 by Robert Collins
move duplicate scratch logic into a scratch transport
287
class ScratchTransport(LocalTransport):
288
    """A transport that works in a temporary dir and cleans up after itself.
289
    
290
    The dir only exists for the lifetime of the Python object.
291
    Obviously you should not put anything precious in it.
292
    """
293
1442.1.42 by Robert Collins
rebuild ScratchBranch on top of ScratchTransport
294
    def __init__(self, base=None):
295
        if base is None:
296
            base = tempfile.mkdtemp()
1442.1.41 by Robert Collins
move duplicate scratch logic into a scratch transport
297
        super(ScratchTransport, self).__init__(base)
298
299
    def __del__(self):
1442.1.42 by Robert Collins
rebuild ScratchBranch on top of ScratchTransport
300
        shutil.rmtree(self.base, ignore_errors=True)
1442.1.41 by Robert Collins
move duplicate scratch logic into a scratch transport
301
        mutter("%r destroyed" % self)
1530.1.1 by Robert Collins
Minimal infrastructure to test TransportTestProviderAdapter.
302
303
1530.1.3 by Robert Collins
transport implementations now tested consistently.
304
class LocalRelpathServer(Server):
1530.1.1 by Robert Collins
Minimal infrastructure to test TransportTestProviderAdapter.
305
    """A pretend server for local transports, using relpaths."""
306
1530.1.3 by Robert Collins
transport implementations now tested consistently.
307
    def get_url(self):
308
        """See Transport.Server.get_url."""
309
        return "."
310
311
312
class LocalAbspathServer(Server):
1530.1.1 by Robert Collins
Minimal infrastructure to test TransportTestProviderAdapter.
313
    """A pretend server for local transports, using absolute paths."""
314
1530.1.3 by Robert Collins
transport implementations now tested consistently.
315
    def get_url(self):
316
        """See Transport.Server.get_url."""
317
        return os.path.abspath("")
318
319
320
class LocalURLServer(Server):
1530.1.1 by Robert Collins
Minimal infrastructure to test TransportTestProviderAdapter.
321
    """A pretend server for local transports, using file:// urls."""
1530.1.3 by Robert Collins
transport implementations now tested consistently.
322
323
    def get_url(self):
324
        """See Transport.Server.get_url."""
325
        # FIXME: \ to / on windows
326
        return "file://%s" % os.path.abspath("")
1530.1.11 by Robert Collins
Push the transport permutations list into each transport module allowing for automatic testing of new modules that are registered as transports.
327
328
329
def get_test_permutations():
330
    """Return the permutations to be used in testing."""
331
    return [(LocalTransport, LocalRelpathServer),
332
            (LocalTransport, LocalAbspathServer),
333
            (LocalTransport, LocalURLServer),
334
            ]