1
# Copyright (C) 2006 Canonical Ltd
3
# This program is free software; you can redistribute it and/or modify
4
# it under the terms of the GNU General Public License as published by
5
# the Free Software Foundation; either version 2 of the License, or
6
# (at your option) any later version.
8
# This program is distributed in the hope that it will be useful,
9
# but WITHOUT ANY WARRANTY; without even the implied warranty of
10
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11
# GNU General Public License for more details.
13
# You should have received a copy of the GNU General Public License
14
# along with this program; if not, write to the Free Software
15
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
17
"""Smart-server protocol, client and server.
19
This code is fairly complex, so it has been split up into a package of modules,
20
rather than being a single large module. Refer to the individual module
21
docstrings for details.
26
The smart protocol provides a way to send a requests and corresponding
27
responses to communicate with a remote bzr process.
35
At the bottom level there is either a socket, pipes, or an HTTP
36
request/response. We call this layer the *medium*. It is responsible for
37
carrying bytes between a client and server. For sockets, we have the
38
idea that you have multiple requests and get a read error because the other side
39
did shutdown. For pipes we have read pipe which will have a zero read which
40
marks end-of-file. For HTTP server environment there is no end-of-stream
41
because each request coming into the server is independent.
43
So we need a wrapper around pipes and sockets to seperate out requests from
44
substrate and this will give us a single model which is consistent for HTTP,
50
On top of the medium is the *protocol*. This is the layer that deserialises
51
bytes into the structured data that requests and responses consist of.
53
Version one of the protocol (for requests and responses) is described by::
56
RESPONSE := MESSAGE_V1
57
MESSAGE_V1 := ARGS BODY
59
ARGS := ARG [MORE_ARGS] NEWLINE
60
MORE_ARGS := SEP ARG [MORE_ARGS]
63
BODY := LENGTH NEWLINE BODY_BYTES TRAILER
64
LENGTH := decimal integer
65
TRAILER := "done" NEWLINE
67
That is, a tuple of arguments separated by Ctrl-A and terminated with a newline,
68
followed by length prefixed body with a constant trailer. Note that although
69
arguments are not 8-bit safe (they cannot include 0x01 or 0x0a bytes without
70
breaking the protocol encoding), the body is.
72
Version two of the request protocol is::
74
REQUEST_V2 := "bzr request 2" NEWLINE MESSAGE_V1
76
Version two of the response protocol is::
78
RESPONSE_V2 := "bzr request 2" NEWLINE MESSAGE_V1
80
Future versions should follow this structure, like version two does::
82
FUTURE_MESSAGE := VERSION_STRING NEWLINE REST_OF_MESSAGE
84
This is that clients and servers can read bytes up to the first newline byte to
85
determine what version a message is.
87
Request/Response processing
88
---------------------------
90
On top of the protocol is the logic for processing requests (on the server) or
91
responses (on the client).
98
MEDIUM (factory for protocol, reads bytes & pushes to protocol,
99
uses protocol to detect end-of-request, sends written
100
bytes to client) e.g. socket, pipe, HTTP request handler.
105
PROTOCOL(serialization, deserialization) accepts bytes for one
106
request, decodes according to internal state, pushes
107
structured data to handler. accepts structured data from
108
handler and encodes and writes to the medium. factory for
114
HANDLER (domain logic) accepts structured data, operates state
115
machine until the request can be satisfied,
116
sends structured data to the protocol.
118
Request handlers are registered in `bzrlib.smart.request`.
126
CLIENT domain logic, accepts domain requests, generated structured
127
data, reads structured data from responses and turns into
128
domain data. Sends structured data to the protocol.
129
Operates state machines until the request can be delivered
130
(e.g. reading from a bundle generated in bzrlib to deliver a
133
Possibly this should just be RemoteBzrDir, RemoteTransport,
139
PROTOCOL (serialization, deserialization) accepts structured data for one
140
request, encodes and writes to the medium. Reads bytes from the
141
medium, decodes and allows the client to read structured data.
146
MEDIUM (accepts bytes from the protocol & delivers to the remote server.
147
Allows the potocol to read bytes e.g. socket, pipe, HTTP request.
149
The domain logic is in `bzrlib.remote`: `RemoteBzrDir`, `RemoteBranch`, and so
152
There is also an plain file-level transport that calls remote methods to
153
manipulate files on the server in `bzrlib.transport.remote`.
158
Paths are passed across the network. The client needs to see a namespace that
159
includes any repository that might need to be referenced, and the client needs
160
to know about a root directory beyond which it cannot ascend.
162
Servers run over ssh will typically want to be able to access any path the user
163
can access. Public servers on the other hand (which might be over http, ssh
164
or tcp) will typically want to restrict access to only a particular directory
165
and its children, so will want to do a software virtual root at that level.
166
In other words they'll want to rewrite incoming paths to be under that level
167
(and prevent escaping using ../ tricks.)
169
URLs that include ~ should probably be passed across to the server verbatim
170
and the server can expand them. This will proably not be meaningful when
171
limited to a directory?
174
# TODO: _translate_error should be on the client, not the transport because
175
# error coding is wire protocol specific.
177
# TODO: A plain integer from query_version is too simple; should give some
180
# TODO: Server should probably catch exceptions within itself and send them
181
# back across the network. (But shouldn't catch KeyboardInterrupt etc)
182
# Also needs to somehow report protocol errors like bad requests. Need to
183
# consider how we'll handle error reporting, e.g. if we get halfway through a
184
# bulk transfer and then something goes wrong.
186
# TODO: Make each request and response self-validatable, e.g. with checksums.
188
# TODO: get/put objects could be changed to gradually read back the data as it
189
# comes across the network
191
# TODO: What should the server do if it hits an error and has to terminate?
193
# TODO: is it useful to allow multiple chunks in the bulk data?
195
# TODO: If we get an exception during transmission of bulk data we can't just
196
# emit the exception because it won't be seen.
197
# John proposes: I think it would be worthwhile to have a header on each
198
# chunk, that indicates it is another chunk. Then you can send an 'error'
199
# chunk as long as you finish the previous chunk.
201
# TODO: Clone method on Transport; should work up towards parent directory;
202
# unclear how this should be stored or communicated to the server... maybe
203
# just pass it on all relevant requests?
205
# TODO: Better name than clone() for changing between directories. How about
206
# open_dir or change_dir or chdir?
208
# TODO: Is it really good to have the notion of current directory within the
209
# connection? Perhaps all Transports should factor out a common connection
210
# from the thing that has the directory context?
212
# TODO: The server that manages a connection should be quite small and retain
213
# minimum state because each of the requests are supposed to be stateless.
214
# Then we can write another implementation that maps to http.
216
# TODO: What to do when a client connection is garbage collected? Maybe just
217
# abruptly drop the connection?
219
# TODO: Server in some cases will need to restrict access to files outside of
220
# a particular root directory. LocalTransport doesn't do anything to stop you
221
# ascending above the base directory, so we need to prevent paths
222
# containing '..' in either the server or transport layers. (Also need to
223
# consider what happens if someone creates a symlink pointing outside the
226
# TODO: Server should rebase absolute paths coming across the network to put
227
# them under the virtual root, if one is in use. LocalTransport currently
228
# doesn't do that; if you give it an absolute path it just uses it.
230
# XXX: Arguments can't contain newlines or ascii; possibly we should e.g.
231
# urlescape them instead. Indeed possibly this should just literally be
234
# TODO: Probably want some way for server commands to gradually produce body
235
# data rather than passing it as a string; they could perhaps pass an
236
# iterator-like callback that will gradually yield data; it probably needs a
237
# close() method that will always be closed to do any necessary cleanup.
240
# Promote some attributes from submodules into this namespace
241
from bzrlib.smart.request import SmartServerRequestHandler
added
removed
bzrlib/smart/__init__.py
