256
251
Holds documentation on a whole range of things on Bazaar from the
257
252
origination of ideas within the project to information on Bazaar
258
253
features and use cases. Within this directory there is a subdirectory
259
for each translation into a human language. All the documentation
254
for each translation into a human language. All the documentation
260
255
is in the ReStructuredText markup language.
263
Documentation specifically targeted at Bazaar and plugin developers.
258
Documentation specifically targetted at Bazaar and plugin developers.
264
259
(Including this document.)
268
Automatically-generated API reference information is available at
269
<http://starship.python.net/crew/mwh/bzrlibapi/>.
271
See also the `Bazaar Architectural Overview
272
<http://doc.bazaar-vcs.org/developers/overview.html>`_.
275
The Code Review Process
276
#######################
278
All code changes coming in to Bazaar are reviewed by someone else.
279
Normally changes by core contributors are reviewed by one other core
280
developer, and changes from other people are reviewed by two core
281
developers. Use intelligent discretion if the patch is trivial.
283
Good reviews do take time. They also regularly require a solid
284
understanding of the overall code base. In practice, this means a small
285
number of people often have a large review burden - with knowledge comes
286
responsibility. No one likes their merge requests sitting in a queue going
287
nowhere, so reviewing sooner rather than later is strongly encouraged.
296
Please put a "cover letter" on your merge request explaining:
298
* the reason **why** you're making this change
300
* **how** this change achieves this purpose
302
* anything else you may have fixed in passing
304
* anything significant that you thought of doing, such as a more
305
extensive fix or a different approach, but didn't or couldn't do now
307
A good cover letter makes reviewers' lives easier because they can decide
308
from the letter whether they agree with the purpose and approach, and then
309
assess whether the patch actually does what the cover letter says.
310
Explaining any "drive-by fixes" or roads not taken may also avoid queries
311
from the reviewer. All in all this should give faster and better reviews.
312
Sometimes writing the cover letter helps the submitter realize something
313
else they need to do. The size of the cover letter should be proportional
314
to the size and complexity of the patch.
317
Reviewing proposed changes
318
==========================
320
Anyone is welcome to review code, and reply to the thread with their
323
The simplest way to review a proposed change is to just read the patch on
324
the list or in Bundle Buggy. For more complex changes it may be useful
325
to make a new working tree or branch from trunk, and merge the proposed
326
change into it, so you can experiment with the code or look at a wider
329
There are three main requirements for code to get in:
331
* Doesn't reduce test coverage: if it adds new methods or commands,
332
there should be tests for them. There is a good test framework
333
and plenty of examples to crib from, but if you are having trouble
334
working out how to test something feel free to post a draft patch
337
* Doesn't reduce design clarity, such as by entangling objects
338
we're trying to separate. This is mostly something the more
339
experienced reviewers need to help check.
341
* Improves bugs, features, speed, or code simplicity.
343
Code that goes in should not degrade any of these aspects. Patches are
344
welcome that only cleanup the code without changing the external
345
behaviour. The core developers take care to keep the code quality high
346
and understandable while recognising that perfect is sometimes the enemy
349
It is easy for reviews to make people notice other things which should be
350
fixed but those things should not hold up the original fix being accepted.
351
New things can easily be recorded in the Bug Tracker instead.
353
It's normally much easier to review several smaller patches than one large
354
one. You might want to use ``bzr-loom`` to maintain threads of related
355
work, or submit a preparatory patch that will make your "real" change
359
Checklist for reviewers
360
=======================
362
* Do you understand what the code's doing and why?
364
* Will it perform reasonably for large inputs, both in memory size and
365
run time? Are there some scenarios where performance should be
368
* Is it tested, and are the tests at the right level? Are there both
369
blackbox (command-line level) and API-oriented tests?
371
* If this change will be visible to end users or API users, is it
372
appropriately documented in NEWS?
374
* Does it meet the coding standards below?
376
* If it changes the user-visible behaviour, does it update the help
377
strings and user documentation?
379
* If it adds a new major concept or standard practice, does it update the
380
developer documentation?
382
* (your ideas here...)
388
From May 2009 on, we prefer people to propose code reviews through
391
* <https://launchpad.net/+tour/code-review>
393
* <https://help.launchpad.net/Code/Review>
395
Anyone can propose or comment on a merge proposal just by creating a
398
There are two ways to create a new merge proposal: through the web
399
interface or by email.
402
Proposing a merge through the web
403
---------------------------------
405
To create the proposal through the web, first push your branch to Launchpad.
406
For example, a branch dealing with documentation belonging to the Launchpad
407
User mbp could be pushed as ::
409
bzr push lp:~mbp/bzr/doc
411
Then go to the branch's web page, which in this case would be
412
<https://code.launchpad.net/~mbp/bzr/doc>. You can simplify this step by just
417
You can then click "Propose for merging into another branch", and enter your
418
cover letter (see above) into the web form. Typically you'll want to merge
419
into ``~bzr/bzr/trunk`` which will be the default; you might also want to
420
nominate merging into a release branch for a bug fix. There is the option to
421
specify a specific reviewer or type of review, and you shouldn't normally
424
Submitting the form takes you to the new page about the merge proposal
425
containing the diff of the changes, comments by interested people, and
426
controls to comment or vote on the change.
428
Proposing a merge by mail
263
Automatically-generated API reference information is available at
264
<http://starship.python.net/crew/mwh/bzrlibapi/>.
265
(There is an experimental editable version at
266
<http://starship.python.net/crew/mwh/bzrlibapi-oe/>.)
267
See also the `Essential Domain Classes`_
268
section of this guide.
274
The Importance of Testing
275
=========================
277
Reliability is a critical success factor for any Version Control System.
278
We want Bazaar to be highly reliable across multiple platforms while
279
evolving over time to meet the needs of its community.
281
In a nutshell, this is want we expect and encourage:
283
* New functionality should have test cases. Preferably write the
284
test before writing the code.
286
In general, you can test at either the command-line level or the
287
internal API level. See Writing tests below for more detail.
289
* Try to practice Test-Driven Development: before fixing a bug, write a
290
test case so that it does not regress. Similarly for adding a new
291
feature: write a test case for a small version of the new feature before
292
starting on the code itself. Check the test fails on the old code, then
293
add the feature or fix and check it passes.
295
By doing these things, the Bazaar team gets increased confidence that
296
changes do what they claim to do, whether provided by the core team or
297
by community members. Equally importantly, we can be surer that changes
298
down the track do not break new features or bug fixes that you are
301
As of May 2008, Bazaar ships with a test suite containing over 12000 tests
302
and growing. We are proud of it and want to remain so. As community
303
members, we all benefit from it. Would you trust version control on
304
your project to a product *without* a test suite like Bazaar has?
307
Running the Test Suite
308
======================
310
Currently, bzr selftest is used to invoke tests.
311
You can provide a pattern argument to run a subset. For example,
312
to run just the blackbox tests, run::
314
./bzr selftest -v blackbox
316
To skip a particular test (or set of tests), use the --exclude option
317
(shorthand -x) like so::
319
./bzr selftest -v -x blackbox
321
To ensure that all tests are being run and succeeding, you can use the
322
--strict option which will fail if there are any missing features or known
325
./bzr selftest --strict
327
To list tests without running them, use the --list-only option like so::
329
./bzr selftest --list-only
331
This option can be combined with other selftest options (like -x) and
332
filter patterns to understand their effect.
334
Once you understand how to create a list of tests, you can use the --load-list
335
option to run only a restricted set of tests that you kept in a file, one test
336
id by line. Keep in mind that this will never be sufficient to validate your
337
modifications, you still need to run the full test suite for that, but using it
338
can help in some cases (like running only the failed tests for some time)::
340
./bzr selftest -- load-list my_failing_tests
342
This option can also be combined with other selftest options, including
343
patterns. It has some drawbacks though, the list can become out of date pretty
344
quick when doing Test Driven Development.
346
To address this concern, there is another way to run a restricted set of tests:
347
the --starting-with option will run only the tests whose name starts with the
348
specified string. It will also avoid loading the other tests and as a
349
consequence starts running your tests quicker::
351
./bzr selftest --starting-with bzrlib.blackbox
353
This option can be combined with all the other selftest options including
354
--load-list. The later is rarely used but allows to run a subset of a list of
355
failing tests for example.
357
Test suite debug flags
358
----------------------
360
Similar to the global ``-Dfoo`` debug options, bzr selftest accepts
361
``-E=foo`` debug flags. These flags are:
363
:allow_debug: do *not* clear the global debug flags when running a test.
364
This can provide useful logging to help debug test failures when used
365
with e.g. ``bzr -Dhpss selftest -E=allow_debug``
371
In general tests should be placed in a file named test_FOO.py where
372
FOO is the logical thing under test. That file should be placed in the
373
tests subdirectory under the package being tested.
375
For example, tests for merge3 in bzrlib belong in bzrlib/tests/test_merge3.py.
376
See bzrlib/tests/test_sampler.py for a template test script.
378
Tests can be written for the UI or for individual areas of the library.
379
Choose whichever is appropriate: if adding a new command, or a new command
380
option, then you should be writing a UI test. If you are both adding UI
381
functionality and library functionality, you will want to write tests for
382
both the UI and the core behaviours. We call UI tests 'blackbox' tests
383
and they are found in ``bzrlib/tests/blackbox/*.py``.
385
When writing blackbox tests please honour the following conventions:
387
1. Place the tests for the command 'name' in
388
bzrlib/tests/blackbox/test_name.py. This makes it easy for developers
389
to locate the test script for a faulty command.
391
2. Use the 'self.run_bzr("name")' utility function to invoke the command
392
rather than running bzr in a subprocess or invoking the
393
cmd_object.run() method directly. This is a lot faster than
394
subprocesses and generates the same logging output as running it in a
395
subprocess (which invoking the method directly does not).
397
3. Only test the one command in a single test script. Use the bzrlib
398
library when setting up tests and when evaluating the side-effects of
399
the command. We do this so that the library api has continual pressure
400
on it to be as functional as the command line in a simple manner, and
401
to isolate knock-on effects throughout the blackbox test suite when a
402
command changes its name or signature. Ideally only the tests for a
403
given command are affected when a given command is changed.
405
4. If you have a test which does actually require running bzr in a
406
subprocess you can use ``run_bzr_subprocess``. By default the spawned
407
process will not load plugins unless ``--allow-plugins`` is supplied.
409
When writing library functionality, it is often necessary to set up a
410
branch with a certain history. Most current tests do this by inheriting
411
from ``TestCaseWithTransport`` and using the ``make_branch_and_tree``
412
helper to give them a ``WorkingTree`` that they can commit to. However,
413
there is a newer api available from ``TestCaseWithMemoryTransport`` using
414
the ``make_branch_builder`` helper. This helper is preferred, because it
415
can build the changes in memory, rather than on disk. Tests that are
416
explictly testing how we work with disk objects should, of course, use a
417
real ``WorkingTree``. See ``bzrlib/branch_builder.py`` for how to use the
424
We make selective use of doctests__. In general they should provide
425
*examples* within the API documentation which can incidentally be tested. We
426
don't try to test every important case using doctests -- regular Python
427
tests are generally a better solution.
429
Most of these are in ``bzrlib/doc/api``. More additions are welcome.
431
__ http://docs.python.org/lib/module-doctest.html
434
Skipping tests and test requirements
435
------------------------------------
437
In our enhancements to unittest we allow for some addition results beyond
438
just success or failure.
440
If a test can't be run, it can say that it's skipped. This is typically
441
used in parameterized tests - for example if a transport doesn't support
442
setting permissions, we'll skip the tests that relating to that. ::
445
return self.branch_format.initialize(repo.bzrdir)
446
except errors.UninitializableFormat:
447
raise tests.TestSkipped('Uninitializable branch format')
449
Raising TestSkipped is a good idea when you want to make it clear that the
450
test was not run, rather than just returning which makes it look as if it
453
Several different cases are distinguished:
456
Generic skip; the only type that was present up to bzr 0.18.
459
The test doesn't apply to the parameters with which it was run.
460
This is typically used when the test is being applied to all
461
implementations of an interface, but some aspects of the interface
462
are optional and not present in particular concrete
463
implementations. (Some tests that should raise this currently
464
either silently return or raise TestSkipped.) Another option is
465
to use more precise parameterization to avoid generating the test
469
**(Not implemented yet)**
470
The test can't be run because of an inherent limitation of the
471
environment, such as not having symlinks or not supporting
475
The test can't be run because a dependency (typically a Python
476
library) is not available in the test environment. These
477
are in general things that the person running the test could fix
478
by installing the library. It's OK if some of these occur when
479
an end user runs the tests or if we're specifically testing in a
480
limited environment, but a full test should never see them.
483
The test exists but is known to fail, for example because the
484
code to fix it hasn't been run yet. Raising this allows
485
you to distinguish these failures from the ones that are not
486
expected to fail. This could be conditionally raised if something
487
is broken on some platforms but not on others.
489
If the test would fail because of something we don't expect or
490
intend to fix, KnownFailure is not appropriate, and
491
TestNotApplicable might be better.
493
We plan to support three modes for running the test suite to control the
494
interpretation of these results. Strict mode is for use in situations
495
like merges to the mainline and releases where we want to make sure that
496
everything that can be tested has been tested. Lax mode is for use by
497
developers who want to temporarily tolerate some known failures. The
498
default behaviour is obtained by ``bzr selftest`` with no options, and
499
also (if possible) by running under another unittest harness.
501
======================= ======= ======= ========
502
result strict default lax
503
======================= ======= ======= ========
504
TestSkipped pass pass pass
505
TestNotApplicable pass pass pass
506
TestPlatformLimit pass pass pass
507
TestDependencyMissing fail pass pass
508
KnownFailure fail pass pass
509
======================= ======= ======= ========
512
Test feature dependencies
429
513
-------------------------
431
To propose a merge by mail, send a bundle to ``merge@code.launchpad.net``.
433
You can generate a merge request like this::
435
bzr send -o bug-1234.diff
437
``bzr send`` can also send mail directly if you prefer; see the help.
515
Rather than manually checking the environment in each test, a test class
516
can declare its dependence on some test features. The feature objects are
517
checked only once for each run of the whole test suite.
519
For historical reasons, as of May 2007 many cases that should depend on
520
features currently raise TestSkipped.)
524
class TestStrace(TestCaseWithTransport):
526
_test_needs_features = [StraceFeature]
528
This means all tests in this class need the feature. The feature itself
529
should provide a ``_probe`` method which is called once to determine if
532
These should generally be equivalent to either TestDependencyMissing or
533
sometimes TestPlatformLimit.
539
Known failures are when a test exists but we know it currently doesn't
540
work, allowing the test suite to still pass. These should be used with
541
care, we don't want a proliferation of quietly broken tests. It might be
542
appropriate to use them if you've committed a test for a bug but not the
543
fix for it, or if something works on Unix but not on Windows.
546
Testing exceptions and errors
547
-----------------------------
549
It's important to test handling of errors and exceptions. Because this
550
code is often not hit in ad-hoc testing it can often have hidden bugs --
551
it's particularly common to get NameError because the exception code
552
references a variable that has since been renamed.
554
.. TODO: Something about how to provoke errors in the right way?
556
In general we want to test errors at two levels:
558
1. A test in ``test_errors.py`` checking that when the exception object is
559
constructed with known parameters it produces an expected string form.
560
This guards against mistakes in writing the format string, or in the
561
``str`` representations of its parameters. There should be one for
562
each exception class.
564
2. Tests that when an api is called in a particular situation, it raises
565
an error of the expected class. You should typically use
566
``assertRaises``, which in the Bazaar test suite returns the exception
567
object to allow you to examine its parameters.
569
In some cases blackbox tests will also want to check error reporting. But
570
it can be difficult to provoke every error through the commandline
571
interface, so those tests are only done as needed -- eg in response to a
572
particular bug or if the error is reported in an unusual way(?) Blackbox
573
tests should mostly be testing how the command-line interface works, so
574
should only test errors if there is something particular to the cli in how
575
they're displayed or handled.
581
The Python ``warnings`` module is used to indicate a non-fatal code
582
problem. Code that's expected to raise a warning can be tested through
585
The test suite can be run with ``-Werror`` to check no unexpected errors
588
However, warnings should be used with discretion. It's not an appropriate
589
way to give messages to the user, because the warning is normally shown
590
only once per source line that causes the problem. You should also think
591
about whether the warning is serious enought that it should be visible to
592
users who may not be able to fix it.
595
Interface implementation testing and test scenarios
596
---------------------------------------------------
598
There are several cases in Bazaar of multiple implementations of a common
599
conceptual interface. ("Conceptual" because
600
it's not necessary for all the implementations to share a base class,
601
though they often do.) Examples include transports and the working tree,
602
branch and repository classes.
604
In these cases we want to make sure that every implementation correctly
605
fulfils the interface requirements. For example, every Transport should
606
support the ``has()`` and ``get()`` and ``clone()`` methods. We have a
607
sub-suite of tests in ``test_transport_implementations``. (Most
608
per-implementation tests are in submodules of ``bzrlib.tests``, but not
609
the transport tests at the moment.)
611
These tests are repeated for each registered Transport, by generating a
612
new TestCase instance for the cross product of test methods and transport
613
implementations. As each test runs, it has ``transport_class`` and
614
``transport_server`` set to the class it should test. Most tests don't
615
access these directly, but rather use ``self.get_transport`` which returns
616
a transport of the appropriate type.
618
The goal is to run per-implementation only tests that relate to that
619
particular interface. Sometimes we discover a bug elsewhere that happens
620
with only one particular transport. Once it's isolated, we can consider
621
whether a test should be added for that particular implementation,
622
or for all implementations of the interface.
624
The multiplication of tests for different implementations is normally
625
accomplished by overriding the ``test_suite`` function used to load
626
tests from a module. This function typically loads all the tests,
627
then applies a TestProviderAdapter to them, which generates a longer
628
suite containing all the test variations.
634
Some utilities are provided for generating variations of tests. This can
635
be used for per-implementation tests, or other cases where the same test
636
code needs to run several times on different scenarios.
638
The general approach is to define a class that provides test methods,
639
which depend on attributes of the test object being pre-set with the
640
values to which the test should be applied. The test suite should then
641
also provide a list of scenarios in which to run the tests.
643
Typically ``multiply_tests_from_modules`` should be called from the test
644
module's ``test_suite`` function.
647
Essential Domain Classes
648
########################
650
Introducing the Object Model
651
============================
653
The core domain objects within the bazaar model are:
663
Transports are explained below. See http://bazaar-vcs.org/Classes/
664
for an introduction to the other key classes.
669
The ``Transport`` layer handles access to local or remote directories.
670
Each Transport object acts like a logical connection to a particular
671
directory, and it allows various operations on files within it. You can
672
*clone* a transport to get a new Transport connected to a subdirectory or
675
Transports are not used for access to the working tree. At present
676
working trees are always local and they are accessed through the regular
677
Python file io mechanisms.
440
680
-----------------
442
From <https://code.launchpad.net/bzr/+activereviews> you can see all
443
currently active reviews, and choose one to comment on. This page also
444
shows proposals that are now approved and should be merged by someone with
448
Reviews through Bundle Buggy
449
============================
451
The Bundle Buggy tool used up to May 2009 is still available as a review
454
Sending patches for review
455
--------------------------
457
If you'd like to propose a change, please post to the
458
bazaar@lists.canonical.com list with a bundle, patch, or link to a
459
branch. Put ``[PATCH]`` or ``[MERGE]`` in the subject so Bundle Buggy
460
can pick it out, and explain the change in the email message text.
461
Remember to update the NEWS file as part of your change if it makes any
462
changes visible to users or plugin developers. Please include a diff
463
against mainline if you're giving a link to a branch.
465
You can generate a merge request like this::
467
bzr send -o bug-1234.patch
469
A ``.patch`` extension is recommended instead of .bundle as many mail clients
470
will send the latter as a binary file.
472
``bzr send`` can also send mail directly if you prefer; see the help.
474
Please do **NOT** put [PATCH] or [MERGE] in the subject line if you don't
475
want it to be merged. If you want comments from developers rather than
476
to be merged, you can put ``[RFC]`` in the subject line.
478
If this change addresses a bug, please put the bug number in the subject
479
line too, in the form ``[#1]`` so that Bundle Buggy can recognize it.
481
If the change is intended for a particular release mark that in the
482
subject too, e.g. ``[1.6]``.
483
Anyone can "vote" on the mailing list by expressing an opinion. Core
484
developers can also vote using Bundle Buggy. Here are the voting codes and
487
:approve: Reviewer wants this submission merged.
488
:tweak: Reviewer wants this submission merged with small changes. (No
490
:abstain: Reviewer does not intend to vote on this patch.
491
:resubmit: Please make changes and resubmit for review.
492
:reject: Reviewer doesn't want this kind of change merged.
493
:comment: Not really a vote. Reviewer just wants to comment, for now.
495
If a change gets two approvals from core reviewers, and no rejections,
496
then it's OK to come in. Any of the core developers can bring it into the
497
bzr.dev trunk and backport it to maintenance branches if required. The
498
Release Manager will merge the change into the branch for a pending
499
release, if any. As a guideline, core developers usually merge their own
500
changes and volunteer to merge other contributions if they were the second
501
reviewer to agree to a change.
503
To track the progress of proposed changes, use Bundle Buggy. See
504
http://bundlebuggy.aaronbentley.com/help for a link to all the
505
outstanding merge requests together with an explanation of the columns.
506
Bundle Buggy will also mail you a link to track just your change.
682
Transports work in URLs. Take note that URLs are by definition only
683
ASCII - the decision of how to encode a Unicode string into a URL must be
684
taken at a higher level, typically in the Store. (Note that Stores also
685
escape filenames which cannot be safely stored on all filesystems, but
686
this is a different level.)
688
The main reason for this is that it's not possible to safely roundtrip a
689
URL into Unicode and then back into the same URL. The URL standard
690
gives a way to represent non-ASCII bytes in ASCII (as %-escapes), but
691
doesn't say how those bytes represent non-ASCII characters. (They're not
692
guaranteed to be UTF-8 -- that is common but doesn't happen everywhere.)
694
For example if the user enters the url ``http://example/%e0`` there's no
695
way to tell whether that character represents "latin small letter a with
696
grave" in iso-8859-1, or "latin small letter r with acute" in iso-8859-2
697
or malformed UTF-8. So we can't convert their URL to Unicode reliably.
699
Equally problematic if we're given a url-like string containing non-ascii
700
characters (such as the accented a) we can't be sure how to convert that
701
to the correct URL, because we don't know what encoding the server expects
702
for those characters. (Although this is not totally reliable we might still
703
accept these and assume they should be put into UTF-8.)
705
A similar edge case is that the url ``http://foo/sweet%2Fsour`` contains
706
one directory component whose name is "sweet/sour". The escaped slash is
707
not a directory separator. If we try to convert URLs to regular Unicode
708
paths this information will be lost.
710
This implies that Transports must natively deal with URLs; for simplicity
711
they *only* deal with URLs and conversion of other strings to URLs is done
712
elsewhere. Information they return, such as from ``list_dir``, is also in
713
the form of URL components.
508
716
Coding Style Guidelines
509
717
#######################
added
removed
doc/developers/HACKING.txt
