Chapter 9. Branching

Table of Contents

1. How To Use Branching
2. Transition Using aenrls
3. Cross Branch Merge
4. Multiple Branch Development
5. Hierarchy of Projects
5.1. Fundamentals
5.2. Incremental Integration
5.3. Super-Project Branching
5.4. Super-Project Testing
5.5. The Next Cycle
5.6. Bug Fixing
6. Conflict Resolution
6.1. Cross Branch Merge
6.2. Insulation
7. Ending A Branch

This chapter describes the concept of branching implemented by Aegis. The process described in previous chapters makes changes to a project's master source.

Branching generalizes this change model, by allowing the baseline to be treated as a change, or the ability to treat a change as a baseline.

Since branchs are sometimes considered as a changes it is useful to expand on the differences. A branch, or trunk, baseline may have children which are either changes or deeper branches. From this perspective the difference is that nothing may be modified directly in a branch. A branch is an integrated baseline with all the associated protection. To modify a branch one must open a change under that branch.

Looking upward from a change under a branch, its parent is the branch baseline, and its grand is the parent of the branch. We will see this used later when we talk about ending a branch.

A significant feature of Aegis branches is that, because they are an extension of the change concept, they are expected to end, and be integrated back into their baseline, or parent ,in time.

The most common case of this is in project releases.

A branch in the being developed state may have changes made to it, and/or deeper branches. This may recurse to any level. Once a branch is complete, no further deeper branches may be created from that branch.

1. How To Use Branching

To access a project branch, the project name has the branch appended, separated by a dot or a hyphen. For example: branch 1 of project "aegis" is referred to as "aegis.1". To reference changes on this branch, use this compound project name wherever you would normally use a project name.

Traditional 2-level project release names are obtained by using a further level of branching. For example: by creating branch 0 of project "aegis.1", there would be a branch accessed as project "aegis.1.0".

By default, these two level of project branching are created automatically when the aenpr(1) command is used. You need to use the -VERSion option to make this deeper or shallower, or have different numbering.

CommandBranches Created
aenpr foofoo, foo.1, foo.1.0
aenpr foo -vers 2.4.1foo, foo.2, foo.2.4, foo.2.4.1
aenpr foo.7foo, foo.7
aegis -npr foo -vers -foo

The last is a special case, to enable a project to be created with no default branches (it's also hard to get the empty string past the alias).

To add branching and release level management to an existing project on uses the aenbr(1) command at any level. Say we already have foo.1.0, which represents version 1.0 of our software. One method of release level management would be to integrate foo.1.0 into its parent foo.1 and then do aenbr -p foo.1 would create foo.1.1 representing version 1.1. Eventually we might want to make a major version release and would integrate foo.1 into its parent foo and then do aenbr -p foo, which would create foo.2. Then if we do aenbr -p foo.2 we create foo.2.0, for development of version 2.0 of our software.

2. Transition Using aenrls

To convert a project from the old-style to the new branching style, use the aenrls(1) command.

If you give no second project name, the new name is generated by removing the numeric suffixes. If you did not give a -VERSion option, the numeric suffixes will be used to determine the next version, by adding one to the previous minor version number. The new project is then created rather like the aenpr(1) command.

The files of the old project are copied across as an implicit change on the newly created branch within the new project. If the new project name already exists, and is a new-style project, the aenrls(1) command will attempt to make the appropriate numbered branches. If the new project already exists and is an old-style project, or it exists and the branch number(s) are already in use, aenrls(1) will emit an error and fail. The aenrls(1) command only works on old-style projects, and always converts them to new style projects.

Planning you branch numbers is essential. If you want to use 3-level branch numbers (e.g. "aegis.2.3.1") at some time in the future, then you must use 3-level version numbers all the way through (e.g. "aegis.2.3.0"). This is because change numbers and branch numbers come from the same common pool of numbers. Once change one has been used (e.g. "aegis.2.3.C001"), then branch one is no longer available (e.g. "aegis.2.3.1.C042" conflicts).

3. Cross Branch Merge

From time to time you will want to merge the changes from one branch into a change. This may be done using a cross-branch merge. This is done by specifying the -BRanch option to the aegis -diff -merge-only command.

The most common cross branch merge is when the project's files are out-of-date. Because it is not possible to use aegis -diff -merge-only directly on the branch, this must be in a change on the branch. As a short-cut, the branch may be specified using the -grandparent option.

4. Multiple Branch Development

It is very common for a bug fix to need to be applied to more than one branch at once. The change could be applied to the common ancestor branch, however this may not be effective in the branch immediately. An alternative is to use the aegis -clone command, which can be used to identically reproduce a change on another branch.

5. Hierarchy of Projects

It would be nice if there was some way to use one project as a sort of "super change" to a "super project", so that large teams (say 1000 people) could work as lots of small teams (say 100 people). As a small team gets their chunk ready, using the facilities provided to-date by Aegis, the small team's baseline is treated as a change to be made to the large team baseline.

This idea can be extended quite naturally to any depth of layering.

After reading Transaction Oriented Configuration Management: A Case Study Peter Fieler, Grace Downey, CMU/SEI-90-TR-23, this is not a new idea. It also provides some ideas for how to do branching sensibly, and was influential in the design of Aegis' branching.

5.1. Fundamentals

Aegis has everything you need to have a super project and a number of sub-projects. All you need to do is create an active branch for each sub-project. Each branch gets a separate baseline, viz

% aenpr gizmo.0.1
project "gizmo": created
project "gizmo.0": created
project "gizmo.0.1": created
%

Now, for each of your desired sub-projects, create another branch

aenbr -p gizmo.0.1 1 # the foo project
aenbr -p gizmo.0.1 2 # the bar project
aenbr -p gizmo.0.1 3 # the baz project

Now, the guys on the foo project set their AEGIS_­PROJECT environment variable to to gizmo.0.1.1, the bar guys use gizmo.0.1.2, and baz uses gizmo.0.1.3. From the developer's point of view they are separate projects. From one level up, though, they are just part of a bigger project.

It helps if you design and implement the build system first. You do this as a change set on the common parent branch. Once it is completed each branch can inherit it from the common parent. This makes integration easier, when it comes time to integrate the sub-projects together.

5.2. Incremental Integration

It is very common that not all of the sub-projects will be ready to be integrated at the same time. This is the normal situation with Aegis branching, and is handled cleanly and simply.

In Aegis each branch is literally a change, all the way down into the internal data structures. Just as each change gets its own development directory, each branch gets its own baseline. Just as a development directory inherits everything its doesn't have from the baseline, so branches inherit everything they don't have from their parent branch (or ultimately from the trunk). Just as you incrementally integrate changes into a branch, you incrementally integrate branches into their parent.

The branches only influence each other when they are integrated, just as changes only influence each other when they are integrated.

There are times when a branch being integrated into its parent is found to be inadequate. Aegis provides a simple mechanism to “bounce” a branch integration. Recall that, for Aegis, branches are the same as changes. Just as you “develop end” a change (see aede(1) for more information) you also aede a branch when development on it is completed.

Once a branch has develop-end (stops being an active branch), it is reviewed as a normal change, and integrated as a normal change. If integration failed, it returns to “being developed” and becomes an active branch once again, until the next aede. As you can see, it is as easy to bounce a branch integration as it is to bounce a change integration.

An unsuccessful branch integration leaves the repository unchanged, just as an unsuccessful change integration leaves it unchanged.

5.3. Super-Project Branching

In many real-world situations it is very important to be able to branch at any point in the past history of the super-project to fix (integration specific) bugs or to customize more the older states of the super-project.

You can create a branch at any time, on any active branch or active branch ancestor. You can populate that branch with historical versions (from any other branch, actually, not just the ancestral line). The method is a little fussy - you can't aecp into a branch directly, you need to do this via a change to that branch. Files not changed by a change on a branch are inherited from the current (i.e. active) state of the parent branch. See the section on Insulation, above.

5.4. Super-Project Testing

Many folks see Aegis' testing features as useful for unit testing individual files or change sets. For large projects, it is common that a specific test tool will be written. However, even large scale integration testing is possible using Aegis.

You can change the test command from being a shell script to being anything to you want - see the test_command field in aepconf(5). Or run the test tool from the shell script. If the integration tests can be automated, it makes sense to preserve them in the repository - they are some of the most valuable regression tests for developers, because they describe correct behavior outside the “box” the developer usually works in.

5.5. The Next Cycle

Once you have a fully-integrated product, what happens on the next cycle? Well, first you may want to finish gizmo.0.1 and integrate it into gizmo.0, and then aenbr -p gizmo.0 2

Then what? Same deal as before, but anything not changed in one of the sub-project branches gets inherited from the ancestor.

aenbr -p gizmo.0.2 1 # the foo project
aenbr -p gizmo.0.2 2 # the bar project
aenbr -p gizmo.0.2 3 # the baz project

Most folks find doing the whole mega-project-build every time tiresome - so don't. Temporarily (via a change set) hack the build configuration to build only the bit you want - obviously a different hack on each sub-project's branch. Just remember to un-hack it (via another change set) before you integrate the sub-project.

5.6. Bug Fixing

The aeclone(1) command lets you clone a change set from one branch to another. So if you have a bug fix that needs to be done on each active branch you can clone it (once you have it fixed the first time). You still have to build review and integrate n times (branches often differ non-trivially). Providing it isn't already in use, you can even ask for the same change number on each branch - handy for syncing with an external bug tracking system.

Alternatively, fix bugs in the common ancestor, and the sub-projects will inherit the fix the next time they integrate something on their branch (assuming they aren't insulated against it).

6. Conflict Resolution

A development directory becomes out of date, compared to the project, when another change is integrated which has a file in common. This situation is detected automatically by aede(1) and you resolve it using aed(1), usually with something like the --merge-only option. Additionally, you can see if you have an out-of-date file from the change files listing, because it will show you the current baseline version in parentheses if you are out-of-date.

Aegis implements branches as very long changes, with sub-changes. A side effect of this is that a branch can become out-of-date in the same way that a development director becomes out of date. When it comes time to aede(1) the branch, you will be told if there are any out-of-date files. Additionally, the project files listing will show out of date files in exactly the same way that the change file listing does.

6.1. Cross Branch Merge

However, unlike a simple change, if you attempt to use the aed --merge-only command in the branch baseline, you will get an error message! How, then, do you resolve the apparent impasse?

The aed(1) command has a number of options designed for just this purpose.

  • The --branch option may be used to specify another branch of the same project, as a source of the file to be differenced against. This is almost what you need.

  • The --grandparent option is a special case of the --branch option, and it means the parent branch of project.

  • The --trunk option is also a special case of the --branch option, and it means the base branch from which the entire branch tree springs.

By creating a new change on the out-of-date branch, and copying in the out-of-date files, you have almost everything required. All that is necessary is to perform a cross-branch merge against the project grandparent, and the necessary merging will be performed. In addition Aegis will remember that it was a cross-branch merge, and once aeipass completes successfully, the branch will be up-to-date once more.

  • Create a new change on the out-of-date branch

  • Use a simple aecp command to copy the out-of-date files. (Do not use any --branch or --delta options.)

  • Use the “aed --merge-only --grandparent” command to perform the merge. At this point, if you use the “ael cf” command, you will notice that this file is tagged in the listing with the new branch edit origin, to be used during aeipass. If it isn't, you have made a mistake.

  • As usual, use your favourite editor to check the merge results, and resolve any conflicts.

  • Build and test as usual.

  • Complete the change as usual.

  • Once aeipass is successful, the branch will be up-to-date (for the files in the change).

6.2. Insulation

One of the stated benefits of using a branch is the insulating effects which branches can provide. However, when you have multiple simultaneous active branches, that insulation will inevitably lead to out-of-date branch files. Now that how to merge them has been described, when should you merge?

In a simple change's development directory, there are times when an aeipass will result in all developers needing to recompile. Depending on what files you are working on, it may be that you need to merge some of your change files immediately, or aecp an earlier version of the files which changed in the project.

Branches can also suffer from exactly the same problems, and are mended by exactly the same alternatives.

6.2.1. Branch Insulated Against Project

If you created a branch to insulate the work being done on the branch from other activities in the project, it follows that when such build problems occurred, you would use an “aecp -delta” command to continue insulating.

This action defers the labour of merging until towards the end of the branch development, sometimes with a quite visible schedule impact.

6.2.2. Project Insulated Against Branch

If you created a branch to insulate the project from work being done in the branch, it follows that you would do a cross branch immediately.

This action amortizes the labour of merging across the life of the branch, often with a number of small delays and less schedule impact.

6.2.3. Mix 'n' Match

Of course, we usually have both these motives, and some more besides, so the answer is usually “it depends”.

7. Ending A Branch

“OK, I give up. I do not understand the ending of branches.”

Usually, you end development of a branch the same way you end development of a simple change. In this example, branch example.1.42 will be ended:

% aede -p example 1 -c 42


aegis: project "example.1": change 42:
file "fubar" in the baseline
has changed since the last
'aegis -DIFFerence' command, you
need to do a merge

%

Oops. Something went wrong. Don't panic!

I'm going to assume, for the purposes of explanation, that there have been changes in one of the ancestor branches, and thus require a merge, just like file fubar, above.

You need to bring file fubar up-to-date. How? You do it in a change set, like everything else.

At his point you need to do 5 things: (1) create a new change on example.1.42, (2) copy fubar into it, (3) merge fubar with branch "example.1" (4) end development of the change and integrate it, and (5) now you can end example.1.42

The -GrandParent option is a special case of the -BRanch option. You are actually doing a cross-branch merge, just up-and-down rather than sideways.

% aem -gp fubar
%

And manually fix any conflicts... naturally.

At this point, have a look at the file listing, it will show you something you have never seen before - it will show you what it is going to set the branch's edit_number_origin to for each file, at aeipass.

% ael cf
Type   Action Edit        File Name
------ ------ -------     -----------
source modify 1.3         fubar
              {cross 1.2}

Now finish the change as usual... aeb, aed, aede, aerpass, aeib, ..., aeipass nothing special here.

One small tip: merge the files one at a time. It makes keeping track of where you are up to much easier.

Now you can end development of the branch, because all of the files are up-to-date.

In some cases, Aegis has detected a logical conflict where you, the human, know there is none. Remember that the aem command saves the old version of the file with a ,B suffix (`B' for backup). If you have a file like this, just use

% mv fubar,B fubar
%

to discard everything from the ancestor branch, and use the current branch.