Chapter 3. The Change Development Cycle

Table of Contents

1. The Developer
1.1. Before You Start
1.2. The First Change
1.3. The Second Change
1.4. The Third and Fourth Changes
1.5. Developer Command Summary
2. The Reviewer
2.1. Before You Start
2.2. The First Change
2.3. The Second Change
2.4. Reviewer Command Summary
3. The Integrator
3.1. Before You Start
3.2. The First Change
3.3. The Other Changes
3.4. Integrator Command Summary
3.5. Minimum Integrations
4. The Administrator
4.1. Before You Start
4.2. The First Change
4.3. The Second Change
4.4. The Third Change
4.5. The Fourth Change
4.6. Administrator Command Summary
5. What to do Next
6. Common Questions
6.1. Insulation
6.2. Partial Check-In
6.3. Multiple Active Branches
6.4. Collaboration

As a change to a project is developed using Aegis, it passes through several states. Each state is characterized by different quality requirements, different sets of applicable Aegis commands, and different responsibilities for the people involved.

These people may be divided into four categories: developers, reviewers, integrators and administrators. Each has different responsibilities, duties and permissions; although one person may belong to more than one category, depending on how a project is administered.

This chapter looks at each of these categories, by way of an example project undergoing its first four changes. This example will be examined from the perspective of each category of people in the following sections.

There are six hypothetical users in the example: the developers are Pat, Jan and Sam; the reviewers are Robyn and Jan; the integrator is Isa; and the administrator is Alex.[8] There need not have been this many people involved, but it keeps things slightly cleaner for this example.

The project is called "example". It implements a very simple calculator. Many features important to a quality product are missing, checking for divide-by-zero for example. These have been omitted for brevity.

1. The Developer

The developer role is the coal face.[9] This is where new software is written, and bugs are fixed. This example shows only the addition of new functionality, but usually a change will include modifications of existing code, similar to bug-fixing activity.

1.1. Before You Start

Have you configured your account to use Aegis? See the User Setup section of the Tips and Traps chapter for how to do this.

1.2. The First Change

While the units of change, unoriginally, are called "changes", this also applies to the start of a project - a change to nothing, if you like. The developer of this first change will be Pat.

First, Pat has been told by the project administrator that the change has been created. How Alex created this change will be detailed in the Administrator section, later in this chapter. Pat then acquires the change and starts work.

pat% aedb -l -p example.1.0
Project "example.1.0"
List of Changes

Change  State           Description
------- -------         -------------
  10    awaiting_       Create initial skeleton.
        development
pat% aedb example.1.0 10
aegis: project "example.1.0": change 10: development directory "/u/pat/
        example.1.0.C010"
aegis: project "example.1.0": change 10: user "pat" has begun development
pat% aecd
aegis: project "example.1.0": change 10: /u/pat/example.1.0.C010
pat%

At this point Aegis has created a development directory for the change and Pat has changed directory to the development directory.[10]

Five files will be created by this change.

pat% aenf aegis.conf Howto.cook gram.y lex.l main.c
aegis: project "example.1.0": change 10: file "Howto.cook" added
aegis: project "example.1.0": change 10: file "aegis.conf" added
aegis: project "example.1.0": change 10: file "gram.y" added
aegis: project "example.1.0": change 10: file "lex.l" added
aegis: project "example.1.0": change 10: file "main.c" added
pat%

The contents of the aegis.conf file will not be described in this section, mostly because it is a rather complex subject; so complex it requires four chapters to describe: the History Tool chapter, the Dependency Maintenance Tool chapter, the Difference Tools chapter and the Project Attributes chapter. The contents of the Howto.cook file will not be described in this section, as it is covered in the Dependency Maintenance Tool chapter.

The file main.c will have been created by Aegis as an empty file. Pat edits it to look like this

#include <stdio.h>

static void
usage()
{
        fprintf(stderr, "usage: example\n");
        exit(1);
}

void
main(argc, argv)
        int     argc;
        char    **argv;
{
        if (argc != 1)
                usage();
        yyparse();
        exit(0);
}

The file gram.y describes the grammar accepted by the calculator. This file was also created empty by Aegis, and Pat edits it to look like this:

%token  DOUBLE
%token  NAME

%union
{
        int     lv_int;
        double  lv_double;
}

%type <lv_double> DOUBLE expr
%type <lv_int> NAME

%left '+' '-'
%left '*' '/'
%right UNARY

%%

example
        : /* empty */
        | example command '\n'
                { yyerrflag = 0; fflush(stderr); fflush(stdout); }
command
        : expr
		{ printf("%g\n", $1); }
        | error
expr
        : DOUBLE
                { $$ = $1; }
        | '(' expr ')'
                { $$ = $2; }
        | '-' expr
                %prec UNARY
                { $$ = -$2; }
        | expr '*' expr
                { $$ = $1 * $3; }
        | expr '/' expr
                { $$ = $1 / $3; }
        | expr '+' expr
                { $$ = $1 + $3; }
        | expr '-' expr
                { $$ = $1 - $3; }

The file lex.l describes a simple lexical analyzer. It will be processed by lex(1) to produce C code implementing the lexical analyzer. This kind of simple lexer is usually hand crafted, but using lex allows the example to be far smaller. Pat edits the file to look like this:

%{
#include <math.h>
#include <libaegis/gram.h>
%}
%%
[ \t]+      ;
[0-9]+(\.[0-9]*)?([eE][+-]?[0-9]+)?    {
                yylval.lv_double = atof(yytext);
                return DOUBLE;
        }
[a-z]   {
                yylval.lv_int = yytext[0] - 'a';
                return NAME;
        }
\n      |
.       return yytext[0];

Note how the gram.h file is included using the #include <filename> form. This is very important for builds in later changes, and is discussed more fully in the Using Cook section of the Dependency Maintenance Tool chapter.

The files are processed, compiled and linked together using the aeb command; this is known as building a change. This is done through Aegis so that Aegis can know the success or failure of the build. (Build success is a precondition for a change to leave the being developed state.) The build command is in the aegis.conf file so vaguely described earlier. In this example it will use the cook(1) command which in turn will use the Howto.cook file, also alluded to earlier. This file describes the commands and dependencies for the various processing, compiling and linking.

pat% aeb
aegis: project "example.1.0": change 10: development build started
aegis: cook -b Howto.cook project=example.1.0 change=10
        version=1.0.C010 -nl
cook: yacc -d gram.y
cook: mv y.tab.c gram.c
cook: mv y.tab.h gram.h
cook: cc -I. -I/projects/example/branch.1./branch0/baseline -O -c gram.c
cook: lex lex.l
cook: mv lex.yy.c lex.c
cook: cc -I. -I/projects/example/branch.1/branch.0/baseline -O -c lex.c
cook: cc -I. -I/projects/example/baseline -O -c main.c
cook: cc -o example gram.o lex.o main.o -ll -ly
aegis: project "example.1.0": change 10: development build complete
pat%

The example program is built, and Pat could even try it out:

pat% example
1 + 2
3
^D
pat%

At this point the change is apparently finished. The command to tell Aegis this is the develop end command:

pat% aede
aegis: project "example.1.0": change 10: no current 'aegis -DIFFerence'
        registration
pat%

It didn't work, because Aegis thinks you have missed the difference step.

The difference step is used to produce files useful for reviewing changes, mostly in the form of context difference files between the project baseline and the development directory. Context differences allow reviewers to see exactly what has changed, and not have to try to track them down and inevitably miss obscure but important edits to large or complex files.

pat% aed
aegis: set +e; diff -c /dev/null /u/pat/example.1.0.C010/Howto.cook >
        /u/pat/example.1.0.C010/Howto.cook,D; test $? -eq 0 -o $? -eq 1
aegis: set +e; diff -c /dev/null /u/pat/example.1.0.C010/aegis.conf >
        /u/pat/example.1.0.C010/aegis.conf,D; test $? -eq 0 -o $? -eq 1
aegis: set +e; diff -c /dev/null /u/pat/example.1.0.C010/gram.y >
        /u/pat/example.1.0.C010/gram.y,D; test $? -eq 0 -o $? -eq 1
aegis: set +e; diff -c /dev/null /u/pat/example.1.0.C010/lex.l >
        /u/pat/example.1.0.C010/lex.l,D; test $? -eq 0 -o $? -eq 1
aegis: set +e; diff -c /dev/null /u/pat/example.1.0.C010/main.c >
        /u/pat/example.1.0.C010/main.c,D; test $? -eq 0 -o $? -eq 1
aegis: project "example.1.0": change 10: difference complete
pat%

Doing a difference for a new file may appear a little pedantic, but when a change consists of tens of files, so modifications of existing files and some new, there is a temptation for reviewers to use "more *,D" and thus completely miss the new files if it were not for this pedanticism.[11]

So that reviewers, and conscientious developers, may locate and view all of these difference files, the command

pat% more `find . -name "*,D" -print | sort`
...examines each file...
pat%

could be used, however this is a little too long winded for most users, and so the aedmore alias does exactly this. There is a similar aedless alias for those who prefer the less command.

So now Pat is done, let's try to sign off again:

pat% aede
aegis: project "example.1.0": change 10: no current 'aegis -Test'
        registration
pat%

It didn't work, again. This time Aegis is reminding Pat that every change must be accompanied by at least one test. This is so that the project team can be confident at all times that a project works.[12] Making this a precondition to leave the being developed state means that a reviewer can be sure that a change builds and passes its tests before it can ever be reviewed. Pat adds the truant test:

pat% aent
aegis: project "example.1.0": change 10: file "test/00/t0001a.sh" new
        test
pat%

The test file is in a weird place, eh? This is because many flavors of Unix™ are slow at searching directories, and so Aegis limits itself to 100 tests per directory. Whatever the name, Pat edits the test file to look like this:

#!/bin/sh
#
# test simple arithmetic
#
tmp=/tmp/$$
here=`pwd`
if [ $? -ne 0 ]; then exit 1; fi

fail()
{
        echo FAILED 1>&2
        cd $here
        rm -rf $tmp
        exit 1
}

pass()
{
        cd $here
        rm -rf $tmp
        exit 0
}
trap "fail" 1 2 3 15

mkdir $tmp
if [ $? -ne 0 ]; then exit 1; fi
cd $tmp
if [ $? -ne 0 ]; then fail; fi

#
# with input like this
#
cat > test.in << 'foobar'
1
(24 - 22)
-(4 - 7)
2 * 2
10 / 2
4 + 2
10 - 3
foobar
if [ $? -ne 0 ]; then fail; fi

#
# the output should look like this
#
cat > test.ok << 'foobar'
1
2
3
4
5
6
7
foobar
if [ $? -ne 0 ]; then fail; fi

#
# run the calculator
# and see if the results match
#
$here/example < test.in > test.out
if [ $? -ne 0 ]; then fail; fi
diff test.ok test.out
if [ $? -ne 0 ]; then fail; fi

#
# this much worked
#
pass

There are several things to notice about this test file:

  • It is a Bourne shell script. All test files are Bourne shell scripts because they are the most portable.[13] (Actually, Aegis likes test files not to be executable, it passes them to the Bourne shell explicitly when running them.)

  • It makes the assumption that the current directory is either the development directory or the baseline. This is valid, aegis always runs tests this way; if you run one manually, you must take care of this yourself.

  • It checks the exit status of each and every command. It is essential that even unexpected and impossible failures are handled.

  • A temporary directory is created for temporary files. It cannot be assumed that a test will be run from a directory which is writable; it is also easier to clean up after strange errors, since you need only throw the directory away, rather than track down individual temporary files. It mostly protects against rogue programs scrambling files in the current directory, too.

  • Every test is self-contained. The test uses auxiliary files, but they are not separate source files (figuring where they are when some are in a change and some are in the baseline can be a nightmare). If a test wants an auxiliary file, it must construct the file itself, in a temporary directory.

  • Two functions have been defined, one for success and one for failure. Both forms remove the temporary directory. A test is defined as passing if it returns a 0 exit status, and failing if it returns anything else.

  • Tests are treated just like any other source file, and are subject to the same process. They may be altered in another change, or even deleted later if they are no longer useful.

The most important feature to note about this test, after ignoring all of the trappings, is that it doesn't do much you wouldn't do manually! To test this program manually you would fire it up, just as the test does, you would give it some input, just as the test does, and you would compare the output against your expectations of what it will do, just as the test does.

The difference with using this test script and doing it manually is that most development contains many iterations of the "build, test, think, edit, build, test..." cycle. After a couple of iterations, the manual testing, the constant re-typing, becomes obviously unergonomic. Using a shell script is more efficient, doesn't forget to test things later, and is preserved for posterity (i.e. adds to the regression test suite).

This efficiency is especially evident when using commands[14] such as

pat% aeb && aet ; vi aegis.log
...
pat% !aeb
...
pat%

It is possible to talk to the shell extremely rarely, and then only to re-issue the same command, using a work pattern such as this.

As you have already guessed, Pat now runs the test like this:

pat% aet
aegis: sh /u/pat/example.1.0.C010/test/00/t0001a.sh
aegis: project "example.1.0": change 10: test "test/00/t0001a.sh"
        passed
aegis: project "example.1.0": change 10: passed 1 test
pat%

Finally, Pat has built the change, prepared it for review and tested it. It is now ready for sign off.

pat% aede
aegis: project "example.1.0": change 10: no current 'aegis -Build'
        registration
pat%

Say what? The problem is that the use of aent canceled the previous build registration. This was because Aegis is decoupled from the dependency maintenance tool (cook in this case), and thus has no way of knowing whether or not the new file in the change would affect the success or failure of a build.[15] All that is required is to re-build, re-test, re-difference (yes, the test gets differenced, too) and sign off.

pat% aeb
aegis: logging to "/u/pat/example.1.0.C010/aegis.log"
aegis: project "example.1.0": change 10: development build started
aegis: cook -b Howto.cook project=example.1.0 change=10
        version=1.0.C001 -nl
cook: "all" is up-to-date
aegis: project "example.1.0": change 10: development build complete
pat% aet
aegis: logging to "/u/pat/example.1.0.C010/aegis.log"
aegis: sh /u/pat/example..1.0.C010/test/00/t0001a.sh
aegis: project "example.1.0": change 10: test "test/00/t0001a.sh"
        passed
aegis: project "example.1.0": change 10: passed 1 test
pat% aed
aegis: logging to "/u/pat/example.1.0.C010/aegis.log"
aegis: set +e; diff -c /dev/null /u/pat/example.1.0.C010/test/00/
        t0001a.sh > /u/pat/example.1.0.C010/test/00/t0001a.sh,D; test
        $? -eq 0 -o $? -eq 1
aegis: project "example.1.0": change 10: difference complete
pat% aede
aegis: sh /opt/aegis/lib/de.sh example.1.0 10 pat
aegis: project "example.1.0": change 10: development completed
pat%

The change is now ready to be reviewed. This section is about developers, so we will have to leave this change at this point in its history. Some time in the next day or so Pat receives electronic mail that this change has passed review, and another later to say that it passed integration. Pat is now free to develop another change, possibly for a different project.

1.3. The Second Change

The second change was created because someone wanted to name input and output files on the command line, and called the absence of this feature a bug. When Jan arrived for work, and lists the changes awaiting development, the following list appeared:

jan% aedb -l -p example.1.0
Project "example.1.0"
List of Changes

Change  State           Description
------  ------          ------------
  11    awaiting_       Add input and output file names to the
        development     command line.
  12    awaiting_       add variables
        development
  13    awaiting_       add powers
        development
jan%

The first on the list is chosen.

jan% aedb -c 11 -p example.1.0
aegis: project "example.1.0": change 11: development directory "/u/
        jan/example.1.0.C011"
aegis: project "example.1.0": change 11: user "jan" has begun
        development
jan% aecd
aegis: project "example.1.0": change 11: /u/jan/example.002
jan%

The best way to get details about a change is to used the "change details" listing.

jan% ael cd
Project "example.1.0", Change 11
Change Details

NAME
        Project "example.1.0", Change 11.

SUMMARY
        file names on command line

DESCRIPTION
        Optional input and output files may be specified on the
        command line.

CAUSE
        This change was caused by internal_bug.

STATE
        This change is in 'being_developed' state.

FILES
        Change has no files.

HISTORY
        What            When            Who     Comment
        ------          ------          -----   ---------
        new_change      Fri Dec 11      alex
                        14:55:06 1992
        develop_begin   Mon Dec 14      jan
                        09:07:08 1992
jan%

Through one process or another, Jan determines that the main.c file is the one to be modified. This file is copied into the change:

jan% aecp main.c
aegis: project "example.1.0": change 11: file "main.c" copied
jan%

This file is now extended to look like this:

#include <stdio.h>

static void
usage()
{
        fprintf(stderr, "usage: example [ <infile> [ <outfile> ]]\n");
        exit(1);
}

void
main(argc, argv)
        int     argc;
        char    **argv;
{
        char    *in = 0;
        char    *out = 0;
        int     j;

        for (j = 1; j < argc; ++j)
        {
                char *arg = argv[j];
                if (arg[0] == '-')
                        usage();
                if (!in)
                        in = arg;
                else if (!out)
                        out = arg;
                else
                        usage();
        }

        if (in && !freopen(in, "r", stdin))
        {
                perror(in);
                exit(1);
        }
        if (out && !freopen(out, "w", stdout))
        {
                perror(out);
                exit(1);
        }

        yyparse();
        exit(0);
}

A new test is also required,

jan% aent
aegis: project "example.1.0": change 11: file "test/00/t0002a.sh" new
        test
jan%

which is edited to look like this:

#!/bin/sh
#
# test command line arguments
#
tmp=/tmp/$$
here=`pwd`
if [ $? -ne 0 ]; then exit 1; fi

fail()
{
        echo FAILED 1>&2
        cd $here
        rm -rf $tmp
        exit 1
}

pass()
{
        cd $here
        rm -rf $tmp
        exit 0
}
trap "fail" 1 2 3 15

mkdir $tmp
if [ $? -ne 0 ]; then exit 1; fi
cd $tmp
if [ $? -ne 0 ]; then fail; fi

#
# with input like this
#
cat > test.in << 'foobar'
1
(24 - 22)
-(4 - 7)
2 * 2
10 / 2
4 + 2
10 - 3
foobar
if [ $? -ne 0 ]; then fail; fi

#
# the output should look like this
#
cat > test.ok << 'foobar'
1
2
3
4
5
6
7
foobar
if [ $? -ne 0 ]; then fail; fi

#
# run the calculator
# and see if the results match
#
# (Use /dev/null for input in case input redirect fails;
# don't want the test to hang!)
#
$here/example test.in test.out < /dev/null
if [ $? -ne 0 ]; then fail; fi
diff test.ok test.out
if [ $? -ne 0 ]; then fail; fi
$here/example test.in < /dev/null > test.out.2
if [ $? -ne 0 ]; then fail; fi
diff test.ok test.out.2
if [ $? -ne 0 ]; then fail; fi

#
# make sure complains about rubbish
# on the command line
#
$here/example -trash < test.in > test.out
if [ $? -ne 1 ]; then fail; fi

#
# this much worked
#
pass

Now it is time for Jan to build and test the change. Through the magic of static documentation, this works first time, and here is how it goes:

jan% aeb
aegis: logging to "/u/pat/example.1.0.C011/aegis.log"
aegis: project "example.1.0": change 11: development build started
aegis: cook -b /projects/example/baseline/Howto.cook
        project=example.1.0 change=11 version=1.0.C011 -nl
cook: cc -I. -I/projects/example/baseline -O -c main.c
cook: cc -o example main.o /projects/example/baseline/gram.o
        /projects/example/baseline/lex.o -ll -ly
aegis: project "example.1.0": change 11: development build complete
jan% aet
aegis: logging to "/u/pat/example.1.0.C011/aegis.log"
aegis: sh /u/jan/example.1.0.C011/test/00/t0002a.sh
aegis: project "example.1.0e": change 11: test "test/00/t0002a.sh"
        passed
aegis: project "example.1.0": change 11: passed 1 test
jan%

All that remains if to difference the change and sign off.

jan% aed
aegis: logging to "/u/pat/example.1.0.C011/aegis.log"
aegis: set +e; diff -c /projects/example/main.c /u/jan/
        example.1.0.C011/main.c > /u/jan/example.1.0.C011/main.c,D; test $?
        -eq 0 -o $? -eq 1
aegis: project "example.1.0": change 11: difference complete
jan% aedmore
...examines the file...
jan%

Note how the context difference shows exactly what has changed. And now the sign-off:

jan% aede
aegis: project "example.1.0": change 11: no current 'aegis -Test
        -BaseLine' registration
jan%

No, it wasn't enough. Tests must not only pass against a new change, but must fail against the project baseline. This is to establish, in the case of bug fixes, that the bug has been isolated and fixed. New functionality will usually fail against the baseline, because the baseline can't do it (if it could, you wouldn't be adding it!). So, Jan needs to use a variant of the aet command.

jan% aet -bl
aegis: sh /u/jan/example.1.0.C011/test/00/t0002a.sh
usage: example
FAILED
aegis: project "example.1.0": change 11: test "test/00/t0002a.sh" on
        baseline failed (as it should)
aegis: project "example.1.0": change 11: passed 1 test
jan%

Running the regression tests is also a good idea

jan% aet -reg
aegis: logging to "/u/pat/example.1.0.C011/aegis.log"
aegis: sh /projects/example/baseline/test/00/t0001a.sh
aegis: project "example.1.0": change 11: test "test/00/t0001a.sh"
        passed
aegis: project "example.1.0": change 11: passed 1 test
jan%

Now Aegis will be satisfied

jan% aede
aegis: sh /opt/aegis/lib/aegis/de.sh example.1.0 11 jan
aegis: project "example.1.0": change 11: development completed
jan%

Like Pat in the change before, Jan will receive email that this change passed review, and later that it passed integration.

1.4. The Third and Fourth Changes

This section will show two people performing two changes, one each. The twist is that they have a file in common.

First Sam looks for a change to work on and starts, like this:

sam% aedb -l
Project "example.1.0"
List of Changes

Change  State           Description
------- -------         -------------
  12    awaiting_       add powers
        development
  13    awaiting_       add variables
        development
sam% aedb 12
aegis: project "example.1.0": change 12: development directory "/u/
        sam/example.1.0.C012"
aegis: project "example.1.0": change 12: user "sam" has begun
        development
sam% aecd
aegis: project "example.1.0": change 12: /u/sam/example.1.0.C012
sam%

A little sniffing around reveals that only the gram.y grammar file needs to be altered, so it is copied into the change.

sam% aecp gram.y
aegis: project "example.1.0": change 12: file "gram.y" copied
sam%

The grammar file is changed to look like this:

%token DOUBLE
%token NAME
%union
{
        double  lv_double;
        int     lv_int;
};

%type <lv_double> DOUBLE expr
%type <lv_int> NAME
%left '+' '-'
%left '*' '/'
%right '^'
%right UNARY

%%
example
        : /* empty */
        | example command '\n'
                { yyerrflag = 0; fflush(stderr); fflush(stdout); }
        ;

command
        : expr
                { printf("%g\n", $1); }
        | error
        ;

expr
        : DOUBLE
        | '(' expr ')'
                { $$ = $2; }
        | '-' expr
                %prec UNARY
                { $$ = -$2; }
        | expr '^' expr
                { $$ = pow($1, $3); }
        | expr '*' expr
                { $$ = $1 * $3; }
        | expr '/' expr
                { $$ = $1 / $3; }
        | expr '+' expr
                { $$ = $1 + $3; }
        | expr '-' expr
                { $$ = $1 - $3; }
        ;

The changes are very small. Sam checks to make sure using the difference command:

sam% aed
aegis: logging to "/u/sam/example.1.0.C012/aegis.log"
aegis: set +e; diff -c /projects/example/baseline/gram.y /u/sam/
        example.1.0.C012/gram.y > /u/sam/example.1.0.C012/gram.y,D; test $?
        -eq 0 -o $? -eq 1
aegis: project "example.1.0": change 12: difference complete
sam% aedmore
...examines the file...
sam%

The difference file looks like this

*** /projects/example/baseline/gram.y
--- /u/sam/example.1.0.C012/gram.y
***************
*** 1,5 ****
--- 1,6 ----
  %{
  #include <stdio.h>
+ #include <math.h>
  %}
  %token DOUBLE
  %token NAME

***************
*** 13,18 ****
--- 14,20 ----
  %type <lv_int> NAME
  %left '+' '-'
  %left '*' '/'
+ %right '^'
  %right UNARY
  %%
  example

***************
*** 32,37 ****
--- 34,41 ----
        | '-' expr
                %prec UNARY
                { $$ = -$2; }
+       | expr '^' expr
+               { $$ = pow($1, $3); }
        | expr '*' expr
                { $$ = $1 * $3; }
        | expr '/' expr

These are the differences Sam expected to see.

At this point Sam creates a test. All good software developers create the tests first, don't they?

sam% aent
aegis: project "example.1.0": change 12: file "test/00/t0003a.sh" new
        test
sam%

The test is created empty, and Sam edit it to look like this:

:
here=`pwd`
if test $? -ne 0 ; then exit 1; fi
tmp=/tmp/$$
mkdir $tmp
if test $? -ne 0 ; then exit 1; fi
cd $tmp
if test $? -ne 0 ; then exit 1; fi

fail()
{
        echo FAILED 1>&2
        cd $here
        chmod u+w `find $tmp -type d -print`
        rm -rf $tmp
        exit 1
}

pass()
{
        cd $here
        chmod u+w `find $tmp -type d -print`
        rm -rf $tmp
        exit 0
}
trap "fail" 1 2 3 15

cat > test.in << 'end'
5.3 ^ 0
4 ^ 0.5
27 ^ (1/3)
end
if test $? -ne 0 ; then fail; fi

cat > test.ok << 'end'
1
2
3
end
if test $? -ne 0 ; then fail; fi

$here/example test.in < /dev/null > test.out 2>&1
if test $? -ne 0 ; then fail; fi

diff test.ok test.out
if test $? -ne 0 ; then fail; fi

$here/example test.in test.out.2 < /dev/null
if test $? -ne 0 ; then fail; fi

diff test.ok test.out.2
if test $? -ne 0 ; then fail; fi

# it probably worked
pass

Everything is ready. Now the change can be built and tested, just like the earlier changes.

sam% aeb
aegis: logging to "/u/sam/example.1.0.C012/aegis.log"
aegis: project "example1.0": change 12: development build started
aegis: cook -b /projects/example/baseline/Howto.cook
        project=example.1.0 change=12 version=1.0.C012 -nl
cook: yacc -d gram.y
cook: mv y.tab.c gram.c
cook: mv y.tab.h gram.h
cook: cc -I. -I/projects/example/baseline -O -c gram.c
cook: cc -I. -I/projects/example/baseline -O -c /projects/
        example/baseline/lex.c
cook: cc -o example gram.o lex.o /projects/example/baseline/
        main.o -ll -ly -lm
aegis: project "example": change 3: development build complete
sam%

Notice how the yacc run produces a gram.h which logically invalidates the lex.o in the baseline, and so the lex.c file in the baseline is recompiled, using the gram.h include file from the development directory, leaving a new lex.o in the development directory. This is the reason for the use of

#include <filename>

directives, rather then the double quote form.

Now the change is tested.

sam% aet
aegis: logging to "/u/sam/example.1.0.C012/aegis.log"
aegis: sh /u/sam/example.1.0.C012/test/00/t0003a.sh
aegis: project "example.1.0": change 12: test "test/00/t0003a.sh"
        passed
aegis: project "example.1.0": change 12: passed 1 test
sam%

The change must also be tested against the baseline, and fail. Sam knows this, and does it here.

sam% aet -bl
aegis: logging to "/u/sam/example.1.0.C012/aegis.log"
aegis: sh /u/sam/example.1.0.C012/test/00/t0003a.sh
1,3c1,6
< 1
< 2
< 3
---
> syntax error
> 5.3
> syntax error
> 4
> syntax error
> 27
FAILED
aegis: project "example.1.0": change 12: test "test/00/t0003a.sh" on
        baseline failed (as it should)
aegis: project "example.1.0": change 12: passed 1 test
sam%

Running the regression tests is also a good idea.

sam% aet -reg
aegis: logging to "/u/sam/example.1.0.C012/aegis.log"
aegis: sh /projects/example/baseline/test/00/t0001a.sh
aegis: project "example.1.0": change 12: test "test/00/t0001a.sh"
        passed
aegis: sh /projects/example/baseline/test/00/t0002a.sh
aegis: project "example.1.0": change 12: test "test/00/t0002a.sh"
        passed
aegis: project "example.1.0": change 12: passed 2 tests
sam%

A this point Sam has just enough time to get to the lunchtime aerobics class in the staff common room.

Earlier the same day, Pat arrived for work a little after Sam, and also looked for a change to work on.

pat% aedb -l
Project "example.1.0"
List of Changes

Change  State           Description
------- -------         -------------
  13    awaiting_       add variables
        development
pat%

With such a wide choice, Pat selected change 13.

pat% aedb 13
aegis: project "example.1.0": change 13: development directory "/u/
        pat/example.1.0.C013"
aegis: project "example.1.0": change 13: user "pat" has begun
        development
pat% aecd
aegis: project "example.1.0": change 13: /u/pat/example.1.0.C013
pat%

To get more information about the change, Pat then uses the "change details" listing:

pat% ael cd
Project "example.1.0", Change 13
Change Details

NAME
        Project "example.1.0", Change 13.

SUMMARY
        add variables

DESCRIPTION
        Enhance the grammar to allow variables. Only single
        letter variable names are required.

CAUSE
        This change was caused by internal_enhancement.

STATE
        This change is in 'being_developed' state.

FILES
        This change has no files.

HISTORY
        What            When            Who     Comment
        ------          ------          -----   ---------
        new_change      Mon Dec 14      alex
                        13:08:52 1992
        develop_begin   Tue Dec 15      pat
                        13:38:26 1992
pat%

To add the variables the grammar needs to be extended to understand them, and a new file for remembering and recalling the values of the variables needs to be added.

pat% aecp gram.y
aegis: project "example.1.0": change 13: file "gram.y" copied
pat% aenf var.c
aegis: project "example.1.0": change 13: file "var.c" added
pat%

Notice how Aegis raises no objection to both Sam and Pat having a copy of the gram.y file. Resolving this contention is the subject of this section.

Pat now edits the grammar file.

pat% vi gram.y
...edit the file...
pat% aed
aegis: logging to "/u/pat/example.1.0.C013/aegis.log"
aegis: set +e; diff -c /projects/example/baseline/gram.y /u/pat/
        example.1.0.C013/gram.y > /u/pat/example.1.0.C013/gram.y,D; test $?
        -eq 0 -o $? -eq 1
aegis: project "example.1.0": change 13: difference complete
pat%

The difference file looks like this

...hey, someone fill me in!...

The new var.c file was created empty by Aegis, and Pat edits it to look like this:

static double memory[26];

void
assign(name, value)
        int     name;
        double  value;
{
        memory[name] = value;
}

double
recall(name)
        int     name;
{
        return memory[name];
}

Little remains except to build the change.

pat% aeb
aegis: logging to "/u/pat/example.1.0.C013/aegis.log"
aegis: cook -b /example.proj/baseline/Howto.cook
        project=example.1.0 change=13 version=1.0.C013 -nl
cook: yacc -d gram.y
cook: mv y.tab.c gram.c
cook: mv y.tab.h gram.h
cook: cc -I. -I/projects/example/baseline -O -c gram.c
cook: cc -I. -I/projects/example/baseline -O -c /projects/
        example/baseline/lex.c
cook: cc -I. -I/projects/example/baseline -O -c var.c
cook: cc -o example gram.o lex.o /projects/example/baseline/
        main.o var.o -ll -ly -lm
aegis: project "example.1.0": change 13: development build complete
pat%

A new test for the new functionality is required and Pat creates one like this.

:
here=`pwd`
if test $? -ne 0 ; then exit 1; fi
tmp=/tmp/$$
mkdir $tmp
if test $? -ne 0 ; then exit 1; fi
cd $tmp
if test $? -ne 0 ; then exit 1; fi

fail()
{
        echo FAILED 1>&2
        cd $here
        chmod u+w `find $tmp -type d -print`
        rm -rf $tmp
        exit 1
}
pass()
{
        cd $here
        chmod u+w `find $tmp -type d -print`
        rm -rf $tmp
        exit 0
}
trap "fail" 1 2 3 15

cat > test.in << 'end'
a = 1
a + 1
c = a * 40 + 5
c / (a + 4)
end
if test $? -ne 0 ; then fail; fi

cat > test.ok << 'end'
2
9
end
if test $? -ne 0 ; then fail; fi

$here/example test.in < /dev/null > test.out 2>&1
if test $? -ne 0 ; then fail; fi

diff test.ok test.out
if test $? -ne 0 ; then fail; fi

$here/example test.in test.out.2 < /dev/null
if test $? -ne 0 ; then fail; fi

diff test.ok test.out.2
if test $? -ne 0 ; then fail; fi

# it probably worked
pass

The new files are then differenced:

pat% aed
aegis: logging to "/u/pat/example.1.0.C013/aegis.log"
aegis: set +e; diff -c /projects/example/baseline/gram.y /u/pat/
        example.1.0.C013/gram.y > /u/pat/example.1.0.C013/gram.y,D; test $?
        -eq 0 -o $? -eq 1
aegis: set +e; diff -c /dev/null /u/pat/example.1.0.C013/test/00/
        t0004a.sh > /u/pat/example.1.0.C013/test/00/t0004a.sh,D; test
        $? -eq 0 -o $? -eq 1
aegis: set +e; diff -c /dev/null /u/pat/example.1.0.C013/var.c > /u/
        pat/example.1.0.C013/var.c,D; test $? -eq 0 -o $? -eq 1
aegis: project "example.1.0": change 13: difference complete
pat%

Notice how the difference for the gram.y file is still current, and so is not run again.

The change is tested.

pat% aet
aegis: logging to "/u/pat/example.1.0.C013/aegis.log"
aegis: sh /u/pat/example.1.0.C013/test/00/t0001a.sh
aegis: project "example.1.0": change 13: test "test/00/t0004a.sh"
        passed
aegis: project "example.1.0": change 13: passed 2 tests
pat%

The change is tested against the baseline.

pat% aet -bl
aegis: logging to "/u/pat/example.1.0.C013/aegis.log"
aegis: sh /u/pat/example.1.0.C013/test/00/t0001a.sh
1,2c1,4
< 2
< 9
---
> syntax error
> syntax error
> syntax error
> syntax error
FAILED
aegis: project "example.1.0": change 13: test "test/00/t0004a.sh" on
        baseline failed (as it should)
pat%

And the regression tests

pat% aet -reg
aegis: logging to "/u/pat/example.1.0.C013/aegis.log"
aegis: sh /projects/example/baseline/test/00/t0001a.sh
aegis: project "example.1.0": change 13: test "test/00/t0001a.sh"
        passed
aegis: sh /projects/example/baseline/test/00/t0002a.sh
aegis: project "example.1.0": change 13: test "test/00/t0002a.sh"
        passed
aegis: project "example.1.0": change 13: passed 2 tests
pat%

Note how test 3 has not been run, in any form of testing. This is because test 3 is part of another change, and is not yet integrated with the baseline.

All is finished for this change,

pat% aede
aegis: sh /opt/aegis/lib/de.sh example.1.0 13 pat
aegis: project "example.1.0": change 13: development completed
pat%

Anxious to get this change into the baseline, Pat now wanders down the hall in search of a reviewer, but more of that in the next section.

Some time later, San returns from aerobics feeling much improved. All that is required for change 12 is to do develop end, or is it?

sam% aede
aegis: project "example.1.0": change 12: file "gram.y" in baseline
        has changed since last 'aegis -DIFFerence' command
sam%

A little sleuthing on Sam's part with the Aegis list command will reveal how this came about. The way to resolve this problem is with the difference command, but the merge variant - this will merge the new baseline version, and Sam's edit together.

sam% aem
aegis: logging to "/u/pat/example.1.0.C012/aegis.log"
aegis: co -u'1.1' -p /projects/example/history/gram.y,v > /tmp/
        aegis.14594
/projects/example/history/gram.y,v    stdout revision 1.1 (unlocked)
aegis: (diff3 -e /projects/example/baseline/gram.y /tmp/
        aegis.14594 /u/sam/example.003/gram.y | sed -e '/^w$/d'
        -e '/^q$/d'; \techo '1,$p' ) | ed - /projects/example/
        baseline/gram.y,B > /u/sam/example.003/gram.y
aegis: project "example.1.0": change 12: merge complete
aegis: project "example.1.0": change 12: file "gram.y" was out of
        date and has been merged, see "gram.y,B" for original source
aegis: new 'aegis -Build' required
sam%

This was caused by the conflict between change 13, which is now integrated, and change 12; both of which are editing the gram.y file. Sam examines the gram.y file, and is satisfied that it contains an accurate merge of the edit done by change 13 and the edits for this change. The merged source file looks like this:

%{
#include <stdio.h>
#include <math.h>
%}
%token DOUBLE
%token NAME
%union
{
        double  lv_double;
        int     lv_int;
};

%type <lv_double> DOUBLE expr
%type <lv_int> NAME
%left '+' '-'
%left '*' '/'
%right '^'
%right UNARY

%%
example
        : /* empty */
        | example command '\n'
                { yyerrflag = 0; fflush(stderr); fflush(stdout); }
        ;

command
        : expr
                { printf("%g\n", $1); }
        | NAME '=' expr
                { assign($1, $3); }
        | error
        ;

expr
        : DOUBLE
        | NAME
                { extern double recall(); $$ = recall($1); }
        | '(' expr ')'
                { $$ = $2; }
        | '-' expr
                %prec UNARY
                { $$ = -$2; }
        | expr '^' expr
                { $$ = pow($1, $3); }
        | expr '*' expr
                { $$ = $1 * $3; }
        | expr '/' expr
                { $$ = $1 / $3; }
        | expr '+' expr
                { $$ = $1 + $3; }
        | expr '-' expr
                { $$ = $1 - $3; }
        ;

The automatic merge worked because most such conflicts are actually working on logically separate portions of the file. Two different areas of the grammar in this case. In practice, there is rarely a real conflict, and it is usually small enough to detect fairly quickly.

Sam now rebuilds:

sam% aeb
aegis: logging to "/u/sam/example.1.0.C012/aegis.log"
aegis: project "example.1.0": change 12: development build started
aegis: cook -b /projects/example/baseline/Howto.cook
        project=example.1.0 change=12 version=1.0.C012 -nl
cook: rm gram.c
cook: rm gram.h
cook: yacc -d gram.y
cook: mv y.tab.c gram.c
cook: mv y.tab.h gram.h
cook: rm gram.o
cook: cc -I. -I/projects/example/baseline -O -c gram.c
cook: rm lex.o
cook: cc -I. -I/projects/example/baseline -O -c /projects/
        example/baseline/lex.c
cook: rm example
cook: cc -o example gram.o lex.o /projects/example/baseline/
        main.o /projects/example/baseline/var.o -ll -ly -lm
aegis: project "example.1.0": change 12: development build complete
sam%

Notice how the list of object files linked has also adapted to the addition of another file in the baseline, without any extra work by Sam.

All that remains is to test the change again.

sam% aet
aegis: /bin/sh /u/sam/example.1.0.C012/test/00/t0003a.sh
aegis: project "example.1.0": change 12: test "test/00/t0003a.sh"
        passed
aegis: project "example.1.0": change 12: passed 1 test
sam%

And test against the baseline,

sam% aet -bl
aegis: /bin/sh /u/sam/example.1.0.C012/test/00/t0003a.sh
1,3c1,6
< 1
< 2
< 3
---
> syntax error
> 5.3
> syntax error
> 4
> syntax error
> 27
FAILED
aegis: project "example.1.0": change 12: test "test/00/t0003a.sh" on
        baseline failed (as it should)
aegis: project "example.1.0": change 12: passed 1 test
sam%

Perform the regression tests, too. This is important for a merged change, to make sure you didn't break the functionality of the code you merged with.

sam% aet -reg
aegis: logging to "/u/sam/example.1.0.C012/aegis.log"
aegis: /bin/sh /projects/example/baseline/test/00/
        t0001a.sh
aegis: project "example.1.0": change 12: test "test/00/t0001a.sh"
        passed
aegis: /bin/sh /projects/example/baseline/test/00/
        t0002a.sh
aegis: project "example.1.0": change 12: test "test/00/t0002a.sh"
        passed
aegis: /bin/sh /projects/example/baseline/test/00/
        t0004a.sh
aegis: project "example.1.0": change 12: test "test/00/t0004a.sh"
        passed
aegis: project "example.1.0": change 12: passed 3 tests
sam%

All done, or are we?

sam% aede
aegis: project "example.1.0": change 12: no current 'aegis -Diff'
        registration
sam%

The difference we did earlier, which revealed that we were out of date, does not show the differences since the two changes were merged, and possibly further edited.

sam% aed
aegis: logging to "/u/sam/example.1.0.C012/aegis.log"
aegis: set +e; diff /projects/example/baseline/gram.y /u/pat/
        example.1.0.C012/gram.y > /u/pat/example.1.0.C012/gram.y,D;
	test $? -le 1
aegis: project "example.1.0": change 12: difference complete
sam%

This time everything will run smoothly,

sam% aede
aegis: project "example.1.0": change 12: development completed
sam%

Some time soon Sam will receive email that this change passed review, and later that it passed integration.

Within the scope of a limited example, you have seen most of what Aegis can do. To get a true feeling for the program you need to try it in a similarly simple case. You could even try doing this example manually.

1.5. Developer Command Summary

Only a few of the Aegis commands available to developers have been used in the example. The following table (very tersely) describes the Aegis commands most useful to developers.

CommandDescription
aebBuild
aecaedit Change Attributes
aecdChange Directory
aecleanClean a development directory
aeclonecopy a whole change
aecpCopy File
aecpuCopy File Undo
aedDifference
aedbDevelop Begin
aedbuDevelop Begin Undo
aedeDevelop End
aedeuDevelop End Undo
aelList Stuff
aenfNew File
aenfuNew File Undo
aentNew Test
aentuNew Test Undo
aermRemove File
aermuRemove File Undo
aetTest

You will want to read the manual entries for all of these commands. Note that all Aegis commands have a -Help option, which will give a result very similar to the corresponding man output. Most Aegis commands also have a -List option, which usually lists interesting context sensitive information.

2. The Reviewer

The role of a reviewer is to check another user's work. You are helped in this by Aegis, because changes can never reach the being reviewed state without several preconditions:

  • The change is known to build. You know that it compiled successfully, so there is no need to search for syntax errors.

  • The change has tests, and those tests have been run, and have passed.

This information allows you to concentrate on implementation issues, completeness issues, and local standards issues.

To help the reviewer, a set of "comma D" files is available in the change development directory. Every file which is to be added to the baseline, removed from the baseline, or changed in some way, has a corresponding "comma D" file.

2.1. Before You Start

Have you configured your account to use Aegis? See the User Setup section of the Tips and Traps chapter for how to do this.

2.2. The First Change

Robyn finds out what changes are available for review by asking Aegis:

robyn% aerpass -l -p example.1.0

Project "example.1.0"
List of Changes

Change  State           Description
------- -------         -------------
  10    being_reviewed  Place under Aegis
robyn%

Any of the above changes could be reviewed, Robyn chooses the first.

robyn% aecd -p example.1.0 -c 10
aegis: project "example": change 1: /u/pat/example.1.0.C010
robyn% aedmore
...examines each file...
robyn%

The aedmore command walks the development directory tree to find all of the "comma D" files, and displays them using more(1). There is a corresponding aedless for those who prefer the less(1) command.

Once the change has been reviewed and found acceptable, it is passed:

robyn% aerpass -p example.1.0 10
aegis: sh /opt/aegis/lib/rp.sh example.1.0 10 pat robyn
aegis: project "example.1.0": change 10: passed review
robyn%

Some time soon Isa will notice the email notification and commence integration of the change.

2.3. The Second Change

Most reviews have the same pattern as the first.

robyn% aerpass -l -p example.1.0

Project "example.1.0"
List of Changes

Change  State           Description
------- -------         -------------
  11    being_reviewed  file names on command line
robyn%

Always change directory to the change's development directory, otherwise you will not be able to review the files.

robyn% aecd -p example.1.0 -c 11
aegis: project "example.1.0": change 11: /u/jan/example.1.0.C011
robyn%

Another useful way of finding out about a change is the "list change details" command, viz:

robyn% ael cd -p example.1.0 -c 11

Project "example.1.0", Change 11
Change Details

NAME
        Project "example.1.0", Change 11.

SUMMARY
        file names on command line

DESCRIPTION
        Optional input and output files may be specified on
        the command line.

CAUSE
        This change was caused by internal_bug.

STATE
        This change is in 'being_reviewed' state.

FILES
        Type    Action  Edit    File Name
        ------- ------- ------- -----------
        source  modify  1.1     main.c
        test    create          test/00/t0002a.sh

HISTORY
        What            When            Who     Comment
        ------          ------          -----   ---------
        new_change      Fri Dec 11      alex
                        14:55:06 1992
        develop_begin   Mon Dec 14      jan
                        09:07:08 1992
        develop_end     Mon Dec 14      jan
                        11:43:23 1992
robyn%

Once Robyn knows what the change is meant to be doing, the files are then examined:

robyn% aedmore
...examines each file...
robyn%

Once the change is found to be acceptable, it is passed:

robyn% aerpass -p example.1.0 11
aegis: sh /opt/aegis/lib/rp.sh example.1.0 11 jan robyn
aegis: project "example.1.0": change 11: passed review
robyn%

Some time soon Isa will notice the email notification and commence integration of the change.

The reviews of the third and fourth changes will not be given here, because they are almost identical to the other changes. If you want to know how to fail a review, see the aerfail(1) manual entry.

2.4. Reviewer Command Summary

Only a few of the Aegis commands available to reviewers have been used in this example. The following table (very tersely) describes the Aegis commands most useful to reviewers.

CommandDescription
aecdChange Directory
aerpassReview Pass
aerpuReview Pass Undo
aerfailReview Fail
aelList Stuff

You will want to read the manual entries for all of these commands. Note that all Aegis commands have a -Help option, which will give a result very similar to the corresponding man output. Most Aegis commands also have a -List option, which usually lists interesting context sensitive information.

3. The Integrator

This section shows what the integrator must do for each of the changes shown to date. The integrator does not have the ability to alter anything in the change; if a change being integrated is defective, it is simply failed back to the developer. This documented example has no such failures, in order to keep it manageably small.

3.1. Before You Start

Have you configured your account to use Aegis? See the User Setup section of the Tips and Traps chapter for how to do this.

3.2. The First Change

The first change of a project is often the trickiest, and the integrator is the last to know. This example goes smoothly, and you may want to consider using the example project as a template.

The integrator for this example project is Isa. Isa knows there is a change ready for integration from the notification which arrived by email.

isa% aeib -l -p example.1.0

Project "example.1.0"
List of Changes

Change  State           Description
------- -------         -------------
  10    awaiting_       Place under Aegis
        integration
isa% aeib example.1.0 10
aegis: project "example.1.0": change 10: link baseline to integration
        directory
aegis: project "example.1.0": change 10: apply change to integration
        directory
aegis: project "example.1.0": change 10: integration has begun
isa%

The integrator must rebuild and retest each change. This ensures that it was no quirk of the developer's environment which resulted in the success at the development stage.

isa% aeb
aegis: logging to "/projects/example/delta.001/aegis.log"
aegis: project "example.1.0": change 10: integration build started
aegis: cook -b Howto.cook project=example.1.0 change=10
        version=1.0.D001 -nl
cook: yacc -d gram.y
cook: mv y.tab.c gram.c
cook: mv y.tab.h gram.h
cook: cc -I. -O -c gram.c
cook: lex lex.l
cook: mv lex.yy.c lex.c
cook: cc -I. -O -c lex.c
cook: cc -I. -O -c main.c
cook: cc -o example gram.o lex.o main.o -ll -ly
aegis: project "example.1.0": change 10: integration build complete
isa%

Notice how the above build differed from the builds that were done while in the being developed state; the extra baseline include is gone. This is because the integration directory will shortly be the new baseline, and must be entirely internally consistent and self-sufficient.

You are probably wondering why this isn't all rolled into the one Aegis command. It is not because there may be some manual process to be performed after the build and before the test. This may be making a command set-uid-root (as in the case of Aegis itself) or it may require some tinkering with the local oracle or ingress database. Instructions for the integrator may be placed in the description field of the change attributes.

The change is now re-tested:

isa% aet
aegis: logging to "/projects/example/delta.001/aegis.log"
aegis: sh /project/example/delta.001/test/00/t0001a.sh
aegis: project "example": change 1: test "test/00/t0001a.sh"
        passed
aegis: project "example": change 1: passed 1 test
isa%

The change builds and tests. Once Isa is happy with the change, perhaps after browsing the files, Isa then passes the integration, causing the history files to be updated and the integration directory becomes the baseline.

isa% aeipass
aegis: logging to "/projects/example/delta.001/aegis.log"
aegis: ci -u -m/dev/null -t/dev/null /projects/example/delta.001/
        Howto.cook /projects/example/history/Howto.cook,v;
        rcs -U /projects/example/history/Howto.cook,v
/projects/example/history/Howto.cook,v  
        /projects/example/delta.001/Howto.cook
initial revision: 1.1
done
RCS file: /projects/example/history/Howto.cook,v
done
aegis: rlog -r /projects/example/history/Howto.cook,v | awk
        '/^revision/ {print $2}' > /tmp/aegis.15309
...lots of similar RCS output...
aegis: project "example.1.0": change 10: remove development directory
aegis: sh /opt/aegis/lib/ip.sh example.1.0 10 pat robyn isa
aegis: project "example.1.0": change 10: integrate pass
isa%

All of the staff involved, will receive email to say that the change has been integrated. This notification is a shell script, so USENET could be usefully used instead.

You should note that the development directory has been deleted. It is expected that each development directory will only contain files necessary to develop the change. You should keep "precious" files somewhere else.

3.3. The Other Changes

There is no difference to integrating any of the later changes. The integration process is very simple, as it is a cut-down version of what the developer does, without all the complexity.

Your project may elect to have the integrator also monitor the quality of the reviews. An answer to "who will watch the watchers" if you like.

It is also a good idea to rotate people out of the integrator position after a few weeks in a busy project, this is a very stressful position. The position of integrator gives a unique perspective to software quality, but the person also needs to be able to say "NO!" when a cruddy change comes along.

3.4. Integrator Command Summary

Only a few of the Aegis commands available to integrators have been used in this example. The following table (very tersely) describes the Aegis commands most useful to integrators.

CommandDescription
aebBuild
aecdChange Directory
aedDifference
aeibIntegrate Begin
aeibuIntegrate Begin Undo
aeifailIntegrate Fail
aelList Stuff
aetTest
aeipassIntegrate Pass

You will want to read the manual entries for all of these commands. Note that all Aegis commands have a -Help option, which will give a result very similar to the corresponding man output. Most Aegis commands also have a -List option, which usually lists interesting context sensitive information.

3.5. Minimum Integrations

The aegis --integrate-begin command provides a --minimum option which may be used for various reasons. The term minimum may be a bit counter intuitive. One might think it means to the minimum amount of work, however it actually means use a minimum of files from the baseline in populating the delta directory. This normally leads to actually building everything in the project from sources and, as such, might be considered the most robust of builds.

Note that any change which removes a file, whether by aerm or aemv, results in an implicit minimum integration. This is intended to ensure nothing in the project references the removed file.

A project may adopt a policy that a product release should be based on a minimum integration. Such a policy may be a reflection of local confidence, or lack therof, in the projects DMT (Dependency Maintenance Tool) or build system. Or it may be based on a validation process wishing to make a simple statement on how the released package was produced.

Another, more transient, reason a to require a minimum integration might be when upgrading a third party library, compiler or maybe even OS level. Any of these events would signal the need for a minimum integration to ensure everything is rebuilt using the new resources.

The cost of a minimum integration varies according to type and size of the project. For very large projects, especially those building large numbers of binaries, the cost can be large. However large projects also require significant time to fully populate the delta directory. A minimum integration only copies those files under aegis control, skipping all “produced” files. In the case where a file upon which everything depends is changed, everything will be built anyway so the copy of the already built files is a waste of time. This means that sometimes a minimum can be as cheap as a normal integration.

4. The Administrator

The previous discussion of developers, reviewers and integrators has covered many aspects of the production of software using Aegis. The administrator has responsibility for everything they don't, but there is very little left.

These responsibilities include:

  • access control: The administrator adds and removes all categories of user, including administrators. This is on a per-project basis, and has nothing to do with Unix™ user administration. This simply nominates which users may do what.

  • change creation: The administrator adds (and sometimes removes) changes to the system. At later stages, developers may alter some attributes of the change, such as the description, to say what they fixed.

  • project creation: Aegis does not limit who may create projects, but when a project is created the user who created the project is set to be the administrator of that project.

All of these things will be examined

4.1. Before You Start

Have you configured your account to use Aegis? See the User Setup section of the Tips and Traps chapter for how to do this.

4.2. The First Change

Many things need to happen before development can begin on the first change; the project must be created, the staff but be given access permissions, the branches created, and the change must be created.

alex% aenpr example -dir /projects/example -version -
aegis: project "example": project directory "/projects/example"
aegis: project "example": created
alex%

Once the project has been created, the project attributes are set. Alex will set the desired project attributes using the -Edit option of the aepa command. This will invoke an editor (vi(1) by default) to edit the project attributes. Alex edits them to look like this:

description = "Aegis Documentation Example Project";
developer_may_review = false;
developer_may_integrate = false;
reviewer_may_integrate = false;

The project attributes are set as follows:

alex% aepa -edit -p example
...edit as above...
aegis: project "example.1.0": attributes changed
alex% ael p
List of Projects

Project     Directory           Description
-------     -----------         -------------
example     /projects/example   Aegis Documentation Example
                                Project
alex%

The various staff must be added to the project. Developers are the only staff who may actually edit files.

alex% aend pat jan sam -p example
aegis: project "example": user "pat" is now a developer
aegis: project "example": user "jan" is now a developer
aegis: project "example": user "sam" is now a developer
alex%

Reviewers may veto a change. There may be overlap between the various categories, as show here for Jan:

alex% aenrv robyn jan -p example
aegis: project "example": user "robyn" is now a reviewer
aegis: project "example": user "jan" is now a reviewer
alex%

The next role we need to fill is an integrator.

alex% aeni isa -p example
aegis: project "example": user "isa" is now an integrator
alex%

Once the staff have been given access, it is time to create the working branch. Branches inherit their attributes and staff lists from their parent branches when they are first created, which is why we set all that stuff first.

alex% aegis -nbr -p example 1
aegis: project "example.1": created
alex% aegis -nbr -p example.1 0
aegis: project "example.1.0": created
alex%

This is for versioning; see the Branching chapter for more information. For the moment, we will simply work on branch 1.0. Notice how the branches appear as projects in the project listing; in general branches can be used interchangeably with projects.

alex% ael p
List of Projects

Project     Directory           Description
-------     -----------         -------------
example     /projects/example   Aegis Documentation Example
                                Project
example.1   /projects/example/  Aegis Documentation Example
            branch.1            Project, branch.1.
example.1.0 /projects/example/  Aegis Documentation Example
            branch.1/branch.0   Project, branch.1.0.
alex%

Once the working branch has been created, Alex creates the first change. The -Edit option of the aenc command is used, to create the attributes of the change. They are edited to look like this:

brief_description = "Create initial skeleton.";
description = "A simple calculator using native \
floating point precision.  \
The four basic arithmetic operators to be provided, \
using conventional infix notation.  \
Parentheses and negation also required.";
cause = internal_enhancement;

The change is created as follows:

alex% aenc -edit -p example.1.0
...edit as above...
aegis: project "example.1.0": change 10: created
alex%

Notice that the first change number is “10”. This is done so that changes 1 to 9 could be used as bug-fix branches at some future time. See the Branching chapter for more information. You can over-ride this is you need to.

The above was written almost a decade ago. There is a newer command, tkaenc, which uses a GUI and is much easier to use, with a much less fiddly interface. You may want to try that command, instead, for most routine change creation.

At this point, Alex walks down the hall to Pat's office, to ask Pat to develop the first change. Pat has had some practice using Aegis, and can be relied on to do the rest of the project configuration speedily.

4.3. The Second Change

Some time later, Alex patiently sits through the whining and grumbling of an especially pedantic user. The following change description is duly entered:

brief_description = "file names on command line";
description = "Optional input and output files may be \
specified on the command line.";
cause = internal_bug;

The pedantic user wanted to be able to name files on the command line, rather than use I/O redirection. Also, having a bug in this example is useful. The change is created as follows:

alex% aenc -edit -p example.1.0
...edit as above...
aegis: project "example.1.0": change 11: created
alex%

At some point a developer will notice this change and start work on it.

4.4. The Third Change

Other features are required for the calculator, and also for this example. The third change adds exponentiation to the calculator, and is described as follows:

brief_description = "add powers";
description = "Enhance the grammar to allow exponentiation.  \
No error checking required.";
cause = internal_enhancement;

The change is created as follows:

alex% aenc -edit -p example.1.0
...edit as above...
aegis: project "example.1.0": change 12: created
alex%

At some point a developer will notice, and this change will be worked on.

4.5. The Fourth Change

A fourth change, this time adding variables to the calculator is added.

brief_description = "add variables";
description = "Enhance the grammar to allow variables.  \
Only single letter variable names are required.";
cause = internal_enhancement;

The change is created as follows:

alex% aenc -edit -p example.1.0
...edit as above...
aegis: project "example.1.0": change 13: created
alex%

At some point a developer will notice, and this change will be worked on.

4.6. Administrator Command Summary

Only a few of the Aegis commands available to administrators have been used in this example. The following table lists the Aegis commands most useful to administrators.

CommandDescription
aecaedit Change Attributes
aelList Stuff
aenaNew Administrator
aencNew Change
aencuNew Change Undo
aendNew Developer
aeniNew Integrator
aenprNew Project
aenrvNew Reviewer
aepaedit Project Attributes
aeraRemove Administrator
aerdRemove Developer
aeriRemove Integrator
aermprRemove Project
aerrvRemove Reviewer

You will want to read the manual entries for all of these commands. Note that all Aegis commands have a -Help option, which will give a result very similar to the corresponding man output. Most Aegis commands also have a -List option, which usually lists interesting context sensitive information.

5. What to do Next

This chapter has given an overview of what using Aegis feels like. As a next step in getting to know Aegis, it would be a good idea if you created a project and went through this same exercise. You could use this exact example, or you could use a similar small project. The idea is simply to run through many of the same steps as in the example. Typos and other natural events will ensure that you come across a number of situations not directly covered by this chapter.

If you have not already done do, a printed copy of the section 1 and 5 manual entries will be invaluable. If you don't want to use that many trees, they will be available on-line, by using the "-Help" option of the appropriate command variant. Try:

% aedb -help
...manual entry...
%

Note that this example has not demonstrated all of the available functionality. One item of particular interest is that tests, like any other source file, may be copied into a change and modified, or even deleted, just like any other source file.

6. Common Questions

There are a number of questions which are frequently asked by people evaluating Aegis. This section attempts to address some of them.

6.1. Insulation

The repository model used by Aegis is of the “push” type - that is, changes to the baseline are “pushed” onto the developer as soon as they are integrated. Many configuration management systems have a “pull” model, where the developer elects when to cope with changes in the repository. At first glance, Aegis does not appear to have a “pull” equivalent.

It is possible to insulate your change from the baseline as much or as little as required. The aecp(1) command, used to copy files into a change, has a --read-only option. The files copied in this way are marked as insulation (i.e. you don't intend to change them). If you have not un-copied them at develop end time, the aede(1) command will produce a suitable error message, reminding you to un-copy the insulation and verify that your change still builds and tests successfully with the (probably) now-different baseline.

6.1.1. Copy Read-Only

It is possible to select the degree of insulation. By using “aecp .” at the top of a development directory, the complete project source tree will be copied, thus completely insulating you. Mind you, it comes at the cost of a complete build.

If you are working on a library, and only want the rest of the library to remain fixed, simply copy the whole library (aecp library/fred), and allow the rest to track the baseline. This comes at a smaller cost, because more of the baseline's object files can be taken advantage of.

6.1.2. Branches

It is also possible to create a sub-branch (see the Branching chapter). This does not itself automatically insulate, however the first change of a branch intended to insulate would copy and integrate but not modify the files to be insulated. You need to remember to perform a cross-branch merge with the parent branch before integrating the branch back into the parent branch.

6.1.3. Builds

You can also insulate yourself from baseline change by being selective about what you choose to build. You can do this by giving specific build targets on the aeb(1) command line, or you could copy the build tool's configuration file and butcher it. Remember to change it back before you aede(1) your change!

6.1.4. Mix-and-Match

Some or all of the above techniques may be combined to provide an insulation technique best suited to your project and development policy. E.g. changing the build configuration file for a branch dedicated to working on a small portion of a large project; towards the ed of the development, change the build configuration file back and perform integration testing.

6.1.5. Disadvantages

There is actually a down-side to insulating your changes from the evolution of the baseline. By noticing and adapting to the baseline, you have much less merging to do at the end of your change set. Each integration will typically be be modest, but the cumulative effect could be substantial, and add a huge unexpected (and un-budgeted for) time penalty.

It also means that if there are integration problems between your work and the changes which were integrated before yours, or if your work shows up a bug in their work, the project find this out late, rather than early. The literature, based on industrial experience, indicates that the earlier problems are found the cheaper they are to fix.

Insulated development directories also use more disk space. While disk space is relatively cheap these days, it can still add up to a substantial hit for a large development team. Un-insulated development directories can take advantage of the pre-compiled objects and libraries in the baseline.

6.2. Partial Check-In

In the course of developing new functionality, it is very common to come across a pre-existing bug which the new functionality exposes. It is common for such bugs to be fixed by the developer in the same development directory, in order to get the new functionality to a testable state.

There are two common courses of action at this point: simply include the bug fix with the rest of the change, or integrate the bug fix earlier than the rest of the change. Combining the bug fix with the rest of the change can have nasty effects on statistics: it can hide the true bug level from your metrics program, and it also denies Aegis the opportunity of having accurate test correlations (see aet(1) for more information.) It also denies the rest of the development team the use of the bug fix, or worse, it allows the possibility that more than one team member will fix the bug, wasting development effort and time.

Many configuration management systems allow you to perform a partial check-in of a work area. This means that you can check-in just the bug fix, but continue to work on the unfinished portions of the functionality you are implementing.

Because Aegis insists on atomic change sets which are known to build and test successfully, such a partial check-in is not allowed - because Aegis can't know for certain that it works.

Instead, you are able to clone a change (see aeclone(1) for more information). This gives you a new change, and a second development directory, with exactly the same files. You then remove from this second change all of the files not related to the bug fix (using aecpu(1), aenfu(11), etc). You then create a test, build, difference, run the test, develop end, all as usual.

The original change will then need to be merged with the baseline, because the bug fix change will have been integrated before it. Usually this is straight-forward, as you already have the changes (some merge tools make this harder than others). Often, all that is required is to merge, and then say “aecpu -unch” to un-copy all files which are (now) unchanged when compared to the current baseline.

6.3. Multiple Active Branches

Some companies have multiple branches active at the same time, for different customers or distributions, etc.

They often need to make the same change to more than one branch. Some configuration management systems allow you to check-in the same file multiple times, once to each active branch. Aegis does not let you do this, because you need to convince Aegis that the change set will build and test cleanly on each branch. It is quite common for the change to require non-trivial edits to work on each branch.

6.3.1. Cloning

Aegis instead provides two mechanisms to handle this. The first, and simplest to understand, is to clone the change onto each relevant branch (rather than onto the same branch, as mentioned above for bug fixes). Then build and test as normal.

6.3.2. Ancestral

The second technique is more subtle. Perform the fix as a change to the common ancestor of both branches. This assumes that neither branch is insulated against the relevant area of code, and that earlier changes to the branch do not mask it in some way (otherwise a cross-branch merge with the common ancestor will be needed to propagate the fix).

6.4. Collaboration

It is often the case that difficult problems are tackled by small groups of 2 or 3 staff working together. In order to do this, they often work in a shared work area with group-writable or global-write permissions. However, this tends to give security auditor heart attacks.

Aegis has several different ways to achieve the same ends, and still not give the auditors indigestion.

6.4.1. Change Owner

The simplest method available is to change the ownership of a change from one developer to the next. A new development directory is created for the new developer, and the source files are copied across.[16] This allows a kind of serial collaboration between developers.

6.4.2. Branch

The other possibility is to create a branch to perform the work in, rather than a simple change. (A branch in Aegis is literally just a big change, which has lots of sub-changes.) This allows parallel collaboration between developers.

6.4.3. View Path Hacking

Aegis usually provides a “view path” to the build tool. This specifies where to look for source files and derived files, in order to union together the development directory, and the baseline, and the branch's ancestors' baselines. If you run the build by hand, rather than through Aegis, you can add another developer's development directory to the view path, after your own development directory, but before the baseline.

This has many of the advantages of the branch method, but none of the safeguards. In particular, if the other developer edits a file, and it no longer compiles, your development directory will not, either.



[8] The names are deliberately gender-neutral. Finding such a name starting with "I" is not easy!

[9] I thought this expression was fairly common English usage, until I had a query. "The Coal Face" is an expression meaning "where the real work is done" in reference to old-style coal mining which was hard, tiring, hot, very dangerous, and bad for your health even if you were lucky enough not to be killed. It was a 14-hour per day job, and you walked to and from work in the dark, even in summer. Unlike the mine owners, who rode expensive horses and saw sunlight most days of the week.

[10] The default directory in which to place new development directories is configurable for each user.

[11] This is especially true when you use a tool such as fcomp(1) which gives a complete file listing with the inserts and deletes marked in the margin. This tool is also available from the author of Aegis.

[12] As discussed in the How Aegis Works chapter, aegis has the objective of ensuring that projects always work, where "works" is defined as passing all tests in the project's baseline. A change "works" if it passes all of its accompanying tests.

[13] Portable for Aegis' point of view: Bourne shell is the most widely available shell. Of course, if you are writing code to publish on USENET or for FTP, portability of the tests will be important from the developer's point of view also.

[14] This is a csh specific example, unlike most others.

[15] Example: in addition to the executable file "example" shown here, the build may also produce an archive file of the project's source for export. The addition of one more file may push the size of this archive beyond a size limit; the build would thus fail because of the addition of a test.

[16] For the technically minded: the chown(2) system call has semantics which vary too widely between Unix™ variants and file-systems to be useful.