Chapter 2. How Aegis Works

Table of Contents

1. The Model
1.1. The Baseline
1.2. The Change Mechanism
1.3. Change States
1.4. The Software Engineers
1.5. The Change Process
2. Philosophy
2.1. Development
2.2. Post Development
2.3. Minimalism
2.4. Overlap
2.5. Design Goals
3. Security
4. Scalability
5. When (not) to use Aegis
5.1. Building
5.2. Testing
5.3. Reviewing
6. Further Work
6.1. Code Coverage Tool
6.2. Virtual File System

Before you will be able to exploit Aegis fully, you will need to know what Aegis does and why.

The Aegis program provides a change control mechanism and a repository, a subset of the functionality which CASE vendors call Software Configuration Management (SCM). In order to fit into a software engineering environment, or any place software is written, Aegis needs a clearly defined place in the scheme of things.

This chapter describes the model of the software development process embodied in the Aegis program, some of the deliberate design decisions made for Aegis, some of the things Aegis will and wont do for you, and the situations where Aegis is most and least useful.

1. The Model

The model of the software development process used by Aegis evolved and grew with time in a commercial software development environment, and it has continued to be used and developed.

The model described here is generic, and can be adapted in a variety of ways. These will be described at the relevant points in the text.

1.1. The Baseline

Most CASE systems revolve around a repository: a place where stuff is kept. This stuff is the raw material that is processed in some way to produce the final product, whatever that may be. This stuff is the preferred form for editing or composing or whatever.

In the Aegis program, the repository is known as the baseline and the units of stuff are Unix™ files. The Aegis program makes no distinction between text and binary files, so both are supported.

The history mechanism which must be included in any repository function is not provided by the Aegis program. It is instead provided by some other per-project configurable software, such as RCS. This means that the user may select the history tool most suited to any given project. It also means that Aegis is that much smaller to test and maintain.

The structure of the baseline is dictated by the nature of each project. The Aegis program attempts to make as few arbitrary rules as possible. There is one mandatory file in the your project baseline. The mandatory file is called aegis.conf by default, and contains the per-project configuration information. The name of this file may be changed if you want to call it something different. It ios also common (though not mandatory, and the name may be changed) to have a directory called test which contains all of the test scripts. The contents and structure of the test directory (or whatever you call it) are controlled by a test filename pattern you supply to Aegis. Tests are treated just like any other source file, and are subject to the same process.

The baseline in Aegis has one particular attribute: it always works. It is always there to show off to visiting big-wigs, it is always there to grab a copy of and ship a "pre-release snapshot" to some overly anxious customer, it is always there to let upper management "touch and feel" the progress being made towards the next release.

You may claim that "works" is comfortably fuzzy, but it is not. The baseline contains not only the source of a project, but also the tests for a project. Tests are treated just like any other source file, and are subject to the same process. A baseline is defined to "work" if and only if it passes all of its own tests. The Aegis program has mandatory testing, to ensure that all changes to the baseline are accompanied by tests, and that those tests have been run and are known to pass. This means that no change to the baseline may result in the baseline ceasing to work.[2]

The model may be summarized briefly: it consists of a baseline (master source), updated through the agency of an integrator who is in turn fed changes by a team of developers These terms will be explained in the following sections. See figure 1 for a picture of how files flow around the system.

Figure 2.1.  Flow of Files through the Model

Flow of Files through the Model

The baseline is a set of files including the source files for a projects, and also all derived files (such as generated code, binary files from the compiler, etc), and all of the tests. Tests are treated just like any other source file, and are subject to the same process. All files in the baseline are consistent with each other.

Thus the baseline may be considered to be the closure of the source files, in mathematical terms. That is, it is the source files and all implications flowing from those source files, such as object files and executables. All files in the baseline are consistent with each other; this means that development builds can take object files from the baseline rather than rebuild them within the development directory.

The baseline is readable by all staff, and usually writable by no-one. When it is necessary to write to the baseline, this is done by Aegis, as will be shown below.

In many ways, the baseline may be thought of as a database, and all derived files are projections (views) of the source files. Passing its own tests may be thought of as input validation of fields. This is a powerful concept, and indeed the implementation of the Aegis program performs many of the locking and synchronization tasks demanded of a database engine.

All of the files forming this database are text files. This means that they may be repaired with an ordinary text editor, should remedial action be necessary. The format is documented in section 5 of the reference manual. Should you wish to perform some query not yet available in Aegis, the files are readily accessible to members of the appropriate Unix™ group.

Tests are treated just like any other source file, and are subject to the same process.

1.2. The Change Mechanism

Any changes to the baseline are made by atomic increments, known (unoriginally) as "changes". A change is a collection of files to be added to, modified in, or deleted from, the baseline. These files must all be so altered simultaneously for the baseline to continue to "work".[3]

For example, if the calling interface to a function were changed in one file, all calls to that function in any other file must also change for the baseline to continue to work. All of the files must be changed simultaneously, and thus must all be included in the one change. Other files which would logically be included in such an change include the reference manual entry for the function, the design document relating to that area of functionality, the relevant user documentation, tests would have to be included for the functionality, and existing tests may need to be revised.

Changes must be accompanied by tests. These tests will either establish that a bug has been fixed (in the case of a bug fix) or will establish that new functionality works (in the case of an enhancement).

Tests are shell scripts, and as such are capable of testing anything which has functionality accessible from the command line. The ability to run background processes allows even client-server models to be tested. Tests are thus text files, and are treated as source files; they may be modified by the same process as any other source file. Tests usually need to be revised as a project grows and adapts to changing requirements, or to be extended as functionality is extended. Tests can even be deleted if the functionality they test has been deleted; tests are deleted by the same process as any other source file.

1.3. Change States

As a change is developed using Aegis, it passes through six states. Many Aegis commands relate to transitions between these states, and Aegis performs any validation at these times.

The six states of a change are described as follows, although the various state transitions, and their conditions, will be described later.

1.3.1. Awaiting Development

A change is in this state after it has been created, but before it has been assigned to a developer. This state can't be skipped: a change can't be immediately assigned to a developer by an administrator, because this disempowers the staff.

The Aegis program is not a progress tracking tool, nor is it a work scheduling tool; plenty of both already exist.

1.3.2. Being Developed

A change is in this state after it has been assigned to a developer, by the developer. This is the coal face: all development is carried out in this state. Files can be edited in no other state, this particularly means that only developers can develop, reviewers and integrators only have the power to veto a change. Staff roles will be described more fully in a later section.

To advance to the next state, the change must build successfully, it must have tests, and it must pass those tests.[4]

The new tests must also fail against the baseline; this is to establish that tests for bug-fixes actually reproduce the bug and then demonstrate that it is gone. New functionality added by a change will naturally fail when tested in the old baseline, because it is not there.

When these conditions are met, the Aegis program marks all of the changes files as locked, simultaneously. If any one of them is already locked, you can't leave the being developed state, because the file is part of a change which is somewhere between being reviewed and being integrated

If any one of them is out-of-date with respect to the baseline, the lock is not taken, either. Locking the files at this state transition means that popular files may be modified simultaneously in many changes, but that only differences to the latest version are ever submitted for integration. The Aegis program provides a mechanism, described later, for bringing out-of-date files in changes up-to-date without losing the edits made by the developer.

1.3.3. Awaiting Review

The default configuration for an Aegis project does not use this state, because for small-ish projects it can be tedious. For larger projects, however, it assists in coordinating reviewers when you use email notification that a review is required to several potential reviewers.

To enable this state, you need to change the develop_­end_­action field of the project attributes. See aepa(1) for more information, or tkaepa(1) for a GUI interface.

It is also possible, by a different setting of the same project attribute, to skip the code review step altogether. This can be of benefit to one-person projects where independent code review would be impossible.

The rest of this description will assume the awaiting review state is not being used, but code reviews are being used, to simplify matters. Once you are more familiar with Aegis, enabling the use of the awaiting review state will be simple and will behave intuitively.

1.3.4. Being Reviewed

A change is in this state after a developer has indicated that development is complete. The change is inspected, usually by a second party (or parties), to ensure that it matches the change description as to what it is meant to be doing, and meets other project or company standards you may have.

The style of review, and who may review, is not dictated by the Aegis program. A number of alternative have been observed:

  • You may have a single person who coordinates review panels of, say, 4 peers, with this coordinator the only person allowed to sign-off review passes or fails.

  • You may allow any of the developers to review any other developer's changes.

  • You may require that only senior staff, familiar with large portions of the code, be allowed to review.

The Aegis program enforces that a developer may not review their own code. This ensures that at least one person other than the developer has scrutinized the code, and eliminates a rather obvious conflict of interest. It is possible to turn this requirement off on a per-project basis, but this is only desirable for projects with a one person team (or maybe two). The Aegis program has no way of knowing that the user passing a review has actually looked at, and understood, the code.

The reviewer knows certain things about a change for it to reach this state: it has passed all of the conditions required to reach this state. The change compiles, it has tests and it passes those tests, and the changes are to the current version of the baseline. The reviewer may thus concentrate on issues of completeness, implementation, and standards - to name only a few.

1.3.4.1. Customizing Code Review Policy

It is possible to require more than one reviewer for a change. By setting the review_­policy_­command of the project configuration file, you can pass a shell script (or other command) the relevant change details, and the exit status will be used to determine of the change advances to the awaiting integration state, or requires additional code reviewers first.

Because it is a program, it is possible to implement almost any policy you can think of, including particular reviewers for particular areas of code, or that there must be 3 different reviewers, etc.

1.3.5. Awaiting Integration

A change is in this state after a reviewer has indicated that a change is acceptable to the reviewer(s). This is essentially a queue, as there may be many developers, but only one integration may proceed at any one time.

The issue of one integration at a time is a philosophical one: all of the changes in the queue are physically independent; because of the Develop End locking rules they do not have intersecting sets of files. The problem comes when one change would break another, in these cases the integrator needs to know which to bounce and which to accept. Integrating one change at a time greatly simplifies this, and enforces the "only change one thing at a time" maxim, occasionally at the expense of integrator throughput.

1.3.6. Being Integrated

A change is in this state when the integration of the change back into the baseline is commenced. A (logical) copy of the baseline is taken, and the change is applied to that copy. In this state, the change is compiled and tested once again.

The additional compilation has two purposes: it ensures that the successful compile performed by the developer was not a fluke of the developer's environment, and it also allows the baseline to be the closure of the sources files. That is, all of the implications flowing from the source files, such as object files and linked programs or libraries. It is not possible for Aegis to know which files these are in the development directory, because Aegis is decoupled from the build mechanism (this will discussed later).

To advance to the next state, the integration copy must have been compiled, and the tests included in the change must have been run and passed.

The integrator also has the power of veto. A change may fail an integration because it fails to build or fails tests, and also just because the integrator says so. This allows the being integrated state to be another review state, if desired. The being integrated state is also the place to monitor the quality of reviews and reviewers.

Should a faulty change manage to reach this point, it is to be hoped that the integration process, and the integrator's sharp eyes, will detect it.

While most of this task is automated, this step is necessary to ensure that some strange quirk of the developer's environment was not responsible for the change reaching this stage. The change is built once more, and tested once more. If a change fails to build or test, it is returned to the developer for further work; the integrator may also choose to fail it for other reasons. If the integrator passes that change, the integrated version becomes the new baseline.

1.3.7. Completed

A change reaches this state when integration is complete. The (logical) copy of the baseline used during integration has replaced the previous copy of the baseline, and the file histories have been updated. Once in this state, a change may never leave it, unlike all other states.

If you wish to remove a change which is in this state from the baseline, you will have to submit another change.

1.4. The Software Engineers

The model of software development used by Aegis has four different roles for software engineers to fill. These four roles may be overlapping sets of people, or be distinct, as appropriate for your project.

1.4.1. Developer

This is the coal-face. This role is where almost everything is done. This is the only role allowed to edit a source file of a project.

Most staff will be developers. There is nothing stopping a developer from also being an administrator, except for the possible conflict of interests with respect to testing exemptions.

A developer may edit many of the attributes of a change while it is being developed. This is mostly useful to update the description of the change to say why it was done and what was actually done. A developer may not grant testing exemptions (but they may be relinquished).

1.4.2. Reviewer

The role of the reviewer is to check a developer's work. This review may consist of a peer examining the code, or it may be handled by a single member of staff setting up and scheduling multi-person review panels. The Aegis program does not mandate what style of review, it only requires that a reviewer pass or fail each change. If it passes, an integrator will handle it next, otherwise it is returned to the developer for further work.

In a large team, the reviewers are usually selected from the more senior members of the team, because of their depth of experience at spotting problems, but also because this is an opportunity for more senior members of staff to coach juniors on the finer points of the art.

The Aegis programs makes some of the reviewer's task easier, because the reviewer knows several specific things about a change before it comes up for review: it builds, it has tests, and they have run successfully. There is also optional (per project) additional conditions imposed at the end of development, such as line length limits, or anything else which is automatically testable. The Aegis program also provides a difference listing to the reviewer, so that each and every edit, to each and every file, can be pointed out to the reviewer.

There is nothing stopping a reviewer from being either an administrator or a developer. The Aegis program specifically prevents a developer from reviewing his own work, to avoid conflicts of interest. (It is possible for this restriction to be waived, but that only makes sense for one person projects.)

It will occasionally be necessary to arbitrate between a developer and a reviewer. The appropriate person to do this would have line responsibility above both staff involved. Thus it is desirable that supervisors/managers not be reviewers, except in very small teams.

1.4.3. Integrator

The role of the integrator is to take a change which has already been reviewed and integrate it with the baseline, to form a new baseline. The integrator is thus the last line of defense for the baseline.

There is nothing preventing an integrator from being an administrator, a developer or a reviewer. The Aegis program specifically prevents a developer or reviewer from integrating his own work, eliminating any conflict of interests. (It is possible for this restriction to be waived, but that only makes sense for one and two person projects.)

It will occasionally be necessary to arbitrate between an integrator and a reviewer and/or a developer. The appropriate person to do this would have line responsibility above all of the staff involved. Thus it is desirable that supervisors/mangers not be integrators, except in very small teams.

The baseline is readable by all developers, but not writable. All updates of the baseline to reflect changes produced by developers are performed through the agency of the integrator.

1.4.4. Administrator

The project administrator has the following duties:

  • Create new changes. These may be the result of some customer bug reporting mechanism, it may be the result of new functionality being requested.

  • Grant testing exemptions. By default, Aegis insists that all changes be accompanied by tests. The project administrator may grant case-by-case exemptions, or a project-wide exemption.

  • Add or remove staff. The four roles described in this section may be assigned to, or removed from, specific Unix™ logins by the project administrator.

  • Edit project attributes. There are many attributes attached to a project, only a project administrator may alter them.

  • Edit change attributes. There are many attributes attached to a change, only a project administrator may alter all of them.

A project usually has only one or two administrators at any one time.

1.5. The Change Process

This section will examine the progression of a change through the six change states. Most of the attention will be given to the conditions which must be met in order to progress from one state to the next, as this is where the software development model employed by Aegis is most often expressed.

See figure 2 for a picture of how all of the states and transitions fit together.

Figure 2.2.  Change States and Transitions

Change States and Transitions

1.5.1. New Change

A project administrator creates a change. This change will consist mostly of a description at this time. The project administrator is not able (through Aegis) to assign it to a specific developer.

The change is awaiting development; it is in the awaiting development state.

1.5.2. New Change Undo

It is possible to abandon a change if it is in the awaiting development state. All record of the change, including its description, will be deleted.

It is called new change undo to emphasize the state it must be in to delete it.

1.5.3. Develop Begin

A developer, for whatever reason, scans the list of changes awaiting development. Having selected a change, the developer then assigns that change to herself.

The change is now being developed; it is in the being developed state.

A number of Aegis commands only work in this state, including commands to include files and tests in the change (be they new files to be added to the baseline, files in the baseline to be modified, or files to be deleted from the baseline), commands to build the change, commands to test the change, and commands to difference the change.

The process of taking sources files, the preferred form for editing of a project, and transforming them, through various manipulations and translations, into a "finished" product is known as building. In the Unix™ world this usually means things like compiling and linking a program, but as fancy graphical programs become more wide-spread, the source files could be the binary output from a graphical Entity-Relationship-Diagram editor, which would then be run through a database schema generator.

The process of testing a change has three aspects. The most intuitive is that a test must be run to determine of the functionality works. The second requirement is that the test be run against the baseline and fail; this is to ensure that bugs are not just fixed, but reproduced as well. The third requirement is optional: all or some of the tests already in the baseline may also be run. Tests consist of Unix™ shell scripts - anything that can be done in a shell script can be tested.

In preparation for review, a change is differenced. This usually consists of automatically comparing the present contents of the baseline with what the change proposes to do to the baseline, on a file-by-file basis. The results of the difference, such as Unixdiff -c output, is kept in a difference file, for examination by the reviewer(s). The benefit of this procedure is that reviewers may examine these files to see every change the developer made, rather than only the obvious ones. The differencing commands are per-project configurable, and other validations, such as line length restrictions, may also be imposed at this time.

To leave this state, the change must have source files, it must have tests, it must have built successfully, it must have passed all its own tests, and it must have been differenced.

1.5.4. Develop Begin Undo

It is possible to return a change from the being developed state to the awaiting development state. This is usually desired if a developer selected the wrong change by mistake. It also provides a method to start over on a change for some other reason.

1.5.5. Develop End

When the conditions for the end of development have been met (the change must have source files, it must have tests, it must have built successfully, it must have passed all its own tests, and it must have been differenced) the developer may cause the change to leave the being developed state and enter the being reviewed state. The Aegis program will check to see that all the conditions are met at this time. There is no history kept of unsuccessful develop end attempts.

Most of these preconditions are determined by the use of time stamps which are recorded for various operations, in addition to file system timestamps on the files themselves. Logical sequencing (e.g. tests being run after building after editing) is also verified.

Note that there are 3 kinds of tests

  1. If a change contains a new test or a test which is being modified, this test must pass against the code compiled and linked in the change. This is simply referred to as a “test”. Changes may be granted an exemption from such tests.

  2. If a change contains a new test and the change is a bug fix, this test must fail against the old code in the baseline. This is to confirm that the bug has been fixed. This is referred to as a “baseline test”. Changes may be granted an exemption from such tests.

  3. Tests which already exist in the baseline may be run against the code compiled and linked in the change. These tests must pass. This is to confirm that the project has not regressed, which is why these tests are referred to as “regression tests”. Changes may be granted an exemption from such tests.

A successful develop end command results in the change advancing from the being developed state to the being reviewed state. (It is also possible to advance to the awaiting review state or the awaiting integration state. See aede(1) or aepattr(5) for more information.)

1.5.6. Develop End Undo

There are many times when a developer thinks that a change is completed, and goes hunting for a reviewer. Half way down the hall, she thinks of something that should have been included.

It is possible for a developer to rescind a Develop End to allow further work on a change. No reason need be given. This request may be issued to a change in either the being reviewed or awaiting integration states.

1.5.7. Review Pass

This event is used to notify Aegis that the change has been examined, by a method unspecified as discussed above, and has been found to be acceptable.

1.5.8. Review Pass Undo

The reviewer of a change may rescind a Review Pass while the change remains in the awaiting integration state. No reason need be supplied. The change will be returned to the being reviewed state.

1.5.9. Review Fail

This event is used to notify Aegis that the change has been examined, by a method unspecified as discussed above, and has been found to be unacceptable.

A file containing a brief summary of the problems must be given, and will be included in the change's history.

The change will be returned to the being developed state for further work.

It is not the responsibility of any reviewer to fix a defective change.

1.5.10. Integrate Begin

This command is used to commence integration of a change into the project baseline.

Whether a physical copy of the baseline is used, or a logical copy using links, is controlled by the project configuration file. The change is then applied to this copy.

The integrator must then issue build and test commands as appropriate. This is not automated as some integrator tasks may be required in and around these commands.

1.5.11. Integrate Begin Undo

This command is used to return a change to the integration queue, without prejudice. No reason need be given.

This is usually done when a particularly important change is in the queue, and the current integration is expected to take a long time.

1.5.12. Integrate Pass

This command is used to notify Aegis that the change being integrated is acceptable.

The current baseline is replaced with the integration copy, and the history is updated.

1.5.13. Integrate Fail

This command is used to notify Aegis that an integration is unacceptable, usually because it failed to build or test in some way, or sometimes because the integrator found a deficiency.

A file containing a brief summary of the problems must be given, and the summary will be included in the change's history.

The change will be returned to the being developed state for further work. The integration copy of the baseline is deleted, leaving the original baseline unchanged.

It is not the responsibility of any integrator to fix a defective change, or even diagnose what the defect may be.

2. Philosophy

The philosophy is simple, and that makes some of the implementation complex.

  • When a change is in the being developed state, the aegis program is a developer's tool. Its purpose is to make it as easy for a developer to develop changes as possible.

  • When a change leaves (or attempts to leave) the being developed state, the aegis program is protecting the project baseline, and does not exist to make the developer happy.

  • The aegis program attempts to adhere to the Unix™ minimalist philosophy. Least unnecessary output, least command line length, least dependence on specific 3rd party tools.

  • No overlap in functionality of cooperating tools. (I.e. no internal build mechanism, no internal history mechanism, etc.)

2.1. Development

During the development of a change, the aegis program exists to help the developer. It helps him navigate around his change and the project, it copies file for him, and keeps track of the versions. It can even tell him what changes he has made.

2.2. Post Development

When a change has left the "being developed" state, or when it is attempting to leave that state, the aegis program ceases to attempt to help the developer and proceeds to defend the project baseline. The model used by aegis states that "the baseline always works", and aegis attempts to guarantee this.

2.3. Minimalism

The idea of minimalism is to help the user out. It is the intention that the aegis program can work out unstated command line options for itself, in cases where it is "safe" to do so. This means a number of defaulting mechanisms, all designed to help the user.

2.4. Overlap

It was very tempting while writing the aegis program to have it grow and cover source control and dependency maintenance roles. Unfortunately, this would have meant that the user would have been trapped with whatever the aegis program provided, and the aegis program is already plenty big. To add this functionality would have diverted effort, resulting in an inferior result. It would also have violated the underlying Unix™ philosophy.

2.5. Design Goals

A number of specific ideas molded the shape of the aegis program. These include:

The Unix™ philosophy of writing small tools for specific tasks with little or no overlap. Tools should be written with the expectation of use in pipes or scripts, or other combinations.

  • Stay out of the way. If it is possible to let a project do whatever it likes, write the code to let it. It is not possible to anticipate even a fraction of the applications of a software tool.

  • People. The staff using aegis should be in charge of the development process. They should not feel that some machine is giving them orders.

  • Users aren't psychic. Feedback must be clear, accurate and appropriate.

3. Security

Access to project data is controlled by the Unix™ group mechanism. The group may be selected as suitable for your project, as may the umask.

All work done by developers (build, difference, etc) is all with a default group of the project's group, irrespective of the user's default group. Directories (when BSD semantics are available) are all created so that their contents default to the correct group. This ensures that reviewers and integrators are able to examine the change.

Other Unix™ users not in the project's group may be excluded, or not, by the appropriate setting of the project umask. This umask is used by all Aegis actions, assuring appropriate default behaviour.

A second aspect of security is to ensure that developers are unable to deliberately deceive Aegis. Should the files be tampered with at any later date, Aegis will notice.

4. Scalability

How big can a project get before Aegis chokes? There are a huge number of variables in this question.

The most obvious bottleneck is the integrator. An artificial "big project" example: Assume that the average integration takes an hour to build and test. A full-time integrator could be expected to get 7 or 8 of these done per day (this was the observed average on one project the author was involved in). Assume that the average time for a developer to develop a change is two weeks; a figure recommended by many text books as "the most you can afford to throw away". These two assumptions mean that for this "big project" Aegis can cope with 70 to 80 developers, before integrations become a bottleneck.

The more serious bottle neck is the dependency maintenance tool. Seventy developers can churn out a staggering volume of code. It takes a very long time to wade through the file times and the rules, just to find the one or two files which changed. This can easily push the integrate build time past the one hour mark. Developers also become very vocal when build times are this long.

5. When (not) to use Aegis

The aegis program is not a silver bullet; it will not solve all of your problems. Aegis is suitable for some kinds of projects, useful for others, and useless for a few.

The software development process embodied by Aegis has the following attributes:

  • Each change set is applied atomically.

  • Each change set must build successfully before it will be accepted. (This can be trivial, if desired.)

  • Each change set must test successfully before it will be accepted. (This can be disabled, if desired.)

  • Each change set must pass a peer review before it will be accepted. (This can be a rubber stamp, if desired.)

The most difficult thing about Aegis program is that it takes management buy-in. It takes effort to convince many people that the model used by aegis has benefits, and you need management backing you up when some person comes along with a way of developing software "without the extra work" imposed by the model used by Aegis.

5.1. Building

If the source code to your software product doesn't build, it isn't a product. However, many software shops commit changes to their repository without preconditions, and then do a daily build (or worse, weekly). The problem here is that "pollution" by defective changes is already in your product before it is detected. Aegis will not let it be committed in the first place.

If your product is entirely composed of scripts or HTML, you can make the build step completely trivial: "exit 0" is usually used for this purpose. Thus, this requirement, while usually highly desirable, may be avoided if desired.

5.2. Testing

There is extra up-front work: writing tests. The win is that the tests hang around forever, catching minor and major slips before they become embarrassing "features" in a released product. Prevention is cheaper than cure in this case, the tests save work down the track. See the Testing chapter for more information.

5.3. Reviewing

Code reviews of some sort are normal in most software houses. Often, unfortunately, time pressures or other political pressures mean that code reviews manage not to happen. Yet the literature repeatedly cites reviews as one of the most important factors in removing defects before they reach your code repository. Aegis requires a code review before it will commit code into your product; again, the idea is to remove defects before they pollute the product.

6. Further Work

The Aegis program is far from finished. A number of features are known to be lacking.

At the date of this writing, Aegis is being actively supported and improved.

6.1. Code Coverage Tool

It would be very helpful if a code coverage tool could be used to analyze tests included with changes to ensure that the tests actually exercised the lines of code changed in the change.

Another use of the code coverage tool would be to select regression tests based on the object files recompiled by a change, and those regression tests which exercise those files.

While there is freeware C code coverage tool available, based on GNU C, the interfacing and semantics still need more thought.

[Note]Note:

A fairly good approximation is already available using the --suggest option of the aet(1) command. It works on the correlation of sources file versus tests in the various change sets. See aet(1) for more information.

6.2. Virtual File System

There is almost sufficient information in the Aegis data base to create a virtual file system, overlaying the development directory atop the baseline.[5] This could be implemented similarly to automounters, intercepting file system operations by pretending to be an NFS server. Many commercial CASE products provide such a facility.

Such a virtual file system has a number of advantages: you don't need such a capable DMT, for starters; it only needs the dynamic include dependencies, and does not need a search path.[6] Second, many horrible and dumb compilers, notably FORTRAN and "fourth" GLs, don't have adequate include semantics; overlaying the two directories make this much easier to deal with.[7] Many graphical tools, such as bubble chart drawers, etc, when they do actually have include files, have no command line specifiable search path.

The disadvantage is that this adds significant complexity to an already large program. Also, implementation is limited to NFS capable systems, or would have to be rewritten for a variety of other systems. The semantics of interactions between the daemon and other Aegis commands, while clearly specifiable, are challenging to implement. Performance could also be a significant factor.

The question is "is it really necessary?" If the job can be done without it, does the effort of writing such a beast result in significant productivity gains?

The addition of the create_symlinks_before_build field to the project configuration file has greatly reduced the need for this functionality. However, it does not provide copy-on-write semantics, nor automatic aecp functionality; which a virtual file system could do.



[2] Well, mostly. It is possible for this restriction to be relaxed if you feel there are special circumstances for a particular change. The danger is that a change will be integrated with the baseline when that change is not actually of acceptable quality.

[3] Whether to allow several logically independent changes to be included in the one change is a policy decision for individual projects to make, and is not dictated by the Aegis program. It is a responsibility of reviewers to ensure that all new and changed functionality is tested and documented.

[4] It is possible for these testing requirements to be waived on either a per-project or per-change basis. How is described in a later section. The power to waive this requirement is not automatically granted to developers, as experience has shown that it is usually abused.

[5] Reminiscent of Sun's TFS, but not the same. Similar to AT&T's 3D-FS. Similar to TeamNet. Similar to ClearCase, but I wasn't thinking of the time-travel aspects which they implement.

[6] Discussed in the Dependency Maintenance Tool chapter.

[7] There are other ways, discussed in the Tips and Traps chapter.