Aegis AProject Change Supervisor - Aegis 4.24aegis.sourceforge.net/user-guide.pdf · Aegis AProject Change Supervisor User Guide Peter Miller [email protected]

.

AegisA Project Change Supervisor

User Guide

Peter [email protected]

User Guide Aegis

.

DEDICATIONS

This user guide is dedicated to my wifeMary Therese Miller

for all her love and supportdespite the computers.

And to my grandmotherJean Florence Pelham

1905 — 1992Always in our hearts.

This document describes Aegis version 4.24and was prepared 10 March 2008.

This document describing the Aegis program, and the Aegis program itself, areCopyright © 1991, 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002,2003, 2004, 2005, 2006, 2007, 2008 Peter Miller

This program is free software; you can redistribute it and/or modify it under the terms ofthe GNU General Public License as published by the Free Software Foundation; eitherversion 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WAR-RANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FORA PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this pro-gram. If not, see <http://www.gnu.org/licenses/>.

Page 2 (bl/lib/en/user-guide/c1.0.so) Peter Miller

Aegis UserGuide

Table of Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.1. Year 2000 Status . . . . . . . . . . . . . . . . . . . . . . . . 31.2. What does aegis do?. . . . . . . . . . . . . . . . . . . . . . . 31.3. Why use aegis? . . . . . . . . . . . . . . . . . . . . . . . . . 31.4. How to use this manual . . . . . . . . . . . . . . . . . . . . . . 41.5. GNU GPL . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2. How Aegis Works . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1. The Model . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1.1. The Baseline. . . . . . . . . . . . . . . . . . . . . . . . 52.1.2. The Change Mechanism. . . . . . . . . . . . . . . . . . . . 62.1.3. Change States. . . . . . . . . . . . . . . . . . . . . . . 62.1.4. The Software Engineers. . . . . . . . . . . . . . . . . . . . 82.1.5. The Change Process. . . . . . . . . . . . . . . . . . . . . 10

2.2. Philosophy . . . . . . . . . . . . . . . . . . . . . . . . . . 132.2.1. Development . . . . . . . . . . . . . . . . . . . . . . . . 132.2.2. Post Development . . . . . . . . . . . . . . . . . . . . . . 132.2.3. Minimalism . . . . . . . . . . . . . . . . . . . . . . . . 132.2.4. Overlap . . . . . . . . . . . . . . . . . . . . . . . . . 132.2.5. Design Goals . . . . . . . . . . . . . . . . . . . . . . . 13

2.3. Security . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.4. Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . 142.5. When (not) to use Aegis . . . . . . . . . . . . . . . . . . . . . . 15

2.5.1. Building . . . . . . . . . . . . . . . . . . . . . . . . . 152.5.2. Testing . . . . . . . . . . . . . . . . . . . . . . . . . 152.5.3. Reviewing . . . . . . . . . . . . . . . . . . . . . . . . 15

2.6. Further Work . . . . . . . . . . . . . . . . . . . . . . . . . . 162.6.1. Code Coverage Tool . . . . . . . . . . . . . . . . . . . . . 162.6.2. Virtual File System. . . . . . . . . . . . . . . . . . . . . . 16

3. The Change Development Cycle . . . . . . . . . . . . . . . . . . . . . 173.1. The Developer . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.1.1. Before You Start . . . . . . . . . . . . . . . . . . . . . . 183.1.2. The First Change. . . . . . . . . . . . . . . . . . . . . . 183.1.3. The Second Change. . . . . . . . . . . . . . . . . . . . . 253.1.4. The Third and Fourth Changes. . . . . . . . . . . . . . . . . . 303.1.5. Developer Command Summary. . . . . . . . . . . . . . . . . . 42

3.2. The Reviewer . . . . . . . . . . . . . . . . . . . . . . . . . 433.2.1. Before You Start . . . . . . . . . . . . . . . . . . . . . . 433.2.2. The First Change. . . . . . . . . . . . . . . . . . . . . . 433.2.3. The Second Change. . . . . . . . . . . . . . . . . . . . . 433.2.4. Reviewer Command Summary. . . . . . . . . . . . . . . . . . 45

3.3. The Integrator . . . . . . . . . . . . . . . . . . . . . . . . . 463.3.1. Before You Start . . . . . . . . . . . . . . . . . . . . . . 463.3.2. The First Change. . . . . . . . . . . . . . . . . . . . . . 463.3.3. The Other Changes. . . . . . . . . . . . . . . . . . . . . . 473.3.4. Integrator Command Summary. . . . . . . . . . . . . . . . . . 483.3.5. Minimum Integrations . . . . . . . . . . . . . . . . . . . . . 48

3.4. The Administrator. . . . . . . . . . . . . . . . . . . . . . . . 493.4.1. Before You Start . . . . . . . . . . . . . . . . . . . . . . 493.4.2. The First Change. . . . . . . . . . . . . . . . . . . . . . 493.4.3. The Second Change. . . . . . . . . . . . . . . . . . . . . 513.4.4. The Third Change. . . . . . . . . . . . . . . . . . . . . . 51

Peter Miller (bl/lib/en/user-guide/main.ms) Page mi

User Guide Aegis

3.4.5. The Fourth Change. . . . . . . . . . . . . . . . . . . . . . 513.4.6. Administrator Command Summary. . . . . . . . . . . . . . . . 51

3.5. What to do Next . . . . . . . . . . . . . . . . . . . . . . . . . 533.6. Common Questions . . . . . . . . . . . . . . . . . . . . . . . 53

3.6.1. Insulation. . . . . . . . . . . . . . . . . . . . . . . . . 533.6.2. Partial Check-In. . . . . . . . . . . . . . . . . . . . . . . 543.6.3. Multiple Active Branches . . . . . . . . . . . . . . . . . . . . 543.6.4. Collaboration . . . . . . . . . . . . . . . . . . . . . . . 54

4. The History Tool . . . . . . . . . . . . . . . . . . . . . . . . . . 564.1. History File Names. . . . . . . . . . . . . . . . . . . . . . . . 564.2. Interfacing . . . . . . . . . . . . . . . . . . . . . . . . . . 56

4.2.1. history_create_command. . . . . . . . . . . . . . . . . . . . 564.2.2. history_get_command. . . . . . . . . . . . . . . . . . . . . 564.2.3. history_put_command. . . . . . . . . . . . . . . . . . . . . 564.2.4. history_query_command. . . . . . . . . . . . . . . . . . . . 564.2.5. history_content_limitation . . . . . . . . . . . . . . . . . . . 564.2.6. history_tool_trashes_file . . . . . . . . . . . . . . . . . . . . 574.2.7. Quoting Filenames. . . . . . . . . . . . . . . . . . . . . . 574.2.8. Templates . . . . . . . . . . . . . . . . . . . . . . . . . 57

4.3. Using aesvt . . . . . . . . . . . . . . . . . . . . . . . . . . 584.3.1. history_create_command. . . . . . . . . . . . . . . . . . . . 584.3.2. history_put_command. . . . . . . . . . . . . . . . . . . . . 584.3.3. history_get_command. . . . . . . . . . . . . . . . . . . . . 584.3.4. history_query_command. . . . . . . . . . . . . . . . . . . . 584.3.5. Templates . . . . . . . . . . . . . . . . . . . . . . . . . 584.3.6. Binary Files. . . . . . . . . . . . . . . . . . . . . . . . 58

4.4. Using SCCS. . . . . . . . . . . . . . . . . . . . . . . . . . 594.4.1. history_create_command. . . . . . . . . . . . . . . . . . . . 594.4.2. history_get_command. . . . . . . . . . . . . . . . . . . . . 594.4.3. history_put_command. . . . . . . . . . . . . . . . . . . . . 594.4.4. history_query_command. . . . . . . . . . . . . . . . . . . . 594.4.5. Templates . . . . . . . . . . . . . . . . . . . . . . . . . 594.4.6. Binary Files. . . . . . . . . . . . . . . . . . . . . . . . 60

4.5. Using RCS . . . . . . . . . . . . . . . . . . . . . . . . . . 614.5.1. history_create_command. . . . . . . . . . . . . . . . . . . . 614.5.2. history_get_command. . . . . . . . . . . . . . . . . . . . . 614.5.3. history_put_command. . . . . . . . . . . . . . . . . . . . . 614.5.4. history_query_command. . . . . . . . . . . . . . . . . . . . 624.5.5. merge_command . . . . . . . . . . . . . . . . . . . . . . 624.5.6. Referential Integrity . . . . . . . . . . . . . . . . . . . . . 624.5.7. Templates . . . . . . . . . . . . . . . . . . . . . . . . . 634.5.8. Binary Files. . . . . . . . . . . . . . . . . . . . . . . . 634.5.9. history_put_trashes_files . . . . . . . . . . . . . . . . . . . . 63

4.6. Using fhist . . . . . . . . . . . . . . . . . . . . . . . . . . 644.6.1. history_create_command. . . . . . . . . . . . . . . . . . . . 644.6.2. history_get_command. . . . . . . . . . . . . . . . . . . . . 644.6.3. history_put_command. . . . . . . . . . . . . . . . . . . . . 644.6.4. history_query_command. . . . . . . . . . . . . . . . . . . . 644.6.5. Templates . . . . . . . . . . . . . . . . . . . . . . . . . 644.6.6. Capabilities . . . . . . . . . . . . . . . . . . . . . . . . 644.6.7. Binary Files. . . . . . . . . . . . . . . . . . . . . . . . 65

4.7. Detecting History File Corruption. . . . . . . . . . . . . . . . . . . 664.7.1. General Method. . . . . . . . . . . . . . . . . . . . . . . 664.7.2. Configuration Commands . . . . . . . . . . . . . . . . . . . 66

Page mii (bl/lib/en/user-guide/main.ms) Peter Miller

Aegis UserGuide

4.7.3. An Alternative . . . . . . . . . . . . . . . . . . . . . . . 664.7.4. Aegis’ Database. . . . . . . . . . . . . . . . . . . . . . . 67

5. The Dependency Maintenance Tool . . . . . . . . . . . . . . . . . . . . 685.1. Required Features. . . . . . . . . . . . . . . . . . . . . . . . 68

5.1.1. View Paths . . . . . . . . . . . . . . . . . . . . . . . . 685.1.2. Dynamic Include File Dependencies. . . . . . . . . . . . . . . . 68

5.2. Development Directory Style . . . . . . . . . . . . . . . . . . . . 695.2.1. View Path . . . . . . . . . . . . . . . . . . . . . . . . . 695.2.2. Link the Baseline . . . . . . . . . . . . . . . . . . . . . . 705.2.3. Copy All Sources . . . . . . . . . . . . . . . . . . . . . . 705.2.4. Obsolete Features. . . . . . . . . . . . . . . . . . . . . . 71

5.3. Using Cook . . . . . . . . . . . . . . . . . . . . . . . . . . 725.3.1. Invoking Cook . . . . . . . . . . . . . . . . . . . . . . . 725.3.2. The Recipe File. . . . . . . . . . . . . . . . . . . . . . . 725.3.3. The Recipe for C . . . . . . . . . . . . . . . . . . . . . . 735.3.4. The Recipe for Yacc . . . . . . . . . . . . . . . . . . . . . 745.3.5. The Recipe for Lex . . . . . . . . . . . . . . . . . . . . . . 745.3.6. Recipes for Documents. . . . . . . . . . . . . . . . . . . . 745.3.7. Templates . . . . . . . . . . . . . . . . . . . . . . . . . 74

5.4. Using Cake . . . . . . . . . . . . . . . . . . . . . . . . . .755.4.1. Invoking Cake . . . . . . . . . . . . . . . . . . . . . . . 755.4.2. The Rules File . . . . . . . . . . . . . . . . . . . . . . . 755.4.3. The Rule for C. . . . . . . . . . . . . . . . . . . . . . . 755.4.4. The Rule for Yacc . . . . . . . . . . . . . . . . . . . . . . 765.4.5. The Rule for Lex . . . . . . . . . . . . . . . . . . . . . . 765.4.6. Rules for Documents. . . . . . . . . . . . . . . . . . . . . 76

5.5. Using Make . . . . . . . . . . . . . . . . . . . . . . . . . .775.5.1. Invoking Make . . . . . . . . . . . . . . . . . . . . . . . 775.5.2. The Rule File . . . . . . . . . . . . . . . . . . . . . . . 775.5.3. The Rule for C. . . . . . . . . . . . . . . . . . . . . . . 785.5.4. The Rule for Yacc . . . . . . . . . . . . . . . . . . . . . . 785.5.5. The Rule for Lex . . . . . . . . . . . . . . . . . . . . . . 785.5.6. Rules for Documents. . . . . . . . . . . . . . . . . . . . . 795.5.7. Other Makes . . . . . . . . . . . . . . . . . . . . . . . . 795.5.8. Templates . . . . . . . . . . . . . . . . . . . . . . . . . 795.5.9. GNU Make VPATH Patch . . . . . . . . . . . . . . . . . . . 795.5.10. GNU Make’s VPATH+ . . . . . . . . . . . . . . . . . . . . 79

5.6. Building Executable Scripts. . . . . . . . . . . . . . . . . . . . . 805.7. GNU Autoconf . . . . . . . . . . . . . . . . . . . . . . . . . 80

5.7.1. The Sources. . . . . . . . . . . . . . . . . . . . . . . . 805.7.2. Building . . . . . . . . . . . . . . . . . . . . . . . . . 805.7.3. Tesing . . . . . . . . . . . . . . . . . . . . . . . . . . 815.7.4. An Optimization . . . . . . . . . . . . . . . . . . . . . . 815.7.5. Signed-off-by . . . . . . . . . . . . . . . . . . . . . . . 815.7.6. Importing the Next Upstream Tarball . . . . . . . . . . . . . . . . 825.7.7. Importing the Next Upstream Patch . . . . . . . . . . . . . . . . 82

5.8. No Build Required. . . . . . . . . . . . . . . . . . . . . . . . 825.8.1. Why This May Not Be Such A Good Idea. . . . . . . . . . . . . . 83

6. The Difference Tools . . . . . . . . . . . . . . . . . . . . . . . . . 846.1. Binary Files . . . . . . . . . . . . . . . . . . . . . . . . . . 846.2. Interfacing . . . . . . . . . . . . . . . . . . . . . . . . . . 84

6.2.1. diff_command . . . . . . . . . . . . . . . . . . . . . . . 846.2.2. merge_command . . . . . . . . . . . . . . . . . . . . . . 84

6.3. When No Diff is Required . . . . . . . . . . . . . . . . . . . . . 85

Peter Miller (bl/lib/en/user-guide/main.ms) Page miii

User Guide Aegis

6.4. Using diff and merge . . . . . . . . . . . . . . . . . . . . . . . 866.4.1. diff_command . . . . . . . . . . . . . . . . . . . . . . . 866.4.2. merge_command . . . . . . . . . . . . . . . . . . . . . . 86

6.5. Using fhist . . . . . . . . . . . . . . . . . . . . . . . . . . 866.5.1. diff_command . . . . . . . . . . . . . . . . . . . . . . . 866.5.2. merge_command . . . . . . . . . . . . . . . . . . . . . . 86

7. The Project Attributes . . . . . . . . . . . . . . . . . . . . . . . . . 887.1. Description and Access. . . . . . . . . . . . . . . . . . . . . . 887.2. Notification Commands . . . . . . . . . . . . . . . . . . . . . . 88

7.2.1. Notification by email . . . . . . . . . . . . . . . . . . . . . 887.2.2. Notification by USENET. . . . . . . . . . . . . . . . . . . . 89

7.3. Exemption Controls . . . . . . . . . . . . . . . . . . . . . . . 897.3.1. One Person Projects. . . . . . . . . . . . . . . . . . . . . 897.3.2. Two Person Projects . . . . . . . . . . . . . . . . . . . . . 897.3.3. Larger Projects. . . . . . . . . . . . . . . . . . . . . . . 907.3.4. RSS Feeds . . . . . . . . . . . . . . . . . . . . . . . . 90

8. Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 928.1. Why Bother? . . . . . . . . . . . . . . . . . . . . . . . . . 92

8.1.1. Projects for which Aegis’ Testing is Most Suitable. . . . . . . . . . . . 928.1.2. Projects for which Aegis’ Testing is Useful. . . . . . . . . . . . . . 928.1.3. Projects for which Aegis’ Testing is Least Useful. . . . . . . . . . . . 93

8.2. Writing Tests . . . . . . . . . . . . . . . . . . . . . . . . . 958.2.1. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 958.2.2. General Guidelines. . . . . . . . . . . . . . . . . . . . . . 958.2.3. Bourne Shell. . . . . . . . . . . . . . . . . . . . . . . . 968.2.4. Perl . . . . . . . . . . . . . . . . . . . . . . . . . . 978.2.5. Batch Testing . . . . . . . . . . . . . . . . . . . . . . . 99

9. Branching. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1009.1. How To Use Branching . . . . . . . . . . . . . . . . . . . . . . 1009.2. Transition Using aenrls . . . . . . . . . . . . . . . . . . . . . . 1009.3. Cross Branch Merge . . . . . . . . . . . . . . . . . . . . . . . 1019.4. Multiple Branch Development . . . . . . . . . . . . . . . . . . . . 1019.5. Hierarchy of Projects . . . . . . . . . . . . . . . . . . . . . . . 101

9.5.1. Fundamentals . . . . . . . . . . . . . . . . . . . . . . . 1019.5.2. Incremental Integration . . . . . . . . . . . . . . . . . . . . 1019.5.3. Super-Project Branching. . . . . . . . . . . . . . . . . . . . 1029.5.4. Super-Project Testing . . . . . . . . . . . . . . . . . . . . . 1029.5.5. The Next Cycle. . . . . . . . . . . . . . . . . . . . . . . 1029.5.6. Bug Fixing . . . . . . . . . . . . . . . . . . . . . . . . 102

9.6. Conflict Resolution. . . . . . . . . . . . . . . . . . . . . . . . 1029.6.1. Cross Branch Merge . . . . . . . . . . . . . . . . . . . . . 1039.6.2. Insulation. . . . . . . . . . . . . . . . . . . . . . . . . 103

9.7. Ending A Branch . . . . . . . . . . . . . . . . . . . . . . . . 10310. Tips and Traps . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

10.1. Renaming Include Files. . . . . . . . . . . . . . . . . . . . . . 10510.2. Symbolic Links . . . . . . . . . . . . . . . . . . . . . . . . 10510.3. User Setup. . . . . . . . . . . . . . . . . . . . . . . . . . 105

10.3.1. The .cshrc or .profile files . . . . . . . . . . . . . . . . . . . 10510.3.2. The AEGIS_PATH environment variable . . . . . . . . . . . . . . 10510.3.3. The .aegisrc file . . . . . . . . . . . . . . . . . . . . . . 10610.3.4. The defaulting mechanism. . . . . . . . . . . . . . . . . . . 106

10.4. The Project Owner . . . . . . . . . . . . . . . . . . . . . . . 10610.5. USENET Publication Standards. . . . . . . . . . . . . . . . . . . 106

10.5.1. CHANGES. . . . . . . . . . . . . . . . . . . . . . . . 106

Page miv (bl/lib/en/user-guide/main.ms) Peter Miller

Aegis UserGuide

10.5.2. MANIFEST . . . . . . . . . . . . . . . . . . . . . . . 10610.5.3. Makefile . . . . . . . . . . . . . . . . . . . . . . . . . 10610.5.4. patchlevel.h . . . . . . . . . . . . . . . . . . . . . . . . 10610.5.5. Building Patch Files. . . . . . . . . . . . . . . . . . . . . 106

10.6. Heterogeneous Development . . . . . . . . . . . . . . . . . . . . 10810.6.1. Projectaegis.confFile . . . . . . . . . . . . . . . . . . . . 10810.6.2. Change Attribute . . . . . . . . . . . . . . . . . . . . . . 10810.6.3. Network Files. . . . . . . . . . . . . . . . . . . . . . . 10910.6.4. DMT Implications . . . . . . . . . . . . . . . . . . . . . 10910.6.5. Test Implications. . . . . . . . . . . . . . . . . . . . . . 10910.6.6. Cross Compiling. . . . . . . . . . . . . . . . . . . . . . 11010.6.7. File Version by Architecture . . . . . . . . . . . . . . . . . . 110

10.7. Reminders. . . . . . . . . . . . . . . . . . . . . . . . . . 11010.7.1. Awaiting Development . . . . . . . . . . . . . . . . . . . . 11010.7.2. Being Developed . . . . . . . . . . . . . . . . . . . . . . 11010.7.3. Being Reviewed . . . . . . . . . . . . . . . . . . . . . . 11010.7.4. Awaiting Integration . . . . . . . . . . . . . . . . . . . . . 110

11. Geographically Distributed Development . . . . . . . . . . . . . . . . . . 11111.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 111

11.1.1. Risk Reduction . . . . . . . . . . . . . . . . . . . . . . 11111.1.2. What to Send. . . . . . . . . . . . . . . . . . . . . . . 11111.1.3. Methods and Topologies . . . . . . . . . . . . . . . . . . . 11111.1.4. The Rest of this Chapter. . . . . . . . . . . . . . . . . . . . 112

11.2. Manual Operation. . . . . . . . . . . . . . . . . . . . . . . . 11211.2.1. Manual Send. . . . . . . . . . . . . . . . . . . . . . . 11211.2.2. Sending Baselines . . . . . . . . . . . . . . . . . . . . . 11211.2.3. Sending Branches. . . . . . . . . . . . . . . . . . . . . . 11211.2.4. Manual Receive . . . . . . . . . . . . . . . . . . . . . . 11311.2.5. Getting Started. . . . . . . . . . . . . . . . . . . . . . . 113

11.3. Sneaker Net . . . . . . . . . . . . . . . . . . . . . . . . . 11311.4. Automatic Operation. . . . . . . . . . . . . . . . . . . . . . . 114

11.4.1. Sending. . . . . . . . . . . . . . . . . . . . . . . . . 11411.4.2. Receiving . . . . . . . . . . . . . . . . . . . . . . . . 114

11.5. World Wide Web . . . . . . . . . . . . . . . . . . . . . . . . 11411.5.1. Server . . . . . . . . . . . . . . . . . . . . . . . . . 11411.5.2. Browser . . . . . . . . . . . . . . . . . . . . . . . . . 11411.5.3. Hands-Free Tracking . . . . . . . . . . . . . . . . . . . . . 114

11.6. Security. . . . . . . . . . . . . . . . . . . . . . . . . . . 11511.6.1. Trojan Horses. . . . . . . . . . . . . . . . . . . . . . . 11511.6.2. PGP . . . . . . . . . . . . . . . . . . . . . . . . . . 11511.6.3. Sorcerer’s Apprentice . . . . . . . . . . . . . . . . . . . . 115

11.7. Patches . . . . . . . . . . . . . . . . . . . . . . . . . . . 11611.7.1. Send. . . . . . . . . . . . . . . . . . . . . . . . . . 11611.7.2. Receive . . . . . . . . . . . . . . . . . . . . . . . . . 11611.7.3. Limitations. . . . . . . . . . . . . . . . . . . . . . . . 116

12. Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . 11812.1. Software Configuration Management. . . . . . . . . . . . . . . . . 11812.2. Reviewing . . . . . . . . . . . . . . . . . . . . . . . . . . 118

13. Appendix A: New Project Quick Reference. . . . . . . . . . . . . . . . . . 11913.1. Create the Project. . . . . . . . . . . . . . . . . . . . . . . . 119

13.1.1. Add the Staff . . . . . . . . . . . . . . . . . . . . . . . 11913.1.2. Project Attributes . . . . . . . . . . . . . . . . . . . . . . 119

13.2. Create Change One. . . . . . . . . . . . . . . . . . . . . . . 11913.3. Develop Change One. . . . . . . . . . . . . . . . . . . . . . . 119

Peter Miller (bl/lib/en/user-guide/main.ms) Page mv

User Guide Aegis

13.4. Review The Change . . . . . . . . . . . . . . . . . . . . . . . 12013.5. Integrate the Change. . . . . . . . . . . . . . . . . . . . . . . 12013.6. What to do Next . . . . . . . . . . . . . . . . . . . . . . . . 121

14. Appendix B: Glossary . . . . . . . . . . . . . . . . . . . . . . . . 12215. Appendix D: Why is Aegis Set-Uid-Root? . . . . . . . . . . . . . . . . . . 124

15.1. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 12415.2. Source Details. . . . . . . . . . . . . . . . . . . . . . . . . 125

16. Appendix I: Internationalization and Localization. . . . . . . . . . . . . . . . 12616.1. The “.po” Files. . . . . . . . . . . . . . . . . . . . . . . . . 12616.2. Checking the Code . . . . . . . . . . . . . . . . . . . . . . . 12616.3. Translators Welcome . . . . . . . . . . . . . . . . . . . . . . . 126

Page mvi () Peter Miller

Aegis UserGuide

1. Intr oduction

Aegis is a CASE tool with a difference. Inthespirit of theUNIX® Operating System, Aegis is asmall component designed to work with otherprograms.

Many CASE systems attempt to provide every-thing, from bubble charts to source control tocompilers. Usersare trapped with the compo-nents supplied by the CASE system, and if youdon’t like one of the components (it may be toolimited, for instance), then that is just tough.

In contrast,UNIX provides many components of aCASE system - compilers, editors, dependencytools (such as make), source control (such asSCCS). You may substitute the tool of yourchoice - gcc, jove, cake, rcs (to name a few) if youdon’t like the ones supplied with the system.

Aegis adds to this list with software configurationmanagement (SCM), and consistent withUNIX

philosophy, Aegis does not dictate the choice ofany of the other tools (although it may stretchthem to their limits).

1.1. Year 2000 Status

Aegis does not suffer from Year 2000 problems.

• Aegis stores dates internally in Unix style (i.e.seconds offset), so internal storage of times anddates does not suffer from any Y2K problems.

• Aegis always uses the ANSI C standardstrf-time function to display times and dates.(Thisassumes that your vendor has supplied a compli-ant strftime .) This means that displayingdates does not assume fixed field widths, nor willit display the year 2000 as “100”.

• There is no user-input of years at any time, sothere is no issue surrounding “guessing” the cen-tury.

1.2. Whatdoes aegis do?

Just what is software configuration management?This question is sufficiently broad as to require abook in answer. In essence, the aegis program isa project change supervisor. It provides a frame-work within which a team of developers maywork on many changes to a program indepen-dently, and the aegis program coordinates inte-grating these changes back into the master sourceof the program, with as little disruption as possi-ble. Resolutionof contention for source files, amajor headache for any project with more thanone developer, is one of the aegis program’s majorfunctions.

It should be noted that the aegis program is adeveloper’s tool, in the same sense as make orRCS are developer’s tools. It is not a manager’stool - it does not provide progress tracking or helpwith work allocation.

1.3. Why use aegis?

So why should you use the aegis program?Theaegis program uses a particular model of thedevelopment of software projects. This modelhas a master source (or baseline) of a project, con-sisting of several (possibly several hundred) files,and a team of developers creating changes to bemade to this baseline. When a change is com-plete, it is integrated with the baseline, to becomethe new baseline. Eachchange must be atomicand self-contained, no change is allowed to causethe baseline to cease to work. "Working" isdefined as passing its own tests. The tests areconsidered part of the baseline.Aegis providessupport for the developer so that an entire copy ofthe baseline need not be taken to change a fewfi les, only those files which are to be changedneed to be copied.

The win in using the aegis program is that thereareO(n) interactions between developers and thebaseline. Contrastthis with a master sourcewhich is being edited directly by the developers -there isO(n!) interactions between developers -this makes adding "just one" more developer apotential disaster.

Another win is that the project baseline alwaysworks. Always having a working baseline meansthat a version is always available for demonstra-tions, or those "pre-release snapshots" we arealways forced to provide.

The above advantages are all very well - for man-agement types.Why should Joe Average Pro-grammer use the aegis program? Recall that RCSprovides file locking, but only for one file at atime. Theaegis program provides the file lock-ing, atomically, for the set of files in the change.Recall also that correct RCS usage locks the filethe instant you start editing it. This makes popu-lar files a project bottleneck.The aegis programallows concurrent editing, and a resolution mech-anism just before the change must be integrated,meaning fewer delays for J.A.Programmer.

Peter Miller (bl/lib/en/user-guide/c1.4.so) Page 3

User Guide Aegis

1.4. How to use this manual

This manual assumes the reader is already famil-iar with the UNIX operating system, and withdeveloping software using theUNIX operating sys-tem and the tools available; terms such asRCSandSCCSandmake(1) are not explained.

There is also the assumption that the reader isfamiliar with the issues surrounding team devel-opment of software; coordination and multipleversion issues, for example, are not explained.

This manual is broken into a number of sections.

Chapter 2describes how aegis works and some of thereasoning behind the design and implemen-tation of the aegis program. Look here foranswers to "Why does it..." questions.

Chapter 3is a worked example of how particular usersinteract with the aegis program. Look herefor answers to "How do I..." questions.

Chapter 4is a discussion of how aegis interacts withthe History Tool, and provides templatesand suggestions for history tools known towork with aegis.

Chapter 5is a discussion of how aegis interacts withthe Dependency Maintenance Tool (DMT),and provides templates and suggestions forDMTs known to work with aegis.

Chapter 6is a discussion of how aegis interacts withthe Difference Tools, and provides tem-plates and suggestions for difference toolsknown to work with aegis.

Chapter 7describes the project attributes and how thevarious parameters may be used for particu-lar projects.

Chapter 8describes managing tests and testing withAegis.

Chapter 9describes the branching mechanism used inAegis.

Chapter 10is a collection of helpful hints on how touse aegis effectively, based on real-worldexperience. Thisis of most use when ini-tially placing projects under the supervisionof the aegis program.

Chapter 11describes how to manage geographicallydistributed development using Aegis.

Appendix Ais a quick reference for placing an existingproject under aegis.

Appendix Bis a glossary of terms.

Appendix Dis a description of why Aegis must be set-uid-root, for system administrators who areconcerned about the security issues.

Appendix Iis a brief look at internationalization andlocalization if Aegis.

1.5. GNUGPL

Aegis is distributed under the terms and condi-tions of the GNU General Public License.Pro-grams which are developed using Aegis are notautomatically subject to the GNU GPL.Onlyprograms which are derivative works based onGNU GPL code are automatically subject to theGNU GPL. We still encourage software authorsto distribute their work under terms like those ofthe GNU GPL, but doing so is not required to useAegis.


Aegis UserGuide

2. How Aegis Works

Before you will be able to exploit Aegis fully, youwill need to know what Aegis does and why.

The Aegis program provides a change controlmechanism and a repository, a subset of the func-tionality which CASE vendors call Software Con-figuration Management (SCM). In order to fitinto a software engineering environment, or anyplace software iswritten, Aegis needs a clearlydefined place in the scheme of things.

This chapter describes the model of the softwaredevelopment process embodied in the Aegis pro-gram, some of the deliberate design decisionsmade for Aegis, some of the things Aegis will andwont do for you, and the situations where Aegis ismost and least useful.

2.1. TheModel

The model of the software development processused by Aegis evolved and grew with time in acommercial software development environment,and it has continued to be used and developed.

The model described here is generic, and can beadapted in a variety of ways. Thesewill bedescribed at the relevant points in the text.

2.1.1. TheBaseline

Most CASE systems revolve around a repository:a place wherestuff is kept. Thisstuff is the rawmaterial that is processed in some way to producethe final product, whatever that may be.This stuffis the preferred form for editing or composing orwhatever.

In the Aegis program, the repository is known asthe baselineand the units ofstuff are UNIX fi les.The Aegis program makes no distinction betweentext and binary files, so both are supported.

The history mechanism which must be includedin any repository function is not provided by theAegis program. It is instead provided by someother per-project configurable software, such asRCS. Thismeans that the user may select the his-tory tool most suited to any giv en project. It alsomeans that Aegis is that much smaller to test andmaintain.

The structure of the baseline is dictated by thenature of each project. The Aegis programattempts to make as few arbitrary rules as possi-ble. There is one mandatory file in the yourproject baseline. The mandatory file is calledaegis.confby default, and contains the per-projectconfiguration information.The name of this file

may be changed if you want to call it somethingdifferent. Itios also common (though not manda-tory, and the name may be changed) to have adirectory calledtestwhich contains all of the testscripts. Thecontents and structure of thetestdirectory (or whatever you call it) are controlledby a test filename pattern you supply to Aegis.Tests are treated just like any other source file,and are subject to the same process.

The baseline in Aegis has one particular attribute:it always works. It is always there to show off tovisiting big-wigs, it is always there to grab a copyof and ship a "pre-release snapshot" to someoverly anxious customer, it is always there to letupper management "touch and feel" the progressbeing made towards the next release.

You may claim that "works" is comfortably fuzzy,but it is not. Thebaseline contains not only thesource of a project, but also the tests for a project.Tests are treated just like any other source file,and are subject to the same process.A baseline isdefined to "work" if and only if it passes all of itsown tests. TheAegis program has mandatorytesting, to ensure that all changes to the baselineare accompanied by tests, and that those testshave been run and are known to pass.This meansthat no change to the baseline may result in thebaseline ceasing to work1.

The model may be summarized briefly: it consistsof abaseline(master source), updated through theagency of an integrator, who is in turn fedchangesby a team ofdevelopers. These termswill be explained in the following sections.Seefigure 1 for a picture of how files flow around thesystem.

The baseline is a set of files including the sourcefi les for a projects, and also all derived files (suchas generated code, binary files from the compiler,etc), and all of the tests.Tests are treated just likeany other source file, and are subject to the sameprocess. Allfi les in the baseline are consistentwith each other.

Thus the baseline may be considered to be theclosureof the source files, in mathematical terms.That is, it is the source files and all implicationsflowing from those source files, such as objectfi les and executables. Allfi les in the baseline areconsistent with each other; this means that

1 Well, mostly. It is possible for this restrictionto be relaxed if you feel there are special circum-stances for a particular change.The danger is that achange will be integrated with the baseline whenthat change is not actually of acceptable quality.


User Guide Aegis

baseline

developmentdirectory

integrator

integratebegin

integrationdirectory

integratepass

developmentdirectory

Figure 1: Flow of Files through the Model

development builds can take object files from thebaseline rather than rebuild them within the devel-opment directory.

The baseline is readable by all staff, and usuallywritable by no-one.When it is necessary to writeto the baseline, this is done by Aegis, as will beshown below.

In many ways, the baseline may be thought of asa database, and all derived files are projections(views) of the source files. Passing its own testsmay be thought of as input validation of fields.This is a powerful concept, and indeed the imple-mentation of the Aegis program performs manyof the locking and synchronization tasksdemanded of a database engine.

All of the files forming this database are text files.This means that they may be repaired with anordinary text editor, should remedial action benecessary. The format is documented in section 5of the reference manual. Should you wish to per-form some query not yet available in Aegis, thefi les are readily accessible to members of the

appropriateUNIX group.

Tests are treated just like any other source file,and are subject to the same process.

2.1.2. TheChange Mechanism

Any changes to the baseline are made by atomicincrements, known (unoriginally) as "changes".A change is a collection of files to be added to,modified in, or deleted from, the baseline.Thesefi les must all be so altered simultaneously for thebaseline to continue to "work".2

For example, if the calling interface to a functionwere changed in one file, all calls to that functionin any other file must also change for the baselineto continue to work. All of the files must bechanged simultaneously, and thus must all beincluded in the one change.Other files whichwould logically be included in such an changeinclude the reference manual entry for the func-tion, the design document relating to that area offunctionality, the relevant user documentation,tests would have to be included for the functional-ity, and existing tests may need to be revised.

Changes must be accompanied by tests.Thesetests will either establish that a bug has been fixed(in the case of a bug fix) or will establish that newfunctionality works (in the case of an enhance-ment).

Tests are shell scripts, and as such are capable oftesting anything which has functionality accessi-ble from the command line.The ability to runbackground processes allows even client-servermodels to be tested.Tests are thus text files, andare treated as source files; they may be modifiedby the same process as any other source file.Tests usually need to be revised as a project growsand adapts to changing requirements, or to beextended as functionality is extended. Tests canev en be deleted if the functionality they test hasbeen deleted; tests are deleted by the same pro-cess as any other source file.

2.1.3. ChangeStates

As a change is developed using Aegis, it passesthrough six states.Many Aegis commands relateto transitions between these states, and Aegis per-forms any validation at these times.

2 Whether to allow sev eral logically independentchanges to be included in the one change is a policydecision for individual projects to make, and is notdictated by the Aegis program. It is a responsibilityof reviewers to ensure that all new and changedfunctionality is tested and documented.


Aegis UserGuide

The six states of a change are described as fol-lows, although the various state transitions, andtheir conditions, will be described later.

2.1.3.1. Awaiting Development

A change is in this state after it has been created,but before it has been assigned to a developer.This state can’t be skipped: a change can’t beimmediately assigned to a developer by an admin-istrator, because this disempowers the staff.

The Aegis program is not a progress tracking tool,nor is it a work scheduling tool; plenty of bothalready exist.

2.1.3.2. BeingDeveloped

A change is in this state after it has been assignedto a developer, by the developer. This is the coalface: all development is carried out in this state.Files can be edited in no other state, this particu-larly means that only developers can develop,reviewers and integrators only have the power toveto a change.Staff roles will be described morefully in a later section.

To advance to the next state, the change mustbuild successfully, it must have tests, and it mustpass those tests.3

The new tests must alsofail against the baseline;this is to establish that tests for bug-fixes actuallyreproduce the bug and then demonstrate that it isgone. New functionality added by a change willnaturally fail when tested in the old baseline,because it is not there.

When these conditions are met, the Aegis pro-gram marks all of the changes files as locked,simultaneously. If any one of them is alreadylocked, you can’t leave thebeing developedstate,because the file is part of a change which is some-where betweenbeing reviewed and being inte-grated.

If any one of them is out-of-date with respect tothe baseline, the lock is not taken, either. Lockingthe files at this state transition means that popularfi les may be modified simultaneously in manychanges, but that only differences to the latest ver-sion are ever submitted for integration. TheAegisprogram provides a mechanism, described later,for bringing out-of-date files in changes up-to-

3 It is possible for these testing requirements tobe waived on either a per-project or per-changebasis. How is described in a later section.Thepower to waive this requirement is not automati-cally granted to developers, as experience hasshown that it is usually abused.

date without losing the edits made by the devel-oper.

2.1.3.3. Awaiting Review

The default configuration for an Aegis projectdoes not use this state, because for small-ishprojects it can be tedious.For larger projects,however, it assists in coordinating reviewers whenyou use email notification that a review isrequired to several potential reviewers.

To enable this state, you need to change thedevelop_end_actionfield of the project attributes.Seeaepa(1) for more information, ortkaepa(1)for a GUI interface.

It is also possible, by a different setting of thesame project attribute, to skip the code reviewstep altogether. This can be of benefit to one-per-son projects where independent code reviewwould be impossible.

The rest of this description will assume theawait-ing review state is not being used, but codereviewsare being used, to simplify matters.Onceyou are more familiar with Aegis, enabling theuse of theawaiting review state will be simpleand will behave intuitively.

2.1.3.4. BeingReviewed

A change is in this state after a developer hasindicated that development is complete.Thechange is inspected, usually by a second party (orparties), to ensure that it matches the changedescription as to what it is meant to be doing, andmeets other project or company standards youmay have.

The style of review, and who may review, is notdictated by the Aegis program. A number ofalternative hav ebeen observed:

• You may have a single person who coordinatesreview panels of, say, 4 peers, with this coordina-tor the only person allowed to sign-off reviewpasses or fails.

• You may allow any of the developers to reviewany other developer’s changes.

• You may require that only senior staff, familiarwith large portions of the code, be allowed toreview.

The Aegis program enforces that a developer maynot review their own code.This ensures that atleast one person other than the developer hasscrutinized the code, and eliminates a rather obvi-ous conflict of interest.It is possible to turn this


User Guide Aegis

requirement off on a per-project basis, but this isonly desirable for projects with a one person team(or maybe two). TheAegis program has no wayof knowing that the user passing a review hasactually looked at, and understood, the code.

The reviewer knows certain things about a changefor it to reach this state: it has passed all of theconditions required to reach this state.Thechange compiles, it has tests and it passes thosetests, and the changes are to the current version ofthe baseline. The reviewer may thus concentrateon issues of completeness, implementation, andstandards - to name only a few.

2.1.3.4.1. CustomizingCode Review Policy

It is possible to require more than one reviewerfor a change. By setting there view_policy_-commandof the project configuration file, youcan pass a shell script (or other command) the rel-evant change details, and the exit status will beused to determine of the change advances to theawaiting integration state, or requires additionalcode reviewers first.

Because it is a program, it is possible to imple-ment almost any policy you can think of, includ-ing particular reviewers for particular areas ofcode, or that there must be 3 different reviewers,etc.

2.1.3.5. Awaiting Integration

A change is in this state after a reviewer has indi-cated that a change is acceptable to thereviewer(s). Thisis essentially a queue, as theremay be many dev elopers, but only one integrationmay proceed at any one time.

The issue of one integration at a time is a philo-sophical one: all of the changes in the queue arephysically independent; because of theDevelopEnd locking rules they do not have intersectingsets of files. The problem comes when onechange would break another, in these cases theintegrator needs to know which to bounce andwhich to accept.Integrating one change at a timegreatly simplifies this, and enforces the "onlychange one thing at a time" maxim, occasionallyat the expense of integrator throughput.

2.1.3.6. BeingIntegrated

A change is in this state when the integration ofthe change back into the baseline is commenced.A (logical) copy of the baseline is taken, and thechange is applied to that copy. In this state, thechange is compiled and tested once again.

The additional compilation has two purposes: itensures that the successful compile performed bythe developer was not a fluke of the developer’senvironment, and it also allows the baseline to bethe closure of the sources files. Thatis, all of theimplications flowing from the source files, suchas object files and linked programs or libraries.Itis not possible for Aegis to know which files theseare in the development directory, because Aegis isdecoupled from the build mechanism (this willdiscussed later).

To advance to the next state, the integration copymust have been compiled, and the tests includedin the change must have been run and passed.

The integrator also has the power of veto. Achange may fail an integration because it fails tobuild or fails tests, and also just because the inte-grator says so.This allows thebeing integratedstate to be another review state, if desired.Thebeing integratedstate is also the place to monitorthe quality of reviews and reviewers.

Should a faulty change manage to reach thispoint, it is to be hoped that the integration pro-cess, and the integrator’s sharp eyes, will detect it.

While most of this task is automated, this step isnecessary to ensure that some strange quirk of thedeveloper’s environment was not responsible forthe change reaching this stage.The change isbuilt once more, and tested once more. If achange fails to build or test, it is returned to thedeveloper for further work; the integrator mayalso choose to fail it for other reasons. If the inte-grator passes that change, the integrated versionbecomes the new baseline.

2.1.3.7. Completed

A change reaches this state when integration iscomplete. The(logical) copy of the baseline usedduring integration has replaced the previous copyof the baseline, and the file histories have beenupdated. Oncein this state, a change may neverleave it, unlike all other states.

If you wish to remove a change which is in thisstate from the baseline, you will have to submitanother change.

2.1.4. TheSoftware Engineers

The model of software development used byAegis has four different roles for software engi-neers to fill. Thesefour roles may be overlappingsets of people, or be distinct, as appropriate foryour project.


Aegis UserGuide

2.1.4.1. Developer

This is the coal-face. Thisrole is where almostev erything is done. This is the only role allowedto edit a source file of a project.

Most staff will be developers. Thereis nothingstopping a developer from also being an adminis-trator, except for the possible conflict of interestswith respect to testing exemptions.

A dev eloper may edit many of the attributes of achange while it is being developed. This ismostly useful to update the description of thechange to say why it was done and what was actu-ally done. A dev eloper may not grant testingexemptions (but they may be relinquished).

2.1.4.2. Reviewer

The role of the reviewer is to check a developer’swork. This review may consist of a peer examin-ing the code, or it may be handled by a singlemember of staff setting up and scheduling multi-person review panels. TheAegis program doesnot mandate what style of review, it only requiresthat a reviewer pass or fail each change. If itpasses, an integrator will handle it next, otherwiseit is returned to the developer for further work.

In a large team, the reviewers are usually selectedfrom the more senior members of the team,because of their depth of experience at spottingproblems, but also because this is an opportunityfor more senior members of staff to coach juniorson the finer points of the art.

The Aegis programs makes some of thereviewer’s task easier, because the reviewerknows several specific things about a changebefore it comes up for review: it builds, it hastests, and they hav e run successfully. There isalso optional (per project) additional conditionsimposed at the end of development, such as linelength limits, or anything else which is automati-cally testable.The Aegis program also provides adifference listing to the reviewer, so that each andev ery edit, to each and every file, can be pointedout to the reviewer.

There is nothing stopping a reviewer from beingeither an administrator or a developer. The Aegisprogram specifically prevents a developer fromreviewing his own work, to avoid conflicts ofinterest. (It is possible for this restriction to bewaiv ed, but that only makes sense for one personprojects.)

It will occasionally be necessary to arbitratebetween a developer and a reviewer. The

appropriate person to do this would have lineresponsibility above both staff inv olved. Thusitis desirable that supervisors/managers not bereviewers, except in very small teams.

2.1.4.3. Integrator

The role of the integrator is to take a changewhich has already been reviewed and integrate itwith the baseline, to form a new baseline. Theintegrator is thus the last line of defense for thebaseline.

There is nothing preventing an integrator frombeing an administrator, a dev eloper or a reviewer.The Aegis program specifically prevents a devel-oper or reviewer from integrating his own work,eliminating any conflict of interests.(It is possi-ble for this restriction to be waived, but that onlymakes sense for one and two person projects.)

It will occasionally be necessary to arbitratebetween an integrator and a reviewer and/or adeveloper. The appropriate person to do thiswould have line responsibility above all of thestaff inv olved. Thusit is desirable that supervi-sors/mangers not be integrators, except in verysmall teams.

The baseline is readable by all developers, but notwritable. All updates of the baseline to reflectchanges produced by developers are performedthrough the agency of the integrator.

2.1.4.4. Administrator

The project administrator has the followingduties:

• Create new changes. Thesemay be the result ofsome customer bug reporting mechanism, it maybe the result of new functionality being requested.

• Grant testing exemptions. By default, Aegisinsists that all changes be accompanied by tests.The project administrator may grant case-by-caseexemptions, or a project-wide exemption.

• Add or remove staff. Thefour roles described inthis section may be assigned to, or removed from,specificUNIX logins by the project administrator.

• Edit project attributes. There are manyattributes attached to a project, only a projectadministrator may alter them.

• Edit change attributes. There are manyattributes attached to a change, only a projectadministrator may alter all of them.

A project usually has only one or two administra-tors at any one time.


User Guide Aegis

2.1.5. TheChange Process

This section will examine the progression of achange through the six change states.Most of theattention will be given to the conditions whichmust be met in order to progress from one state tothe next, as this is where the software develop-ment model employed by Aegis is most oftenexpressed.

See figure 2 for a picture of how all of the statesand transitions fit together.

2.1.5.1. NewChange

A project administrator creates a change.Thischange will consist mostly of a description at thistime. The project administrator is not able(through Aegis) to assign it to a specific devel-oper.

The change is awaiting development; it is in theaw aiting development state.

2.1.5.2. NewChange Undo

It is possible to abandon a change if it is in theawaiting developmentstate. All record of thechange, including its description, will be deleted.

It is called new change undo to emphasize thestate it must be in to delete it.

2.1.5.3. Develop Begin

A dev eloper, for whatever reason, scans the list ofchanges awaiting development. Having selected achange, the developer then assigns that change toherself.

The change is now being developed; it is in thebeing developed state.

A number of Aegis commands only work in thisstate, including commands to include files andtests in the change (be they new files to be addedto the baseline, files in the baseline to be modi-fied, or files to be deleted from the baseline),commands to build the change, commands to testthe change, and commands to difference thechange.

The process of taking sources files, the preferredform for editing of a project, and transformingthem, through various manipulations and transla-tions, into a "finished" product is known as build-ing. In the UNIX world this usually means thingslike compiling and linking a program, but asfancy graphical programs become more wide-spread, the source files could be the binary outputfrom a graphical Entity-Relationship-Diagram

newchange

aw aitingdevelopment

developbegin

beingdeveloped

developend

beingreviewed

reviewpass

aw aitingintegration

integratebegin

beingintegrated

integratepass

completed

newchangeundo

developbeginundo

developendundo

developendundo

integratebeginundo

reviewfail

reviewpass

undo

integratefail

Figure 2: Change States and Transitions

editor, which would then be run through adatabase schema generator.

The process of testing a change has three aspects.The most intuitive is that a test must be run todetermine of the functionality works. Thesecondrequirement is that the test be run against thebaseline and fail; this is to ensure that bugs arenot just fixed, but reproduced as well. The thirdrequirement is optional: all or some of the testsalready in the baseline may also be run.Tests


Aegis UserGuide

consist ofUNIX shell scripts - anything that can bedone in a shell script can be tested.

In preparation for review, a change is differenced.This usually consists of automatically comparingthe present contents of the baseline with what thechange proposes to do to the baseline, on a file-by-file basis. The results of the difference, suchasUNIX diff -c output, is kept in a difference file,for examination by the reviewer(s). Thebenefitof this procedure is that reviewers may examinethese files to see every change the developermade, rather than only the obvious ones. The dif-ferencing commands are per-project configurable,and other validations, such as line length restric-tions, may also be imposed at this time.

To leave this state, the change must have sourcefi les, it must have tests, it must have built success-fully, it must have passed all its own tests, and itmust have been differenced.

2.1.5.4. Develop Begin Undo

It is possible to return a change from the beingdeveloped state to the awaiting development state.This is usually desired if a developer selected thewrong change by mistake. It also provides amethod to start over on a change for some otherreason.

2.1.5.5. Develop End

When the conditions for the end of developmenthave been met (the change must have source files,it must have tests, it must have built successfully,it must have passed all its own tests, and it musthave been differenced) the developer may causethe change to leave the being developed state andenter the being reviewed state. The Aegis pro-gram will check to see that all the conditions aremet at this time. There is no history kept ofunsuccessful develop end attempts.

Most of these preconditions are determined by theuse of time stamps which are recorded for variousoperations, in addition to file system timestampson the files themselves. Logicalsequencing (e.g.tests being run after building after editing) is alsoverified.

Note that there are 3 kinds of tests

1. If a change contains a new test or a testwhich is being modified, this test must passagainst the code compiled and linked in thechange. Thisis simply referred to as a“test”. Changesmay be granted an exemp-tion from such tests.

2. If a change contains a new test and thechange is a bug fix, this test mustfail againstthe old code in the baseline. This is to con-fi rm that the bug has been fixed. This isreferred to as a “baseline test”.Changesmay be granted an exemption from suchtests.

3. Tests which already exist in the baseline maybe run against the code compiled and linkedin the change.These tests must pass. This isto confirm that the project has not regressed,which is why these tests are referred to as“regression tests”. Changes may be grantedan exemption from such tests.

A successful develop end command results in thechange advancing from thebeing developedstateto thebeing reviewedstate. (Itis also possible toadvance to theawaiting review state or theawait-ing integration state. Seeaede(1) or aepattr(5)for more information.)

2.1.5.6. Develop End Undo

There are many times when a developer thinksthat a change is completed, and goes hunting for areviewer. Half way down the hall, she thinks ofsomething that should have been included.

It is possible for a developer to rescind aDevelopEnd to allow further work on a change.No rea-son need be given. Thisrequest may be issued toa change in either thebeing reviewedor awaitingintegrationstates.

2.1.5.7. Review Pass

This event is used to notify Aegis that the changehas been examined, by a method unspecified asdiscussed above, and has been found to be accept-able.

2.1.5.8. Review Pass Undo

The reviewer of a change may rescind aReviewPass while the change remains in theawaitingintegration state. Noreason need be supplied.The change will be returned to thebeing reviewedstate.

2.1.5.9. Review Fail

This event is used to notify Aegis that the changehas been examined, by a method unspecified asdiscussed above, and has been found to be unac-ceptable.

A file containing a brief summary of the problemsmust be given, and will be included in the


User Guide Aegis

change’s history.

The change will be returned to thebeing devel-opedstate for further work.

It is not the responsibility of any reviewer to fix adefective change.

2.1.5.10. IntegrateBegin

This command is used to commence integrationof a change into the project baseline.

Whether a physical copy of the baseline is used,or a logical copy using links, is controlled by theproject configuration file. The change is thenapplied to this copy.

The integrator must then issue build and test com-mands as appropriate. This is not automated assome integrator tasks may be required in andaround these commands.

2.1.5.11. IntegrateBegin Undo

This command is used to return a change to theintegration queue, without prejudice. No reasonneed be given.

This is usually done when a particularly importantchange is in the queue, and the current integrationis expected to take a long time.

2.1.5.12. IntegratePass

This command is used to notify Aegis that thechange being integrated is acceptable.

The current baseline is replaced with the integra-tion copy, and the history is updated.

2.1.5.13. IntegrateFail

This command is used to notify Aegis that anintegration is unacceptable, usually because itfailed to build or test in some way, or sometimesbecause the integrator found a deficiency.

A file containing abrief summary of the problemsmust be given, and the summary will be includedin the change’s history.

The change will be returned to thebeing devel-opedstate for further work. Theintegration copyof the baseline is deleted, leaving the originalbaseline unchanged.

It is not the responsibility of any integrator to fixa defective change, or even diagnose what thedefect may be.


Aegis UserGuide

2.2. Philosophy

The philosophy is simple, and that makes some ofthe implementation complex.

• When a change is in thebeing developedstate,the aegis program is a developer’s tool. Its pur-pose is to make it as easy for a developer todevelop changes as possible.

• When a change leaves (or attempts to leave) thebeing developedstate, the aegis program is pro-tecting the project baseline, and does not exist tomake the developer happy.

• The aegis program attempts to adhere to theUNIX minimalist philosophy. Least unnecessaryoutput, least command line length, least depen-dence onspecific3rd party tools.

• No overlap in functionality of cooperating tools.(I.e. no internal build mechanism, no internal his-tory mechanism, etc.)

2.2.1. Development

During the development of a change, the aegisprogram exists to help the developer. It helps himnavigate around his change and the project, itcopies file for him, and keeps track of the ver-sions. Itcan even tell him what changes he hasmade.

2.2.2. Post Development

When a change has left the "being developed"state, or when it is attempting to leave that state,the aegis program ceases to attempt to help thedeveloper and proceeds to defend the projectbaseline. Themodel used by aegis states that "thebaseline always works", and aegis attempts toguarantee this.

2.2.3. Minimalism

The idea of minimalism is to help the user out.Itis the intention that the aegis program can workout unstated command line options for itself, incases where it is "safe" to do so. This means anumber of defaulting mechanisms, all designed tohelp the user.

2.2.4. Overlap

It was very tempting while writing the aegis pro-gram to have it grow and cover source control anddependency maintenance roles.Unfortunately,this would have meant that the user would havebeen trapped with whatever the aegis programprovided, and the aegis program is already plentybig. To add this functionality would have div erted

effort, resulting in an inferior result. It would alsohave violated the underlyingUNIX philosophy.

2.2.5. DesignGoals

A number of specific ideas molded the shape ofthe aegis program. These include:

The UNIX philosophy of writing small tools forspecific tasks with little or no overlap. Toolsshould be written with the expectation of use inpipes or scripts, or other combinations.

• Stay out of the way. If it is possible to let aproject do whatever it l ikes, write the code to letit. It is not possible to anticipate even a fractionof the applications of a software tool.

• People. Thestaff using aegis should be incharge of the development process.They shouldnot feel that some machine is giving them orders.

• Users aren’t psychic. Feedbackmust be clear,accurate and appropriate.


User Guide Aegis

2.3. Security

Access to project data is controlled by theUNIX

group mechanism.The group may be selected assuitable for your project, as may the umask.

All work done by developers (build, difference,etc) is all with a default group of the project’sgroup, irrespective of the user’s default group.Directories (when BSD semantics are available)are all created so that their contents default to thecorrect group. This ensures that reviewers andintegrators are able to examine the change.

Other UNIX users not in the project’s group maybe excluded, or not, by the appropriate setting ofthe project umask. This umask is used by allAegis actions, assuring appropriate defaultbehaviour.

A second aspect of security is to ensure thatdevelopers are unable to deliberately deceiveAegis. Shouldthe files be tampered with at anylater date, Aegis will notice.

2.4. Scalability

How big can a project get before Aegis chokes?There are a huge number of variables in this ques-tion.

The most obvious bottleneck is the integrator. Anartificial "big project" example: Assume that theav erage integration takes an hour to build and test.A full-time integrator could be expected to get 7or 8 of these done per day (this was the observedav erage on one project the author was involvedin). Assumethat the average time for a developerto develop a change is two weeks; a figure recom-mended by many text books as "the most you canafford to throw away". Thesetwo assumptionsmean that for this "big project" Aegis can copewith 70 to 80 developers, before integrationsbecome a bottleneck.

The more serious bottle neck is the dependencymaintenance tool.Seventy developers can churnout a staggering volume of code.It takes a verylong time to wade through the file times and therules, just to find the one or two files whichchanged. Thiscan easily push the integrate buildtime past the one hour mark.Developers alsobecome very vocal when build times are this long.


Aegis UserGuide

2.5. When(not) to use Aegis

The aegis program is not a silver bullet; it will notsolve all of your problems.Aegis is suitable forsome kinds of projects, useful for others, and use-less for a few.

The software development process embodied byAegis has the following attributes:

• Each change set is applied atomically.

• Each change set must build successfully beforeit will be accepted. (This can be trivial, ifdesired.)

• Each change set must test successfully before itwill be accepted. (This can be disabled, ifdesired.)

• Each change set must pass a peer review beforeit will be accepted. (This can be a rubberstamp, if desired.)

The most difficult thing about Aegis program isthat it takes management buy-in. It takes effort toconvince many people that the model used byaegis has benefits, and you need managementbacking you up when some person comes alongwith a way of developing software "without theextra work" imposed by the model used by Aegis.

2.5.1. Building

If the source code to your software productdoesn’t build, it isn’t a product. However, manysoftware shops commit changes to their repositorywithout preconditions, and then do a daily build(or worse, weekly). The problem here is that"pollution" by defective changes is alreadyinyour productbefore it is detected.Aegis will notlet it be committed in the first place.

If your product is entirely composed of scripts orHTML, you can make the build step completelytrivial: "exit 0" is usually used for this purpose.Thus, this requirement, while usually highlydesirable, may be avoided if desired.

2.5.2. Testing

There is extra up-front work: writing tests. Thewin is that the tests hang around forever, catchingminor and major slips before they become embar-rassing "features" in a released product.Preven-tion is cheaper than cure in this case, the testssave work down the track. See thetestingchapterfor more information.

2.5.3. Reviewing

Code reviews of some sort are normal in mostsoftware houses. Often, unfortunately, time pres-sures or other political pressures mean that codereviews manage not to happen.Yet the literaturerepeatedly cites reviews as one of the most impor-tant factors in removing defects before they reachyour code repository. Aegis requires a codereview before it will commit code into your prod-uct; again, the idea is to remove defectsbeforethey pollute the product.


User Guide Aegis

2.6. Further Work

The Aegis program is far from finished. Anum-ber of features are known to be lacking.

At the date of this writing, Aegis is beingactively supported and improved.

2.6.1. CodeCoverage Tool

It would be very helpful if a code coverage toolcould be used to analyze tests included withchanges to ensure that the tests actually exercisedthe lines of code changed in the change.

Another use of the code coverage tool would be toselect regression tests based on the object filesrecompiled by a change, and those regressiontests which exercise those files.

While there is freeware C code coverage toolavailable, based on GNU C, the interfacing andsemantics still need more thought.

Note: A fairly good approximation is alreadyavailable using the--suggest option of theaet(1)command. Itworks on the correlation of sourcesfi le versus tests in the various change sets.Seeaet(1) for more information.

2.6.2. Virtual File System

There is almost sufficient information in theAegis data base to create a virtual file system,overlaying the development directory atop thebaseline4. This could be implemented similarly toautomounters, intercepting file system operationsby pretending to be an NFS server. Many com-mercial CASE products provide such a facility.

Such a virtual file system has a number of advan-tages: you don’t need such a capable DMT, forstarters; it only needs the dynamic include depen-dencies, and does not need a search path5. Sec-ond, many horrible and dumb compilers, notablyFORTRAN and "fourth" GLs, don’t hav e ade-quate include semantics; overlaying the two direc-tories make this much easier to deal with6. Manygraphical tools, such as bubble chart drawers, etc,when they do actually have include files, have nocommand line specifiable search path.

4 Reminiscent of Sun’s TFS, but not the same.Similar to AT&T’ s 3D-FS. Similar to TeamNet.Similar to ClearCase, but I wasn’t thinking of thetime-travel aspects which they implement.

5 Discussed in theDependency MaintenanceTool chapter.

6 There are other ways, discussed in theTips andTr apschapter.

The disadvantage is that this adds significantcomplexity to an already large program. Also,implementation is limited to NFS capable sys-tems, or would have to be rewritten for a varietyof other systems. The semantics of interactionsbetween the daemon and other Aegis commands,while clearly specifiable, are challenging toimplement. Performancecould also be a signifi-cant factor.

The question is "is it really necessary?" If the jobcan be done without it, does the effort of writingsuch a beast result in significant productivitygains?

The addition of thecreate_symlinks_before_buildfield to the project configuration file has greatlyreduced the need for this functionality. Howev er,it does not provide copy-on-write semantics, norautomaticaecpfunctionality; which a virtual filesystem could do.


Aegis UserGuide

3. TheChange Development Cycle

As a change to a project is developed using Aegis,it passes through several states. Each state ischaracterized by different quality requirements,different sets of applicable Aegis commands, anddifferent responsibilities for the people involved.

These people may be divided into four categories:developers, reviewers, integrators and administra-tors. Eachhas different responsibilities, dutiesand permissions; although one person may belongto more than one category, depending on how aproject is administered.

This chapter looks at each of these categories, byway of an example project undergoing its firstfour changes. This example will be examinedfrom the perspective of each category of people inthe following sections.

There are six hypothetical users in the example:the developers are Pat, Jan and Sam; the reviewersare Robyn and Jan; the integrator is Isa; and theadministrator is Alex7. There need not have beenthis many people involved, but it keeps thingsslightly cleaner for this example.

The project is called "example". It implements avery simple calculator. Many features importantto a quality product are missing, checking fordivide-by-zero for example. Thesehave beenomitted for brevity.

7 The names are deliberately gender-neutral.Finding such a name starting with "I" is not easy!


User Guide Aegis

3.1. TheDeveloper

The developer role is the coal face8. This is where new software is written, and bugs are fixed. Thisexam-ple shows only the addition of new functionality, but usually a change will include modifications of existingcode, similar to bug-fixing activity.

3.1.1. Before You Start

Have you configured your account to use Aegis? SeetheUser Setupsection of theTips and Trapschapterfor how to do this.

3.1.2. TheFirst Change

While the units of change, unoriginally, are called "changes", this also applies to the start of a project - achange to nothing, if you like. Thedeveloper of this first change will be Pat.

First, Pat has been told by the project administrator that the change has been created.How Alex createdthis change will be detailed in the "Administrator" section, later in this chapter. Pat then acquires thechange and starts work.

pat% aedb -l -p example.1.0Project "example.1.0"List of Changes

Change State Description------- ------- -------------

10 awaiting_ Create initial skeleton.development

pat% aedb example.1.0 10aegis: project "example.1.0": change 10: development directory "/u/pat/

example.1.0.C010"aegis: project "example.1.0": change 10: user "pat" has begun developmentpat% aecdaegis: project "example.1.0": change 10: /u/pat/example.1.0.C010pat%

At this point Aegis has created a development directory for the change and Pat has changed directory to thedevelopment directory9.

Five files will be created by this change.

pat% aenf aegis.conf Howto.cook gram.y lex.l main.caegis: project "example.1.0": change 10: file "Howto.cook" addedaegis: project "example.1.0": change 10: file "aegis.conf" addedaegis: project "example.1.0": change 10: file "gram.y" addedaegis: project "example.1.0": change 10: file "lex.l" addedaegis: project "example.1.0": change 10: file "main.c" addedpat%

The contents of theaegis.conffi le will not be described in this section, mostly because it is a rather com-plex subject; so complex it requires four chapters to describe: theHistory Tool chapter, the DependencyMaintenance Tool chapter, theDifference Toolschapter and theProject Attributeschapter. The contents ofthe Howto.cookfi le will not be described in this section, as it is covered in theDependency MaintenanceTool chapter.

The filemain.cwill have been created by Aegis as an empty file. Pat edits it to look like this

8 I thought this expression was fairly common English usage, until I had a query. "The Coal Face" is anexpression meaning "where thereal work is done" in reference to old-style coal mining which was hard, tiring,hot, very dangerous, and bad for your health even if you were lucky enough not to be killed. It was a 14-hourper day job, and you walked to and from work in the dark, even in summer. Unlike the mine owners, who rodeexpensive horses and saw sunlight most days of the week.

9 The default directory in which to place new dev elopment directories is configurable for each user.


Aegis UserGuide

#include <stdio.h>

static voidusage(){

fprintf(stderr, "usage: example\n");exit(1);

}

voidmain(argc, argv)

int argc;char **argv;

{if (argc != 1)

usage();yyparse();exit(0);

}

The file gram.y describes the grammar accepted by the calculator. This file was also created empty byAegis, and Pat edits it to look like this:

%token DOUBLE%token NAME

%union{

int lv_int;double lv_double;

}

%type <lv_double> DOUBLE expr%type <lv_int> NAME

%left ’+’ ’-’%left ’*’ ’/’%right UNARY

%%


User Guide Aegis

example: /* e mpty */| e xample command ’\n’

{ y yerrflag = 0; fflush(stderr); fflush(stdout); }command

: e xpr{ p rintf("%g\n", $1); }

| e rrorexpr

: D OUBLE{ $$ = $ 1; }

| ’ (’ expr ’)’{ $$ = $ 2; }

| ’ -’ expr%prec UNARY{ $$ = - $2; }

| e xpr ’*’ expr{ $$ = $1 * $ 3; }

| e xpr ’/’ expr{ $$ = $1 / $ 3; }

| e xpr ’+’ expr{ $$ = $1 + $ 3; }

| e xpr ’-’ expr{ $$ = $1 - $ 3; }

The file lex.l describes a simple lexical analyzer. It will be processed bylex(1) to produce C code imple-menting the lexical analyzer. This kind of simple lexer is usually handcrafted, but using lex allows theexample to be far smaller. Pat edits the file to look like this:

%{#include <math.h>#include <libaegis/gram.h>%}%%[ \ t]+ ;[0-9]+(\.[0-9]*)?([eE][+-]?[0-9]+)? {

yylval.lv_double = atof(yytext);return DOUBLE;

}[a-z] {

yylval.lv_int = yytext[0] - ’a’;return NAME;

}\n |. r eturn yytext[0];

Note how the gram.hfi le is included using the#include < filename> form. This is very importantfor builds in later changes, and is discussed more fully in theUsing Cooksection of theDependency Main-tenance Toolchapter.

The files are processed, compiled and linked together using theaebcommand; this is known asbuilding achange. Thisis done through Aegis so that Aegis can know the success or failure of the build. (Build suc-cess is a precondition for a change to leave the being developedstate.) Thebuild command is in theaegis.conffi le so vaguely described earlier. In this example it will use thecook(1) command which in turnwill use theHowto.cookfi le, also alluded to earlier. This file describes the commands and dependencies forthe various processing, compiling and linking.


Aegis UserGuide

pat% aebaegis: project "example.1.0": change 10: development build startedaegis: cook -b Howto.cook project=example.1.0 change=10

version=1.0.C010 -nlcook: yacc -d gram.ycook: mv y.tab.c gram.ccook: mv y.tab.h gram.hcook: cc -I. -I/projects/example/branch.1./branch0/baseline -O -c gram.ccook: lex lex.lcook: mv lex.yy.c lex.ccook: cc -I. -I/projects/example/branch.1/branch.0/baseline -O -c lex.ccook: cc -I. -I/projects/example/baseline -O -c main.ccook: cc -o example gram.o lex.o main.o -ll -lyaegis: project "example.1.0": change 10: development build completepat%

The example program is built, and Pat could even try it out:

pat% example1 + 23ˆDpat%

At this point the change is apparently finished. Thecommand to tell Aegis this is thedevelop endcom-mand:

pat% aedeaegis: project "example.1.0": change 10: no current ’aegis -DIFFerence’

registrationpat%

It didn’t work, because Aegis thinks you have missed the difference step.

The difference step is used to produce files useful for reviewing changes, mostly in the form of context dif-ference files between the project baseline and the development directory. Context differences allow review-ers to see exactly what has changed, and not have to try to track them down and inevitably miss obscure butimportant edits to large or complex files.

pat% aedaegis: set +e; diff -c /dev/null /u/pat/example.1.0.C010/Howto.cook >

/u/pat/example.1.0.C010/Howto.cook,D; test $? -eq 0 -o $? -eq 1aegis: set +e; diff -c /dev/null /u/pat/example.1.0.C010/aegis.conf >

/u/pat/example.1.0.C010/aegis.conf,D; test $? -eq 0 -o $? -eq 1aegis: set +e; diff -c /dev/null /u/pat/example.1.0.C010/gram.y >

/u/pat/example.1.0.C010/gram.y,D; test $? -eq 0 -o $? -eq 1aegis: set +e; diff -c /dev/null /u/pat/example.1.0.C010/lex.l >

/u/pat/example.1.0.C010/lex.l,D; test $? -eq 0 -o $? -eq 1aegis: set +e; diff -c /dev/null /u/pat/example.1.0.C010/main.c >

/u/pat/example.1.0.C010/main.c,D; test $? -eq 0 -o $? -eq 1aegis: project "example.1.0": change 10: difference completepat%

Doing a difference for a new file may appear a little pedantic, but when a change consists of tens of files, somodifications of existing files and some new, there is a temptation for reviewers to use "more *,D" and thuscompletely miss the new files if it were not for this pedanticism10.

So that reviewers, and conscientious developers, may locate and view all of these difference files, the com-mand

10 This is especially true when you use a tool such asfcomp(1) which gives a complete file listing with theinserts and deletes marked in the margin. Thistool is also available from the author of Aegis.


User Guide Aegis

pat% more ‘find . -name "*,D" -print | sort‘...examines each file...pat%

could be used, however this is a little too long winded for most users, and so theaedmorealias does exactlythis. Thereis a similaraedlessalias for those who prefer theless(1) command.

So now Pat is done, let’s try to sign off again:

pat% aedeaegis: project "example.1.0": change 10: no current ’aegis -Test’

registrationpat%

It didn’t work, again. Thistime Aegis is reminding Pat that every change must be accompanied by at leastone test. This is so that the project team can be confident at all times that a project works11. Making this aprecondition to leave thebeing developedstate means that a reviewer can be sure that a change builds andpasses its tests before it can ever be reviewed. Pat adds the truant test:

pat% aentaegis: project "example.1.0": change 10: file "test/00/t0001a.sh" new

testpat%

The test file is in a weird place, eh?This is because many flavors ofUNIX are slow at searching directories,and so Aegis limits itself to 100 tests per directory. Whatever the name, Pat edits the test file to look likethis:

#!/bin/sh## t est simple arithmetic#tmp=/tmp/$$here=‘pwd‘if [ $? -ne 0 ]; then exit 1; fi

fail(){

echo FAILED 1>&2cd $hererm -rf $tmpexit 1

}

pass(){

cd $hererm -rf $tmpexit 0

}trap "fail" 1 2 3 15

mkdir $tmpif [ $? -ne 0 ]; then exit 1; ficd $tmpif [ $? -ne 0 ]; then fail; fi

11 As discussed in theHow Aegis Works chapter, aegis has the objective of ensuring that projects alwayswork, where "works" is defined as passing all tests in the project’s baseline. Achange "works" if it passes all ofits accompanying tests.


Aegis UserGuide

## with input like this#cat > test.in << ’foobar’1(24 - 22)-(4 - 7)2 * 210 / 24 + 210 - 3foobarif [ $? -ne 0 ]; then fail; fi

## t he output should look like this#cat > test.ok << ’foobar’1234567foobarif [ $? -ne 0 ]; then fail; fi

## r un the calculator# and see if the results match#$here/example < test.in > test.outif [ $? -ne 0 ]; then fail; fidiff test.ok test.outif [ $? -ne 0 ]; then fail; fi

## t his much worked#pass

There are several things to notice about this test file:

• It is a Bourne shell script. All test files are Bourne shell scripts because they are the most portable.12

(Actually, Aegis likes test files not to be executable, it passes them to the Bourne shell explicitlywhen running them.)

• It makes the assumption that the current directory is either the development directory or the baseline.This is valid, aegis always runs tests this way; if you run one manually, you must take care of thisyourself.

• It checks the exit status of each and every command.It is essential that even unexpected and impossiblefailures are handled.

• A temporary directory is created for temporary files. It cannot be assumed that a test will be run from adirectory which is writable; it is also easier to clean up after strange errors, since you need only throwthe directory away, rather than track down individual temporary files. It mostly protects againstrogue programs scrambling files in the current directory, too.

12 Portable for Aegis’ point of view: Bourne shell is the most widely available shell. Of course, if you arewriting code to publish on USENET or for FTP, portability of the tests will be important from the developer’spoint of view also.


User Guide Aegis

• Every test is self-contained. The test uses auxiliary files, but they are not separate source files (figuringwhere they are when some are in a change and some are in the baseline can be a nightmare).If a testwants an auxiliary file, it must construct the file itself, in a temporary directory.

• Two functions have been defined, one for success and one for failure. Bothforms remove the temporarydirectory. A test is defined as passing if it returns a 0 exit status, and failing if it returns anythingelse.

• Tests are treated just like any other source file, and are subject to the same process.They may be alteredin another change, or even deleted later if they are no longer useful.

The most important feature to note about this test, after ignoring all of the trappings, is that it doesn’t domuch you wouldn’t do manually! To test this program manually you would fire it up, just as the test does,you would give it some input, just as the test does, and you would compare the output against your expecta-tions of what it will do, just as the test does.

The difference with using this test script and doing it manually is that most development contains manyiterations of the "build, test,think, edit, build, test..." cycle. Aftera couple of iterations, the manual testing,the constant re-typing, becomes obviously unergonomic. Usinga shell script is more efficient, doesn’t for-get to test things later, and is preserved for posterity (i.e. adds to the regression test suite).

This efficiency is especially evident when using commands13 such as

pat% aeb && aet ; vi aegis.log...pat% !aeb...pat%

It is possible to talk to the shell extremely rarely, and then only to re-issue the same command, using a workpattern such as this.

As you have already guessed, Pat now runs the test like this:

pat% aetaegis: sh /u/pat/example.1.0.C010/test/00/t0001a.shaegis: project "example.1.0": change 10: test "test/00/t0001a.sh"

passedaegis: project "example.1.0": change 10: passed 1 testpat%

Finally, Pat has built the change, prepared it for review and tested it. It is now ready for sign off.

pat% aedeaegis: project "example.1.0": change 10: no current ’aegis -Build’

registrationpat%

Say what? The problem is that the use ofaentcanceled the previous build registration. Thiswas becauseAegis is decoupled from the dependency maintenance tool (cook in this case), and thus has no way ofknowing whether or not the new file in the change would affect the success or failure of a build14. All thatis required is to re-build, re-test, re-difference (yes, the test gets differenced, too) and sign off.

13 This is acshspecific example, unlike most others.14 Example: in addition to the executable file "example" shown here, the build may also produce an archive

fi le of the project’s source for export. Theaddition of one more file may push the size of this archive beyond asize limit; the build would thus fail because of the addition of a test.


Aegis UserGuide

pat% aebaegis: logging to "/u/pat/example.1.0.C010/aegis.log"aegis: project "example.1.0": change 10: development build startedaegis: cook -b Howto.cook project=example.1.0 change=10

version=1.0.C001 -nlcook: "all" is up-to-dateaegis: project "example.1.0": change 10: development build completepat% aetaegis: logging to "/u/pat/example.1.0.C010/aegis.log"aegis: sh /u/pat/example..1.0.C010/test/00/t0001a.shaegis: project "example.1.0": change 10: test "test/00/t0001a.sh"

passedaegis: project "example.1.0": change 10: passed 1 testpat% aedaegis: logging to "/u/pat/example.1.0.C010/aegis.log"aegis: set +e; diff -c /dev/null /u/pat/example.1.0.C010/test/00/

t0001a.sh > /u/pat/example.1.0.C010/test/00/t0001a.sh,D; test$? -eq 0 -o $? -eq 1

aegis: project "example.1.0": change 10: difference completepat% aedeaegis: sh /usr/local/lib/aegis/de.sh example.1.0 10 pataegis: project "example.1.0": change 10: development completedpat%

The change is now ready to be reviewed. Thissection is about developers, so we will have to leave thischange at this point in its history. Some time in the next day or so Pat receives electronic mail that thischange has passed review, and another later to say that it passed integration. Pat is now free to developanother change, possibly for a different project.

3.1.3. TheSecond Change

The second change was created because someone wanted to name input and output files on the commandline, and called the absence of this feature a bug. WhenJan arrived for work, and lists the changes awaitingdevelopment, the following list appeared:

jan% aedb -l -p example.1.0Project "example.1.0"List of Changes

Change State Description------ ------ ------------

11 awaiting_ Add input and output file names to thedevelopment command line.

12 awaiting_ add variablesdevelopment

13 awaiting_ add powersdevelopment

jan%

The first on the list is chosen.

jan% aedb -c 11 -p example.1.0aegis: project "example.1.0": change 11: development directory "/u/

jan/example.1.0.C011"aegis: project "example.1.0": change 11: user "jan" has begun

developmentjan% aecdaegis: project "example.1.0": change 11: /u/jan/example.002jan%

The best way to get details about a change is to used the "change details" listing.


User Guide Aegis

jan% ael cdProject "example.1.0", Change 11Change Details

NAMEProject "example.1.0", Change 11.

SUMMARYfile names on command line

DESCRIPTIONOptional input and output files may be specified on thecommand line.

CAUSEThis change was caused by internal_bug.

STATEThis change is in ’being_developed’ state.

FILESChange has no files.

HISTORYWhat When Who Comment------ ------ ----- ---------new_change Fri Dec 11 alex

14:55:06 1992develop_begin Mon Dec 14 jan

09:07:08 1992jan%

Through one process or another, Jan determines that themain.cfi le is the one to be modified. Thisfi le iscopied into the change:

jan% aecp main.caegis: project "example.1.0": change 11: file "main.c" copiedjan%

This file is now extended to look like this:

#include <stdio.h>

static voidusage(){

fprintf(stderr, "usage: example [ <infile> [ <outfile> ]]\n");exit(1);

}

voidmain(argc, argv)

int argc;char **argv;

{char *in = 0;char *out = 0;int j;


Aegis UserGuide

for (j = 1; j < argc; ++j){

char *arg = argv[j];if (arg[0] == ’-’)

usage();if (!in)

in = arg;else if (!out)

out = arg;else

usage();}

if (in && !freopen(in, "r", stdin)){

perror(in);exit(1);

}if (out && !freopen(out, "w", stdout)){

perror(out);exit(1);

}

yyparse();exit(0);

}

A new test is also required,

jan% aentaegis: project "example.1.0": change 11: file "test/00/t0002a.sh" new

testjan%

which is edited to look like this:

#!/bin/sh## t est command line arguments#tmp=/tmp/$$here=‘pwd‘if [ $? -ne 0 ]; then exit 1; fi

fail(){

echo FAILED 1>&2cd $hererm -rf $tmpexit 1

}

pass(){

cd $hererm -rf $tmpexit 0

}trap "fail" 1 2 3 15


User Guide Aegis

mkdir $tmpif [ $? -ne 0 ]; then exit 1; ficd $tmpif [ $? -ne 0 ]; then fail; fi

## with input like this#cat > test.in << ’foobar’1(24 - 22)-(4 - 7)2 * 210 / 24 + 210 - 3foobarif [ $? -ne 0 ]; then fail; fi

## t he output should look like this#cat > test.ok << ’foobar’1234567foobarif [ $? -ne 0 ]; then fail; fi

## r un the calculator# and see if the results match## ( Use /dev/null for input in case input redirect fails;# don’t want the test to hang!)#$here/example test.in test.out < /dev/nullif [ $? -ne 0 ]; then fail; fidiff test.ok test.outif [ $? -ne 0 ]; then fail; fi$here/example test.in < /dev/null > test.out.2if [ $? -ne 0 ]; then fail; fidiff test.ok test.out.2if [ $? -ne 0 ]; then fail; fi

## make sure complains about rubbish# on t he command line#$here/example -trash < test.in > test.outif [ $? -ne 1 ]; then fail; fi

## t his much worked#pass

Now it is time for Jan to build and test the change. Through the magic of static documentation, this worksfi rst time, and here is how it goes:


Aegis UserGuide

jan% aebaegis: logging to "/u/pat/example.1.0.C011/aegis.log"aegis: project "example.1.0": change 11: development build startedaegis: cook -b /projects/example/baseline/Howto.cook

project=example.1.0 change=11 version=1.0.C011 -nlcook: cc -I. -I/projects/example/baseline -O -c main.ccook: cc -o example main.o /projects/example/baseline/gram.o

/projects/example/baseline/lex.o -ll -lyaegis: project "example.1.0": change 11: development build completejan% aetaegis: logging to "/u/pat/example.1.0.C011/aegis.log"aegis: sh /u/jan/example.1.0.C011/test/00/t0002a.shaegis: project "example.1.0e": change 11: test "test/00/t0002a.sh"

passedaegis: project "example.1.0": change 11: passed 1 testjan%

All that remains if to difference the change and sign off.

jan% aedaegis: logging to "/u/pat/example.1.0.C011/aegis.log"aegis: set +e; diff -c /projects/example/main.c /u/jan/

example.1.0.C011/main.c > /u/jan/example.1.0.C011/main.c,D; test $?-eq 0 -o $? -eq 1

aegis: project "example.1.0": change 11: difference completejan% aedmore...examines the file...jan%

Note how the context difference shows exactly what has changed. And now the sign-off:

jan% aedeaegis: project "example.1.0": change 11: no current ’aegis -Test

-BaseLine’ registrationjan%

No, it wasn’t enough. Tests must not only pass against a new change, but must fail against the project base-line. Thisis to establish, in the case of bug fixes, that the bug has been isolatedand fixed. New functional-ity will usually fail against the baseline, because the baseline can’t do it (if it could, you wouldn’t be addingit!). So,Jan needs to use a variant of theaetcommand.

jan% aet -blaegis: sh /u/jan/example.1.0.C011/test/00/t0002a.shusage: exampleFAILEDaegis: project "example.1.0": change 11: test "test/00/t0002a.sh" on

baseline failed (as it should)aegis: project "example.1.0": change 11: passed 1 testjan%

Running the regression tests is also a good idea

jan% aet -regaegis: logging to "/u/pat/example.1.0.C011/aegis.log"aegis: sh /projects/example/baseline/test/00/t0001a.shaegis: project "example.1.0": change 11: test "test/00/t0001a.sh"

passedaegis: project "example.1.0": change 11: passed 1 testjan%

Now Aegis will be satisfied

jan% aedeaegis: sh /usr/local/lib/aegis/aegis/de.sh example.1.0 11 janaegis: project "example.1.0": change 11: development completedjan%


User Guide Aegis

Like Pat in the change before, Jan will receive email that this change passed review, and later that it passedintegration.

3.1.4. TheThird and Fourth Changes

This section will show two people performing two changes, one each. The twist is that they hav ea file incommon.

First Sam looks for a change to work on and starts, like this:

sam% aedb -lProject "example.1.0"List of Changes


12 awaiting_ add powersdevelopment


sam% aedb 12aegis: project "example.1.0": change 12: development directory "/u/

sam/example.1.0.C012"aegis: project "example.1.0": change 12: user "sam" has begun

developmentsam% aecdaegis: project "example.1.0": change 12: /u/sam/example.1.0.C012sam%

A l ittle sniffing around reveals that only thegram.ygrammar file needs to be altered, so it is copied into thechange.

sam% aecp gram.yaegis: project "example.1.0": change 12: file "gram.y" copiedsam%

The grammar file is changed to look like this:

%token DOUBLE%token NAME%union{

double lv_double;int lv_int;

};

%type <lv_double> DOUBLE expr%type <lv_int> NAME%left ’+’ ’-’%left ’*’ ’/’%right ’ˆ’%right UNARY

%%example

: /* e mpty */| e xample command ’\n’

{ y yerrflag = 0; fflush(stderr); fflush(stdout); };


Aegis UserGuide

command: e xpr

{ p rintf("%g\n", $1); }| e rror;

expr: D OUBLE| ’ (’ expr ’)’

{ $$ = $ 2; }| ’ -’ expr

%prec UNARY{ $$ = - $2; }

| e xpr ’ˆ’ expr{ $$ = p ow($1, $3); }

| e xpr ’*’ expr{ $$ = $1 * $ 3; }

| e xpr ’/’ expr{ $$ = $1 / $ 3; }

| e xpr ’+’ expr{ $$ = $1 + $ 3; }

| e xpr ’-’ expr{ $$ = $1 - $ 3; }

;

The changes are very small. Sam checks to make sure using the difference command:

sam% aedaegis: logging to "/u/sam/example.1.0.C012/aegis.log"aegis: set +e; diff -c /projects/example/baseline/gram.y /u/sam/

example.1.0.C012/gram.y > /u/sam/example.1.0.C012/gram.y,D; test $?-eq 0 -o $? -eq 1

aegis: project "example.1.0": change 12: difference completesam% aedmore...examines the file...sam%

The difference file looks like this

*** /projects/example/baseline/gram.y--- /u/sam/example.1.0.C012/gram.y****************** 1,5 ****--- 1,6 ----

%{#include <stdio.h>

+ #include <math.h>%}%token DOUBLE%token NAME

****************** 13,18 ****--- 14,20 ----

%type <lv_int> NAME%left ’+’ ’-’%left ’*’ ’/’

+ %right ’ˆ’%right UNARY%%example


User Guide Aegis

****************** 32,37 ****--- 34,41 ----

| ’ -’ expr%prec UNARY{ $$ = - $2; }

+ | expr ’ˆ’ expr+ { $$ = pow($1, $3); }

| e xpr ’*’ expr{ $$ = $1 * $ 3; }

| e xpr ’/’ expr

These are the differences Sam expected to see.

At this point Sam creates a test. All good software developers create the tests first, don’t they?

sam% aentaegis: project "example.1.0": change 12: file "test/00/t0003a.sh" new

testsam%

The test is created empty, and Sam edit it to look like this:

:here=‘pwd‘if test $? -ne 0 ; then exit 1; fitmp=/tmp/$$mkdir $tmpif test $? -ne 0 ; then exit 1; ficd $tmpif test $? -ne 0 ; then exit 1; fi

fail(){

echo FAILED 1>&2cd $herechmod u+w ‘find $tmp -type d -print‘rm -rf $tmpexit 1

}

pass(){

cd $herechmod u+w ‘find $tmp -type d -print‘rm -rf $tmpexit 0

}trap "fail" 1 2 3 15

cat > test.in << ’end’5.3 ˆ 04 ˆ 0 .527 ˆ (1/3)endif test $? -ne 0 ; then fail; fi

cat > test.ok << ’end’123endif test $? -ne 0 ; then fail; fi


Aegis UserGuide

$here/example test.in < /dev/null > test.out 2>&1if test $? -ne 0 ; then fail; fi

diff test.ok test.outif test $? -ne 0 ; then fail; fi

$here/example test.in test.out.2 < /dev/nullif test $? -ne 0 ; then fail; fi

diff test.ok test.out.2if test $? -ne 0 ; then fail; fi

# it p robably workedpass

Everything is ready. Now the change can be built and tested, just like the earlier changes.

sam% aebaegis: logging to "/u/sam/example.1.0.C012/aegis.log"aegis: project "example1.0": change 12: development build startedaegis: cook -b /projects/example/baseline/Howto.cook

project=example.1.0 change=12 version=1.0.C012 -nlcook: yacc -d gram.ycook: mv y.tab.c gram.ccook: mv y.tab.h gram.hcook: cc -I. -I/projects/example/baseline -O -c gram.ccook: cc -I. -I/projects/example/baseline -O -c /projects/

example/baseline/lex.ccook: cc -o example gram.o lex.o /projects/example/baseline/

main.o -ll -ly -lmaegis: project "example": change 3: development build completesam%

Notice how the yacc run produces agram.hwhich logically invalidates thelex.o in the baseline, and so thelex.c fi le in the baseline is recompiled, using thegram.hinclude file from the development directory, leav-ing a newlex.oin the development directory. This is the reason for the use of

#include < filename>

directives, rather then the double quote form.

Now the change is tested.

sam% aetaegis: logging to "/u/sam/example.1.0.C012/aegis.log"aegis: sh /u/sam/example.1.0.C012/test/00/t0003a.shaegis: project "example.1.0": change 12: test "test/00/t0003a.sh"

passedaegis: project "example.1.0": change 12: passed 1 testsam%

The change must also be tested against the baseline, and fail. Samknows this, and does it here.


User Guide Aegis

sam% aet -blaegis: logging to "/u/sam/example.1.0.C012/aegis.log"aegis: sh /u/sam/example.1.0.C012/test/00/t0003a.sh1,3c1,6< 1< 2< 3---> syntax error> 5.3> syntax error> 4> syntax error> 27FAILEDaegis: project "example.1.0": change 12: test "test/00/t0003a.sh" on

baseline failed (as it should)aegis: project "example.1.0": change 12: passed 1 testsam%

Running the regression tests is also a good idea.

sam% aet -regaegis: logging to "/u/sam/example.1.0.C012/aegis.log"aegis: sh /projects/example/baseline/test/00/t0001a.shaegis: project "example.1.0": change 12: test "test/00/t0001a.sh"

passedaegis: sh /projects/example/baseline/test/00/t0002a.shaegis: project "example.1.0": change 12: test "test/00/t0002a.sh"

passedaegis: project "example.1.0": change 12: passed 2 testssam%

A this point Sam has just enough time to get to the lunchtime aerobics class in the staff common room.

Earlier the same day, Pat arrived for work a little after Sam, and also looked for a change to work on.

pat% aedb -lProject "example.1.0"List of Changes



pat%

With such a wide choice, Pat selected change 13.

pat% aedb 13aegis: project "example.1.0": change 13: development directory "/u/

pat/example.1.0.C013"aegis: project "example.1.0": change 13: user "pat" has begun

developmentpat% aecdaegis: project "example.1.0": change 13: /u/pat/example.1.0.C013pat%

To get more information about the change, Pat then uses the "change details" listing:

pat% ael cdProject "example.1.0", Change 13Change Details


Aegis UserGuide


SUMMARYadd variables

DESCRIPTIONEnhance the grammar to allow variables. Only singleletter variable names are required.

CAUSEThis change was caused by internal_enhancement.

STATEThis change is in ’being_developed’ state.

FILESThis change has no files.

HISTORYWhat When Who Comment------ ------ ----- ---------new_change Mon Dec 14 alex

13:08:52 1992develop_begin Tue Dec 15 pat

13:38:26 1992pat%

To add the variables the grammar needs to be extended to understand them, and a new file for rememberingand recalling the values of the variables needs to be added.

pat% aecp gram.yaegis: project "example.1.0": change 13: file "gram.y" copiedpat% aenf var.caegis: project "example.1.0": change 13: file "var.c" addedpat%

Notice how Aegis raises no objection to both Sam and Pat having a copy of thegram.yfi le. Resolvingthiscontention is the subject of this section.

Pat now edits the grammar file.

pat% vi gram.y...edit the file...pat% aedaegis: logging to "/u/pat/example.1.0.C013/aegis.log"aegis: set +e; diff -c /projects/example/baseline/gram.y /u/pat/

example.1.0.C013/gram.y > /u/pat/example.1.0.C013/gram.y,D; test $?-eq 0 -o $? -eq 1

aegis: project "example.1.0": change 13: difference completepat%

The difference file looks like this

... hey, someone fill me in!...

The newvar.c fi le was created empty by Aegis, and Pat edits it to look like this:

static double memory[26];


User Guide Aegis

voidassign(name, value)

int name;double value;

{memory[name] = value;

}

doublerecall(name)

int name;{

return memory[name];}

Little remains except to build the change.

pat% aebaegis: logging to "/u/pat/example.1.0.C013/aegis.log"aegis: cook -b /example.proj/baseline/Howto.cook

project=example.1.0 change=13 version=1.0.C013 -nlcook: yacc -d gram.ycook: mv y.tab.c gram.ccook: mv y.tab.h gram.hcook: cc -I. -I/projects/example/baseline -O -c gram.ccook: cc -I. -I/projects/example/baseline -O -c /projects/

example/baseline/lex.ccook: cc -I. -I/projects/example/baseline -O -c var.ccook: cc -o example gram.o lex.o /projects/example/baseline/

main.o var.o -ll -ly -lmaegis: project "example.1.0": change 13: development build completepat%

A new test for the new functionality is required and Pat creates one like this.

:here=‘pwd‘if test $? -ne 0 ; then exit 1; fitmp=/tmp/$$mkdir $tmpif test $? -ne 0 ; then exit 1; ficd $tmpif test $? -ne 0 ; then exit 1; fi

fail(){

echo FAILED 1>&2cd $herechmod u+w ‘find $tmp -type d -print‘rm -rf $tmpexit 1

}pass(){

cd $herechmod u+w ‘find $tmp -type d -print‘rm -rf $tmpexit 0

}trap "fail" 1 2 3 15


Aegis UserGuide

cat > test.in << ’end’a = 1a + 1c = a * 40 + 5c / (a + 4 )endif test $? -ne 0 ; then fail; fi

cat > test.ok << ’end’29endif test $? -ne 0 ; then fail; fi

$here/example test.in < /dev/null > test.out 2>&1if test $? -ne 0 ; then fail; fi

diff test.ok test.outif test $? -ne 0 ; then fail; fi

$here/example test.in test.out.2 < /dev/nullif test $? -ne 0 ; then fail; fi

diff test.ok test.out.2if test $? -ne 0 ; then fail; fi

# it p robably workedpass

The new files are then differenced:

pat% aedaegis: logging to "/u/pat/example.1.0.C013/aegis.log"aegis: set +e; diff -c /projects/example/baseline/gram.y /u/pat/

example.1.0.C013/gram.y > /u/pat/example.1.0.C013/gram.y,D; test $?-eq 0 -o $? -eq 1

aegis: set +e; diff -c /dev/null /u/pat/example.1.0.C013/test/00/t0004a.sh > /u/pat/example.1.0.C013/test/00/t0004a.sh,D; test$? -eq 0 -o $? -eq 1

aegis: set +e; diff -c /dev/null /u/pat/example.1.0.C013/var.c > /u/pat/example.1.0.C013/var.c,D; test $? -eq 0 -o $? -eq 1

aegis: project "example.1.0": change 13: difference completepat%

Notice how the difference for thegram.yfi le is still current, and so is not run again.

The change is tested.

pat% aetaegis: logging to "/u/pat/example.1.0.C013/aegis.log"aegis: sh /u/pat/example.1.0.C013/test/00/t0001a.shaegis: project "example.1.0": change 13: test "test/00/t0004a.sh"

passedaegis: project "example.1.0": change 13: passed 2 testspat%

The change is tested against the baseline.


User Guide Aegis

pat% aet -blaegis: logging to "/u/pat/example.1.0.C013/aegis.log"aegis: sh /u/pat/example.1.0.C013/test/00/t0001a.sh1,2c1,4< 2< 9---> syntax error> syntax error> syntax error> syntax errorFAILEDaegis: project "example.1.0": change 13: test "test/00/t0004a.sh" on

baseline failed (as it should)pat%

And the regression tests

pat% aet -regaegis: logging to "/u/pat/example.1.0.C013/aegis.log"aegis: sh /projects/example/baseline/test/00/t0001a.shaegis: project "example.1.0": change 13: test "test/00/t0001a.sh"

passedaegis: sh /projects/example/baseline/test/00/t0002a.shaegis: project "example.1.0": change 13: test "test/00/t0002a.sh"

passedaegis: project "example.1.0": change 13: passed 2 testspat%

Note how test 3 has not been run, in any form of testing. This is because test 3 is part of another change,and is not yet integrated with the baseline.

All is finished for this change,

pat% aedeaegis: sh /usr/local/lib/aegis/de.sh example.1.0 13 pataegis: project "example.1.0": change 13: development completedpat%

Anxious to get this change into the baseline, Pat now wanders down the hall in search of a reviewer, butmore of that in the next section.

Some time later, San returns from aerobics feeling much improved. All that is required for change 12 is todo develop end, or is it?

sam% aedeaegis: project "example.1.0": change 12: file "gram.y" in baseline

has changed since last ’aegis -DIFFerence’ commandsam%

A l ittle sleuthing on Sam’s part with the Aegis list command will reveal how this came about. The way toresolve this problem is with the difference command, but the merge variant - this will merge the new base-line version, and Sam’s edit together.


Aegis UserGuide

sam% aemaegis: logging to "/u/pat/example.1.0.C012/aegis.log"aegis: co -u’1.1’ -p /projects/example/history/gram.y,v > /tmp/

aegis.14594/projects/example/history/gram.y,v --> stdout revision 1.1 (unlocked)aegis: (diff3 -e /projects/example/baseline/gram.y /tmp/

aegis.14594 /u/sam/example.003/gram.y | sed -e ’/ˆw$/d’-e ’/ˆq$/d’; echo ’1,$p’ ) | ed - /projects/example/baseline/gram.y,B > /u/sam/example.003/gram.y

aegis: project "example.1.0": change 12: merge completeaegis: project "example.1.0": change 12: file "gram.y" was out of

date and has been merged, see "gram.y,B" for original sourceaegis: new ’aegis -Build’ requiredsam%

This was caused by the conflict between change 13, which is now integrated, and change 12; both of whichare editing thegram.yfi le. Samexamines thegram.yfi le, and is satisfied that it contains an accurate mergeof the edit done by change 13 and the edits for this change. The merged source file looks like this:

%{#include <stdio.h>#include <math.h>%}%token DOUBLE%token NAME%union{

double lv_double;int lv_int;

};

%type <lv_double> DOUBLE expr%type <lv_int> NAME%left ’+’ ’-’%left ’*’ ’/’%right ’ˆ’%right UNARY

%%example

: /* e mpty */| e xample command ’\n’

{ y yerrflag = 0; fflush(stderr); fflush(stdout); };

command: e xpr

{ p rintf("%g\n", $1); }| N AME ’=’ expr

{ a ssign($1, $3); }| e rror;


User Guide Aegis

expr: D OUBLE| N AME

{ e xtern double recall(); $$ = recall($1); }| ’ (’ expr ’)’

{ $$ = $ 2; }| ’ -’ expr

%prec UNARY{ $$ = - $2; }

| e xpr ’ˆ’ expr{ $$ = p ow($1, $3); }

| e xpr ’*’ expr{ $$ = $1 * $ 3; }

| e xpr ’/’ expr{ $$ = $1 / $ 3; }

| e xpr ’+’ expr{ $$ = $1 + $ 3; }

| e xpr ’-’ expr{ $$ = $1 - $ 3; }

;

The automatic merge worked because most such conflicts are actually working on logically separate por-tions of the file. Two different areas of the grammar in this case.In practice, there is rarely a real conflict,and it is usually small enough to detect fairly quickly.

Sam now rebuilds:

sam% aebaegis: logging to "/u/sam/example.1.0.C012/aegis.log"aegis: project "example.1.0": change 12: development build startedaegis: cook -b /projects/example/baseline/Howto.cook

project=example.1.0 change=12 version=1.0.C012 -nlcook: rm gram.ccook: rm gram.hcook: yacc -d gram.ycook: mv y.tab.c gram.ccook: mv y.tab.h gram.hcook: rm gram.ocook: cc -I. -I/projects/example/baseline -O -c gram.ccook: rm lex.ocook: cc -I. -I/projects/example/baseline -O -c /projects/

example/baseline/lex.ccook: rm examplecook: cc -o example gram.o lex.o /projects/example/baseline/

main.o /projects/example/baseline/var.o -ll -ly -lmaegis: project "example.1.0": change 12: development build completesam%

Notice how the list of object files linked has also adapted to the addition of another file in the baseline,without any extra work by Sam.

All that remains is to test the change again.

sam% aetaegis: /bin/sh /u/sam/example.1.0.C012/test/00/t0003a.shaegis: project "example.1.0": change 12: test "test/00/t0003a.sh"

passedaegis: project "example.1.0": change 12: passed 1 testsam%

And test against the baseline,


Aegis UserGuide

sam% aet -blaegis: /bin/sh /u/sam/example.1.0.C012/test/00/t0003a.sh1,3c1,6< 1< 2< 3---> syntax error> 5.3> syntax error> 4> syntax error> 27FAILEDaegis: project "example.1.0": change 12: test "test/00/t0003a.sh" on

baseline failed (as it should)aegis: project "example.1.0": change 12: passed 1 testsam%

Perform the regression tests, too. This is important for a merged change, to make sure you didn’t break thefunctionality of the code you merged with.

sam% aet -regaegis: logging to "/u/sam/example.1.0.C012/aegis.log"aegis: /bin/sh /projects/example/baseline/test/00/

t0001a.shaegis: project "example.1.0": change 12: test "test/00/t0001a.sh"

passedaegis: /bin/sh /projects/example/baseline/test/00/


passedaegis: /bin/sh /projects/example/baseline/test/00/


passedaegis: project "example.1.0": change 12: passed 3 testssam%

All done, or are we?

sam% aedeaegis: project "example.1.0": change 12: no current ’aegis -Diff’

registrationsam%

The difference we did earlier, which revealed that we were out of date, does not show the differences sincethe two changes were merged, and possibly further edited.

sam% aedaegis: logging to "/u/sam/example.1.0.C012/aegis.log"aegis: set +e; diff /projects/example/baseline/gram.y /u/pat/

example.1.0.C012/gram.y > /u/pat/example.1.0.C012/gram.y,D;test $? -le 1

aegis: project "example.1.0": change 12: difference completesam%

This time everything will run smoothly,

sam% aedeaegis: project "example.1.0": change 12: development completedsam%

Some time soon Sam will receive email that this change passed review, and later that it passed integration.

Within the scope of a limited example, you have seen most of what Aegis can do.To get a true feeling forthe program you need to try it in a similarly simple case.You could even try doing this example manually.


User Guide Aegis

3.1.5. Developer Command Summary

Only a few of the Aegis commands available to developers have been used in the example. Thefollowingtable (very tersely) describes the Aegis commands most useful to developers.

Command Description

aeb Buildaeca editChange Attributesaecd ChangeDirectoryaeclean Cleana dev elopment directoryaeclone copy a whole changeaecp Copy Fileaecpu Copy File Undoaed Differenceaedb Develop Beginaedbu Dev elop Begin Undoaede Develop Endaedeu Develop End Undoael ListStuffaenf New Fileaenfu New File Undoaent New Testaentu New Test Undoaerm Remove Fileaermu Remove File Undoaet Test

You will want to read the manual entries for all of these commands.Note that all Aegis commands have a−Help option, which will give a result very similar to the correspondingman(1) output. Most Aegis com-mands also have a−List option, which usually lists interesting context sensitive information.


Aegis UserGuide

3.2. TheReviewer

The role of a reviewer is to check another user’s work. You are helped in this by Aegis, because changescan never reach thebeing reviewedstate without several preconditions:

• The change is known to build. You know that it compiled successfully, so there is no need to search forsyntax errors.

• The change has tests, and those tests have been run, and have passed.

This information allows you to concentrate on implementation issues, completeness issues, and local stan-dards issues.

To help the reviewer, a set of "comma D" files is available in the change development directory. Every filewhich is to be added to the baseline, removed from the baseline, or changed in some way, has a correspond-ing "comma D" file.




Robyn finds out what changes are available for review by asking Aegis:

robyn% aerpass -l -p example.1.0

Project "example.1.0"List of Changes


10 being_reviewed Place under Aegisrobyn%

Any of the above changes could be reviewed, Robyn chooses the first.

robyn% aecd -p example.1.0 -c 10aegis: project "example": change 1: /u/pat/example.1.0.C010robyn% aedmore...examines each file...robyn%

The aedmorecommand walks the development directory tree to find all of the "comma D" files, and dis-plays them usingmore(1). Thereis a correspondingaedlessfor those who prefer theless(1) command.

Once the change has been reviewed and found acceptable, it is passed:

robyn% aerpass -p example.1.0 10aegis: sh /usr/local/lib/aegis/rp.sh example.1.0 10 pat robynaegis: project "example.1.0": change 10: passed reviewrobyn%

Some time soon Isa will notice the email notification and commence integration of the change.


Most reviews have the same pattern as the first.


User Guide Aegis

robyn% aerpass -l -p example.1.0



11 being_reviewed file names on command linerobyn%

Always change directory to the change’s dev elopment directory, otherwise you will not be able to reviewthe files.

robyn% aecd -p example.1.0 -c 11aegis: project "example.1.0": change 11: /u/jan/example.1.0.C011robyn%

Another useful way of finding out about a change is the "list change details" command, viz:

robyn% ael cd -p example.1.0 -c 11

Project "example.1.0", Change 11Change Details


SUMMARYfile names on command line

DESCRIPTIONOptional input and output files may be specified onthe command line.

CAUSEThis change was caused by internal_bug.

STATEThis change is in ’being_reviewed’ state.

FILESType Action Edit File Name------- ------- ------- -----------source modify 1.1 main.ctest create test/00/t0002a.sh

HISTORYWhat When Who Comment------ ------ ----- ---------new_change Fri Dec 11 alex

14:55:06 1992develop_begin Mon Dec 14 jan

09:07:08 1992develop_end Mon Dec 14 jan

11:43:23 1992robyn%

Once Robyn knows what the change is meant to be doing, the files are then examined:

robyn% aedmore...examines each file...robyn%


Aegis UserGuide

Once the change is found to be acceptable, it is passed:

robyn% aerpass -p example.1.0 11aegis: sh /usr/local/lib/aegis/rp.sh example.1.0 11 jan robynaegis: project "example.1.0": change 11: passed reviewrobyn%

Some time soon Isa will notice the email notification and commence integration of the change.

The reviews of the third and fourth changes will not be given here, because they are almost identical to theother changes. If you want to know how to fail a review, see theaerfail(1) manual entry.

3.2.4. Reviewer Command Summary

Only a few of the Aegis commands available to reviewers have been used in this example. Thefollowingtable (very tersely) describes the Aegis commands most useful to reviewers.

Command Description

aecd ChangeDirectoryaerpass Review Passaerpu Review Pass Undoaerfail Review Failael ListStuff



User Guide Aegis

3.3. TheIntegrator

This section shows what the integrator must do for each of the changes shown to date.The integrator doesnot have the ability to alter anything in the change; if a change being integrated is defective, it is simplyfailed back to the developer. This documented example has no such failures, in order to keep it manageablysmall.




The first change of a project is often the trickiest, and the integrator is the last to know. This example goessmoothly, and you may want to consider using the example project as a template.

The integrator for this example project is Isa. Isa knows there is a change ready for integration from thenotification which arrived by email.

isa% aeib -l -p example.1.0



10 awaiting_ Place under Aegisintegration

isa% aeib example.1.0 10aegis: project "example.1.0": change 10: link baseline to integration

directoryaegis: project "example.1.0": change 10: apply change to integration

directoryaegis: project "example.1.0": change 10: integration has begunisa%

The integrator must rebuild and retest each change.This ensures that it was no quirk of the developer’senvironment which resulted in the success at the development stage.

isa% aebaegis: logging to "/projects/example/delta.001/aegis.log"aegis: project "example.1.0": change 10: integration build startedaegis: cook -b Howto.cook project=example.1.0 change=10

version=1.0.D001 -nlcook: yacc -d gram.ycook: mv y.tab.c gram.ccook: mv y.tab.h gram.hcook: cc -I. -O -c gram.ccook: lex lex.lcook: mv lex.yy.c lex.ccook: cc -I. -O -c lex.ccook: cc -I. -O -c main.ccook: cc -o example gram.o lex.o main.o -ll -lyaegis: project "example.1.0": change 10: integration build completeisa%

Notice how the above build differed from the builds that were done while in thebeing developedstate; theextra baseline include is gone. This is because the integration directory will shortly be the new baseline,and must be entirely internally consistent and self-sufficient.

You are probably wondering why this isn’t all rolled into the one Aegis command. It is not because theremay be some manual process to be performed after the build and before the test. This may be making acommand set-uid-root (as in the case of Aegis itself) or it may require some tinkering with the local oracleor ingress database. Instructions for the integrator may be placed in the description field of the change


Aegis UserGuide

attributes.

The change is now re-tested:

isa% aetaegis: logging to "/projects/example/delta.001/aegis.log"aegis: sh /project/example/delta.001/test/00/t0001a.shaegis: project "example": change 1: test "test/00/t0001a.sh"

passedaegis: project "example": change 1: passed 1 testisa%

The change builds and tests. Once Isa is happy with the change, perhaps after browsing the files, Isa thenpasses the integration, causing the history files to be updated and the integration directory becomes thebaseline.

isa% aeipassaegis: logging to "/projects/example/delta.001/aegis.log"aegis: ci -u -m/dev/null -t/dev/null /projects/example/delta.001/

Howto.cook /projects/example/history/Howto.cook,v;rcs -U /projects/example/history/Howto.cook,v

/projects/example/history/Howto.cook,v <--/projects/example/delta.001/Howto.cook

initial revision: 1.1doneRCS file: /projects/example/history/Howto.cook,vdoneaegis: rlog -r /projects/example/history/Howto.cook,v | awk

’/ˆrevision/ {print $2}’ > /tmp/aegis.15309...lots of similar RCS output...aegis: project "example.1.0": change 10: remove development directoryaegis: sh /usr/local/lib/aegis/ip.sh example.1.0 10 pat robyn isaaegis: project "example.1.0": change 10: integrate passisa%

All of the staff inv olved, will receive email to say that the change has been integrated. Thisnotification is ashell script, so USENET could be usefully used instead.

You should note that the development directory has been deleted. It is expected that each develop-ment directory will only contain files necessary to develop the change.You should keep "precious" filessomewhere else.

3.3.3. TheOther Changes

There is no difference to integrating any of the later changes. The integration process is very simple, as it isa cut-down version of what the developer does, without all the complexity.

Your project may elect to have the integrator also monitor the quality of the reviews. An answer to "whowill watch the watchers" if you like.

It is also a good idea to rotate people out of the integrator position after a few weeks in a busy project, thisis a very stressful position.The position of integrator gives a unique perspective to software quality, but theperson also needs to be able to say "NO!" when a cruddy change comes along.


User Guide Aegis

3.3.4. IntegratorCommand Summary

Only a few of the Aegis commands available to integrators have been used in this example. Thefollowingtable (very tersely) describes the Aegis commands most useful to integrators.

Command Description

aeb Buildaecd ChangeDirectoryaed Differenceaeib Integrate Beginaeibu Integrate Begin Undoaeifail Integrate Failael ListStuffaet Testaeipass Integrate Pass


3.3.5. Minimum Integrations

The aegis --integrate-begincommand provides a--minimum option which may be used for various rea-sons. Theterm minimum may be a bit counter intuitive. One might think it means to theminimumamount of work, however it actually means use aminimum of files from the baseline in populating thedelta directory. This normally leads to actually building everything in the project from sources and, assuch, might be considered the most robust of builds.

Note that any change which removes a file, whether byaermor aemv, results in an implicitminimum inte-gration. Thisis intended to ensure nothing in the project references the removed file.

A project may adopt a policy that a product release should be based on a minimum integration. Sucha pol-icy may be a reflection of local confidence, or lack therof, in the projects DMT (Dependency MaintenanceTool) or build system. Or it may be based on a validation process wishing to make a simple statement onhow the released package was produced.

Another, more transient, reason a to require a minimum integration might be when upgrading a third partylibrary, compiler or maybe even OS lev el. Any of these events would signal the need for a minimum inte-gration to ensure everything is rebuilt using the new resources.

The cost of aminimum integration varies according to type and size of the project.For very large projects,especially those building large numbers of binaries, the cost can be large. However large projects alsorequire significant time to fully populate the delta directory. A minimum integration only copies those filesunder aegis control, skipping all “produced” files. In the case where a file upon which everything dependsis changed, everything will be built anyway so the copy of the already built files is a waste of time.Thismeans that sometimes a minimum can be as cheap as a normal integration.


Aegis UserGuide

3.4. TheAdministrator

The previous discussion of developers, reviewers and integrators has covered many aspects of the produc-tion of software using Aegis. Theadministrator has responsibility for everything they don’t, but there isvery little left.

These responsibilities include:

• access control: The administrator adds and removes all categories of user, including administrators.Thisis on a per-project basis, and has nothing to do withUNIX user administration. This simply nominateswhich users may do what.

• change creation: The administrator adds (and sometimes removes) changes to the system. At later stages,developers may alter some attributes of the change, such as the description, to say what they fixed.

• project creation: Aegis does not limit who may create projects, but when a project is created the user whocreated the project is set to be the administrator of that project.

All of these things will be examined




Many things need to happen before development can begin on the first change; the project must be created,the staff but be given access permissions, the branches created, and the change must be created.

alex% aenpr example -dir /projects/example -version -aegis: project "example": project directory "/projects/example"aegis: project "example": createdalex%

Once the project has been created, the project attributes are set.Alex will set the desired project attributesusing the-Edit option of theaepa command. Thiswill invoke an editor (vi(1) by default) to edit theproject attributes. Alex edits them to look like this:

description = "Aegis Documentation Example Project";developer_may_review = false;developer_may_integrate = false;reviewer_may_integrate = false;

The project attributes are set as follows:

alex% aepa -edit -p example...edit as above...aegis: project "example.1.0": attributes changedalex% ael pList of Projects

Project Directory Description------- ----------- -------------example /projects/example Aegis Documentation Example

Projectalex%

The various staff must be added to the project. Developers are the only staff who may actually edit files.

alex% aend pat jan sam -p exampleaegis: project "example": user "pat" is now a developeraegis: project "example": user "jan" is now a developeraegis: project "example": user "sam" is now a developeralex%

Reviewers may veto a change. There may be overlap between the various categories, as show here for Jan:


User Guide Aegis

alex% aenrv robyn jan -p exampleaegis: project "example": user "robyn" is now a revieweraegis: project "example": user "jan" is now a revieweralex%

The next role we need to fill is an integrator.

alex% aeni isa -p exampleaegis: project "example": user "isa" is now an integratoralex%

Once the staff hav e been given access, it is time to create the working branch. Branches inherit theirattributes and staff l ists from their parent branches when they are first created, which is why we set all thatstuff f irst.

alex% aegis -nbr -p example 1aegis: project "example.1": createdalex% aegis -nbr -p example.1 0aegis: project "example.1.0": createdalex%

This is for versioning; see theBranchingchapter for more information.For the moment, we will simplywork on branch 1.0. Notice how the branches appear as projects in the project listing; in general branchescan be used interchangeably with projects.

alex% ael pList of Projects

Project Directory Description------- ----------- -------------example /projects/example Aegis Documentation Example

Projectexample.1 /projects/example/ Aegis Documentation Example

branch.1 Project, branch.1.example.1.0 /projects/example/ Aegis Documentation Example

branch.1/branch.0 Project, branch.1.0.alex%

Once the working branch has been created, Alex creates the first change.The -Edit option of theaenccommand is used, to create the attributes of the change. They are edited to look like this:

brief_description = "Create initial skeleton.";description = "A simple calculator using native \floating point precision. \The four basic arithmetic operators to be provided, \using conventional infix notation. \Parentheses and negation also required.";cause = internal_enhancement;

The change is created as follows:

alex% aenc -edit -p example.1.0...edit as above...aegis: project "example.1.0": change 10: createdalex%

Notice that the first change number is “10”. This is done so that changes 1 to 9 could be used as bug-fixbranches at some future time. See theBranchingchapter for more information.You can over-ride this isyou need to.

The above was written almost a decade ago.There is a newer command,tkaenc, which uses a GUI and ismuch easier to use, with a much less fiddly interface. You may want to try that command, instead, for mostroutine change creation.

At this point, Alex walks down the hall to Pat’s off i ce, to ask Pat to develop the first change.Pat has hadsome practice using Aegis, and can be relied on to do the rest of the project configuration speedily.


Aegis UserGuide


Some time later, Alex patiently sits through the whining and grumbling of an especially pedantic user. Thefollowing change description is duly entered:

brief_description = "file names on command line";description = "Optional input and output files may be \specified on the command line.";cause = internal_bug;

The pedantic user wanted to be able to name files on the command line, rather than use I/O redirection.Also, having a bug in this example is useful. The change is created as follows:


At some point a developer will notice this change and start work on it.

3.4.4. TheThird Change

Other features are required for the calculator, and also for this example. Thethird change adds exponentia-tion to the calculator, and is described as follows:

brief_description = "add powers";description = "Enhance the grammar to allow exponentiation. \No error checking required.";cause = internal_enhancement;



At some point a developer will notice, and this change will be worked on.

3.4.5. TheFourth Change

A fourth change, this time adding variables to the calculator is added.

brief_description = "add variables";description = "Enhance the grammar to allow variables. \Only single letter variable names are required.";cause = internal_enhancement;



At some point a developer will notice, and this change will be worked on.

3.4.6. Administrator Command Summary

Only a few of the Aegis commands available to administrators have been used in this example. Thefollow-ing table lists the Aegis commands most useful to administrators.

Command Description

aeca editChange Attributesael ListStuff


User Guide Aegis

aena New Administratoraenc New Changeaencu New Change Undoaend New Dev eloperaeni New Integratoraenpr New Projectaenrv New Revieweraepa editProject Attributesaera Remove Administratoraerd Remove Dev eloperaeri Remove Integratoraermpr Remove Projectaerrv Remove Reviewer



Aegis UserGuide

3.5. What to do Next

This chapter has given an overview of what usingAegis feels like. Asa next step in getting to knowAegis, it would be a good idea if you created aproject and went through this same exercise. Youcould use this exact example, or you could use asimilar small project. The idea is simply to runthrough many of the same steps as in the example.Typos and other natural events will ensure thatyou come across a number of situations notdirectly covered by this chapter.

If you have not already done do, a printed copy ofthe section 1 and 5 manual entries will be invalu-able. If you don’t want to use that many trees,they will be available on-line, by using the"-Help" option of the appropriate command vari-ant. Try:

% aedb -help...manual entry...%

Note that this example has not demonstrated all ofthe available functionality. One item of particularinterest is that tests, like any other source file,may be copied into a change and modified, orev en deleted, just like any other source file.

3.6. CommonQuestions

There are a number of questions which are fre-quently asked by people evaluating Aegis. Thissection attempts to address some of them.

3.6.1. Insulation

The repository model used by Aegis is of the“push” type - that is, changes to the baseline are“pushed” onto the developer as soon as they areintegrated. Many configuration management sys-tems have a “pull” model, where the developerelects when to cope with changes in the reposi-tory. At first glance, Aegis does not appear tohave a “pull” equivalent.

It is possible to insulate your change from thebaseline as much or as little as required.Theaecp(1) command, used to copy files into achange, has a--read-only option. Thefi lescopied in this way are marked as insulation (i.e.you don’t intend to change them).If you have notun-copied them at develop end time, theaede(1)command will produce a suitable error message,reminding you to un-copy the insulation and ver-ify that your change still builds and tests success-fully with the (probably) now-different baseline.

3.6.1.1. CopyRead-Only

It is possible to select the degree of insulation.By using “aecp . ” at the top of a developmentdirectory, the complete project source tree will becopied, thus completely insulating you.Mindyou, it comes at the cost of a complete build.

If you are working on a library, and only want therest of the library to remain fixed, simply copy thewhole library (aecp library/fred ), andallow the rest to track the baseline. This comes ata smaller cost, because more of the baseline’sobject files can be taken advantage of.

3.6.1.2. Branches

It is also possible to create a sub-branch (see theBranchingchapter). Thisdoes not itself automat-ically insulate, however the first change of abranch intended to insulate would copy and inte-gratebut not modifythe files to be insulated.Youneed to remember to perform across-branchmerge with the parent branch before integratingthe branch back into the parent branch.

3.6.1.3. Builds

You can also insulate yourself from baselinechange by being selective about what you chooseto build. You can do this by giving specific buildtargets on theaeb(1) command line, or you couldcopy the build tool’s configuration file andbutcher it. Remember to change it back beforeyouaede(1) your change!

3.6.1.4. Mix-and-Match

Some or all of the above techniques may be com-bined to provide an insulation technique bestsuited to your project and development policy.E.g. changing the build configuration file for abranch dedicated to working on a small portion ofa large project; towards the ed of the develop-ment, change the build configuration file back andperform integration testing.

3.6.1.5. Disadvantages

There is actually a down-side to insulating yourchanges from the evolution of the baseline.Bynoticing and adapting to the baseline, you havemuch less merging to do at the end of yourchange set. Each integration will typically be bemodest, but the cumulative effect could be sub-stantial, and add a huge unexpected (and un-bud-geted for) time penalty.


User Guide Aegis

It also means that if there are integration problemsbetween your work and the changes which wereintegrated before yours, or if your work shows upa bug in their work, the project find this out late,rather than early. The literature, based on indus-trial experience, indicates that the earlier prob-lems are found the cheaper they are to fix.

Insulated development directories also use moredisk space. While disk space is relatively cheapthese days, it can still add up to a substantial hitfor a large development team. Un-insulateddevelopment directories can take advantage of thepre-compiled objects and libraries in the baseline.

3.6.2. Partial Check-In

In the course of developing new functionality, it isvery common to come across a pre-existing bugwhich the new functionality exposes. Itis com-mon for such bugs to be fixed by the developer inthe same development directory, in order to getthe new functionality to a testable state.

There are two common courses of action at thispoint: simply include the bug fix with the rest ofthe change, or integrate the bug fix earlier thanthe rest of the change. Combining the bug fixwith the rest of the change can have nasty effectson statistics: it can hide the true bug level f romyour metrics program, and it also denies Aegis theopportunity of having accurate test correlations(seeaet(1) for more information.) It also deniesthe rest of the development team the use of thebug fix, or worse, it allows the possibility thatmore than one team member will fix the bug,wasting development effort and time.

Many configuration management systems allowyou to perform a partial check-in of a work area.This means that you can check-in just the bug fix,but continue to work on the unfinished portions ofthe functionality you are implementing.

Because Aegis insists on atomic change setswhich are known to build and test successfully,such a partial check-in is not allowed - becauseAegis can’t know for certain that it works.

Instead, you are able toclone a change (seeaeclone(1) for more information).This gives youa new change, and a second development direc-tory, with exactly the same files. You thenremove from this second change all of the filesnot related to the bug fix (using aecpu(1),aenfu(11), etc). You then create a test, build, dif-ference, run the test, develop end, all as usual.

The original change will then need to be mergedwith the baseline, because the bug fix change willhave been integrated before it.Usually this isstraight-forward, as you already have the changes(some merge tools make this harder than others).Often, all that is required is to merge, and thensay “aecpu -unch ” to un-copy all files whichare (now) unchanged when compared to the cur-rent baseline.

3.6.3. Multiple Active Branches

Some companies have multiple branches active atthe same time, for different customers or distribu-tions,etc.

They often need to make the same change to morethan one branch. Some configuration manage-ment systems allow you to check-in the same filemultiple times, once to each active branch. Aegisdoes not let you do this, because you need to con-vince Aegis that the change set will build and testcleanly on each branch. It is quite common forthe change to require non-trivial edits to work oneach branch.

3.6.3.1. Cloning

Aegis instead provides two mechanisms to handlethis. Thefi rst, and simplest to understand, is toclone the change onto each relevant branch (ratherthan onto the same branch, as mentioned abovefor bug fixes). Thenbuild and test as normal.

3.6.3.2. Ancestral

The second technique is more subtle. Perform thefix as a change to the common ancestor of bothbranches. Thisassumes that neither branch isinsulated against the relevant area of code, andthat earlier changes to the branch do not mask itin some way (otherwise a cross-branch mergewith the common ancestor will be needed to prop-agate the fix).

3.6.4. Collaboration

It is often the case that difficult problems are tack-led by small groups of 2 or 3 staff workingtogether. In order to do this, they often work in ashared work area with group-writable or global-write permissions.However, this tends to givesecurity auditor heart attacks.

Aegis has several different ways to achieve thesame ends, and still not give the auditors indiges-tion.


Aegis UserGuide

3.6.4.1. ChangeOwner

The simplest method available is to change theownership of a change from one developer to thenext. A new dev elopment directory is created forthe new dev eloper, and the source files are copiedacross15. This allows a kind of serial collabora-tion between developers.

3.6.4.2. Branch

The other possibility is to create a branch to per-form the work in, rather than a simple change.(Abranch in Aegis is literally just a big change,which has lots of sub-changes.) This allows par-allel collaboration between developers.

3.6.4.3. View Path Hacking

Aegis usually provides a “view path” to the buildtool. Thisspecifies where to look for source filesand derived files, in order to union together thedevelopment directory, and the baseline, and thebranch’s ancestors’ baselines.If you run the buildby hand, rather than through Aegis, you can addanother developer’s dev elopment directory to theview path, after your own development directory,but before the baseline.

This has many of the advantages of the branchmethod, but none of the safeguards. Inparticular,if the other developer edits a file, and it no longercompiles, your development directory will not,either.

15 For the technically minded: thechown(2) sys-tem call has semantics which vary too widelybetweenUNIX variants and file-systems to be useful.


User Guide Aegis

4. TheHistory Tool

Aegis is decoupled from the history mechanism.This allows you to use the history mechanism ofyour choice, SCCS or RCS, for example. Youmay even wish to write your own.

The intention of this is that you may use a historymechanism which suits your special needs, or theone that comes free with your flavour of UNIX

operating system.

Aegis uses the history mechanism for file historyand so does not require many of the features ofSCCS or RCS. This simplistic approach cansometimes make the interface to these utilitieslook a little strange.

4.1. History File Names

In order to track project source file renames andyet preserve a continuous history, the name ofeach source file and the name of each correspond-ing history file have nothing in common.The his-tory file will have the same name (both on thelocal repository and any remote repository it is in)no matter how many times the source file isrenamed.

Each source file is assigned universally uniqueidentifier (UUID) when it is first created.Thisattribute, unlike the source file’s name, isimmutable and thus is suitable for use when form-ing the name of the history file.

4.2. Interfacing

The history mechanism interface is found in theproject configuration file calledaegis.conf, rela-tive to the root of the baseline.It is a source fileand subject to the same controls as any othersource file. The history fields of the file aredescribed as follows

4.2.1. history_create_command

This field is used to create a new history. Thecommand is always executed as the project owner.Substitutions available for the command stringare:

${Input}absolute path of source file

${History}absolute path of history file

In addition, all substitutions described inaesub(5)are available.

This command should be identical to thehistory_-put_commandotherwise mysterious things can

happen when branches are ended.

4.2.2. history_get_command

This field is used to get a file from history. Thecommand may be executed by developers. Sub-stitutions available for the command string are:


${Edit}edit number, as giv en by the history_-query_command.

${Output}absolute path of destination file


4.2.3. history_put_command

This field is used to add a new change to the his-tory. The command is always executed as theproject owner. Substitutions available for thecommand string are:




This command should be identical to thehistory_-create_commandotherwise mysterious things canhappen when branches are ended.

4.2.4. history_query_command

This field is used to query the topmost edit of ahistory file. Resultto be printed on the standardoutput. This command may be executed bydevelopers. Substitutionsavailable for the com-mand string are:



4.2.5. history_content_limitation

This field describes the content style which thehistory tool is capable of working with.

ascii_textThe history tool can only cope with fileswhich contain printable ASCII characters,plus space, tab and newline. Thefi le mustend with a newline. Thisis the default.


Aegis UserGuide

international_textThe history tool can only cope with fileswhich do not contain the NUL character.The file must end with a newline.

binary_capableThe history tool can cope with files withoutany limitation on the form of the contents.

When a file is added to the history (by either thehistory_create_commandor the history_put_-commandfield) it is examined for conformance tothis limitation. If there is a problem, the file isencoded in either the MIME quoted printable orthe MIME Base 64 encoding (see RFC 1521),whichever is smaller, before being given to thehistory tool. The file in the baseline isunchanged.

On extract (thehistory_get_commandfield) theencoding is reversed, using information attachedto the change file information. This is becauseeach put could use a different encoding (althoughin practice, file contents rarely change that dra-matically, and the same encoding is likely to bededuced every time).

4.2.6. history_tool_trashes_file

Many history tools (e.g. RCS) can modify thecontents of the file when it is committed.Whilethere are usually options to turn this off, they areseldom used.The problem is: if the commitchanges the file, the source in the repository nowno longer matches the object file in the repository- i.e. the history tool has compromised the refer-ential integrity of the repository.

By default, when this happens Aegis issues a fatalerror (at intergate passtime). You can turn thisinto a warning if you are convinced this is irrele-vant. This would only make sense if the substi-tion only ever occurs in comments.See aep-conf(5) for more information on the values forthis field.

4.2.7. QuotingFilenames

The default setting is for Aegis to reject filenameswhich contain shell special characters.Thisensures that filenames may be substituted into thecommands without worrying about whether this issafe. If you set theshell_safe_filenamesfield ofthe projectaegis.conffi le to false , you willneed to surround filenames with the${quotefilename} substitution. Thiswill only quote file-names which actually need to be quoted, so usersusually will not notice. This applies to all of thevarious filenames in the commands in the sections

which follow.

4.2.8. Templates

The source distribution contains numerous config-uration examples in a directory calledlib/con-fig.example/ which is installed into/usr/local/share/aegis/config.example/by default.In the interests of accuracy, it may be best to copyconfigurations from there, rather than copy-typethe ones below.


User Guide Aegis

4.3. Usingaesvt

The aesvt(1) command is distributed with Aegis.It supports binary files, has versy small historyfi les, and has good end-to-end behaviour. Theentries for the commands are listed below.


This command is used to create a new file history.This command is always executed as the projectowner.

The following substitutions are available:

${Input}absolute path of the source file

${History}absolute path of the history file

The entry in theaegis.conffi le looks like this:

history_create_command ="aesvt -checkin ""-history $history ""-f $input";


It is essential that thehistory_create_commandand thehistory_put_commandare identical. It isa historical accident that there are two separatecommands: before Aegis supported branches, thiswas not a requirement.


This command is used to get a specific edit backfrom history. This command is always executedas the project owner.



${Edit}edit number, as giv en by history_query_-command

${Output}absolute path of the destination file


history_get_command ="aesvt -checkout ""-history $history ""-edit $edit ""-o $output";


This command is used to query what the historymechanism calls the top-most edit of a historyfi le. The result may be any arbitrary string, itneed not be anything like a number, just so longas it uniquely identifies the edit for use by thehis-tory_get_commandat a later date. The edit num-ber is to be printed on the standard output.Thiscommand is always executed as the project owner.




history_query_command ="aesvt -query ""-history $history";

4.3.5. Templates

The lib/config.example/aesvtfi le in the Aegis dis-tribution (installed as/usr/local/share/aegis/con-fig.example/aesvtby default) contains all of theabove commands, so that you may readily insertthem into your project configuration file.

Also, there are some subtleties to writing thecommands, which are not present in the aboveexamples. Inparticular, being able to support filenames which contain characters which are specialto the shell requires the use of the ${quote} sub-stitution around all of the files names in the com-mands.

In addition, it is possible to store meta-date witheach version. For example: “Descrip-tion=${quote ($version) ${changedescription}} ” i nserts the version numberand the brief description into the file’s log. Thismeans that using theaesvt -listoption will pro-vide quite useful summaries.

4.3.6. BinaryFiles

Theaesvt(1) command is able to cope with binaryfi les. Set

history_content_limitation =binary_capable;

so that Aegis knows that no encoding is required.


Aegis UserGuide

4.4. UsingSCCS

The entries for the commands are listed below.SCCS uses a slightly different model than Aegiswants, so some maneuvering is required.Thecommand strings in this section assume that theSCCS commandsccs is in the command searchPATH, but you may like to hard-wire the path, orset PATH at the start of each command. (It is alsopossible that you need to say “delta” instead of“sccs delta”. if this is the case, this commandneeds to be in the path.)You should also notethat the strings are always handed to the Bourneshell to be executed, and are set to exit with anerror immediately a sub-command fails.

One further assumption is that theae-sccs-put(1)command, which is distributed with Aegis, is inthe command search path.This insulates some ofthe weirdness that SCCS carries on with, andmakes the commands below comprehensible.


This command is used to create a new project his-tory. The command is always executed as theproject owner.





history_create_command ="ae-sccs-put -y$version -G$input "" $ {d $h}/s.${b $h}";

It is important that thehistory_create_commandand thehistory_put_commandbe the same.Thisis necessary for branching to work correctly.


This command is used to get a specific edit backfrom history. The command may be executed bydevelopers.






history_get_command ="get -r’$e’ -s -p -k "" $ {d $h}/s.${b $h} > $o";


This command is used to add a new "top-most"entry to the history file. Thiscommand is alwaysexecuted as the project owner.





history_put_command ="ae-sccs-put -y$version -G$input "" $ {d $h}/s.${b $h}";

Note that the SCCS file is left in thenot-editstate,and that the source file is left in the baseline.

It is important that thehistory_create_commandand thehistory_put_commandbe the same.Thisis necessary for branching to work correctly.


This command is used to query what the historymechanism calls the top-most edit of a historyfi le. The result may be any arbitrary string, itneed not be anything like a number, just so longas it uniquely identifies the edit for use by thehis-tory_get_commandat a later date. The edit num-ber is to be printed on the standard output.Thiscommand may be executed by developers.




history_query_command ="get -t -g ${d $h}/s.${b $h}";

Note that "get" reports the edit number on stdout.

4.4.5. Templates

The lib/config.example/sccsfi le in the Aegis dis-tribution contains all of the above commands(installed as /usr/local/share/aegis/example.con-fig/sccsby default) so that you may readily insertthem into your project configuration file (calledaegis.confby default, seeaepconf(5) for how tocall it something else).


User Guide Aegis


In addition, it is possible to have a much moreuseful description for the−y option. For exam-ple: “-y${quote ($version) ${changedescription}} ” i nserts the version numberand the brief description into the file’s log. Thismeans that using thesccs prs(1) command willprovide quite useful summaries.

4.4.6. BinaryFiles

SCCS is unable to cope with binary files. How-ev er, Aegis will transparently encode all suchfi les, if you leave the history_content_limitationfield unset.


Aegis UserGuide

4.5. UsingRCS

The entries for the commands are listed below.RCS uses a slightly different model than aegiswants, so some maneuvering is required.Thecommand strings in this section assume that theRCS commandsci andco andrcs andrlog are inthe command search PATH, but you may like tohard-wire the paths, or set PATH at the start ofeach. You should also note that the strings arealways handed to the Bourne shell to be executed,and are set to exit with an error immediately asub-command fails.

In these commands, the RCS file is keptunlocked, since only the owner will be checkingchanges in. The RCS functionality for coordinat-ing shared access is not required.

One advantage of using RCS version 5.6 or lateris that binary files are supported, should you wantto have binary files in the baseline.


This command is used to create a new file history.This command is always executed as the projectowner.





history_create_command ="ci -u -d -M -m$c -t/dev/null \$i $h,v; rcs -U $h,v";

The "ci -u " option is used to specify that anunlocked copy will remain in the baseline.The"ci -d " option is used to specify that the filetime rather than the current time is to be used forthe new revision. The"ci -M " option is used tospecify that the mode date on the original file isnot to be altered. The "ci -t " option is used tospecify that there is to be no description text forthe new RCS file. The"ci -m " option is used tospecify that the change number is to be stored inthe file log if this is actually an update (typicallyfrom aenfafteraermon the same file name).The"rcs -U " option is used to specify that the newRCS file is to have unstrict locking.

It is essential that thehistory_create_commandand thehistory_put_commandare identical. It isa historical accident that there are two separatecommands: before Aegis supported branches, this

was not a requirement.


This command is used to get a specific edit backfrom history. This command is always executedas the project owner.






history_get_command ="co -r’$e’ -p $h,v > $o";

The "co -r option is used to specify the edit tobe retrieved. The"co -p option is used to spec-ify that the results be printed on the standard out-put; this is because the destination filename willnever look anything like the history source file-name.







history_put_command ="ci -u -d -M -m$c -t/dev/null \$i $h,v; rcs -U $h,v";

Uses ci to deposit a new revision, using -d and -Mas described for history_create_command.The-m flag stores the change number in the file log,which allows rlog(1) to be used to find the Aegischange numbers to which each revision of the filecorresponds.

The "ci -u " option is used to specify that anunlocked copy will remain in the baseline.The"ci -d " option is used to specify that the filetime rather than the current time is to be used forthe new revision. The"ci -M " option is used tospecify that the mode date on the original file is


User Guide Aegis

not to be altered. The "ci -m " option is used tospecify that the change number is to be stored inthe file log, which allows rlog to be used to findthe change numbers to which each revision of thefi le corresponds. You might want to use-m$p,$c instead which stores both the projectname and the change number. Or -m$version ,which will be composed of the branch and thedelta. Thesemake it much easier to track changesacross branches.

It is essential that thehistory_create_commandand thehistory_put_commandare identical. It isa historical accident that there are two separatecommands: before Aegis supported branches, thiswas not a requirement.


This command is used to query what the historymechanism calls the top-most edit of a historyfi le. The result may be any arbitrary string, itneed not be anything like anumber, just so longas it uniquely identifies the edit for use by thehis-tory_get_commandat a later date.The edit num-ber is to be printed on the standard output.Thiscommand is always executed as the project owner.




history_query_command ="rlog -r $h,v | ""awk ’/ˆrevision/ {print $$2}’";

4.5.5. merge_command

RCS also provides amerge program, which canbe used to provide a three-way merge.

All of the command substitutions described inaesub(5) are available. In addition, the followingsubstitutions are also available:

${ORiginal}The absolute path name of a file containingthe version originally copied. Usually in atemporary file.

${Most_Recent}The absolute path name of a file containingthe most recent version. Usuallyin thebaseline.

${Input}The absolute path name of the edited ver-sion of the file. Usuallyin the developmentdirectory. Aegis usually moves the original

source file aside, so that the output mayhave the source file’s name.

${Output}The absolute path name of the file in whichto write the difference listing.Usually inthe development directory, usually thename of a change source file.


merge_command ="set +e; ""merge -p -L baseline -L C$c "" $ mr $orig $in > $out; ""test $? -le 1";

The "merge -L " options are used to specifylabels for the baseline and the development direc-tory, respectively, when conflict lines are insertedinto the result. The "merge -p " options is usedto specify that the results are to be printed on thestandard output.

It is important that this command does not moveits input and output files around, otherwise thiscontradicts the warnings Aegis may issue to theuser. (In previous versions of Aegis, this was nec-essary, howev er this is no longer the case.)

Warning: The version of diff3(1) available toRCS merge(1) has a huge impact on its perfor-mance and utility. You need to grab and installGNU diff to get the best results.Unfortunatelythe diff tool used by RCSmerge(1) is determinedat compile time. This means that you need tobuild and install GNU diff packagebefore youbuild and install GNU RCS package.

4.5.6. Referential Integrity

Many history tools (including RCS) can modifythe contents of the file when it is committed.While there are usually options to turn this off,they are seldom used.The problem is: if the com-mit changes the file, the source in the repositorynow no longer matches the object file in therepository -i.e. the history tool has compromisedthe referential integrity of the repository.

history_put_trashes_file = warn;

If you use RCS keyword substitution, you willneed this line. (The default is to report a fatalerror.)

Another reason for this option is that it tells Aegisit needs to recalculate the file’s fingerprint after acheckin.


Aegis UserGuide

4.5.7. Templates

The lib/config.example/rcsfi le in the Aegis distri-bution (installed as /usr/local/share/aegis/con-fig.example/rcs by default) contains all of theabove commands, so that you may readily insertthem into your project configuration file.


In addition, it is possible to have a much moreuseful description for the−m option. For exam-ple: “-m${quote ($version) ${changedescription}} ” i nserts the version numberand the brief description into the file’s log. Thismeans that using therlog(1) command will pro-vide quite useful summaries.

4.5.8. BinaryFiles

RCS (version 5.6 and later) is able to cope withbinary files. It does so by saving a whole copy ofthe file at each check-in.

If you want Aegis to transparently encode all suchfi les, simply leave the history_content_limitationfield unset.

If you want to check-in binary files, add the−kboption to each of thercs -U commands in thefields above, and also set


so that Aegis knows that no encoding is desired.

4.5.9. history_put_trashes_files

If you use RCS keywords, such as $id $ or$log $, this will result in the file in the baselinebeing changed by RCS at integrate pass.This isafter the build. Theresult is that the source filesno longer match the object files. Oops.

While such mechanism are essential when usingonly a simple history tool, far more informationmay be obtained using the file history report (aerfile_history filename), rendering such crudemethods unnecessary.

In addition to expected expansions in file headercomments, this can also be very destructive if, forexample, such a string appeared in a uuencodedor MIME base 64 encoded file.

If you wish to prevent RCS from performingkeyword expansion, used thercs -kb option.

If, however, you wish to keep using keywordexpansion, set

history_tool_trashes_file = warning;

to cause Aegis to warn you, rather than fail.


User Guide Aegis

4.6. Usingfhist

The fhist program was written by David I. Belland is admirably suited to providing a historymechanism with out the "cruft" that SCCS andRCS impose.

Please note that the[# edit #] feature needsto be avoided, or the-Forced_Update(-fu) flagneeds to be used in addition to the-Condi-tional_Update(-cu) flag, otherwise updates willcomplain that “Input file "XXX" contains editAinstead ofB for module "YYY"”

The history_create_commandand the his-tory_put_commandare intentionally identical.This minimizes problems when using branches.


This command is used to create a new project his-tory. The command is always executed as theproject owner.





history_create_command ="fhist ${b $h} -create -cu ""-i $i -p ${d $h} -r";

Note that the source file is left in the baseline.


This command is used to get a specific edit backfrom history. The command may be executed bydevelopers.






history_get_command ="fhist ${b $h} -e ’$e’ -o $o ""-p ${d $h}";

Note that the destination filename willneverlookanything like the history source filename, so the-p is essential.







history_put_command ="fhist ${b $h} -create -cu ""-i $i -p ${d $h} -r";

Note that the source file is left in the baseline.


This command is used to query what the historymechanism calls the "top-most" edit of a historyfi le. The result may be any arbitrary string, itneed not be anything like a number, just so longas it uniquely identifies the edit for use by thehis-tory_get_commandat a later date. The edit num-ber is to be printed on the standard output.Thiscommand may be executed by developers.




history_query_command ="fhist ${b $h} -l 0 ""-p ${d $h} -q";

4.6.5. Templates

The lib/config.example/fhistfi le in the Aegis dis-tribution (installed as/usr/local/share/aegis/con-fig.example/fhistby default) contains all of theabove commands, so that you may readily insertthem into your project configuration file.

4.6.6. Capabilities

By default, FHist is unable to cope will NULcharacters in its input files, however this is theonly limitation. By default, Aegis expects thathistory tools are only able to cope with printableASCII text. To tell it ontherwise, set

history_content_limitation =international_text;

in the projectaegis.conffi le.


Aegis UserGuide

Aegis will transparently encode binary files (fileswhich contain NUL characters) on entry and exitfrom the history tool. This means that you mayhave binary files in your project without configur-ing anything special.

4.6.7. BinaryFiles

FHist (version 1.7 and later) has support forbinary files. The fhist −binary option may beused to specify that the file is binary, that it maycontain NUL characters. It is essential that youhave consistent presence or absence of the−binary option for each file when combined withthe −CReate, −Update, −Conditional_Update and−Extract options.Failure to do so will produceinconsistent results.

This means that you have to always use the−binary option in thehistory_create_commandand history_put_commandfields. You have todecide right at the very beginning if your projecthistory will ever hav e binary files, or will neverhave binary files. You can’t change your mindlater. If you choose to use the −binary option, set


However, Aegis would transparently encode allsuch files, if you leave the history_content_-limitation field set for international text. In somecases, Aegis’ encoding will be more efficient thanfhist’s. Andyou have the advantage of being ableto change your mind later.


User Guide Aegis

4.7. DetectingHistory File Corruption

When you have files which exist for long periodsof time, particularly files such as the ones typi-cally used by history tools, which are generallyappended to, without modification of the bulk ofthe file, there is a very real possibility that a blockof the file could become corrupted over theyears.16 Unless you access the file versions con-tained within that block, you have no way ofknowing whether or not the history file is OK.(Arguably, the operating system should check forthis, but many do not, and in any case the errormay not be detectable at that level.)

Using Aegis, you can add a simple checksum toyour history files which will detect many cases ofcorruption such as this, for all of the commonlyused history tools.Note: it cannot detect all cor-ruptions (nothing can) but it will detect more thanmany operating systems will.

You don’t need to use this technique with SCCSor aesvt(1), they already have checksums in theirfi les.

4.7.1. GeneralMethod

In general, you need to do three things:

1. You need to create some kind of checksum ofyour history file each time you modify it.Something like md5sum(1) from the GNUFileutils would be good. Store the checksumin a file next to the history file. Thiswould bedone in the history_create_commandandhistory_put_commandfields of the projectaegis.conffi le.

2. Eachtime the file is read, you need to verifythe file’s checksum. Usethe same checksumutility as before, and then compare it using,say, cmp(1); it it fails (either an IO error, orthe checksum doesn’t compare equal) thendon’t proceed with the history file access.You may need to repair or replace the disk.You will need to restore from backup (yester-day’s backup, see below). This would bedone at the beginning of thehistory_create_-command, history_put_command, history_-get_command and history_query_commandfields of the projectaegis.conffi le.

3. Becauseyou may not actually interact withthe file for years at a time, you need to checkthe file fingerprints much more often.Daily

16 See also Saltzer, J.H. et al (1981)End-to-endarguments in system design, http://web.mit.edu/-Saltzer/www/publications/endtoend/endtoend.pdf

or at least weekly is suggested.You do thiswith acron(1) job run nightly which comparesall of the history files with theirmd5sum(1)checksums. Emailfailures to the systemadministrator and the project administrators.By doing this nightly, you not only avoidbacking-up corrupted files, you will alwaysknow on which backup tape the good copyresides - yesterday’s.

4.7.2. Configuration Commands

In order to implement this, you need to modifysome fields of your projectaegis.conffi le as fol-lows:

history_create_commandYou need to test if the history file and itschecksum file exist, and check the checksumif this is the case. Then, use whichever his-tory tool you choose (see the previous sec-tions of this chapter).If it succeeds, runmd5sum(1) over the history file (not thesource file) and store the checksum in a filenext to the history tool’s file. Using thesame filename plus a.md5sum extensionmakes thecron(1) job easier to write.

history_put_commandYou need to test if the file exists (it may, forexample, be an old project to which you haverecently added this technique) and check thechecksum if this is the case. Then, use yourhistory tool as normal.If it succeeds, runmd5sum(1) over the history file (not thesource file) as in the create case.

history_get_commandYou need to test if the file exists (it may, forexample, be an old project to which you haverecently added this technique) and check thechecksum if this is the case.Then use yourhistory tool as normal.

history_head_commandThis command is only used ataeipassfi le,immediately after one of thehistory_create_-command or history_put_commandcom-mands. Itis up to you whether you thinkyou need to add a guard as for thehistory_-get_command field.

4.7.3. AnAlternati ve

Rather than runmd5sum(1) on the history fileseach time you modify them, you could usegzip(1)to obtain some minor compression, but it alsoprovides and Adler32 checksum of the file. Forfi les with long histories, this can be tedious to


Aegis UserGuide

unpack every time you need to extract an old ver-sion, but such operations are frequently I/Obound, and so there may be no perceived slow-ness by the user..

4.7.4. Aegis’Database

In addition to your history files, Aegis maintains adatabase of file meta-data. In order to add achecksum to the various file making up thedatabase, turn on thecompressed_databaseproject attribute. In addition to compressing thedatabase (a minor savings) it also adds an Adler32checksum.

You can check this in thecron(1) job by usinggzcat(1) sent to/dev/null.


User Guide Aegis

5. TheDependency Maintenance Tool

Aegis can place heavy demands on the depen-dency maintenance tool, so it is important thatyou select an appropriate one. This chapter talksabout what features a dependency maintenancetool requires, and gives examples of how to usethe various alternatives.

5.1. Required Features

The heart of any DMT is an inference engine.This inference engine accepts agoal of what youwant it to construct and a set ofrules for how toconstruct things, and attempts to construct whatyou asked for given the rules you specified. Thisis exactly a description of an expert system, andthe DMT needs to be an expert system for con-structing files. Somethinglike PROLOG is prob-ably ideal.

Aegis is capable of supporting a wide variety ofdevelopment directory styles. The differentdevelopment directory styles place differentdemands on the dependency maintenance tool.Development directory styles will be described inthe next section, but here is a quick summary:

copy of all sources:This is what CVS does, and what manyother VC tool do. Because you have acomplete copy of all source files, thedependency maintenance tool only needs tobe aware of one directory tree.

copy of everything:This is a small optimization of the previouscase to cut down the time required for thatfi rst build,because the derived files from theintegration build can be reused.

link all sourcesThe is an optimization of the "copy allsources" case, because linking a file is sig-nificantly faster than making a copy of afi le. The dependency maintenance toolonly needs to be aware of one directorytree.

link everythingThis is an optimization of the previous case,again reusing derived files from the integra-tion build, except that you need to ensurethat your dependency maintenance tool isconfigured to remove the derived file out-puts of each rule before creating them, toavoid corrupting the baseline or getting"permission denied" error.

view pathThis is the most efficient developmentdirectory style, and it scales much betterthan any of the above, but the dependencymaintenance tool must be able to cope witha hierarchy of parallel source directorytrees. Thesetrees for a "view path", a listof directories that programs search below tofind the files of interest. The vpathstatements of GNU Make arealmost, but not quite, capa-ble of being used in thisway.

5.1.1. View Paths

For the union of all files in a project and all filesin a change (remembering that a change onlycopies those files it is modifying, plus it may addor remove files) for all files you must be able tosay to the dependency maintenance tool,

"If and only if the file is up-to-date inthe baseline, use the baseline copy ofthe file, otherwise construct the file inthe development directory".

The presence of a source file in the change makesthe copy in the baseline out-of-date.

Most DMTs with this capability implement it byusing some sort of search path, allowing a hierar-chy of directories to be scanned with little or nomodification to the rules.

If your DMT of choice does not provide this func-tionality, the development_directory_style.-source_file_symlinkfield of the project configura-tion file may be set totrue, which tells Aegis tomaintain symbolic links in the development direc-tory for all source files in the baseline which arenot present in the development directory. (Seeaepconf(5) and aeb(1) for more information.)This incurs a certain amount of overhead whenAegis maintains these links, but a similar amountof work is done within DMTs which have searchpath functionality.

5.1.2. DynamicInclude File Dependencies

Include file dependencies are very important,because a change may alter an include file, and allof the sources in the baseline which use thatinclude file must be recompiled.

Consider the example given earlier: the includefi le describing the interface definition of a func-tion is copied into a change and edited, and so isthe source file defining the function. It is


Aegis UserGuide

essential that all source files in the baseline whichinclude that file are recompiled, which will usu-ally result in suitable diagnostic errors if any ofthe clients of the altered function have yet to beincluded in the change.

There are two ways of handling include filedependencies:

• They can be kept in a file, and the file can bemaintained by suitable programs (maintaining itmanually never works, that’s just human nature).

• They can be determined by the DMT when it isscanning the rules to determine what needs updat-ing.

5.1.2.1. StaticFile

Keeping include dependencies in a file has a num-ber of advantages:

• Most existing DMTs have the ability to includeother rules files, so that when performing a devel-opment build from a baseline rules file, it couldinclude a dependencies file in the developmentdirectory.

• Reading a file is much faster than scanning all ofthe source files.

Keeping include dependencies in a file has a num-ber of disadvantages:

• The file is independent of the DMT, it is eithergenerated before the DMT is invoked, in whichcase it may do more work than is necessary, or itmay be invoked after the DMT (or after the DMThas scanned its rules), in which case it may wellbe out-of-date when the DMT needs it.

For example, the use ofgcc -M produces "dot d"fi les, which may be merged to construct such anincludable dependency file. This happens afterthe DMT has read and applied the rules, but pos-sibly before the DMT has finished executing.17

• Many tools which can generate this information,such as thegcc -Moption, are triggered by sourcefi les, and are unable to manage a case where it isan include file which is changing, to include a dif-ferent set of other include files. In this case, theinaccurate dependencies file may contain refer-ences to the old set of nested include files, someof which may no longer exist, This causes theDMT to incorrectly generate an error stating thatthe old include file is missing, when it is actually

17 See theUsing Make section for how GNUMake may be used.It effectively combines bothmethods: keeping.d fi les and dynamically updatingthem. Becauseit combines both methods, it hassome of the advantages and disadvantages of both.

no longer required.

If a DMT can only support this kind of includefi le dependencies, it is not suitable for use withAegis.

5.1.2.2. Dynamic

In order for a DMT to be suitable for use withAegis, it is essential that rules for the DMT maybe specified in such a way that include file depen-dencies are determined "on the fly" when theDMT is determining if a given rule is applicable,and before the rule is applied.

This method suffers from the problem beingrather slow; but this is amenable to some cachingand the losses of performance are not as bad ascould be imagined.

This method has the advantage of correctness inall cases, where a static file may at times be out-of-date.

5.2. Development Directory Style

The project configuration file, usually calledaegis.conf, contains a field calleddevelopment_-directory_stylewhich controls how the projectsources are presented to the DMT.

Seeaepconf(5) for a complete description of thisfield.

There is a correspondingintegration_directory_-stylefield, which defaults to the same value as thedevelopment_directory_style. It is usually a verybad idea if these two are different.

5.2.1. View Path

By not settingdevelopment_directory_styleat allthe only source files present in the developmentdirectory are source files being create and/ormodified.

By using information provided by the$search_path substitution, the build can accessthe unchanged source files in the branch baselineand deeper branch baselines.The great thingabout this approach is that there are also "precom-piled" object files on the viewpath, so if an objectfi le does not need to be compiled (there are nosoure files in the development directory that haveanything to do with it) then the build can simplylink the unchanged object files in the baselinewithout recompiling.

This build method scales the best, and is theAegis default.


User Guide Aegis

The difficulties of finding a DMT which is capa-ble of coping with a view path means that this isnot the only work area style. All other methodsscale less well than a view path; some scalemuchless well.

5.2.2. Link the Baseline

The first two sub-fields of interest in thedevelopment_directory_styleare source_file_linkandsource_file_symlink.

source_file_link = true; This field is true if hardlinks are to be used for project source files(which are not part of the change) so thatthe work area has a complete set of sourcefi les.

source_file_symlink = true;This field is true ifsymbolic links are to be used for projectsource files (which are not part of thechange) so that the work area has a com-plete set of source files.

By using these settings, all source files are presentin the development directory. They will be read-only. As you decide to modify files in the changeset, theaecpcommand will remove the link andreplace it with a read-write copy of the file.

You need both these sub-fields set, because hardlinks are not allowed to cross file system bound-aries. Aegis will use hard links in preference tosoft links when it can.

Maintaining the hard links can be time consumingfor large projects, and add quite a noticeabledelay before builds start doing anything. Butseethe −assume-symbolic-linksoption of theaeb(1)command; use it sparingly.

The biggest penalty with this method is that theinitial build for a change set for a large projectcan bevery time consuming.Recall that the base-line has a complete "prebuild" already available.To take advantage of these pre-built derived files,there are a few more sub-fields:

derived_file_copy = true; This field is true ifcopies are to be used for non-source fileswhich are present in the project baseline butwhich are not present in the work area, sothat the work area has a complete set ofderived files.

derived_at_start_only = true;This settign causesthe above fields controling the appearanceof derived files to be acted upon only whenthe development directory is created (ataedb(1) time).

Copying files can be very time consuming andalso eats a lot of disk space. If you are preparedto change your build slightly, it is possible to usethe following fields:

derived_file_link = true; This field is true if hardlinks are to be used for non-source fileswhich are present in the project baseline butwhich are not present in the work area, sothat the work area has a complete set ofderived files.

derived_file_symlink = true;This field is true ifsymbolic links are to be used for non-source files which are present in the projectbaseline but which are not present in thework area, so that the work area has a com-plete set of derived files.

Just as for source files, hard links will be used inpreference to symbolic links if possible.

Note thatevery rule in your Makefile (or whateveryour DMT uses)must remove its outputs beforedoing enything else, to break the links to the filesin the baseline, otherwise you will corrupt thebaseline. Aegis tries very hard to ensure that thebaseline files (and thus the links) are read-only, sothat you get an error from the build if you forgetto break a link.

This development directory style is called "archstyle" after Tom Lord’s arch (tla) which doessomething very similar.

If you are placing an existing project under Aegis,do the above three things one step at a time.Firstget the source files available and integrate that.Ina second change set get derived file copies work-ing. In a third change set (if you do it at all)change the build and use derived file links.

5.2.3. CopyAll Sources

The sub-fields of interest in thedevelopment_-directory_styleis source_file_copy.

source_file_copy = true; This field says copiesare to be used for project source files(which are not part of the change) so thatthe work area has a complete set of sourcefi les. Filemodification time attributes willbe preserved.

By using this setting, all source files are present inthe development directory. They will be read-only. As you decide to modify files in the changeset, theaecpcommand will remove the file andreplace it with a read-write copy of the file.


Aegis UserGuide

Maintaining the copies can be time consuming forlarge projects, and add quite a noticeable delaybefore builds start doing anything. But see the−assume-symbolic-linksoption of the aeb(1)command; use it sparingly (yes, it applies tocopies as well).

The biggest penalty with this method is that theinitial build for a change set for a large projectcan bevery time consuming. Recall that the base-line has a complete "prebuild" already available.To take advantage of these pre-built derived files,there are a few more sub-fields:

derived_file_copy = true;This says copies are tobe used for non-source files which arepresent in the project baseline but which arenot present in the work area, so that thework area has a complete set of derivedfi les.

derived_at_start_only = true;This setting causesthe above fields controlling the appearanceof derived files to be acted upon only whenthe development directory is created (ataedb(1) time).

This development directory style is called "CVSstyle" after GNU CVS which does somethingvery similar.

5.2.4. ObsoleteFeatures

There are several fields in the aegis.conf fi lewhich are obsolete.Aegis will automaticallytransfer these to create adevelopment_directory_-styleif you haven’t specified one.

create_symlinks_before_build:This is like settingboth development_directory_style.source_-file_symlink and development_directory_-style.derived_file_symlinkat the same time.

remove_symlinks_after_build:This is like settingthe development_directory_style.during_build_-only field.

create_symlinks_before_integration_build:This islike setting both integration_directory_-style.source_file_symlinkand integration_-directory_style.derived_file_symlinkat thesame time.

remove_symlinks_after_integration_build:This islike setting the integration_directory_-style.during_build_onlyfield.

Aegis will print a warning if you use any of thesefields.


User Guide Aegis

5.3. UsingCook

The Cookprogram is the only dependency main-tenance tool, known to the author, which is suf-ficiently capable to supply Aegis’ needs.18 Toolssuch ascakeandGNU Make are described later.They need a special tweak to make them work.

This section describes appropriate contents for theHowto.cookfi le, input to thecook(1) program. Italso discusses thebuild_commandandintegrate_-build_commandand link_baselineand change_-file_command and project_file_commandandlink_integration_directoryfields of the configura-tion file. Seeaepconf(5) for more informationabout this file.

5.3.1. Invoking Cook

Thebuild_commandfield of the configuration fileis used to invoke the relevant build command.Inthis case, it is set as follows

build_command ="cook -b ${s Howto.cook} -nl\

project=$p change=$c version=$v";

This command tells Cook where to find therecipes. The${s Howto.cook} expands to apath into the baseline during development if thefi le is not in the change. Look inaesub(5) formore information about command substitutions.

The recipes which follow will all remove their tar-gets before constructing them, which qualifiesthem for the next entry in the configuration file:

link_integration_directory = true;

The links must be removed first, otherwise thebaseline would cease to be self-consistent.

5.3.2. TheRecipe File

The file containing the recipes is calledHowto.cookand is given to Cook on the com-mand line.

The following items are preamble to the rest ofthe file; they ask Aegis for the source files of theproject and change so that Cook can determinewhat needs to be compiled and linked.

18 The version in use when writing this sectionwas 1.5. All versions from 1.3 onwards are knownto work with the recipes described here.

project_files =[collect_lines aelpf

-p [project] -c [change]];change_files =

[collect_lines aelcf-p [project] -c [change]];

source_files =[stringset [project_files]

[change_files]];

This example continues the one from chapter 3,and thus has a single executable to be linked fromall the object files

object_files =[fromto %.y %.o [match_mask %.y

[source_files]]][fromto %.l %.o [match_mask %.l

[source_files]]][fromto %.c %.o [match_mask %.c

[source_files]]];

It is necessary to determine if this is a develop-ment build, and thus has the baseline for addi-tional ingredients searches, or an integrationbuild, which does not.The version supplied byAegis will tell us this information, because it willbe major.minor.Cchange for development buildsandmajor.minor.Ddeltafor integration builds.

if [match_mask %1C%2 [version]] then{

baseline = [collect aegis -cd -bl-p [project]];

search_list = . [baseline];}

The search_list variable in Cook is the list ofdirectories to search for dependencies; it defaultsto only the current directory. The resolvebuiltinfunction of Cook may be used to ask Cook for thename of the file actually used to resolve depen-dencies, so that recipe bodies may reference theappropriate file:

example: [object_files]{

[cc] -o example[resolve [object_files]]-ly -ll;

}

This recipe says that to Cook the example pro-gram, you need the object files determined earlier,and them link them together. Object files whichwere up to date in the baseline are used whereverpossible, but files which were out of date are con-structed in the current directory and those will belinked.


Aegis UserGuide

5.3.3. TheRecipe for C

Next we need to tell Cook how to manage Csources. Onthe surface, this is a simple recipe:

%.o: %.c{

rm %.o;[cc] [cc_flags] -c %.c;

}

Unfortunately it has forgotten about finding theinclude file dependencies. The Cook packageincludes a program calledc_incl which is used tofind them. The recipe now becomes

%.o: %.c: [collect c_incl -eia %.c]{

rm %.o;[cc] [cc_flags] -c %.c;

}

The file may not always be present to be removed(causing a fatal error), and it is irritating toexecute a redundant command, so the remove ismangled to look like this:

%.o: %.c: [collect c_incl -eia %.c]{

if [exists %.o] thenrm %.o

set clearstat;[cc] [cc_flags] -c %.c;

}

The "set clearstat" clause tells Cook that the com-mand will invalidate parts of itsstatcache, and tolook at the command for what to invalidate.

Another thing this recipe needs is to use the base-line for include files not in a change, and so therecipe is altered again:

%.o: %.c: [collect c_incl -eia[prepost "-I" "" [search_list]]

%.c]{


set clearstat;[cc] [cc_flags] [prepost "-I" ""

[search_list]] -c %.c;}

See theCook Reference Manualfor a descriptionof the prepostbuiltin function, and other Cookdetails.

There is one last change that must be made to thisrecipe, it must use the resolve function to refer-ence the appropriate file once Cook has found iton the search list:

%.o: %.c: [collect c_incl -eia[prepost "-I" "" [search_list]]

[resolve %.c]]{


set clearstat;[cc] [cc_flags] [prepost "-I" ""

[search_list]] -c [resolve %.c];}

Only use this last recipe for C sources, the othersare only shown so that the derivation of the recipeis clear; while it is very similar to the original, itlooks daunting at first.

5.3.3.1. CInclude Semantics

The semantics of C include directives make the

#include " filename"

directive dangerous in a project developed withthe Aegis program and Cook.

Depending on the age of your compiler, whetherit is AT&T traditional C or newer ANSI C, thisform of directive will search first in the currentdirectory and then along the search path, or in thedirectory of the including file and then along thesearch path.

The first case is fairly benign, except that compil-ers are rapidly becoming ANSI C compliant, andan operating system upgrade could result in anasty surprise.

The second case is bad news. If the source file isin the baseline and the include file is in thechange, you don’t want the source file to use theinclude file in the baseline.

Always use the

#include < filename>

form of the include directive, and set the includesearch path explicitly on the command line usedby Cook.

Cook is able to dynamically adapt to include filedependencies, because they are not static. Thepresence of an include file in a change means thatany file which includes this include file, whetherthat source file is in the baseline or in the change,must have a dependency on the change’s includefi le. Potentially, files in the baseline will need tobe recompiled, and the object file stored in thechange, not the baseline. Subsequent linkingneeds to pick up the object file in the change, notfrom the baseline.


User Guide Aegis

5.3.4. TheRecipe for Yacc

Having explained the complexities of the recipesin the above section about C, the recipe for yaccwill be given without delay:

%.c %.h: %.y{

if [exists %.c] thenrm %.c

set clearstat;if [exists %.h] then

rm %.hset clearstat;

[yacc] [yacc_flags] -d[resolve %.y];

mv y.tab.c %.c;mv y.tab.h %.h;

}

This recipecould be jazzed up to cope with thelisting file, too, if that was desired, but this is suf-ficient to work with the example.

Cook’s ability to cope with transitive dependen-cies will pick up the generated .c file and con-struct the necessary .o file.

5.3.5. TheRecipe for Lex

The recipe for lex is vary similar to the recipe foryacc.

%.c: %.l{

if [exists %.c] thenrm %.c

set clearstat;[lex] [lex_flags] -d [resolve %.l];mv lex.yy.c %.c;

}

Cook’s ability to cope with transitive dependen-cies will pick up the generated .c file and con-struct the necessary .o file.

5.3.6. Recipesfor Documents

You can format documents, such as user guidesand manual entries with Aegis and Cook, and therecipes are similar to the ones above.

%.ps: %.ms: [collect c_incl -r -eia[prepost "-I" "" [search_list]][resolve %.ms]]

{if [exists %.ps] then

rm %.psset clearstat;

roffpp [prepost "-I" ""[search_list]] [resolve %.ms]| g roff -p -t -ms> [ target];

}

This recipe says to run the document throughgroff, with the pic(1) and tbl(1) filters, use thems(7) macro package, to produce PostScript out-put. Theroffpp program comes with Cook, and islike soelim(1) but it accepts include search pathoptions on the command line.

Manual entries may be handled in a similar way

%.cat: %.man: [collect c_incl -r -eia[prepost "-I" "" [search_list]][resolve %.man]]

{if [exists %.cat] then

rm %.catset clearstat;

roffpp [prepost "-I" ""[search_list]] [resolve %.man]| g roff -Tascii -t -man> [ target];

}

5.3.7. Templates

The lib/config.example/cookfi le in the Aegis dis-tribution contains all of the above commands, sothat you may readily insert them into your projectconfiguration file.


Aegis UserGuide

5.4. UsingCake

This section describes how to use cake as thedependency maintenance tool.The cakepackagewas published in thecomp.sources.unixUSENETnewsgroup volume 12, around February 1988,and is thus easily accessible from the manyarchives around the internet.

It does not have a search path of any form,not even something like VPATH. It does, how-ev er, hav e facilities for dynamic include filedependencies.

5.4.1. Invoking Cake

Thebuild_commandfield of the configuration fileis used to invoke the relevant build command.Inthis case, it is set as follows

build_command ="cake -f ${s Cakefile} \

-DPROJECT=$p -DCHANGE=$c \-DVERSION=$v";

This command tellscakewhere to find the rules.The${s Cakefile} expands to a path into thebaseline during development if the file is not inthe change. Look inaesub(5) for more informa-tion about command substitutions.

The rules which follow will all remove their tar-gets before constructing them, which qualifiesthem for the next entry in the configuration file:


The links must be removed first, otherwise thebaseline would be corrupted by integration builds.

Another field to be set in this file is

development_directory_style ={

source_file_symlink = true;};

which tells Aegis to maintain symbolic linksbetween the development directory and the base-line. This also requires that rules remove theirtargets before constructing them, to ensure thatrules do not attempt to write their results onto theread-only versions in the baseline.

5.4.2. TheRules File

The file containing the rules is calledCakefileandis given to cake on the command line.

The following items are preamble to the rest ofthe file; they ask Aegis for the source files of theproject and change so that cake can determinewhat needs to be compiled and linked.

#define project_files \[[aelpf -p PROJECT \

-c CHANGE]];#define change_files \

[[aelcf -p PROJECT \-c CHANGE]];

#define source_files \project_files change_files

#define CC gcc#define CFLAGS -O

This example parallels the one from chapter 3,and thus has a single executable to be linked fromall the object files

#define object_files \[[sub -i X.c %.o source_files]] \[[sub -i X.y %.o source_files]] \[[sub -i X.l %.o source_files]]

Constructing the program is straightforward

example: object_filesrm -f exampleCC -o example object_files

This rule says that to construct the example pro-gram, you need the object files determined earlier,and them link them together. Object files whichwere up to date in the baseline are used whereverpossible, but files which were out of date are con-structed in the current directory and those will belinked.

5.4.3. TheRule for C

Next we need to tell cake how to manage Csources. Onthe surface, this is a simple rule:

%.o: %.cCC CFLAGS -c %.c

paralleling that found in most makes, however itneeds to delete the target first, and to avoid delet-ing the.o fi le whenever cake thinks it is transitive.

%.o!: %.crm -f %.oCC CFLAGS -c %.c

The -f option to therm command is because thefi le does not always exist.

Unfortunately this rule omits finding the includefi le dependencies. The cake package includes aprogram calledccincl which is used to find them.The rule now becomes

%.o!: %.c* [[ccincl %.c]]rm -f %.oCC CFLAGS -c %.c

This rule is a little quirky about include fileswhich do not yet exist, but must be constructed by


User Guide Aegis

some other rule.You may want to usegcc -MMinstead, which is almost as quirky when used withcake. Another alternative, used by the authorwith far more success, is to use thec_incl pro-gram from thecookpackage, mentioned in an ear-lier section. The gcc -MMunderstands C includesemantics perfectly, the c_incl command cachesits results and thus goes faster, so you will need tofigure which you most want.

5.4.3.1. IncludeDirectives

Unlike cookdescribed in an earlier section, usingcake as described here allows you to continueusing the


form of the include directive. This is because thedevelopment directory appears, to the compiler, tobe a complete copy of the baseline.

5.4.4. TheRule for Yacc

Having explained the complexities of the rules inthe above section about C, the rule for yacc willbe given without delay:

#define YACC yacc#define YFLAGS

%.c! %.h!: %.y if exist %.yrm -f %.c %.h y.tab.c y.tab.hYACC YFLAGS -d %.ymv y.tab.c %.cmv y.tab.h %.h

This rule could be jazzed up to cope with the list-ing file, too, if that was desired, but this is suf-ficient to work with the example.

Cake’s ability to cope with transitive dependen-cies will pick up the generated.c fi le and con-struct the necessary.o fi le.

5.4.5. TheRule for Lex

The rule for lex is vary similar to the rule foryacc.

#define LEX lex#define LFLAGS

%.c!: %.l if exist %.lrm -f %.cLEX LFLAGS %.lmv lex.yy.c %.c

Cake’s ability to cope with transitive dependen-cies will pick up the generated.c fi le and con-struct the necessary.o fi le.

5.4.6. Rulesfor Documents

You can format documents, such as user guidesand manual entries with Aegis and cake, and therules are similar to the ones above.

%.ps!: %.ms* [[soincl %.ms]]rm -f %.psgroff -s -p -t -ms %.ms > %.ps

This rule says to run the document through groff,with the soelim(1) and pic(1) and tbl(1) filters,use the ms(7) macro package, to producePostScript output.

This suffers from many of the problems withinclude files which need to be generated, as doesthe C rule, above. You may want to usec_incl -rfrom thecookpackage, rather than thesoinclsup-plied by thecakepackage.


%.cat!: %.man* [[soincl %.man]]rm -f %.catgroff -Tascii -s -t -man %.man \

> %.cat


Aegis UserGuide

5.5. UsingMake

The make(1) program exists in many forms, usu-ally one is available with eachUNIX version. Theone used in the writing of this section isGNUMake 3.70, available by anonymous FTP fromyour nearest GNU archive site. GNU Make waschosen because it was the most powerful, it iswidely available (usually for little or no cost) anddiscussion of the alternatives (SunOS make, BSD4.3 make, etc), would not be universally applica-ble. "Plainvanilla" make (with no transitive clo-sure, no pattern rules, no functions) is not suf-ficiently capable to satisfy the demands placed onit by Aegis.

With the introduction of thedevelopment_-directory_stylefield of the project configurationfi le, any project which is currently using a "plainvanilla" make may continue to use it, and stillmanage the project using Aegis.

As mentioned earlier in this chapter, makeis notreally sufficient, because it lacks dynamic includedependencies. However, GNU Make has a formof dynamic include dependencies, and it has a fewquirks, but mostly works well.

The other feature lacking inmakeis a search path.While GNU Make has functionality calledVPATH, the implementation leaves something tobe desired, and can’t be used for the search pathfunctionality required by Aegis. Becauseof this,the development_directory_style.source_file_-symlinkfield of the project configuration file is setto true so that Aegis will arrange for the develop-ment directory to be full of symbolic links, mak-ing it appear that the entire project source is ineach change’s dev elopment directory.

5.5.1. Invoking Make

The build_commandfield of the project configu-ration file is used to invoke the relevant buildcommand. Inthis case, it is set as follows

build_command ="gmake -f ${s Makefile} project=$p \

change=$c version=$v";

This command tells make where to find the rules.The${s Makefile} expands to a path into thebaseline during development if the file is not inthe change. Look inaesub(5) for more informa-tion about command substitutions.

The rules which follow will all remove their tar-gets before constructing them, which qualifiesthem for the next entry in the configuration file:


The files must be removed first, otherwise thebaseline would be corrupted by integration builds(or even by dev eloper builds, if your aren’t usinga separate user for the project owner).

Note: if you are migrating an existing projectdonot set this field; only set it after you havechangedall of the Make rules. If in doubt,don’tset this field.

Another field to be set in this file is


source_file_symlink = true;};

which tells Aegis to maintain symbolic linksbetween the development directory and the base-line for source files (but not derived files). Seeaepconf(5) for more information.

5.5.2. TheRule File

The file containing the rules is calledMakefileand is given to make on the command line.

The following items are preamble to the rest ofthe file; they ask Aegis for the source files of theproject and change so that make can determinewhat needs to be compiled and linked.

project_files := \$(shell aelpf -p $(project) \

-c $(change))change_files := \

$(shell aelcf -p $(project) \-c $(change))

source_files := \$(sort $(project_files) \

$(change_files))CC := gccCFLAGS := -O

This example parallels the one from chapter 3,and thus has a single executable to be linked fromall the object files

object_files := \$(patsubst %.y,%.o,$(filter \

%.y,$(source_files))) \$(patsubst %.l,%.o,$(filter \

%.l,$(source_files))) \$(patsubst %.c,%.o,$(filter \

%.c,$(source_files)))

Constructing the program is straightforward,remembering to remove the target first.


User Guide Aegis

example: $(object_files)rm -f example$(CC) -o example $(object_files) \

-ly -ll

This rule says that to make the example program,you need the object files determined earlier, andthem link them together. Object files which wereup to date in the baseline are used wherever possi-ble, but files which were out of date are con-structed in the current directory and those will belinked.

5.5.3. TheRule for C

Next we need to tell make how to manage Csources. Onthe surface, this is a simple rule:

%.o: %.c$(CC) $(CFLAGS) -c $*.c

This example matches the built-in rule for mostmakes. But it forgets to remove the target beforeconstructing it.

%.o: %.crm -f $*.o$(CC) $(CFLAGS) -c $*.c

The target may not yet exist, hence the-f option.

Something missing from this rule is finding theinclude file dependencies. The GNU Make UserGuide describes a method for obtaining includefi le dependencies.A set of dependency files areconstructed, one per.c fi le.

%.d: %.crm -f %.d$(CC) $(CFLAGS) -MM $*.c \| s ed ’s/ˆ$.*$.o :/\1.o \1.d :/’ \> $*.d

These dependency files are then included into theMakefile to inform GNU Make of the dependen-cies.

include $(patsubst \%.o,%.d,$(object_files))

GNU Make has the property of making sure all itsinclude files are up-to-date. If any are not, theyare made, and then GNU Make starts over, and re-reads the Makefile and the include files fromscratch, before proceeding with the operationrequested. Inthis case, it means that our depen-dency construction rule will be applied before anyof the sources are constructed.

This method is occasionally quirky about absentinclude files which you have yet to write, orwhich are generated and don’t yet exist, but this isusually easily corrected, though you do need to

watch out for things which will stall an integra-tion.

The -MM option to the $(CC) command meansthat this rule requires thegcc program in order towork correctly. It may be possible to usec_incl(1) from cook, orccincl(1) from cake tobuild the dependency lists instead; but they don’tunderstand the conditional preprocessing as wellasgccdoes.

This method also suffers when heterogeneousdevelopment is performed.If you include differ-ent files, depending on the environment beingcompiled within, the.d fi les may be incorrect, andGNU Make has no way of knowing this.

5.5.3.1. IncludeDirectives

Unlike cookdescribed in an earlier section, usingGNU Make as described here allows you to con-tinue using the


form of the include directive. This is because thedevelopment directory appears, to the compiler, tobe a complete copy of the baseline.

5.5.4. TheRule for Yacc

Having explained the complexities of the rules inthe above section about C, the rule for yacc willbe given without delay:

%.c %.h: %.yrm -f $*.c $*.h y.tab.c y.tab.h$(YACC) $(YFLAGS) -d $*.ymv y.tab.c $*.cmv y.tab.h $*.h

This rulecould be jazzed up to cope with the list-ing file, too, if that was desired, but this is suf-ficient to work with the example.

GNU Make’s ability to cope with transitive clo-sure will pick up the generated.c fi le and con-struct the necessary.o fi le.

To prevent GNU Make throwing away the transi-tive files, and thus slowing things down in somecases, make them precious:

.PRECIOUS: \$(patsubst %.y,%.c,$(filter \

%.y,$(source_files))) \$(patsubst %.y,%.h,$(filter \

%.y,$(source_files)))

5.5.5. TheRule for Lex

The rule for lex is vary similar to the rule foryacc.


Aegis UserGuide

%.c: %.lrm -f $*.c lex.yy.c$(LEX) $(LFLAGS) $*.lmv lex.yy.c $*.c

GNU Make’s ability to cope with transitive clo-sure will pick up the generated.c fi le and con-struct the necessary.o fi le.

To prevent GNU Make throwing away the transi-tive files, and thus slowing things down in somecases, make them precious:

.PRECIOUS: \$(patsubst %.l,%.c,$(filter \

%.l,$(source_files)))

5.5.6. Rulesfor Documents

You can format documents, such as user guidesand manual entries with Aegis and GNU Make,and the rules are similar to the ones above.

%.ps: %.msrm -f $*.psgroff -p -t -ms $*.ms > $*.ps

This rule says to run the document through groff,with the pic(1) and tbl(1) filters, use thems(7)macro package, to produce PostScript output.

This omits include file dependencies. If this isimportant to you, thec_incl program fromcookcan be used to find them. Filtering its output canthen produce the necessary dependency files to beincluded, rather like the C rules, above.


%.cat: %.manrm $*.catgroff -Tascii -s -t -man $*.man \

> $*.cat

5.5.7. OtherMakes

All of the above discussion assumes that GNUMake and GCC are used. If you do not want todo this, or may not do this because of internalcompany politics, it is possible to perform all ofthe automated features manually.

This may, howev er, rapidly become spectacularlytedious. For example: if a user needs to copy theMakefile into their change for any reason, theywill need to constantly useaed(1) to "catch up"with integrations into the baseline.

Reviewers are also affected: they must check thateach change to theMakefileaccurately reflects theobject list and the dependencies of each sourcefi le.

5.5.8. Templates

The lib/config.example/makefi le in the Aegis dis-tribution contains all of the above commands, sothat you may readily insert them into your projectconfiguration file.

5.5.9. GNUMake VPATH Patch

Version 3.76 and later of GNU Make include thispatch, so you don’t need to read this sectionunless you have GNU Make 3.75 or earlier.

There is a patch available for GNU Make 3.75and earlier which gives it improved VPATHsemantics. Atthe time it was not maintained bythe same person who maintained GNU Make.Since then, the maintaier changed, and the patchhas been incorporated.

The patch is the work of Paul D. Smith<[email protected]> and may befetched By Anonymous FTP from

Host: ftp.wellfleet.comDir: /netman/psmith/gmakeFile: vpath+.READMEFile: vpath+.patch.version

The version numbers track the GNU Make ver-sion numbers.

For a description of the VPATH problem, andhow this patch addresses it, see the README filereferenced.

5.5.10. GNUMake’s VPATH+

In theory, using GNU Make 3.76 or later (or asuitable patched earlier version) is similar tousing Cook. The project configuration file nowrequires

link_integration_directory = false;

which is the default. TheMakefilenow requires

VPATH . bl

Assuming thatbl is a symbolic link to the base-line. The.d fi les continue to be used.


User Guide Aegis

5.6. Building Executable Scripts

Aegis treats source files as, well, source files.This means that it forgets any executable bits (andany other mode bits) you may set on the file.Usually, this isn’t a problem - except for scripts.

So, just how do you get Aegis to give you anexecutable script?Well, you add a build rule.However, since it can’t depend on itself, it needsto depend on something else.

Using a Cook example, we could write

bin/%: script/%.sh{

/* copy the script */cp script/%.sh bin/%;/* make it executable */chmod a+rx bin/%;/* syntax check */bash -n bin/%;

}

There is a small amout of value-added here: wedid a syntax check along the way, which catchesall sorts of problems.

The same technique also works for Perl

bin/%: script/%.pl{

cp script/%.pl bin/%;chmod a+rx bin/%;perl -cw bin/%;

}

The same technique also works for TCL

bin/%: script/%.tcl{

cp script/%.rcl bin/%;chmod a+rx bin/%;procheck -nologo bin/%;

}

The procheck(1) command is part of the TclPropackage.

Many tools have a similar options.

You can also combine this with GNU Autoconf toproduce architecture specific shell scripts fromarchitecture neutral sources.

5.7. GNUAutoconf

If your projects uses GNU Make, GNU Autoconfand GNU Automake, here is a quick and simplemethod to import your project into Aegis andhave it running fairly quickly.

5.7.1. TheSources

Once you have create and Aegis project to, yourfi rst change set should simply contain all of the

source files, without removing or adding any-thing. Theonly additonal file is the Aegis projectconfiguration file, usually calledaegis.confandusually located in the top-level directory.

Follow the directions in the section, above, onusing Makefor how to fill out this file.

Note that if you are working from a tarball, theyusually contain several derivedfi le. Thatis, fileswhich are not primary sourec files, but are insteadderived from other files. Thisis a convenience forthe end-user but a nuisance at this point.Exanpleof derived files in source tarballs includecon-figure , Makefile.in , config.h.in , etc.You will need to exclude them form the firstchange set.

In this first change set, you don’t even try to buildanything.

build_command = "exit 0";

Which will allow the Aegis process to complete.

5.7.2. Building

You actually get your project to buld in the sec-ond change set. Once you have started develop-ment, you will see all of the source files in thedevelopment directory (well, symlinks to them).

In order to get you build to work, you have tobootstrap theMakefile . Using the usual GNUtool chain, this file is generated fromMake-file.in which is in turn generated fromMakefile.am , and this is not presently in thedevelopment directory.

This is done by creating a new primary source filecalledmakefile at the top level

$ aenf makefile$

and setting its contents to be

include Makefile

ifndef srcdir

bogus-default-target: Makefile$(MAKE) $(MAKEFLAGS) $(MAKECMDGOALS)

Makefile: configure Makefile.in config.h.inrm -f config.cache./configure

configure: configure.acautoconf

config.h.in: configure.acautoheader


Aegis UserGuide

Makefile.in: Makefile.amautomake

endif

This works becausemake(1) looks for make-file before Makefile , but also because ourbootstrapping makefile includes the realMakefile if it exists, and the real file’s ruleswill take precedence. Atthis point, GNU Makehas a very useful feature: it will rebuild includefi les which are out-of-date before it does anythingelse. Inoue new dev elopment directory, this willresult in the necessary files being automagicallygenerated and then acted upon.

Things that can go wrong: many projects includefi les such asinstall-sh and missing and mkin-stalldirs in the directibution. You will need toinclude rules for these files in the conditonal partof your bootstrappingmakefile rules.

AUTOMAKE_DIR=/usr/share/automake-1.7

install-sh: $(AUTOMAKE_DIR)/install-shcp $ˆ $@

missing: $(AUTOMAKE_DIR)/missingcp $ˆ $@

mkinstalldirs: $(AUTOMAKE_DIR)/mkinstalldirscp $ˆ $@

You will have to tell the configure rule that itdepends on these files as well.

Other things that can go wrong: some projects usedifferent rules for constructing theconfig.hfi le. You should read the generatedMake-file.in fi le for how, and duplicate into thebootstrappingmakefile fi le. You may alsoneed a rule for theaclocal.m4 fi le, and tell theconfigurerule it depends on it.

There is a templatemakefile installed in the/usr/local/share/aegis/config.exampledirectory.

Now you can set the build command field of theproject configuration file:

build_command ="make ""project=$project ""change=$change ""version=$version";

Aegis watches the eist status of the build com-mand. Beaw are that many build systems whichuse recursive make report false positives, becausethe exist status of the sub-make is often ignoredby the top-level Makefile. This means that Aegismay think your project compiles when, in fact, itdoes not.

If, while trying to get it to build, you discovermore derived files which should not be primarysource files, simply use theaerm(1) command.The aeclean(1) command may come in handly,too.

Once this second change set builds, integrate itvia the usual Aegis process.

5.7.3. Tesing

If the project you are importing has tests, they areprobably executed by saying

$ make checklots of output$

or something similar. Aegis expects each test tobe in a separate shell script. Usually this is sim-ple enough to arrange. See the chapter onTestingfor some hints.

5.7.4. AnOptimization

The first build in a new dev elopment directorycan be quite time consuming.It is possible toshort-ciruit this by using the pre-built object filesin the baseline.To do this, use the following set-ting in the project configuration file:


source_file_symlink = true;derived_file_copy = true;derived_at_start_only = true;

};

This causes Aegis to copy all of the derived fileinto your development directory ataedb time.This is usually much faster than compiling andlinking all over again.

5.7.5. Signed-off-by

It is possible to get the Aegis process to automati-cally append Signed-off-by lines to thechange description. Set the following field in theproject configuration file:

signed_off_by = true;

Only open source projects should use this field.The OSDL definition of the Developer’s Certifi-cate of Origin 1.0 can be found at http://-www.osdl.org/newsroom/press_releases/2004/-2004_05_24_dco.html and is defined to mean:

"By making a contribution to this project, I certifythat:

(a) The contribution was created in whole or inpart by me and I have the right to submit it under


User Guide Aegis

the open source license indicated in the file; or

(b) The contribution is based upon previous workthat, to the best of my knowledge, is coveredunder an appropriate open source license and Ihave the right under that license to submit thatwork with modifications, whether created inwhole or in part by me, under the same opensource license (unless I am permitted to submitunder a different license), as indicated in the file;or

(c) The contribution was provided directly to meby some other person who certified (a), (b) or (c)and I have not modified it."

5.7.6. Importing the Next Upstream Tarball

If you are using Aegis to track your local changes,but the master sourecs are elsewhere, you willneed to track upstream changes when they arereleased.

It is tempting to use theaetar(1) command, but itwill not be able to detect derived files which havebeen added to the tarball.You will need touppack the tarball and remove them manually.

Create a change set in the usual way, and aecd(1)into it. Copy the entire project into your changeset, because you don’t yet know what the tarballwill want to change (and it will includeunchanged files).

$ aecd$ aecp .$

(Yes, that dot is part of the command.)Now youcan unpack the tarball.You need to strip off theleading directory somehow (most polite projectsuse a prefix). The author uses thetardy(1) com-mand, like this:

$ zcat project-x.y.tar.gz | \tardy -rp project-x.y -now | \tar xf -

$

It pays to change that the tarball is the shape youexpectbeforerunning this command.

At this point you have to once again remove all ofthe files which are in the tarball, but which are notprimary source files, such asconfigureandMake-file.in and the like.

$ rm -f configure Makefile.in config.h.in etc$ rm -f aegis.log$

It is useful if you place therm(1) command in ashell script, and tell Aegis it is a source file,

because you will have to do this every time.

Now you can have Aegis add any new files byusing the follwoing command:

$ aenf .$

(Yes, that dot is part of the command.)Note thatif there are no new files, this command will giveyou an error, this is expected.

You will have to work out moved and removedfi les for yourself, and use theaemv(1) andaerm(1) commands.

At this point you should remove all the fileswhich were present in the tarball but which dodnot actually change from the change set.Thefollwoing command does this quickly and simply:

$ aecpu -unchanged$

You change set now contains the minimum set ofdifferences. Goahead and complete it using theusual Aegis process.

5.7.7. Importing the Next Upstream Patch

In contrast to tarballs, patches tend to be far easierto cope with. In general, all that is necessary is touse theaepatch(1) command, something like this:

$ aepatch -receive -file project-x.y.diff$

which will create a change set, check-out onlythose file the patch alters, and copes with createsand removes automagically.

There are two problems with this method.Thelargest problem is patches whicg contains diff forderived fiels as well. This is unfortunatelyverycommon.

The simplest way of coping with this is to add theaepatch −trojan option, which will leave thechange in thebeing developedstate, where youcan examine it and use theaenfu(1) command forany derived files it insisted on creating as primarysource files.

The second problem is much simpler: if a patchonly contains new files, Aegis can’t work out howmuch of the leading path it should ignore on thefi lenames in the patch.You will need to use theaepatch −remove-prefixoption in this case.

5.8. NoBuild Required

For some projects, particularly web sites andthose written exclusively in interpreted languages,it may not be necessary to ever actually build your


Aegis UserGuide

project.

For this kind of project you add the following lineto the project configuration file:

build_command = "exit 0";

For a project configured in this way, the aede(1)and aeipass(1) commands will not check that abuild has been performed.

5.8.1. Why This May Not Be Such A GoodIdea

It isn’t always desirable to configure a project thisway, even when it may initialy appear to be agood idea.

Web sites:You can use the build stage to check theHTML files against the relevant standardsand DTDs. You can also check that all ofyou (internal) links are valid, and don’tpoint to non-existant pages or anchors.Sometimes, if you have the space, you canresolve server side includes, to make itfaster for Apache, by serving static pages.

Interpreted Languages:A whole lot of simple errors, such as syntaxerrors, can be caught by a static check ofthe source files. For example, theperl-c option can syntax check your Perl fileswithout executing them. See also the GNUawk −lint option, the Python built-incompile() function, and thephp -l(lower case L) option.You can also checkthat all include files referenced actuallyexist.

Documentation:Many systems allow documentation to beextracted from the source files, and turnedinto HTML or PDF files (e.g. Doxygen).This is a sensable thing to do at build time.


User Guide Aegis

6. TheDifference Tools

This chapter describes the difference commandsin the project configuration file. Usually thesecommands are used by theaegis -DIFFerencecommand when differencing files, but they maybe used to accomplish some other things.

The default setting is for Aegis to reject filenameswhich contain shell special characters.Thisensures that filenames may be substituted into thecommands without worrying about whether this issafe. If you set theshell_safe_filenamesfield ofthe projectaegis.conffi le to false , you willneed to surround filenames with the${quotefilename} substitution. Thiswill only quote file-names which actually need to be quoted, so usersusually will not notice.This command applies toall of the various filenames in the sections whichfollow.

6.1. Binary Files

Aegis doesn’t particularly care whether your filesare binary or text. However, your difference andmerge tools certainly will. In general, you needformat-specific difference and merge tools foreach of the file formats used in your project.Unfortunately, most vendors of software whichmake use of proprietary file formats do not supplydifference and merge tools.

The simplest compromise is to treat all files astext, with manual repairs for binary files.

A more elegant solution is to use a shell scriptinvoked by the diff_command in the projectaegis.conffi le. Thisshell script examines the fileto determine the file format, and then runs theappropriate difference tool. Similar considera-tions apply to themerge_commandfield.

Please note that this support is not present inAegis itself because (a) it would cause code bloat,and (b) it is entirely possible to do with a shellscript launched bydiff_command.

6.2. Interfacing

The diff command is configured by a field of theproject configuration file (aegis.conf).

6.2.1. diff_command

This command is used byaed(1) to produce a dif-ference listing when file in the development direc-tory was originally copied from the current ver-sion in the baseline19.

19 Or this is logically the case.


${ORiginal}The absolute path name of a file containingthe version originally copied. Usually inthe baseline.

${Input}The absolute path name of the edited ver-sion of the file. Usuallyin the developmentdirectory.

${Output}The absolute path name of the file in whichto write the difference listing. Usually inthe development directory.

An exit status of 0 means successful, even of thefi les differ (and they usually do). An exit statuswhich is non-zero means something is wrong.

The non-zero exit status may be used to overloadthis command with extra tests, such as line lengthlimits. The difference files must be produced inaddition to these extra tests.


This command is used byaed(1) to produce a dif-ference listing when file in the development direc-tory is out of date compared to the current versionin the baseline.


${ORiginal}The absolute path name of a file containingthe version originally copied. Usually in atemporary file.

${Most_Recent}The absolute path name of a file containingthe most recent version. Usuallyin thebaseline.

${Input}The absolute path name of the edited ver-sion of the file. Usuallyin the developmentdirectory. Aegis usually moves the sourcefi le aside, so that the output can replace thesource file.

${Output}The absolute path name of the file in whichto write the difference listing. Usually inthe development directory. This is usuallythe name of a change source file.


Aegis UserGuide

An exit status of 0 means successful, even of thefi les differ (and they usually do). An exit statuswhich is non-zero means something is wrong.

6.3. WhenNo Diff is Required

It is possible to configure a project to omit the diffstep as unnecessary, by the following setting:

diff_command = "exit 0";

This disables all generation, checking and valida-tion of difference files for each change source file.The merge functions of theaediff(1) command areunaffected by this setting.


User Guide Aegis

6.4. Usingdiff and merge

These two tools are available with most flavoursof UNIX, but often in a very limited form.Onesevere limitation is thediff3(1) command, whichoften can only cope with 200 lines of differences.The best alternative is to use GNU diff, which hascontext differences available, and a far morerobustdiff3(1) implementation.

See the earlierInterfacingsection for substitutiondetails.

6.4.1. diff_command

The entry in the configuration file looks like this:

diff_command ="set +e; diff -c $original ""$input > $output; test $? -le 1";

This needs a little explanation:• This command is always executed with theshell’s -e option enabled, causing the shell to exiton the first error. The "set +e" turns this off.• The diff(1) command exits with a status of 0 ifthe files are identical, and a status of 1 if they dif-fer. Any other status means something horriblehappened. The"test" command is used to changethis to the exit status aegis expects.

The−c option says to produce a context diff. Youmay choose to use the−u option, to produce uni-diffs, if your diff command supports it.

You may also wish to consider ignoring whitespace in comparisons, as these tend to be cos-metic changes and not very interesting to codereviewers. The −b option of GNU Diff willignore changes to the amount of white space, andthe−w option will ignore white space altogether.

Binary files will often cause modern versions ofGNU Diff to exit with an exit status of 2, which isprobably reasonable most of the time. If yourproject contains binary files, you may want totreat all files as text files. Usethe GNU Diff −aoption in this case.

A useful alternative, available with more recentversions of GNU Diff, is the−U option. Thisis amore compact form than the−c option, and isable to give the whole file as context.

diff_command ="set +e; diff -U999999 $original ""$input > $output; test $? -le 1";

The exit status must once again be taylored, how-ev er the output will be the whole source for con-text, with changes marked by ‘+’ and ‘-’ in theleft margin. This,reviewers need only search for

/ˆ[-+]/ in order to see all edit made to the file.


Note: Themerge(1) command is better than thisuse of thediff3(1) command. See the RCS chap-ter for more details.


merge_command ="(diff3 -e $MostRecent $original \

$input | sed -e ’/ˆw$$/d’ -e \’/ˆq$$/d’; echo ’1,$$p’ ) | ed - \$MostRecent > $output";

This needs a lot of explanation.• Thediff3(1) command is used to produce an editscript that will incorporate into $MostRecent, allthe changes between $original and $input.Youmay want the−a option, to treat all files asACSII.• The sed(1) command is used to remove the"write" and "quit" commands from the generatededit script.• The ed(1) command is used to apply the gener-ated edit script to the $MostRecent file, and printthe results on the standard output, which are redi-rected into the $output file.

6.5. Usingfhist

The fhist program by David I. Bell also comeswith two other utilities,fcompand fmerge, whichuse the same minimal difference algorithm.

See the earlierInterfacingsection for substitutiondetails.

6.5.1. diff_command


diff_command ="fcomp -w $original $input ""-o $output";

The -w option produces an output of the entirefi le, with insertions and deletions marked by"change bars" in the left margin. Thisis superiorto context difference, as it shows the entire file ascontext.

For more information, see thefcomp(1) manualentry.




Aegis UserGuide

merge_command ="fmerge $original $MostRecent \

$input -o $output -c /dev/null";

The output of this command is similar to the out-put of the merge_command in the last section.Conflicts are marked in the output.For moreinformation, see thefmerge(1) manual entry.


User Guide Aegis

7. TheProject Attributes

The project attributes are manipulated using theaepa(1) command.This command reads a projectattributes file to set the project attributes. Thisfi le can be thought of as having several sections,each of which will be covered by this chapter.You should see theaepattr(5) manual entry formore details.

7.1. Descriptionand Access

The descriptionfield is a string which contains adescription of the project.Large amounts ofprose are not required; a single line is sufficient.

The default_development_directoryP field is astring which contains the pathname of where toplace new development directories. Thepath-name must be absolute. This field is only con-sulted if the uconf(5) field of the same name is notset. Defaultsto $HOME.

The umaskfield is an integer which is set to thefi le permission mode mask.See umask(2) formore information. This value will always beOR’ed with 022, because aegis is paranoid.

7.2. Notification Commands

The develop_end_notify_commandfield is astring which contains a command to be used tonotify that a change requires reviewing. All ofthe substitutions described inaesub(5) are avail-able. Thisfield is optional, if it is not specified nonotification will be issued.This command couldalso be used to notify other management systems,such as progress and defect tracking, in additionto notifying users.

The develop_end_undo_notify_commandfield isa string containing a command used to notify thata change has been withdrawn from review for fur-ther development. All of the substitutionsdescribed inaesub(5) are available. Thisfield isoptional, if it is not specified no notification willbe issued.This command could also be used tonotify other management systems, such asprogress and defect tracking, in addition to notify-ing users.

The re view_pass_notify_commandfield is a stringcontaining the command to notify that the reviewhas passed.All of the substitutions described inaesub(5) are available. Thisfield is optional, if itis not specified no notification will be issued.This command could also be used to notify othermanagement systems, such as progress and defecttracking, in addition to notifying users.

The re view_pass_undo_notify_commandfield is astring containing the command to notify that areview pass has has been rescinded. All of thesubstitutions described inaesub(5) are available.This field is optional, and defaults to thedevelop_end_notify_commandfield if not speci-fied. If neither is specified, no notification will beissued. Thiscommand could also be used tonotify other management systems, such asprogress and defect tracking, in addition to notify-ing users.

The re view_fail_notify_commandfield is a stringcontaining the command to notify that the reviewhas failed. All of the substitutions described inaesub(5) are available. Thisfield is optional, if itis not specified no notification will be issued.This command could also be used to notify othermanagement systems, such as progress and defecttracking, in addition to notifying users.

The integrate_pass_notify_commandfield is astring containing the command to notify that theintegration has passed.All of the substitutionsdescribed inaesub(5) are available. Thisfield isoptional, if it is not specified no notification willbe issued.This command could also be used tonotify other management systems, such asprogress and defect tracking, in addition to notify-ing users.

The integrate_fail_notify_commandfield is astring containing the command to notify that theintegration has failed. All of the substitutionsdescribed inaesub(5) are available. Thisfield isoptional, if it is not specified no notification willbe issued.This command could also be used tonotify other management systems, such asprogress and defect tracking, in addition to notify-ing users.

7.2.1. Notification by email

The aegis command is distributed with a set ofshell scripts to perform these notifications byemail. They are installed into the/usr/local/lib/aegis directory, by default; theactual installed directory at your site is availableas the ${DATa_DIRectory} substitution. Theentries in the project attribute file look like this:


Aegis UserGuide

develop_end_notify_command ="$sh $datadir/de.sh $project $change";

develop_end_undo_notify_command ="$sh $datadir/deu.sh $project $change";

review_pass_notify_command ="$sh $datadir/rp.sh $project $change \$developer $reviewer";

review_pass_undo_notify_command ="$sh $datadir/rpu.sh $project $change \$developer";

review_fail_notify_command ="$sh $datadir/rf.sh $project $change \$developer $reviewer";

integrate_pass_notify_command ="$sh $datadir/ip.sh $project $change \$developer $reviewer $integrator";

integrate_fail_notify_command ="$sh $datadir/if.sh $project $change \$developer $reviewer $integrator";

Please note: the exit status of all these commandswill be ignored.

7.2.2. Notification by USENET

The aegis command is distributed with a set ofshell scripts to perform these notifications byUSENET. They are installed into the/usr/local/lib/aegis directory, by default; theactual installed directory at your site is availableas the ${DATa_DIRectory} substitution. Theentries in the project attribute file look like this:

develop_end_notify_command ="$sh $datadir/de.inews.sh $p $c alt.$p";

develop_end_undo_notify_command ="$sh $datadir/deu.inews.sh $p $c alt.$p";

review_pass_notify_command ="$sh $datadir/rp.inews.sh $p $c alt.$p";

review_pass_undo_notify_command ="$sh $datadir/rpu.inews.sh $p $c alt.$p";

review_fail_notify_command ="$sh $datadir/rf.inews.sh $p $c alt.$p";

integrate_pass_notify_command ="$sh $datadir/ip.inews.sh $p $c alt.$p";

integrate_fail_notify_command ="$sh $datadir/if.inews.sh $p $c alt.$p";

The last argument to each command is the news-group to post the article in, you may want to usesome other group.Note that "$p" is an abbrevia-tion for "$project" and "$c" is an abbreviation for"$change".

7.3. ExemptionControls

The developer_may_review field is a boolean.Ifthis field is true, then a developer may review herown change. Thisis probably only a good ideafor projects of less than 3 people. The idea is foras many people as possible to critically examine achange.

The developer_may_integratefield is a boolean.If this field is true, then a developer may integrateher own change. This is probably only a goodidea for projects of less than 3 people. The idea isfor as many people as possible to critically exam-ine a change.

The re viewer_may_integratefield is a boolean.Ifthis field is true, then a reviewer may integrate achange she reviewed. This is probably only agood idea for projects of less than 3 people.Theidea is for as many people as possible to criticallyexamine a change.

The developers_may_create_changesfield is aboolean. Ifthis field is true then developers maycreate changes, in addition to administrators.This tends to be a very useful thing, since devel-opers find most of the bugs.

The default_test_exemptionfield is a boolean.This field contains what to do when a change iscreated with no test exemption specified. Thedefault is "false", i.e. no testing exemption, testsmust be provided.

This kind of blanket exemption should only be setwhen a project has absolutely no functionalityavailable from the command line; examplesinclude X11 programs.The program could possi-bly be improved by providing access to the func-tionality from the command line, thus allowingtests to be written.

7.3.1. OnePerson Projects

The entries in the project attributes file for a oneperson project look like this:

developer_may_review = true;developer_may_integrate = true;reviewer_may_integrate = true;developers_may_create_changes = true;

All of the staff roles (administrator, dev eloper,reviewer and integrator) are all set to be the sameuser.

7.3.2. Two Person Projects

A two person project has the opportunity for eachto review the other’s work.

The entries in the project attributes file for a twoperson project look like this:

developer_may_review = false;developer_may_integrate = true;reviewer_may_integrate = true;developers_may_create_changes = true;


User Guide Aegis

All of the staff roles (developer, reviewer andintegrator) are all set to allow both users.

7.3.3. Larger Projects

Once you have 3 or more staff on a project, youcan assign all of the roles to separate people.Theidea is for the greatest number of eyes to see eachchange and detect flaws before they reach thebaseline.

The entries in the project attributes file for a threeperson project look like this:

developer_may_review = false;developer_may_integrate = false;reviewer_may_integrate = false;developers_may_create_changes = true;

For smaller teams, everyone may be a reviewer.As the teams get larger, the more experiencedstaff are often the reviewers, rather than everyone.

7.3.4. RSSFeeds

Aegis has the ability to publish RSS 2.0 items toan RSS channel when changesets transition to anew state. Thisis an optional feature that must beenabled and configured via the project-specificattributes.

Project administrators can configure each changeof state to cause an RSS item to be added to aspecified RSS channel.Each transition is individ-ually controlled, allowing each transition to berecorded in separate channels, or all transitions inthe same channel, or some combination thereof.

Generating RSS items for particular state transi-tions is enabled by therss:feedfilenameproject-specific attibute. Theformat of this attribute is:

name = "rss:feedfilename-<filename>";value = "<state> [<state> <state>]";

The name part of this attribute includes afile-name,which is the name of the RSS feed file(channel) to which the item is to be added.Thevaluepart of the attribute is a space-separated listof states that will cause an RSS item to be addedto the specified file. For example,

name = "rss:feedfilename-foo.xml";value = "awaiting_review

awaiting_integration";

will cause items to be added to the channel storedin the file "foo.xml" when a changeset transitionsinto the awaiting_review and awaiting_integrationstates.

The channel description can be specified usingthe rss:feeddescriptionattribute. Theformat of

this attribute is:

name = "rss:feeddescription-<filenane>";value = "<Some description>";

For example,

name = "rss:feeddescription-foo.xml";value = "This is a description";

will cause the <description> sub-element of the<channel> element stored in the file foo.xml tohave the value "This is a description". If thisattribute is not used, the default description is:"Feed of changes in state..."

The channel title can be specified using therss:feedtitleattribute. Theformat of this attributeis:

name = "rss:feedtitle-<filename>";value = <Some title>;

For example,

name = "rss:feedtitle-foo.xml";value = "This is a title";

will cause the <title> sub-element of the <chan-nel> element stored in the file foo.xml to have thevalue "Project ...: This is a title" The title willalways start with the word "Project" and theproject name. If this attribute is supplied, thisdefault title is appended with the test provided.

The channel language can be specified using therss:feedlanguage attribute. The format of thisattribute is:

name = "rss:feedlanguage- filename";value = " language";

For example,

name = "rss:feedlanguage-foo.xml";value = "en-AU";

will cause the <language> sub-element of the<channel> element stored in the file foo.xml tohave the valueen-AU If not specified, the defaultvalue of the language sub-element is "en-US".

7.3.4.1. Serving RSS Channels

aeget is able to serve up RSS channels, with anappropriate URL. An example URL is

http://somehost/cgi-bin/aeget/proj.1.0/?rss+foo.xml

The key aspect of the URL shown is the"?rss+foo.xml" modifier. "foo.xml" should obvi-ously be replaced with the name of your RSSchannel feed (that is, the filename specified withthe "rss:feedfilename" project-specifiedattribute(s).


Aegis UserGuide

In order to read the RSS channels, it is recom-mended to point your RSS aggregator of choice tothe appropriate URL. In order to make determin-ing the URL easy, aeget will also place "RSS"icons next to the individual state links on the mainproject web page ("proj.1.0/?menu") if there is anRSS channel configured to include that changesetstate.

7.3.4.2. Linksin RSS Channels

Links within RSS feed files are stored using aplaceholder ("@@SCRIPTNAME@@") insteadof the serving script in URLs.This is donebecause the code that knows about hte URL of aparticular installation is encapsulated withinaeget.

The placeholder is replaced with the real script-name when the file is served by aeget.


User Guide Aegis

8. Testing

This chapter discusses testing, and using Aegis tomanage your tests and testing.

8.1. Why Bother?

Writing tests is extra work, compared to the waymany small (and some not-so-small) softwareshops operate. For this reason, the testingrequirement may be turned off.

The win is that the tests hang around forever,catching minor and major slips before theybecome embarrassing "features" in a releasedproduct. Prevention is cheaper than cure in thiscase, the tests save work down the track.

All of the "extra work" of writing tests is a long-term win, where old problems never again reap-pear. All of the "extra work" of reviewingchanges means that another pair of eyes sees thecode and finds potential problems before theymanifest themselves in shipped product.All ofthe "extra work" of integration ensures that thebaseline always works, and is always self-consis-tent. All of the "extra work" of having a baselineand separate development directories allows mul-tiple parallel development, with no inter-devel-oper interference; and the baseline always works,it is never in an "in-between" state.In each case,not doing this "extra work" is a false economy.

The existence of these tests, though, is what deter-mines which projects are most suited to Aegis andwhich are not. It should be noted that suitabilityis a continuous scale, not black-and-white.Witheffort and resources, almost anything fits.

8.1.1. Projects for which Aegis’ Testing is MostSuitable

Projects most suited to supervision by Aegis arestraight programs. What the non-systems-pro-grammers out there call "tools" and sometimes"applications". Theseare programs which take apile of input, chew on it, and emit a pile of output.The tests can then compare actual outputs withexpected outputs.

As an example, you could be writing ased(1)look-alike, a public domain clone of theUNIX sedutility. You could write tests which exercise everyfeature (insertion, deletion, etc.)and generate theexpected output with the realUNIX sed. You writethe code, and run the tests; you can immediatelysee if the output matches expectations.

This is a simple example. Morecomplex exam-ples exist, such as Aegis itself. The Aegis

program is used to supervise its own develop-ment. Tests consist of sequences of commandsand expected results are tested for.

Other types of software have been developedusing Aegis: compilers and interpreters, client-server model software, magnetic tape utilities,graphics software such as a ray-tracer. The rangeis vast, but it is not all types of software.

8.1.2. Projects for which Aegis’ Testing is Use-ful

For many years there have been full-screen appli-cations on text terminals. In more recent timesthere is increasing use of graphical interfaces.

In developing these types of programs it is stillpossible to use Aegis, but several options need tobe explored.

8.1.2.1. Testing Via Emulators

There are screen emulators for both full-screentext and X11 available. Usingthese emulators, itis possible to test the user interface, and test viathe user interface. Asyet, the author knows of nofreely available emulators suitable for testing viaAegis. If you find one, please let me know.

8.1.2.2. LimitedTesting

You may choose to use Aegis simply for its abil-ity to provide controlled access to a large source.You still get the history and change mechanisms,the baseline model, the enforced review. Yousimply don’t test all changes, because figuring outwhat is on the screen, and testing it against expec-tations, is too hard.

If the program has a command line interface, inaddition to the full-screen or GUI interface, thefunctionality accessible from the command linemay be tested using Aegis.

It is possible that "limited testing" actually means"no testing", if you have no functionality accessi-ble from the command line.

8.1.2.3. Testing Mode

Another alternative is to provide hooks into yourprogram allowing you to substitute a file for userinput, and to be able to trigger the dump of a"screen image". The simulated user input canthen be fed to the program, and the screen dump(in some terminal-independent form) can be com-pared against expectations.

This is easier for full-screen applications, than forX11 applications. You need to judge the cost-


Aegis UserGuide

benefit trade-off. Cost of development, cost ofstorage space for X11 images, cost ofnot testing.

8.1.2.4. ManualTests

The Aegis program provides a manual test facil-ity. It was originally intended for programs whichrequired some physical action from a user, such as"unplug Ethernet cable now" or "mount tapeXG356B now". It can also be used to have a userconfirm that some on-screen activity has hap-pened.

The problem with manual tests is that they simplydon’t happen. Itis far more pleasant to say "runthe automatic tests" and go for a cup of coffee,than to wait while the computer thinks of mind-less things to ask you to do.This is humannature: if it can be automated, it is more likely tohappen.

8.1.2.5. UnitTests

Many folks think of testing as taking the finalproduct and testing it.It is also possible to buildspecialized unit tests, which exercise specific por-tions of the code. These tests can then be admin-istrated by Aegis, even if the full-blown GUI can-not be.

8.1.3. Projects for which Aegis’ Testing isLeast Useful

Another class of software is things like operatingsystem kernels and firmware; things which are"stand alone". This isolated nature makes it themost difficult to test: to test it you want to providephysical input and watch the physical output.Byits very nature, it is hard to put into a shell script,and thus hard to write an Aegis test for.

The above chapter was written in 1991. At thiswriting (1999) there are projects like ×Linux andoperating systems like VxWorks. Theseare allembedded, and all have excellent network anddownload support. It is entirely possible (withdesign support!) to write automatically testableembedded systems.

8.1.3.1. OperatingSystems

It is not impossible, just that few of us hav e theresources to do it.You need to have a test systemand a testing system: the test system has all of itsinput and outputs connected to the outputs andinputs of the testing system. That is, the testingsystem controls and drives the test system, andwatches what happens.

For example, in the olden days before everyonehad PC and graphics terminals, there were onlyserial interfaces available. Many operating sys-tem vendors tested their products by using com-puters connected to each serial line to simulate"user input". The system can be rebooted thisway, and using dual-ported disks allows differentversions of a kernel to be tried, or other test con-ditions created.

For software houses which write kernels, ordevice drivers for kernels, or some other kernelwork, this is bad news: the Aegis program isprobably not for you. It is possible, but there maybe more cost-effective dev elopment strategies. Ofcourse, you could always use the rest of Aegis,and ignore the testing part.

However, Aegis has been used quite successfullyto develop Linux kernel modules.With suitablesudo(1) configuration to permit access toins-mod(1) &co, developers can write test scriptswhich load device drivers, try them out, andunload them again, all without universal rootaccess.

Also, the advent of modern tools, such asVMware, which allow one operating system to"host" another, may also permit straightforwardtesting of kernels and operating systems.

8.1.3.2. Firmware

Firmware is a similar deal: you need some way todownload the code to be tested into the test sys-tem, and write-protect it to simulate ROM, andhave the necessary hardware to drive the inputsand watch the outputs.

As you can see, this is generally not available torun-of-the-mill software houses, but then theyrarely write firmware, either. Those that do writefi rmware usually have the download capabilities,and some kind of remote operation facility.

However, this omits the possibility of not onlycross compiling your code for the target system,but also compiling your code to run natively onthe build system. The firmware (in the host incar-nation) then falls into one of the categories above,and may be readily tested.This does not relieveyou of also testing the firmware, but it increasesthe probability that the firmware isn’t completelyuseless before you download it.

By using an object oriented language, such asC++, the polymorphism necessary to cope withmultiple environments can be elegantly hiddenbehind a pure abstract base class.Alternatively,


User Guide Aegis

by using a consistent API, you can accomplish thenecessary sleight-of-hand at link time.

The unit test method mentioned earlier is alsovery useful for firmware, even if the device "as awhole" cannot be tested.


Aegis UserGuide

8.2. Writing Tests

This section describes a number of general guide-lines for writing better tests, and some pitfalls tobe avoided.

There are also a number of suggestions for porta-bility of tests in specific scripting languages; thiswill definitely be important if you are writingsoftware to publish on WWW or for FTP. Porta-bility is often requiredwithin an organization,also. Examplesinclude a change in company pol-icy from one 386UNIX to another (e.g. companydoesn’t like Linux, now you must use AT&T’sSVR4 offering), or the development team usegccuntil the company finds out and forces you to usethe prototype-less compiler supplied with theoperating system, or even that the software beingdeveloped must run under bothUNIX and Win-dows NT.

Note, also, that when using Aegis’ heterogeneousbuild support, portability will again featureprominently.

8.2.1. Contributors

I’d like to thank Steven Knight<[email protected]> for writing portions ofthis information.

If other readers have additional testing techniques,or use other scripting languages, contributions arewelcome.

8.2.2. GeneralGuidelines

This section lists a number of general guidelinesfor all aegis tests, regardless of implementationlanguage. Usethis section to guide how youwrite tests if the scripting language you choose isnot specifically covered in greater detail below.

8.2.2.1. Choiceof Scripting Language

The aegis program uses thetest_commandfield ofthe projectaegis.conffi le to specify how tests areexecuted. Thedefault value of thetest_commandfield:

test_command = "$shell $file_name";

specifies that tests be Bourne shell scripts.Youmay, howev er, change the value oftest_commandto specify some other scripting language inter-preter, which allows you to write your test scriptsin whatever scripting language is appropriate foryour project. The Perl or Python scripting lan-guages, for example, could be used to create testscripts that are portable to systems other thanUNIX systems.

This means that if you can write it in your script-ing language of choice, you can test it.Thisincludes such things as client-server model inter-faces, and multi-user synchronization testing.

8.2.2.2. NoExecute Permission

Under aegis, script files do not have execute per-mission set, so they should always be invoked bypassing the script file to the interpreter, notexecuting the test directly:

sh filenameperl filename

8.2.2.3. NoCommand-Line Arguments

Tests should not expect command line arguments.Tests are not passed the name of the project northe number of the change.

8.2.2.4. Identifyingthe Scripting Language

Even though aegis does not execute the test scriptdirectly, it is a good idea to put some indication ofits scripting language into the test script.See thesections below for suggested "magic number"identification of scripts in various languages.

8.2.2.5. Current Directory

Tests are always run with the current directory setto either the development directory of the changeunder test when testing a change, or the integra-tion directory when integrating a change, or thebaseline when performing independent tests.

A test must not make assumptions about where itis being executed from, except to the extent that itis somewhere a build has been performed.A testmust not assume that the current directory iswritable, and must not try to write to it, as thiscould damage the source code of a change underdevelopment, potentially destroying weeks ofwork.

8.2.2.6. CheckExit Status and Return Values

A test script should check the exit status or returnvalue of every single command or function call,ev en those which cannot fail. Checkingthe exitstatus or return value of every statement in thescript ensures that strange permission settings, ordisk space problems, will cause the test to fail,rather than plow on and produce spurious results.See the sections below for specific suggestions onchecking exit status or return values in variousscripting languages.


User Guide Aegis

8.2.2.7. Temporary Directory

Tests should create a temporary subdirectory inthe operating system’s temporary directory (typi-cally /tmp on UNIX systems) and then change itsworking directory (cd) to this directory. This iso-lates any vandalism that the program under testmay indulge in, and serves as a place to writetemporary files.

At the end of the test, it is sufficient to changedirectory out of the temporary subdirectory andthen remove the entire temporary subdirectoryhierarchy, rather than track and remove all testfi les which may or may not be created.

Some UNIX systems provide other temporarydirectories, such as/var/tmp, which may providea better location for a temporary subdirectory fortesting (more file system space available, admin-istrator preference, etc.).Test scripts wishing toaccomodate alternate temporary directoriesshould use the TMPDIR environment variable (orsome other environment variable appropriate tothe operating system hosting the tests) as the loca-tion for creating their temporary subdirectory,with /tmp as a reasonable default if TMPDIR isnot set.

8.2.2.8. Trap Interrupts

Test scripts should catch appropriate interrupts (12 3 and 15 onUNIX systems) and cause the test tofail. The interrupt handler should perform anycleanup the test requires, such as removing thetemporary subdirectory.

8.2.2.9. PAGER

If the program under test invokes pagers on itsoutput, a lamore(1) et al, it should be coded touse the PAGER environment variable. Tests ofsuch programs should always set PAGER to catso that tests always behave the same, irrespectiveof invocation method (either by aegis or from thecommand line).

8.2.2.10. Auxiliary Files

If a test requires extra files as input or output to acommand, it must construct them itself from in-line data. (See the sections below for more spe-cific information about how to use in-line data invarious scripting languages to create files.)

It is almost impossible to determine the locationof an auxiliary file, if that auxiliary file is part ofthe project source.It could be in either thechange under test or the baseline.

8.2.2.11. NewTest Templates

Regardless of your choice of scripting language, itis possible to specify most of the repetitious itemsabove in a file templateused every time a usercreates a new test. Seethe aent(1) command formore information.

Having the machine do it for you means that youare more likely to do it.

8.2.3. Bourne Shell

The Bourne shell is available on all flavors of theUNIX operating system, which allows Bourneshell scripts to be written portably across thosesystems. Hereare some specific guidelines forwriting aegis tests using Bourne shell scripts.

8.2.3.1. MagicNumber

Some indication that the test is a Bourne shellscript is a good idea. While many systems acceptthat a first line starting with a colon is a Bourneshell "magic number", a more widely understood"magic number" is

#! /bin/sh

as the first line of the script file.

8.2.3.2. CheckExit Status

A Bourne shell test script should check the exitstatus of every single command, even those whichcannot fail. Do not rely on, or use, theset -eshelloption (it provides no ability to clean up on error).

Checking the exit status involves testing the con-tents of the$? shell variable. Donot use anifstatement wrapped around an execution of theprogram under test as this will miss core dumpsand other terminations caused by signals.


Bourne shell test scripts should create a tempo-rary subdirectory in/tmp (or the directory speci-fied by the TMPDIR environment variable) andthencd into this directory. At the end of the test,or on interrupt, the script shouldcd out of thetemporary subdirectory and thenrm -rf it.


Use thetrap statement to catch interrupts 1 2 3and 15 and cause the test to fail. Thisshould per-form any cleanup the test requires, such as remov-ing the temporary directory.


Aegis UserGuide


If a test requires extra files as input or output to acommand, it must construct them itself, usingheredocuments:

cat <<EOF >filecontentsof thefileEOF

Seesh(1) for more information.

8.2.3.6. [test ]

You should always use thetest command, ratherthan the square bracket form, as many systems donot have the square bracket form, if you publishto USENET or for FTP.

8.2.3.7. OtherBourne Shell Portability Issues

The above list covers the most common Bourneshell issues that are relevant to most aegis tests.The documentation for the GNU autoconf utility,however, contains a more exhaustive list ofBourne shell portability issues. If you want (orneed) to make your tests as portable as possible,see the documentation for GNU autoconf.

8.2.4. Perl

Perl is a popular open-source scripting languageavailable on a number of operating systems.Hereare some specific guidelines for writing aegis testsusing Perl scripts.

8.2.4.1. MagicNumber

Some indication that the test is a Perl script is agood idea. Because Perl is not installed in thesame location on allUNIX systems, a first-line"magic number" such as:

#! /usr/local/bin/perl

that hard-codes the Perl path name will not beportable if you publish your tests.

If the env(1) program is available, a more portable"magic number" for Perl is:

#! /usr/bin/env perl

8.2.4.2. CheckReturn Values

A Perl test script should check the return valuefrom every subroutine, even those which cannotfail.

A Perl test script should also check the exit statusof every command it executes. Checkingthe exit

status involves testing the contents of the$? vari-able. Seethe Perl documentation on "PredefinedVariables" for details.


Perl test scripts should create a temporary subdi-rectory in /tmp (or the directory specified by the$ENV{TMPDIR} environment variable) andthen chdir into this directory. At the end of thetest, or on interrupt, the script shouldchdir out ofthe temporary subdirectory and then remove itand its hierarchy. A portable way to do thiswithin a Perl script:

use File::Find;finddepth(sub { if (-d $_) {

rmdir($_)} e lse {

unlink($_)} } ,$dir);


Use Perl’s $SIGhash to catch interrupts for HUP,INT, QUIT and TERM and cause the test to fail.This should perform any cleanup the test requires,such as removing the temporary directory. A verysimple example:

$SIG{HUP} =$SIG{INT} =$SIG{QUIT} =$SIG{TERM} =

sub { &cleanup; exit(2) };


If a test requires extra files as input or output to acommand, it must construct them itself, using in-line data such ashere documents See the Perldocumentation for more information.

8.2.4.6. ExitValues

Aegis expects tests to exit with a status of 0 forsuccess, 1 for failure, and 2 for no result. The fol-lowing code fragment will map all failed (non-zero) exit values to an exit status of 1, regardlessof what Perl module called exit:

END { $? = 1 if $? }

A more complete example could check conditionsand set the exit status to 2 to indicate NORESULT.


User Guide Aegis

8.2.4.7. Modules

Perl supports the ability to re-use modules ofcommon routines, and to search several directo-ries for modules. This makes it convenient towrite modules to share code among the tests in aproject.

Any modules that are used by your test scripts(other than the standard modules included byPerl) should be checked in to the project as sourcefi les. Test scripts should then import the mod-ule(s) via the normal Perl mechanism:

use MyTest;

When a test is run, the module file may actuallybe in the baseline directory, not the developmentor integration directories.To make sure that thetest invocation finds the module, thetest_com-mand field in the projectaegis.conffi le shoulduse the Perl-I option to search first the localdirectory and then the baseline:

test_command ="perl -I. -I${BaseLine} \${File_Name}"

or, alternatively, if you had created your Perl testmodules in a subdirectory namedaux:

test_command ="perl -I./aux -I${BaseLine}/aux \${File_Name}"

For details on the conventions involved in writingyour own modules, consult the Perl documenta-tion or other reference work.

Actually, you need to use the ${search_path} sub-stitution. I’ll have to fix this one day.

8.2.4.8. TheTest::Cmd Module

A Test::Cmd module is available on CPAN (theComprehensive Perl Archive Network) that makesit easy to write Perl scripts that conform to aegistest requirements. The Test::Cmd module sup-ports most of the guidelines mentioned above,including creating a temporary subdirectory,cleaning up the temporary subdirectory on exit orinterrupt, writing auxiliary files from in-line con-tents, and provides methods for exiting on suc-cess, failure, or no result. The following exampleillustrates some of its capabilities:

#! /usr/bin/env perluse Test::Cmd;$test = Test::Cmd->new(prog

=> ’program_under_test’,workdir => ’’);

$ret = $test->write(’aux_file’, <<EOF);contents of fileEOF$test->no_result(! $ret =>

sub { print STDERR"Couldn’t write file: $!\\n"});

$test->run(args => ’aux_file’);$test->fail($? != 0);$test->pass;

The various methods supplied by the Test::Cmdmodule have a number of options to control theirbehavior.

The Test::Cmd module manipulates file and pathnames using the operating-system-independentFile::Spec module, so the Test::Cmd module canbe used to write tests that are portable to anyoperating system that runs Perl and the programunder test.

The Test::Cmd module is available on CPAN.See the module’s documentation for details.

8.2.4.9. TheTest and Test::Harness Modules

Perl supplies two modules, Test and Test::Har-ness, to support its own testing infrastructure.Perl’s tests use different conventions than aegistests; specifically, Perl tests do not use the exitstatus to indicate the success or failure of the test,like aegis expects. TheTest::Harness moduleexpects that Perl tests report the success or failureof individual sub-tests on standard output, andalways exit with a status of 0 to indicate the scripttested everything it was supposed to.

This difference makes it awkward to use the Testand Test::Harness modules for aegis tests. Insome circumstances, though, you may be forcedto write tests using the Test and Test::Harnessmodules--for example, if you use aegis to developa Perl module for distribution--but still wish tohave the tests conform to aegis conventions dur-ing development.

This can be done by writing each test to use anenvironment variable to control whether its exitstatus should conform to aegis or Perl conven-tions. Thisis easy when using the Test module towrite tests, as itsonfail method provides anappropriate place to set the exit status to non-zeroif the appropriate environment variable is set.The following code fragment at or near the begin-ning of each Perl test script accomplishes this:


Aegis UserGuide

use Test;BEGIN { plan tests => 3,

onfail => sub {$? = 1 if $ENV{AEGIS_TEST}}

}

(See the documentation for the Test module forinformation about using it to write tests.)

There then needs to be a wrapper Perl scriptaround the execution of the tests to set the envi-ronment variable. Thefollowing script (calledmytest.pl for the sake of example) sets theAEGIS_TEST environment variable expected bythe previous code fragment:

use Test::Harness;$ENV{AEGIS_TEST} = 1;open STDOUT, ">/dev/null" || exit (2);runtests(@ARGV);END { $? = 1 if $?;

print STDERR $?? " FAILED" : "PASSED","\n"; }

It also makes its output more nearly conform toaegis’ examples by redirecting standard output to/dev/null and restricting its reporting of results toa simple FAILED or PASSED on standard erroroutput.

The last piece of the puzzle is to modify thetest_commandfield of the projectaegis.conffi leto have themytest.plscript call the test script:

test_command ="perl -I. -I${BaseLine} mytest.pl \${File_Name}"

The Test and Test::Harness modules are part ofthe standard Perl distribution and do not need tobe downloaded from anywhere. Becausethesemodules are part of the standard distribution, theycan be used by test scripts without being checkedin to the project.

8.2.4.10. Granularity By Steven Knight<[email protected]>

The granularity of Perl and Aegis tests mesh verywell at the individual test file (.t) level. Aegis andTest::Harness are simply different harnesses thatexpect slightly different conventions from thetests they execute: Aegis uses the exit code tocommunicate an aggregate pass/fail/no result sta-tus, Test::Harness examines the output from teststo decide if a failure occurred.

It’s actually pretty easy to accomodate both con-ventions. You can do this as easily as setting thetest_command variable in the project

configuration file to something like the following:

test_command ="perl -MTest::Harness -e ’runtests(\"$fn\"); \END {$$? = 1 if $$? }’";

In reality, you’ll likely need to add variableexpansions to generate -I or other Perl options forthe full Aegis search path.The END block takescare of mapping any non-zero Test::Harness exitcode to the ’1’ that Aegis expects to indicate afailure.

The only thing you really lose here is theTest::Harness aggregation of results and timing atthe end of a multi-test run. This is more than off-set by having Aegis track which tests need to berun for a given change.

Alternatively, you can execute the .t files directly,not through Test::Harness::runtests. Thisis easilyaccomodated using the onfail method from thestandard Perl Test module in each test.Here’s astandard opening block for .t tests

use Test;BEGIN { $| = 1; plan tests => 19,

onfail => sub { $? = 1 if $ENV{AEGIS_TEST} }}END {print "not ok 1\n" unless $loaded;}use Test::Cmd;$loaded = 1;ok(1);

That’s it (modulo specifying the appropriate num-ber of tests). My .t tests now use the proper exitstatus to report a failure back to Aegis. Theonlyother piece is configuring the project’s "test_com-mand" value to set the AEGIS_TEST environ-ment variable.

You can also use an intermediate script that alsoredirects the tests’s STDOUT to /dev/null, if youare used to and like the coarser PASSED/FAILEDstatus.

8.2.5. BatchTesting

The usual “test_command” f ield of the projectaegis.conffi le runs a single test at a time.Whenyou have a multi-CPU machine, or are able to dis-tribute the testing load across a range ofmachines, it is often desirable to do so.The“batch_test_command” of the project configura-tion file is for this purpose.Seeaepconf(5) formore information.


User Guide Aegis

9. Branching

This chapter describes the concept of branchingimplemented by Aegis. Theprocess described inprevious chapters makes changes to a project’smaster source.

Baseline

1 2 3

Branching generalizes this change model, byallowing the baseline to be treated as a change, orthe ability to treat a change as a baseline.

Trunk Baseline

Branch 11.1 1.2

Since branchs are sometimes considered as achanges it is useful to expand on the differences.A branch, or trunk, baseline may have childrenwhich are eitherchanges or deeperbranches.From this perspective the difference is that noth-ing may be modified directly in a branch. Abranch is an integrated baseline with all the asso-ciated protection. To modify a branch one mustopen a change under that branch.

Looking upward from a change under a branch,its parentis the branch baseline, and itsgrandpar-ent is theparent of the branch. We will see thisused later when we talk about ending a branch.

A significant feature of Aegis branches is that,because they are an extension of thechangecon-cept, they are expected to end, and be integratedback intotheir baseline, orparent,in time.

The most common case of this is in projectreleases.

A branch in thebeing developedstate may havechanges made to it, and/or deeper branches.Thismay recurse to any lev el. Oncea branch is com-plete, no further deeper branches may be createdfrom that branch.

9.1. How To Use Branching

To access a project branch, the project name hasthe branch appended, separated by a dot or ahyphen. For example: branch 1 of project "aegis"is referred to as "aegis.1". To reference changeson this branch, use this compound project namewherever you would normally use a project name.

Traditional 2-level project release names areobtained by using a further level of branching.For example: by creating branch 0 of project"aegis.1", there would be a branch accessed asproject "aegis.1.0".

By default, these two lev el of project branchingare created automatically when theaenpr(1) com-mand is used.You need to use the-VERSionoption to make this deeper or shallower, or hav edifferent numbering.

Command BranchesCreated

aenpr foo foo, foo.1,foo.1.0

aenpr foo -vers2.4.1

foo, foo.2,foo.2.4, foo.2.4.1

aenpr foo.7 foo, foo.7fooaegis -npr foo

-vers -

The last is a special case, to enable a project to becreated with no default branches (it’s also hard toget the empty string past the alias).

To add branching and release level managementto an existing project on uses theaenbr(1) com-mand at any lev el. Say we already havefoo.1.0 , which represents version 1.0 of oursoftware. One method of release level manage-ment would be to integratefoo.1.0into its parentfoo.1 and then doaenbr -p foo.1would createfoo.1.1 representing version 1.1.Eventually wemight want to make a major version release andwould integratefoo.1 into its parent fooand thendo aenbr -p foo, which would createfoo.2. Thenif we do aenbr -p foo.2we createfoo.2.0, fordevelopment of version 2.0 of our software.

9.2. Transition Using aenrls

To convert a project from the old-style to the newbranching style, use theaenrls(1) command.

If you give no second project name, the new nameis generated by removing the numeric suffixes. Ifyou did not give a -VERSion option, the numericsuffixes will be used to determine the next ver-sion, by adding one to the previous minor versionnumber. The new project is then created ratherlike theaenpr(1) command.

The files of the old project are copied across as animplicit change on the newly created branchwithin the new project. If the new project namealready exists, and is a new-style project, theaen-rls(1) command will attempt to make the appro-priate numbered branches.If the new projectalready exists and is an old-style project, or itexists and the branch number(s) are already inuse, aenrls(1) will emit an error and fail. Theaenrls(1) command only works on old-styleprojects, and always converts them to new style


Aegis UserGuide

projects.

Planning you branch numbers is essential.If youwant to use 3-level branch numbers (e.g."aegis.2.3.1") at some time in the future, then youmust use 3-level version numbers all the waythrough (e.g. "aegis.2.3.0"). This is becausechange numbers and branch numbers come fromthe same common pool of numbers. Once changeone has been used (e.g. "aegis.2.3.C001"), thenbranch one is no longer available (e.g."aegis.2.3.1.C042" conflicts).

9.3. Cross Branch Merge

From time to time you will want to merge thechanges from one branch into a change.Thismay be done using a cross-branch merge. Thisisdone by specifying the−BRanch option to theaegis -diff -merge-onlycommand.

The most common cross branch merge is whenthe project’s files are out-of-date. Because it isnot possible to useaegis -diff -merge-only directlyon the branch, this must be in a change on thebranch. Asa short-cut, the branch may be speci-fied using the−grandparentoption.

9.4. Multiple Branch Development

It is very common for a bug fix to need to beapplied to more than one branch at once.Thechange could be applied to the common ancestorbranch, however this may not be effective in thebranch immediately. An alternative is to use theaegis -clone command, which can be used toidentically reproduce a change on another branch.

9.5. Hierarchy of Projects

It would be nice if there was some way to use oneproject as a sort of "super change" to a "superproject", so that large teams (say 1000 people)could work as lots of small teams (say 100 peo-ple). As a small team gets their chunk ready,using the facilities provided to-date by Aegis, thesmall team’s baseline is treated as a change to bemade to the large team baseline.

This idea can be extended quite naturally to anydepth of layering.

After readingTr ansaction Oriented ConfigurationManagement: A Case StudyPeter Fieler, GraceDowney, CMU/SEI-90-TR-23, this is not a newidea. It also provides some ideas for how to dobranching sensibly, and was influential in thedesign of Aegis’ branching.

9.5.1. Fundamentals

Aegis has everything you need to have a superproject and a number of sub-projects. All youneed to do is create an active branch for each sub-project. Eachbranch gets a separate baseline,viz

% aenpr gizmo.0.1project "gizmo": createdproject "gizmo.0": createdproject "gizmo.0.1": created%

Now, for each of your desired sub-projects, createanother branch

aenbr -p gizmo.0.1 1 # for the foo projectaenbr -p gizmo.0.1 2 # for the bar projectaenbr -p gizmo.0.1 3 # for the baz project

Now, the guys on thefoo project set theirAEGIS_PROJECT environment variable to togizmo.0.1.1, the bar guys usegizmo.0.1.2, andbazusesgizmo.0.1.3. From the developer’s pointof view they are separate projects.From one levelup, though, they are just part of a bigger project.

It helps if you design and implement the buildsystem first. You do this as a change set on thecommon parent branch. Once it is completedeach branch can inherit it from the common par-ent. Thismakes integration easier, when it comestime to integrate the sub-projects together.

9.5.2. Incremental Integration

It is very common that not all of the sub-projectswill be ready to be integrated at the same time.This is the normal situation with Aegis branching,and is handled cleanly and simply.

In Aegis each branch is literally a change, all theway down into the internal data structures.Just aseach change gets its own development directory,each branch gets its own baseline. Just as a devel-opment directory inherits everything its doesn’thave from the baseline, so branches inherit every-thing theydon’t hav efrom their parent branch (orultimately from the trunk). Just as you incremen-tally integrate changes into a branch, you incre-mentally integrate branches into their parent.

The branches only influence each other when theyare integrated, just as changes only influence eachother when they are integrated.

There are times when a branch being integratedinto its parent is found to be inadequate.Aegisprovides a simple mechanism to “bounce” abranch integration. Recall that, for Aegis,branches are the same as changes. Just as you


User Guide Aegis

“develop end” a change (seeaede(1) for moreinformation) you alsoaedea branch when devel-opment on it is completed.

Once a branch has develop-end (stops being anactivebranch), it is reviewed as a normal change,and integrated as a normal change.If integrationfailed, it returns to “being developed” andbecomes an active branch once again, until thenextaede. As you can see, it is as easy to bouncea branch integration as it is to bounce a changeintegration.

An unsuccessful branch integration leaves therepository unchanged, just as an unsuccessfulchange integration leaves it unchanged.

9.5.3. Super-Project Branching

In many real-world situations it is very importantto be able to branch at any point in the past his-tory of the super-project to fix (integration spe-cific) bugs or to customize more the older statesof the super-project.

You can create a branch at any time, on any activebranch or active branch ancestor. You can popu-late that branch with historical versions (from anyother branch, actually, not just the ancestral line).The method is a little fussy − you can’t aecpintoa branch directly, you need to do this via a changeto that branch.Files not changed by a change ona branch are inherited from the current (i.e.active)state of the parent branch.See the section onInsulation, above.

9.5.4. Super-Project Testing

Many folks see Aegis’ testing features as usefulfor unit testing individual files or change sets.For large projects, it is common that a specifictest tool will be written. However, even largescale integration testing is possible using Aegis.

You can change the test command from being ashell script to being anything to you want - see thetest_commandfield in aepconf(5). Orrun the testtool from the shell script. If the integration testscan be automated, it makes sense to preservethem in the repository − they are some of the mostvaluable regression tests for developers, becausethey describe correct behavior outside the “box”the developer usually works in.

9.5.5. TheNext Cycle

Once you have a fully-integrated product, whathappens on the next cycle? Well, first you maywant to finish gizmo.0.1 and integrate it into

gizmo.0, and thenaenbr -p gizmo.0 2

Then what? Same deal as before, but anythingnot changed in one of the sub-project branchesgets inherited from the ancestor.

aenbr -p gizmo.0.2 1 # for the foo projectaenbr -p gizmo.0.2 2 # for the bar projectaenbr -p gizmo.0.2 3 # for the baz project

Most folks find doing the whole mega-project-build every time tiresome − so don’t. Temporar-ily (via a change set) hack the build configurationto build only the bit you want − obviously a dif-ferent hack on each sub-project’s branch. Justremember to un-hack it (via another change set)before you integrate the sub-project.

9.5.6. BugFixing

Theaeclone(1) command lets you clone a changeset from one branch to another. So if you have abug fix that needs to be done on each activebranch you can clone it (once you have it fixedthe first time). You still have to build review andintegraten times (branches often differ non-triv-ially). Providing it isn’t already in use, you canev en ask for the same change number on eachbranch − handy for syncing with an external bugtracking system.

Alternatively, fix bugs in the common ancestor,and the sub-projects will inherit the fix the nexttime they integrate something on their branch(assuming they aren’t insulated against it).

9.6. Conflict Resolution

A dev elopment directory becomes out of date,compared to the project, when another change isintegrated which has a file in common. This situ-ation is detected automatically byaede(1) andyou resolve it using aed(1), usually with some-thing like the --merge-only option. Additionally,you can see if you have an out-of-date file fromthe change files listing, because it will show youthe current baseline version in parentheses if youare out-of-date.

Aegis implements branches as very long changes,with sub-changes.A side effect of this is that abranch can become out-of-date in the same waythat a development director becomes out of date.When it comes time toaede(1) the branch, youwill be told if there are any out-of-date files.Additionally, theproject files listing will show outof date files in exactly the same way that thechange file listing does.


Aegis UserGuide

9.6.1. Cross Branch Merge

However, unlike a simple change, if you attemptto use theaed --merge-only command in thebranch baseline, you will get an error message!How, then, do you resolve the apparent impasse?

The aed(1) command has a number of optionsdesigned for just this purpose.

• The --branch option may be used to specifyanother branch of the same project, as a sourceof the file to be differenced against. Thisisalmost what you need.

• The --grandparentoption is a special case ofthe --branch option, and it means the parentbranch of project.

• The --trunk option is also a special case of the--branch option, and it means the base branchfrom which the entire branch tree springs.

By creating a new change on the out-of-datebranch, and copying in the out-of-date files, youhave almost everything required.All that is nec-essary is to perform a cross-branch merge againstthe project grandparent, and the necessary merg-ing will be performed. In addition Aegis willremember that it was a cross-branch merge, andonce aeipasscompletes successfully, the branchwill be up-to-date once more.

Create a new change on the out-of-datebranch

Use a simpleaecpcommand to copy the out-of-date files. (Do not use any --branch or--deltaoptions.)

Use the “aed --merge-only --grandparent”command to perform the merge.

At this point, if you use the “ael cf” com-mand, you will notice that this file is tagged inthe listing with the new branch edit origin, tobe used duringaeipass. If it i sn’t, you havemade a mistake.

As usual, use your favourite editor to checkthe merge results, and resolve any conflicts.

Build and test as usual.

Complete the change as usual.

Onceaeipassis successful, the branch will beup-to-date (for the files in the change).

9.6.2. Insulation

One of the stated benefits of using a branch is theinsulating effects which branches can provide.However, when you have multiple simultaneousactive branches, that insulation will inevitably

lead to out-of-date branch files. Now that how tomerge them has been described,whenshould youmerge?

In a simple change’s dev elopment directory, thereare times when anaeipasswill result in all devel-opers needing to recompile.Depending on whatfi les you are working on, it may be that you needto merge some of your change files immediately,or aecp an earlier version of the files whichchanged in the project.

Branches can also suffer from exactly the sameproblems, and are mended by exactly the samealternatives.

9.6.2.1. BranchInsulated Against Project

If you created a branch to insulate the work beingdone on the branch from other activities in theproject, it follows that when such build problemsoccurred, you would use an “aecp -delta” com-mand to continue insulating.

This action defers the labour of merging untiltowards the end of the branch development, some-times with a quite visible schedule impact.

9.6.2.2. Project Insulated Against Branch

If you created a branch to insulate the projectfrom work being done in the branch, it followsthat you would do a cross branch immediately.

This action amortizes the labour of mergingacross the life of the branch, often with a numberof small delays and less schedule impact.

9.6.2.3. Mix’n’ Match

Of course, we usually have both these motives,and some more besides, so the answer is usually“it depends”.

9.7. EndingA Branch

“OK, I give up. I do not understand the ending ofbranches.”

Usually, you end development of a branch thesame way you end development of a simplechange. Inthis example, branchexample.1.42will be ended:

% aede -p example 1 -c 42aegis: project "example.1": change42: file " fubar" in t he baselinehas changed since the last ’aegis-DIFFerence’ command, you need todo a merge%

Oops. Somethingwent wrong. Don’t panic!


User Guide Aegis

I’m going to assume, for the purposes of explana-tion, that there have been changes in one of theancestor branches, and thus require a merge, justlike file fubar, above.

You need to bring filefubarup-to-date. How?You do it in a change set, like everything else.

At his point you need to do 5 things: (1) create anew change on example.1.42, (2) copyfubar intoit, (3) mergefubarwith branch "example.1" (4)end development of the change and integrate it,and (5) now you can end example.1.42

The -GrandParent option is a special case of the-BRanch option.You are actually doing a cross-branch merge, just up-and-down rather than side-ways.

% aem -gp fubar%

And manually fix any conflicts... naturally.

At this point, have a look at the file listing, it willshow you something you have nev er seen before -it will show you what it isgoing toset thebranch’s edit_number_origin to for each file, ataeipass.

% ael cfType Action Edit File Name------ ------ ------- -----------source modify 1.3 fubar

{cross 1.2}

Now finish the change as usual...aeb, aed, aede,aerpass, aeib, ..., aeipassnothing special here.

One small tip: merge the files one at a time. Itmakes keeping track of where you are up to mucheasier.

Now you can end development of the branch,because all of the files are up-to-date.

In some cases, Aegis has detected a logical con-flict where you, the human, know there is none.Remember that theaemcommand saves the oldversion of the file with a,B suffix (‘B’ forbackup). Ifyou have a file like this, just use

% mv fubar,B fubar%

to discard everything from the ancestor branch,and use the current branch.


Aegis UserGuide

10. Tips and Traps

This chapter contains hints for how to use theaegis program more efficiently and documents anumber of pitfalls you may encounter.

This chapter is at present very "ad hoc" with noparticular ordering.Fortunately, it is, as yet,rather small. The final size of this chapter isexpected to be quite large.

10.1. RenamingInclude Files

Renaming include files can be a disaster, eitherfinding all of the clients, or making sure the newcopy is used rather than the old copy still in thebaseline.

Aegis provides some assistance. When theaemv(1) command is used, a file in the develop-ment directory is created in theold location, filledwith garbage. Compileswill fail very diagnosti-cally, and you can change the reference in thesource file, probably afteraecp(1)ing it first.

If you are moving an include file from one direc-tory to another, but leaving the basenameunchanged, create a link20 between the new andold names, but only in the development directory(i.e. replacing the "garbage" file aegis created foryou). Createthe link afteraemv(1) has suc-ceeded. Thisinsulates you from a number ofnasty Catch-22 situations in writing the depen-dency maintenance tool’s rules file.

10.2. SymbolicLinks

If you are on a flavor of UNIX which has symboliclinks, it is often useful to create a symbolic linkfrom the development directory to the baseline.This can make browsing the baseline very simple.

Assuming that the project and change defaults areappropriate, the following command

ln -s ‘aegis -cd -bl‘ bl

is all that is required to create a symbolic linkcalledbl pointing to the baseline. Note that theaecdalias is inappropriate in this case.

This can be done automatically for every change,by placing the line

develop_begin_command ="ln -s $baseline bl";

into the project configuration file.

20 A hard link uses fewer disk blocks.Symboliclinks survive the subject file being deleted andrecreated.

10.3. UserSetup

There are a number of things which users of aegiscan do to make it more useful, or more userfriendly. This section describes just a few ofthem.

10.3.1. The.cshrc or .profile files

The aliases for the various user commands usedthroughout this manual are obtained by appendinga line of the form

. / usr/local/share/aegis/profile

to the.profile fi le in the user’s home directory, ifthey use thesh(1) shell or thebash(1) shell.

If the user uses thecsh(1) shell, append a line ofthe form

source /usr/local/share/aegis/cshrc

to the.cshrcfi le in the user’s home directory.

These days, many systems also provide an/etc/profile.ddirectory, which has symbolic linksto the start-up scripts for various packages. Theseare run automatically for all users. If your systemhas such a thing, arrange for symbolic links

ln -s /usr/local/share/aegis/profile \/etc/profile.d/aegis.sh

ln -s /usr/local/share/aegis/cshrc \/etc/profile.d/aegis.csh

and you will not need to edit every user’s.cshrcor .profile fi le.

10.3.2. TheAEGIS_PATH environment vari-able

If users wish to use aegis for their own projects, inaddition to the "system" projects, theAEGIS_PATH environment variable forms a colonseparated search path of aegis "library" directo-ries. The/usr/local/lib/aegisdirectory is alwaysimplicitly added to this list.

The user should not create this library directory,but let aegis do this for itself (otherwise you willget an error message).

TheAEGIS_PATH environment variable shouldbe set in the.cshrcor .profile fi les in the user’shome directory. Typical setting is

setenv AEGIS_PATH ˜/lib/aegis

and this is the default used in the/usr/local/share/aegis/cshrcfi le.


User Guide Aegis

10.3.3. The.aegisrc file

The.aegisrcfi le in the user’s home directory con-tains a number of useful fields. Seeaeuconf(5)for more information.

10.3.4. Thedefaulting mechanism

In order for you to specify the minimum possibleinformation on the command line, aegis has beendesigned to work most of it out itself.

The default project is the project which you areworking on changes for, if there is only one, oth-erwise it is gleaned from the.aegisrcfi le. Thecommand line overrides any default.

The default change is the one you are working onwithin the (default or specified) project, if there isonly one. The command line overrides anydefault.

10.4. TheProject Owner

For the greatest protection from accidentalchange, it is best if the project is owned by aUNIX

account which is none of the staff. Thisaccountis often named the same as the project, or some-times there is a single umbrella account for allprojects.

When an aegis project is created, the owner is theuser creating the project, and the group is theuser’s default group. The creating user isinstalled as the project’s first administrator.

A new project administrator should be created -an actual user account. TheUNIX passwordshould then be disabled on the project account - itwill never be necessary to use it again.21

The user nominated as project administrator manythen assign all of the other staff roles. Aegis takescare of ensuring that the baseline is owned by theproject account, not any of the other staff, whiledevelopment directories always belong to thedeveloper (but the group will always be theproject group, irrespective of the developer’sdefault group).

All of the staff working on a project should bemembers of the project’s group, to be able tobrowse the baseline, for reviewers to be able toreview changes. Thisuse ofUNIX groups meansthat projects may be as secure or open as desired.

21 Unless bugs in aegis corrupt the database, inwhich case repairs can be accomplished as theproject account using a text editor.

10.5. USENETPublication Standards

If you are writing software to publish onUSENET, a number of the source newsgroupshave publication standards. This sectiondescribes ways of generating the following files,required by many of the newsgroups’ moderators:

MANIFEST List of files in the distribu-tion.

Makefile How to build the distribu-tion.

CHANGES What happened for thisdistribution.

patchlevel.h An identification of thisdistribution.

Each of these files may be generated from infor-mation known to aegis, with the aid of some fairlysimple shell scripts.

10.5.1. CHANGES

Write this section.

Look in theaux/CHANGES.shfi le included in theaegis distribution for an example of one way to dothis.

10.5.2. MANIFEST

Write this section.

Look in theaux/MANIFEST.shandaux/MANI-FEST.awkfi les included in the aegis distributionfor an example of one way to do this.

10.5.3. Makefile

Write this section.

Look in theaux/Makefile.shandaux/Makefile.awkfi les included in the aegis distribution for anexample of one way to do this.

10.5.4. patchlevel.h

Write this section.

Look in theaux/Howto.cookfi le included in theaegis distribution for an example of one way to dothis.

10.5.5. BuildingPatch Files

Thepatchprogram by Larry Wall is one of theenduring marvels of USENET. This sectiondescribes how to build input files for this miracleprogram.

Write this section.

Look in theaux/patches.shfi le included in theaegis distribution for an example of one way to do


Aegis UserGuide

this.


User Guide Aegis

10.6. Heterogeneous Development

The aegis program has support for heterogeneousdevelopment. Itwill enforce that each change bebuilt and tested on each of a list of architectures.It determines which architecture it is currentlyexecuting on by using theuname(2) system call.

Theuname(2) system call can yield unevenresults, depending on the operating systems ven-dor’s interpretation of what it should return22. Tocope with this, each required architecture for aproject is specified as a name and a pattern.

The name is used by aegis internally, and is alsoavailable in the${ARCHitecture}substitution (seeaesub(5) for more information).

The patterns are simple shell file name patterns(seesh(1) for more information) matched againstthe output of theuname(2) system call.

The result ofuname(2) has four fields of interest:sysname, release, versionandmachine. These arestitched together with hyphens to form an archi-tecturevariant to be matched by the pattern.

For example, a system the author commonly usesis "SunOS-4.1.3-8-sun4m" which matches the"SunOS-4.1*-*-sun4*" pattern.A solaris system,a very different beast, matches the"SunOS-5.*-*-sun4*" pattern. Sun’s 386 versionof Solaris matches the "SunOS-5.*-*-i86pc" pat-tern. Aconvex system matches the "Con-vexOS-*-10.*-convex" pattern.

10.6.1. Project aegis.confFile

To require a project to build and test on each ofthese architectures, thearchitecturefield of theprojectaegis.conffi le is set. Seeaepconf(5) formore details on this file. Theabove examples ofarchitectures could be represented as

architecture =[

{name = "sun4";pattern = "SunOS-4.1*-*-sun4*";

},{

name = "sun5";pattern = "SunOS-5.*-*-sun4*";

},

22 For example, SCO 3.2 returns the nodenamein the sysname field, when it should place "SCO"there; Convex and Pyramid scramble it even worse.

{name = "sun5pc";pattern = "SunOS-5.*-*-i86pc";

},{

name = "convex";pattern = "ConvexOS-*-10.*-*";

}];

This would require that all changes build and teston each of the "sun4", "sun5", "sun5pc" and "con-vex" architectures.

It is also possible to haveoptionalarchitectures.This may be used to recognise an environment,but not mandate that it be built every time.

{name = "solaris-8-sparc";pattern = "SunOS-5.8*-*-sun4*";mode = optional;

},

However, once an architecture name appears in achange’s architecture list, it is mandatory for thatchange.

If the architecturefield does not appear in theprojectaegis.conffi le, it defaults to

architecture =[

{name = "unspecified";pattern = "*";

}];

Setting the architectures is usually done as part ofthe first change of a project, but it also may bedone to existing projects. This information iskept in the projectaegis.conffi le, rather than as aproject attribute, because it requires that the DMTconfiguration file and the tests have correspond-ing details (see below).

The lib/config.example/architecturefi le inthe Aegis distribution contains many architecturevariations, so that you may readily insert theminto your project configuration file.

10.6.2. ChangeAttribute

Thearchitectureattribute is inherited by each newchange. Aproject administrator may subse-quently edit the change attributes to grant exemp-tions for specific architectures. Seeaeca(1) forhow to do this.

A build must be successfully performed on eachof the target architectures. Similarly, the testsmust be performed successfully on each. Theserequirements are because there is often


Aegis UserGuide

conditional code present to cope with the vagariesof each architecture, and this needs to be com-piled and tested in each case.

This multiple build and test requirement includesboth development and integration states of eachchange.

10.6.3. Network Files

This method of heterogeneous developmentassumes that the baseline and development direc-tories are available as the same pathname in alltarget architectures.With software such as NFS,this does not present a great problem, howeverNFS locking must also work.

There is also an assumption that all the hostsremotely mounting NFS file systems will agreeon the time, because aegis uses time stamps torecord that various tasks have been performed.Software such astimed(8) is required23.

10.6.4. DMTImplications

This method of heterogeneous developmentassumes that the baseline will have a copy of allobject files for all target architecturessimultane-ously.

This means that the configuration file for theDMT will need to distinguish all the variations ofthe object files in some way. The easiest methodis to have a separate object tree for each architec-ture24. To facilitate this, there is an${ARCHitec-ture} substitution available, which may then bepassed to the DMT using thebuild_commandfield of the projectaegis.conffi le.

The architecture name used by aegis needs to beused by the DMT, so that both aegis and the DMTcan agree on which architecture is currently tar-geted.

10.6.4.1. CookExample

As and example of how to do this, the cookrecipes from the DMT chapter are modified asappropriate. First,thebuild_commandfield of theprojectaegis.conffi le is changed to include the${ARCHitecture}substitution:

23 Some sites manage by runningrdate(8) fromcron(8) every 15 minutes.

24 A tree the same shape as the source tree makesnavigation easier, and users need not think of filenames unique across all directories.

build_command ="cook -b ${s Howto.cook} \project=$p change=$c \version=$v arch=’$arch’ -nl";

Second, the C recipe must be changed to includethe architecture in the path of the result:

[arch]/%.o: %.c: [collect c_incl-eia [prepost "-I" ""[search_list]] [resolve %.c]]

{if [not [exists [arch]]] then

mkdir [arch]set clearstat;

if [exists [target]] thenrm [target]

set clearstat;[cc] [cc_flags] [prepost "-I"

"" [search_list]] -c[resolve %.c];

mv %.o [target];}

Third, the link recipe must be changed to includethe architecture in the name of the result:

[arch]/example: [object_files]{

if [not [exists [arch]]] thenmkdir [arch]

set clearstat;if [exists [target]] then

rm [target]set clearstat;

[cc] -o [target] [resolve[object_files]] -ly -ll;

}

The method used to determine theobject_files variable is the same as before,but the object file names now include the architec-ture:

object_files =[fromto %.y [arch]/%.o

[match_mask %.y [source_files]]][fromto %.l [arch]/%.o

[match_mask %.l [source_files]]][fromto %.c [arch]/%.o

[match_mask %.c [source_files]]];

Note that the form of these recipes precludes per-forming a build in each target architecture simul-taneously, because intermediate files in therecipes may clash. However, aegis preventssimultaneous build, for this and other reasons.

10.6.5. Test Implications

Tests will need to know in which directory the rel-evant binary files reside. Thetest_commandfieldof the projectaegis.conffi le may be changed from


User Guide Aegis

the default

test_command ="$shell $file_name";

to pass the architecture name to the test

test_command ="$shell $file_name $arch";

This will make the architecture name available as$1 within the shell script.Tests should fail ele-gantly when the architecture name is not given, orshould assume a sensible default.

10.6.6. Cross Compiling

If you are cross compiling to a number of differ-ent target architectures, you would not use aegis’heterogeneous development support, since itdepends on theuname(2) system call, whichwould tell it nothing useful when cross compiling.In this case, simply write the DMT configurationfi le to cross compile to all architectures in everybuild.

10.6.7. FileVersion by Architecture

There is no intention of ever providing the facilitywhere a project source file may have different ver-sions depending on the architecture, but all ofthese versions overload the same file name25.

The same effect may be achieved by naming filesby architecture, and using the DMT to compileand link those files in the appropriate architecture.

This has the advantage of making it clear that sev-eral variations of a file exist, one for each archi-tecture, rather than hiding several related but inde-pendent source files behind the one file name.

10.7. Reminders

This section documents some scripts available forreminding users of changes which require theirattention. Thesescripts are installed into the/usr/local/share/aegis/reminddirectory, and maybe run bycron(8) at appropriate intervals. Youwill almost certainly want to customize them foryour site.

10.7.1. Awaiting Development

The/usr/local/share/aegis/remind/awt_dvlp.shscript takes a project name as argument. Itisplaced in the project leader’s per-user crontab. Itis suggested that this script be run weekly, at8AM on Monday. This script will send all

25 Some other SCM tools provide a repositorywith this facility.

developers of the named project email if there areany changes in theawaiting developmentstate inthe named project. No mail is sent if there are nochanges outstanding.

10.7.2. BeingDeveloped

The/usr/local/share/aegis/remind/bng_dvlpd.shscript takes no arguments. Itis placed in eachuser’s per-user crontab. It is suggested that thisscript be run weekly, at 8AM on Monday. Thisscript takes no arguments, and sends email to theuser if they hav eany changes in thebeing devel-opedor being integratedstates. Nomail is sent ifthere are no changes outstanding.

10.7.3. BeingReviewed

The/usr/local/share/aegis/remind/bng_rvwd.shscript takes a project name as argument. Itisplaced in the project leader’s per-user crontab. Itis suggested that this script be run daily at 8AM.This script will send all reviewers of the namedproject email if there are any changes in thebeingre viewedstate in the named project. No mail issent if there are no changes outstanding.

10.7.4. Awaiting Integration

The/usr/local/share/aegis/remind/awt_intgrtn.shscript takes a project name as argument. Itisplaced in the project leader’s per-user crontab. Itis suggested that this script be run daily at 8AM.This script will send all integrators of the namedproject email if there are any changes in theawaiting integrationstate in the named project.No mail is sent if there are no changes outstand-ing.


Aegis UserGuide

11. GeographicallyDistributed Development

This chapter describes various methods of collab-oratively developing software using Aegis, wherethe collaborating sites are separated by adminis-trative domains or even large physical distances.

While many Open Source projects on the Internettypify such development, this chapter will alsodescribe techniques suitable for commercialenterprises who do not wish to compromise theirintellectual property.

11.1. Introduction

The core of the distribution method is theaedist(1) command. In its simplest form, thecommand

aedist -send | aedist -receive

will clone a change set locally. This may appearless than useful (after all, theaeclone(1) com-mand already exists) until you consider situationssuch as

aedist -send | e-mail | a edist -receive

wheree-mailrepresents the sending, transport andreceiving of e-mail. In this example, the changeset would be reproduced on the e-mail recipient’ssystem, rather than locally. Similar mechanismsare also possible for web distribution.

11.1.1. RiskReduction

Receiving change sets in the mail, however,comes with a number of risks:

• You can’t just commit it to your repository,because it may not even compile.

• Even if it does compile, you want to run sometests on it first, to make sure it is working anddoesn’t break anything.

• Finally, you would always check it out, tomake sure it was appropriate, and didn’t domore subtle damage to the source.

While these are normal concerns for distributingsource over the Internet, and also internally withincompanies, they are the heart of the processemployed by Aegis. All of these checks and bal-ances are already present. The receive side sim-ply creates a normal Aegis change, and appliesthe normal Aegis process to it.

• The change set format is unpacked into a pri-vate work area, not directly into the reposi-tory. This is a normal Aegis function.

• The change set is then confirmed to buildagainst the repository. All implications

flowing from the change are exercised. Buildinconsistencies will flag the change for atten-tion by a human, and the change set will notbe committed to the repository. This is a nor-mal Aegis function.

• The change set is tested. If it came accompa-nied by tests, these are run. Also, relevanttests fromthe repository are run.Test incon-sistencies will flag the change for attention bya human, and the change set will not be com-mitted to the repository. This is a normalAegis function.

• Once the change set satisfies these require-ments, it must still be reviewed by a humanbefore being committed, to validate thechange set for suitability and completeness.This is a normal Aegis function.

11.1.2. Whatto Send

While there are many risks involved in receivingchange sets, there also problems in figuring outwhat to send.

At the core of Aegis’ design is a transaction.Think of the source files as rows in a database ta-ble, and each change-set as a transaction againstthat table. The build step represents maintainingreferential integrity of the database, but also rep-resents an input validation step, as does thereview. And like databases, the transactions areall-or-nothing affairs, it is not possible to commit“half” a transaction.

As you can see, Aegis changes are already ele-gantly validated, recorded and tracked, and ide-ally suited to being packaged and sent to remoterepositories.

11.1.3. Methodsand Topologies

In distributed systems such as described in thischapter, there are two clear methods of distribu-tion:

• The “push” method has the change set producerautomatically send the change-set to a regis-tered list of interested consumers. This is sup-ported by Aegis andaedist.

• The “pull” method has the change set producermake the change sets available for interestedconsumers to come and collect. This is sup-ported by Aegis andaedist.

These are two ends of a continuum, and it is pos-sible and common for a mix-and-match approachto be taken.


User Guide Aegis

There are also many ways of arranging how dis-tribution is accomplished, and many of the distri-bution arrangements (commonly called topolo-gies, when you draw the graphs) are supported byAegis andaedist:

• The star topology has a central master reposi-tory, surrounded by contributing satellite repos-itories. Thesatellites are almost always “push”model, however the central master could beeither “push” or “pull” model.

• The snowflake topology is like a hierarchicalstar topology, with contributors feeding stagingposts, which eventually feed the master reposi-tory. Common for large Open Source Internetprojects. Tow ards the master repository isalmost always “push” model, and away fromthe master is almost always “pull” model.

• The network topology is your basic anarchicautonomous collective, with change sets flyingabout peer-to-peer with no particular structure.Often done as a “push” model through an e-mail mailing list.

All of these topologies, and any mixture you candream up, are supported by Aegis andaedist. Thechoice of the right topology depends on yourproject and your team.

11.1.4. TheRest of this Chapter

Aegis is the ideal medium for hosting distributedprojects, for all the above reasons, and the rest ofthis chapter describes a number of different waysof doing this:

The second section will describe how to per-form these actions manually, both send andreceive, as this demonstrates the methodefficiently, and represents a majority of theuse made of the mechanism.

The third section will show how to automatee-mail distribution and receipt. Automated e-mail distribution is probably the next mostcommon use.

The fourth section will show how to configuredistribution and receipt using World WideWeb servers and browsers.

The fifth section deals with security issues,such as validating messages and coping withduplicate storms.

11.2. ManualOperation

This section describes how to useaedistmanu-ally, in order to send and receive change sets.

11.2.1. ManualSend

In order to send a change set to another site, itmust be packaged in a form which captures all ofthe change’s attributes and the contents of thechange’s files. Thispackage must be compressedand encoded in a form which will survive the var-ious mail transport agents it must pass through,and then given to the local mail transport agent.This is done by a single command

% aedist -send -c number | \mail [email protected]

%

All of the usual Aegis command line options areavailable, so you could specify the project on thecommand line if you needed to.

This command will send the sources from thedevelopment directory, if the change is not yetcompleted. Thisis useful for collaborationbetween developers, but it isn’t reviewed and inte-grated, so beware.

It is more usual to send a change which has beencompleted. Inthis case the version of the filewhich was committed is sent. If necessary, thehistory files will be consulted to reconstruct thisinformation. Seethe “Automatic Send” section,below, for more discussion of this.

There are many options for customizing the e-mail message sent [email protected] , seeaedist(1) for more information.

11.2.2. SendingBaselines

In order to send the entire contents of the reposi-tory to someone, you use a very similar com-mand:

% aedist -send -baseline | \mail [email protected]

%

This can be a very large change set, because it isall files of the project.

11.2.3. SendingBranches

There are times when remote developers are notinterested in a blow-by-blow update of yourrepository. Instead they want to have updatesfrom time to time. In order to send them the cur-rent state of your active dev elopment branch, inthis example “example.4.2”, you would use acommand of the form


Aegis UserGuide

% aedist -send -p example.4 -c 2 | \mail [email protected]

%

Notice how the correspondence between branchesand change sets is exploited. Thebaseline of abranch is the development directory of the “superchange” is represents.

Branch change sets like this are smaller thanwhole baselines, because they include only thefi les altered by this branch, rather then the state ofev ery file in the project.

11.2.4. ManualReceive

The simplest form of receiving a change set is tosave it from your e-mail program into a file, andthen

% aedist -receive -file filename...lots of information...%

wherefilenameis where you saved the e-mailmessage. Ifyour e-mail program is able to writeto a pipe, you can use a simpler form. This exam-ple uses the Rand Mail Handler’sshow(1) com-mand:

% show | aedist -receive...lots of information...%

Each of these examples assumes that you haveused the same project name locally as that of thesender (it’s stored in the package). If this isn’t thecase, you will need to use the−project optionto tell aedistwhich project to apply the change to.

The actions performed byaediston receive arenot quite a mirror of what it does on send. In par-ticular, send usually extracts its information fromthe repository, but receivedoes notput thechange set directly into the repository.

On receipt of a change set,aedistcreates a newchange with its own development directory, andunpacks the change set into it, in much the sameway as a change would normally be performed bya dev eloper. (Indeed, the receiver must be anauthorized developer.)

Once the change is unpacked, it goes through theusual development cycle of build, difference andtest. Ifany portion of this fails,aedistwill stopwith a suitable error message. If all goes well,development of the change will end, and it will beleft in thebeing reviewedstate.

At this point, a local reviewer must examine thechange, and it proceeds through the change

integration process as normal.

If there is a problem with the change, it can bedealt with as you would with any other defectivechange − by developing it some more. Or youcan email the sender telling them the problem anduseaedbu(1) andaencu(1) to entirely discard thechange.

11.2.5. GettingStarted

In order to receive a change, you must have aproject to receive it into. Also,changes tend to bethedifferencebetween an existing repository andwhat it is to become.You need some way to getthe starting point of the differences before youcan apply any differences. Thissection describesone way of doing this.

You start by creating a normal Aegis project inthe usual way. That is covered earlier in this UserGuide. Ithelps greatly if you give your localproject exactly the same name as the remoteproject. Itdoesn’t need the same pathname forthe project directory, just the same project name.

Once you have this project created, request theremote repository send you a “baseline” change,as described above. Once you have received this,and it is integrated successfully, you are ready toreceive and apply change sets. This is an inher-ently “pull” activity, as the source may never hav eheard of you before. The initial baseline mayarrive by e-mail, or floppy disk, or you mayretrieve it from the web, it all depends how theproject is being managed.

You will be warned about "potential trojan horse"fi les in the baseline change set. This is normal,because you are receiving all project configura-tions file, build files and test files. All of thesecontain executable commandsthat will be exe-cuted. Cav eat emptor. Make sure you trust thesource.

11.3. Sneaker Net

Another common method of transporting data,sometimes a quite large amount of it, is to write itonto transportable media and carry it.

To write a change set onto a floppy, you woulduse commands such as

% mount /mnt/floppy% aedist -send -no-base64 \

-o /mnt/floppy/ change.set% umount /mnt/floppy%

The above command assumes the floppy is pre-


User Guide Aegis

formatted, and that there is a user-permitting linein the/etc/fstabfi le, as is common for manyLinux distributions. Thechange.setcan be anyfi lename you like, but is usually project-name andchange-number related.

It takes a very sizable change set to fail to fit on a1.44MB floppy, because they are compressed(and change sets exceeding 8MB of source arerare, even for huge projects). The-no-base64option is used to avoid the MIME base 64 encod-ing, which is necessary for e-mail, not not neces-sary in this case. The receive side will automati-cally figure out there is no MIME base 64 encod-ing.

Reading the change set is just as simple, as itclosely follows the other commands for receivingcommands sets.

% mount /mnt/floppy% aedist -rec -f /mnt/floppy/ change.set...lots of output...% umount /mnt/floppy%

This technique will work for any of the disksavailable these days including floppies, Zip, Jaz,etc.

11.4. Automatic Operation

This section describes how to useaedistto auto-matically send change sets via e-mail.

11.4.1. Sending

Change sets can be sent automatically when achange passes integration. You do this by settingthe integrate_pass_notify_commandfield of theproject attributes.

In this example, the “example” project sends allintegrations to all the addresses on theexam-ple-developers mailing list. (The mailinglist is maintained outside of Aegis,e.g.by Major-domo.) Therelevant attribute (edited by using theaepa(1) command) looks like this:

integrate_pass_notify_command ="aedist -p $project -c $change | \mail example-users";

Please note that project attributes are inherited bybranches when they are created. If you don’twant all branches to broadcast all changes, youneed to remember to clear this project attributefrom the branchonce the branch has been created.Alternatively, use the$version substitution todecide who to send the change to.

11.4.2. Receiving

write this section

You need to set up an e-mail alias, with a wrapperaround it - you probablydon’t want "daemon" asa registered developer.

While aedist(1) makes every attempt to spotpotential trojan attacks, you really,really wantPGP validation (or similar industrial strength digi-tal signatures) before you accept this kind ofinput.

11.5. World Wide Web

This section describes how to useaeget(1) andaedist(1) to transport change sets using the WorldWide Web. This requires configuration of theweb server to package and send the change sets,and configuration of the browser to receive andunpack the change sets.You can also automati-cally track a remote site, efficiently downloadingand applying new change sets as they appear.

11.5.1. Server

Aegis has a read-only web interface to itsdatabase, it is a web server CGI interface. Ifyouare running Apache, or similar, all you have to dois copy (or symlink, if you have symlinksenabled) the/usr/local/bin/aeget fi le into the webserver’scgi-bindirectory. For example, thedefault Apache install would need the followingcommand:

ln -s /usr/local/bin/aeget /var/www/cgi-bin/aeget

11.5.2. Browser

You need to set the appropriate mailcap entry, sothatapplication/aegis-change-set ishandled byaedist --receive.

Edit the/etc/mailcapfi le, and add the lines

# Aegisapplication/aegis-change-set;/usr/local/bin/aedist -receive -f %s

You may need to restart your web browser for thisto take effect.

11.5.3. Hands-Free Tracking

Clients of sites using a web server, such as thevarious developers in an open sourec project, it ispossible to automatically "replay" change sets onthe server which have not yet been incorporated atyour site.

The command


Aegis UserGuide

aedist --replay -f name-of-web-server

will automatically download any remote changesets not present in the local repository. It down-loads them by using theaedist(1) command. Ituses commands of the form

aedist --receive -f url-of-change-set

to download the change sets, which have to gothrough all of the usual Aegis process beforevecoming part of your local repository. Thisincludes code review, unless you have configuredthedevelop_end_actionfield of the project con-figuration to begoto_awaiting_development.

If you add this command to acrontab(1) entry,you can check to see if there are change sets tosynchronize with once a day, or however oftenyou set the line to run.

11.6. Security

This section deals with security issues. Securityisn’t just “keep the bad guys out”, it actually cov-ersavailability, integrityandconfidentiality.

Av ailability:refers to the system being available foruse by authorised users. Denial of ser-vice and crashes are examples of badthings in this area.

Integrity:refers to the system being in a knowngood state. Corrupted change sets andun-buildable repositories are examplesof bad things in this area.

Confidentiality:refers to the system being availableonlyto authorised users.For many OpenSource projects, this isn’t a large con-cern, but for corporate users of Aegis,non-disclosure of change-sets as theycross the Internet is a serious require-ment.

As you can see, a strategy of “keep the bad guysout” is necessary, but not sufficient, to satisfysecurity.

This section covers the above security issues, asapplied to the use ofaedistto move change setsaround.

11.6.1. Trojan Horses

“A Trojan horse is an apparently useful programcontaining hidden functions that can exploit theprivileges of the user [running the program], witha resulting security threat.A Trojan horse does

things that the program user did not intend26.”

In order to forestall this threat,aedistwill ceasedevelopment of the change if it detects the poten-tial for a Trojan horse. These include...

• Changing the projectaegis.conffi le. Thisfi lecontains the build command and the differencecommands, both of which would be run beforea reviewer had a chance to confirm they wereacceptable.

• Changing any of the files named in thetro-jan_horse_suspectfield of the projectaegis.conffi le. Thislets you cover things likethe build tool’s configuration file (e.g.theMakefile or the cookbook), and any scripts orcode generators which would be run by thebuild.

This isn’t exhaustive protection, and it depends onkeeping thetrojan horse suspectlist up-to-date.(It accepts patterns, so it’s not too onerous.)Forbetter protection, you need to validate the senderand the message.

11.6.2. PGP

PGP can be used to validate that the source of achange set distribution is really someone youtrust.

anyone want to advise me what to put here?

11.6.3. Sorcerer’s Apprentice

In a push system, with a central master server anda collection of contributors, all of which are usingautomatic send, as described above, a potentialexplosion of redundant messages is possible.Viz:

• Contributor integrates a change, which is dis-patched to the master server.

• Maintainer integrates the change set into themaster repository.

• Master repository automatically dispatches thechange set to all of the contributors.

• Each of the contributors receives and integratesthe change, each of which are dispatched to themaster server.

• The master server is inundated with change setsit already has.

• If these change sets were to be integrated, thestorm repeats, growing exponentially everytime it goes around the loop.

26 Summers, Rita C.,Secure Computing Threatsand Safeguards, McGraw-Hill, 1997.


User Guide Aegis

To prevent this,aedistdoes several things...

• Before the change is built, anaecpu--unchangedis run. If there is nothing left, thechange is abandoned, because you already haveit. (This doesn’t always work, because propa-gation delays may try tore versea subsequentlocal change.)

• When a change set is sent, an RFC 822 styleheader is added to the description. Thisincludes From and Date. When a change set isreceived, a Received line is added.Too manyReceived lines causes the change set to bedropped − for a star topology the maximum is2. (Thisdoesn’t always work, because thedescription could be edited to rip it off again.)(This doesn’t always work, because the main-tainer may edit it in some ways before comit-ting it, and forget to rip off (enough of) theheader.) (Thisdoesn’t always work, becausehierarchical topologies will group change setstogether.) (Thisdoesn’t always work, becausea baseline pull will group change sets together.)

• Set the description to indicate it was receivedby aedist? Use this to influence the decision tosend it off again at integrate pass? How?

11.7. Patches

In the open source community, patches are com-mon way of sharing enhancements to software.This was particularly common before the WorldWide Web, and usenet was the more commonmedium of distribution. Patches also have theadvantage of being fairly small and usually tans-portable by email with few problems.

11.7.1. Send

If you are participating in an open source project,and using Aegis to manage your development, theaepatch −sendcommand may be used to con-struct a patch to send to the other developers.

It is very similar in operation to theaedist(1)command, however it is intended for non-Aegis-using recipients.

To send a change to someone (a completedchange, or one in progress) simply use a com-mand such as

% aepatch -send -c number | \mail [email protected]

%

to send your change as a patch. Note that it willbe compressed (using GNU Zip) and encoded(using MIME base 64), which produces small

fi les which are going to survive email transport.

11.7.2. Receive

The simplest way of receiving a patch and turn itinto a change set is to save it from your e-mailprogram into a file, and then

% aepatch -receive -file filename...lots of information...%

wherefilenameis where you saved the e-mailmessage. Ifyour e-mail program is able to writeto a pipe, you can use a simpler form. This exam-ple uses the Rand Mail Handler’sshow(1) com-mand:

% show | aepatch -receive...lots of information...%

Each of these examples assumes that you havealready set the project name, either viaaeuconf(5)or ae_p(1), or you could use the−project option.

The actions performed byaepatchon receive arenot quite a mirror of what it does on send. In par-ticular, send usually extracts its information fromthe repository, but receivedoes notput thechange set directly into the repository.

On receipt of a change set,aepatchcreates a newchange with its own development directory,copies the files into it, and applies the patch to thefi les. Thereceiver must be an authorized devel-oper.

Once the patch is applied, it goes through theusual development cycle of build, difference andtest. Ifany portion of this fails,aepatchwill stopwith a suitable error message. If all goes well,development of the change will end, and it will beleft in thebeing reviewedstate.

At this point, a local reviewer must examine thechange, and it proceeds through the change inte-gration process as normal.

If there is a problem with the change, it can bedealt with as you would with any other defectivechange − by developing it some more. Or youcan email the sender telling them the problem anduseaedbu(1) andaencu(1) to entirely discard thechange.

11.7.3. Limitations

Despite a great similarity of command line opera-tions and operation, theaepatchcommand shouldnot be thought of as an equivalent for theaedistcommand, or a replacement for it.


Aegis UserGuide

The information provided byaedist −sendis suf-ficiently complete to recreate the change set at theremote end. No information is lost. In contrast,theaepatch −sendcommand is limited to thatinformation a patch file (see thepatch(1) com-mand, from the GNU Diff utils). Thereis noguarantee that theaepatch −sendoutput will begiven to aepatch −receive; it must work withpatch(1), and similar tools.

Conversely, there is no guarantee that the input toaepatch −receivecame fromaepatch −send. Itcan and must be able to cope with the outout of asimplediff -r -N -c command, with no additionalinformation.

All this means, useaedistwherever possible. Theaepatchcommand is to simplify and streamlinecommunication with non-Aegis developers.


User Guide Aegis

12. Further Reading

This chapter contains information about books,articles or web sites relevant to some aspect ofAegis or using Aegis. Thesereferences shouldnot be taken as endorements.

If I’ ve missed a good reference, it isn’t personal,but I can’t and haven’t read everything out there.Email me the information and I’ll add it to thischapter (no advertising, please).

12.1. Software Configuration Management

Eaton, D. (1995), Configuration ManagementFrequently Asked Questions, http://www.dav-eeaton.com/scm/CMFAQ.html

This is an essential first-stop for infor-mation about Software ConfigurationManagement. Ithas an excellent booklist.

Pool, D., CM Today, http://www.cmtoday.com/This is a configuration management por-tal site, with news and other links.

12.2. Reviewing

Baldwin, J. (1992), An Abbreviated C++ CodeInspection Checklist,http://www2.ics.hawaii.edu/˜john-son/FTR/Bib/Baldwin92.html

This web page talks about C++ codeinspections with some useful suggestionsabout how to conduct (rather formal)re views and some for C++ constructs towatch out for.

Page 118 (bl/lib/en/user-guide/cA.0.so) Peter Miller

Aegis UserGuide

13. Appendix A: New Project Quick Reference

For those of you too impatient to read a wholegreat big document about how to use the aegisprogram, this appendix gives a quick look at howto place a project under aegis.

The style here is an itemized list. It does not tryto be exhaustive. For exact details on how to usethe various aegis commands, you should see themanual pages, ditto for the formats and contentsof some files.

Probably the quickest start of all is to copy analready existing project. The project used inchapter 2 is complete, assuming you use theauthor’s "cook" dependency maintenance tool.The entirety of this example may be found, ifslightly obfuscated, in the aegis source filetest/00/t0011a.shdistributed with aegis.

13.1. Create the Project

Theaenprcommand is used to create a project.You must supply the name on the command line.The name should be ten characters or less, sixcharacters or less if you want version numbersincluded.

The user who creates the project is the owner ofthe project, and is set as the administrator. Thedefault group of the user who created the projectis used as the project’s group.

You may want to have a user account which ownsthe project.You must create the project as thisuser, and then use theaenaandaeracommandsto add an appropriate administrator, and removethe owning user as an administrator. After this,the password for the owning user may be dis-abled, because the aegis program will, at appro-priate times, set file ownership to reflect projectownership or execute commands on behalf of theproject ownerasthe project owner.

13.1.1. Addthe Staff

Theaendcommand is used to add developers.Theaenrvcommand is used to add reviewers.Theaenicommand is used to add integrators.These commands may only be performed by aproject administrator.

You will still have to do this, even if the personwho created the project will be among these peo-ple, or even be all of these people.

13.1.2. Project Attributes

Theaepacommand is used to change projectattributes. Theseattributes include the description

of the project, and booleans controlling whether,for example, developers may review their ownwork.

The project attributes file is described in theaepattr(5) manual entry.

13.2. Create Change One

Theaenccommand is used to create a newchange. You will need to construct a changeattributes file with your favorite text editor beforerunning this command.

The change attributes file is described in theaecattr(5) manual entry.

13.3. Develop Change One

This is the most grueling step. Indeed, the inte-gration step will probably reveal things youmissed, and you may return to thebeing devel-opedstate several times.

One of the people you nominated as a developerwill have to use theaedbcommand to commencedevelopment of the first change. Theaecdcom-mand can be used to change directory into thejust-created development directory.

Add files to the change. Theaenfcommand isused to create new files. If you don’t useaenfthen the aegis program has no way of knowingwhether that file lying there in the developmentdirectory is significant to the project, or just ashopping list of the groceries you forgot to buyyesterday.

One particular new file whichmustbe created bythis change is the project configuration file, usu-ally calledaegis.confbut can be named somethingelse. Thisfi le tells Aegis what history mechanismyou wish to use, what dependency maintenancecommand to use, what file difference tools to use,and much more. Theaepconf(5) manual entrydescribes this file.

If you are going to use the "cook" dependencymaintenance tool, another new file you will needto create in this change is the "Howto.cook" file.Some other tool will want some other rules file.

You probably have a prototype or some other"seed" you have sort-of working. Createnew filesfor each source file andthencopy the files fromwherever they are now into the developmentdirectory.

Use theaebcommand to build the change. It willneed to build cleanly before it can advance to the

Peter Miller (bl/lib/en/user-guide/cA.0.so) Page 119

User Guide Aegis

next step.

Use theaedcommand to difference the change. Itwill need to difference cleanly before it canadvance to the next step.

Use theaentcommand to add new tests to thecommand. Itwill need to have tests before it canadvance to the next step.

Most existing projects don’t hav eformal tests.These tests will form a regression test-bed, usedto make sure that future changes never compro-mise existing functionality.

Use theaetcommand to test the change. It willneed to test cleanly before it can advance to thenext step.

Once the change builds, differences and testscleanly, use theaedecommand to end develop-ment.

13.4. Review The Change

One of the people nominated as reviewers willhave to run theaerpasscommand to say that thechange passed review.

The aegis program does not mandate any particu-lar review mechanism: you could use a singlepeer to do the review, you could use a panel, youcould set the project so that developers mayreview their own work effectively eliminating thereview step. Inprojects with as few as two peo-ple, it is always beneficial for someone other thanthe developer to review changes. Itis even bene-ficial for the developer herself to review the nextday.

Should a reviewer actually want toseethechange, theaecdcommand may be used tochange directory to the development directory ofthe change. The difference files all end with a"comma D" suffix, so the

more ‘find . -name "*,D" -print |

sort‘

command may be used to search them out and seethem. Thisis probably fairly useless for the firstchange, but is vital for all subsequent changes.There is a supplied alias for this command, it isaedmoreand there is a similaraedlessalias if youprefer theless(1) command.

There are some facts that a reviewerknowsbecause otherwise the change would not be in the"being reviewed" state: • the change compilescleanly, • the change passes all of its tests. Otherinformation about the change may be obtainedusing the "change_details" variation of theael

command.

Theaerfail command may also be used byreviewers to fail reviews and return a change tothe developer for further work; the reviewer mustsupply a reason for the change history to recordfor all time. Similarly, theaedeucommand maybe used by the developer to resume developmentof a change at any time before it is integrated; nostated reason is required.

13.5. Integratethe Change

A person nominated as an project integrator thenintegrates the change. This involves making acopy of the integration directory, applying themodifications described by the change to thisintegration directory, then building and testing allover again.

This re-build and re-test is to ensure that no spe-cial aspect of the developers environment influ-enced the success up to this point, such as aunique environment variable setting. The re-buildalso ensures that all of the files in the baseline,remembering that this includes source files and allother intermediate files required by the build pro-cess, remain consistent with each other, that thebaseline is self-consistent. The definition of thebaseline is that it passes its own tests, so the testsare run on the baseline.

Use theaeibcommand to begin integration.

Theaebcommand is used to build the integrationcopy of the change.

Theaetcommand is used to test the integrationcopy of the change.

On later changes, the integration may also requiretheaet -blcommand to test the change against thebaseline. Thistests ensures that the testfailsagainst the baseline. This failure is to ensure thatbug fixes are accompanied by tests which repro-duce the bug initially, and that the change hasfixed it. New functionality, naturally, will not bepresent in the old baseline, and so tests of newfunctionality will also fail against the old base-line.

Later changes may also have the regression testsrun, using theaet -regcommand. Thiscan be avery time-consuming step for projects with a longhistory, and thus a large collection of tests. Theaet -suggest command can also be used to run"representative" sets of existing tests, but a fullregression test run is recommended before amajor release, or, say, weekly if it will completeover the weekend. Thiscommand is also

Page 120 (bl/lib/en/user-guide/cA.0.so) Peter Miller

Aegis UserGuide

available to developers, so that they hav efewersurprises from irate integrators.

The integrator may use theaeifail command toreturn a change to its developer for further work;a reason must be supplied, and should include rel-evant excerpts from the build log in the case of abuild failure (not thewholelog!), or a list of thetests which failed for test failures.

Theaeipasscommand may be used to pass anintegration. Whenthe change passes, the file his-tories are updated. In the case of the first change,the history is created, and problems with theproject configuration file’s history commands willbe revealed at this point. The integration won’tpass, and should be failed, so that the developermay effect repairs. There are rarely problems atthis point for subsequent changes, except for diskspace problems.

Once the history is successfully updated, aegisrenames the integration directory as the baseline,and throws the old baseline away. The develop-ment directory is deleted at this time, too.

13.6. Whatto do Next

There, the first change is completed. The wholecycle may now be repeated, starting at "CreateChange," for all subsequent changes, with veryfew differences.

It is recommended that you read theChangeDevelopment Cyclechapter for a full worked example of the first fourchanges of an example project, including some ofthe twists which occur in real-world use of aegis.

Remember, too, the definition:

aegis(ee.j.iz)n. a protection, a defence.

It is not always the case that aegis exists to makelife "easier" for the software engineers. The goalis to have a baseline which always "works",where "works" is defined as passing all of its owntests. Wherever possible, the aegis programattempts to be as helpful and as unintrusive aspossible, but when the "working" definition isthreatened, the aegis program intrudes as neces-sary. (Example: you can’t do an integrate passwithout the integration copy building success-fully.)

All of the "extra work" of writing tests is a long-term win, where old problems never again reap-pear. All of the "extra work" of reviewingchanges means that another pair of eyes sees thecode and finds potential problems before they

manifest themselves in shipped product. All ofthe "extra work" of integration ensures that thebaseline always works, and is always self-consis-tent. All of the "extra work" of having a baselineand separate development directories allows mul-tiple parallel development, with no inter-devel-oper interference; and the baseline always works,it is never in an "in-between" state. In each case,not doing this "extra work" is a false economy.

Peter Miller (bl/lib/en/user-guide/cA.0.so) Page 121

User Guide Aegis

14. Appendix B: Glossary

The following is an alphabetical list of terms usedin this document.

administratorPerson responsible for administering aproject.

aw aiting_developmentThe state a change is in immediately aftercreation.

aw aiting_integrationThe state a change is in after it has passedreview and before it is integrated.

aw aiting reviewAn optional state a change is in after it isdeveloped, but before someone has chosento review it..

baselineThe repository; where the project mastersource is kept.

being developedThe state a change is in when it is beingworked on.

being integratedThe state a change is in when it is beingintegrated with the baseline.

being reviewedThe state a change is in after it is devel-oped.

changeA collection of files to be applied as a sin-gle atomic alteration of the baseline.

change numberEachchangehas a unique number identify-ing it.

completedThe state a change is in after it has beenintegrated with the baseline.

delta numberEach time theaeib(1) command is used tostart integrating achangeinto thebaselineaunique number is assigned. This number isthe delta number. This allows ascendingversion numbers to be generated for thebaseline, independent of change numbers,which are inevitably integrated in a differ-ent order to their creation.

dependency maintenance toolA program or programs external to aegiswhich may be given a set of rules for howto efficiently take a set of source files and

process them to produce the final product.

DMTAbbreviation of Dependency MaintenanceTool.

develop_beginThe command issued to take a change fromtheawaiting developmentstate to thebeingdevelopedstate. Thechange will beassigned to the user who executed the com-mand.

develop_begin_undoThe command issued to take a change fromthebeing developedstate to theawaitingdevelopmentstate. Any files associatedwith the change will be removed from thedevelopment directory and their changeslost.

develop_endThe command issued to take a change fromthebeing developedstate to thebeingre viewedstate, or optionally to theawaitingre viewedstate. Thechange must be knownto build and test successfully.

develop_end_undoThe command issued to take a change fromthebeing reviewedstate back to thebeingdevelopedstate. Thecommand must beexecuted by the original developer.

developerA member of staff allowed to developchanges.

development directoryEach change is given a unique developmentdirectory in which to edit files and buildand test.

history toolA program to save and restore previous ver-sions of a file, usually by storing editsbetween the versions for efficiency.

integrate_passThe command used to take a change fromthebeing integratedstate to thecompletedstate. Thechange must be known to buildand test successfully.

integrate_beginThe command used to take a change fromtheawaiting integrationstate to thebeingintegratedstate.

integrate_begin_undoThe command used to take a change fromthebeing integratedstate to theawaiting

Page 122 (bl/lib/en/user-guide/cB.0.so) Peter Miller

Aegis UserGuide

integrationstate.

integrate_failThe command used to take a change fromthebeing integratedstate back to thebeingdevelopedstate.

integrationThe process of merging thebaselinewiththedevelopment directoryto form a newbaseline. Thisincludes building and testingthe merged directory, before replacing theoriginalbaselinewith the new merged ver-sion.

integration directoryThe directory used duringintegrationtomerge the existingbaselinewith a change’sdevelopment directory.

integratorA staff member who performsintegrations.

new_changeThe command used to create new changes.

new_change_undoThe command used to destroy changes.

review_beginThe command used to take a change fromtheawaiting reviewstate to thebeingre viewedstate.

review_failThe command used to take a change fromthebeing reviewedstate back to thebeingdevelopedstate.

review_passThe command used to take a change fromthebeing reviewedstate to theawaitingintegrationstate.

reviewerA person who may reviewchangesandeither pass or fail them (re view_passorre view_fail respectively).

stateEachchangeis in one of seven states:awaiting development, being developed,awaiting review, being reviewed, awaitingintegration, being integratedor completed.

state transitionThe event resulting in achangechangingfrom one state to another.

Peter Miller (bl/lib/en/user-guide/cB.0.so) Page 123

User Guide Aegis

15. Appendix D: Why is Aegis Set-Uid-Root?

The goal for aegis is to have a project that"works". Thereis a fairly long discussion aboutthis earlier in this User Guide. One of the firstthings that must be done to ensure that a project isnot subject to mystery break downs, is to makesure that the master source of the project cannotbe in any way altered in an unauthorized fashion.Note this says "cannot", a stronger statement than"should not".

Aegis is more complicated than, say, set-group-idRCS, because of the flaw with set-group-id: thebaseline is writable by the entire developmentteam, so if a developer says "this developmentprocess stinks" he can always bypass it, and writethe baseline directly. This is averycommonsource of project disasters.To prevent this, youmust have the baseline read-only, and so the set-group-id trick does not work. (Theidea here isthat there isnoway to bypass the QA portions ofthe process. Sure, set-group-id will prevent acci-dental edits on the baseline, if the developers arenot members of the group, but it does not preventdeliberatecheckin of unauthorized code. Again,the emphasis is on "cannot" rather than "shouldnot".)

Also, using the set-group-id trick, you need multi-ple copies of RCS, one for each project. Aegiscan handle many projects, each with a differentowner and group, with a single set-uid-rootexecutable.

Aegis has no internal model of security, it usesUNIX security, and so becomes each user in turn,soUNIX can determine the permissions.

15.1. Examples

Here are a few examples of the uid changes incommon aegis functions. Unix "permissiondenied" errors are not shown, but it should beclear where they would occur.

new change (aenc):become invoking user and read (edit) thechange attribute file, validate the attributefi le, then become the project owner to writethe change state file and the project statefi le.

develop begin (aedb):become the project owner and read theproject state file and the change state file, tosee if the change exists and is available fordevelopment, and if the invoking user is onthe developer access control list. Become

the invoking user, but set the default groupto the project group, and make a dev elop-ment directory. Become the project owneragain, and update the change state file tosay who is developing it and where.

build (aeb):become the project owner to read theproject and change state files, check that theinvoking user is the developer of thechange, and that the change is in thebeingdevelopedstate. Becomethe invoking user,but set the default group to the projectgroup, to invoke the build command.Become the project owner to update thechange state to remember the build result(the exit status).

copy file into change (aecp):become the project owner to read theproject and change state files. Checkthatthe invoking user is the developer and thatthe change is in thebeing developedstate,and that the file is not already in thechange, and that the file exists in the base-line. Becomethe invoking user, but set thedefault group to the project group, and copythe file from the baseline into the develop-ment directory. Become the project owner,and update the change state file to remem-ber that the file is included in the change.

integrate fail (aeifail):become the project owner to read theproject and change state files. Checkthatin invoking user is the integrator of thechange, and that the change is in thebeingintegratedstate. Becomethe integrator tocollect the integrate fail comments, thenbecome the project owner to delete the inte-gration directory, then become the devel-oper to make the development directorywritable again. Thenbecome the projectowner to write the change state file, toremember that the change is back in thebeing developedstate.

integrate pass (aeipass):become the project owner to read theproject and change state files. Checkthatin invoking user is the integrator of thechange, and that the change is in thebeingintegratedstate. Make the integrationdirectory the new baseline directory andremove the old baseline directory. Writethe change and project states to reflect thenew baseline and the change is in the

Page 124 (bl/lib/en/user-guide/cD.0.so) Peter Miller

Aegis UserGuide

completedstate. Thenbecome the devel-oper to remove the development directory.

All the mucking about with default groups is toensure that the reviewers, other members of thesame group, have access to the files when itcomes time to review the change. The umask isalso set (not shown) so that the desired level of"other" access is enforced.

As can be seen, each of the uid change either (a)allowsUNIX to enforce appropriate security, or (b)usesUNIX security to ensure that unauthorizedtampering of project files cannot occur. Eachproject has an owner and a group: members of thedevelopment team obtain read-only access to theproject files by membership to the appropriategroup, to actually alter project files requires thatthe development procedure embodied by aegis iscarried out.You could have a single account (nota user’s account, usually, for obvious conflicts ofinterest) which owns all project sources, or youcould have one account per project.You can haveone group per project, if you don’t want your var-ious projects to be able to see each other’s work,or you could have a single group for all projects.

15.2. Source Details

For implementation details, see theos_become * f unctions in theaegis/os.cfi le.Theos_become_init function is called veryearly inmain , in theaegis/main.cfi le. After that,all accesses are bracketed byos_become andos_become_undo function calls, sometimesindirectly asproject_become * oruser_become *, etc, functions.You need toactually become each user, because root is notroot over NFS, and thuschown tricks do notwork, and also because duplicating kernel permis-sion checking in aegis is a little non-portable.

Note, also, that most system calls go via the inter-face described in theaegis/glue.hfi le. Thisiso-lates the system calls forUNIX variants which donot have theseteuid function, or do not have acorrectly working one. The code in theaegis/glue.cfi le spawns "proxy" process whichuses thesetuid function to become the user andstay that way. If theseteuid function is avail-able, it is used instead, making aegis moreefficient. Thisisolation, however, makes it possi-ble for a system administrator to audit the aegiscode (for trojans) with some degree of confi-dence. Systemcalls should be confined to theaegis/log.c, aegis/pager.c, aegis/os.candaegis/glue.cfi les. Systemcalls anywhere else are

probably a Bad Thing.

Peter Miller (bl/lib/en/user-guide/cI.0.so) Page 125

User Guide Aegis

16. Appendix I: Internationalization andLocalization

The Aegis source code has been internationalized,which is the process of modifying the originalsource code to permit error messages and othertext to be presented in a language other than theauthor’s native English. Thiswas a large andoften painful task, but it allows a degree of cus-tomization of error messages and otherbehaviours which would not have been otherwisepossible. (Italso makes the job of running aspell-checker over the error messages signifi-cantly easier.)

Localization is the process of translating the errormessages and other text into various different lan-guages or nationalities. This appendix is primar-ily aimed at localizers of Aegis.

16.1. The“.po” Files

The “lib/en/LC_MESSAGES” directory in thesource tree contains the various message filesneeded to localize Aegis. You will find a numberof “.po” files in this directory, which translates“programmer cryptic” into English.You will seethat each message has a comment attached,describing the message and the context in which itis used. Many messages also have “substitutions”described, which are strings similar to shell vari-ables which may be substituted into the message -such as the file name for messages which havesomething to do with a specific file.

The substitution mechanism is the same one as isused for the various commands in the projectaegis.conffi le, and so all of the substitutionsdescribed inaesub(5) are available to the transla-tor. Note frequent use of theplural substitution,which allows grammatically correct error mes-sages to be issued when faced with the singu-lar/plural dichotomy. Other substitutions includethe login name of the executing user, names ofprojects, number and state of changes, etc.

Ideally, the task for a translator is to take the.pofi les and translate themsgstr lines into theappropriate language. The job will, of course, notbe that simple and so references into the codehave been included, so that you can read the codeshould context be necessary to correctly translatethe message.

16.2. Checkingthe Code

There are a number of keywords you need to havefor thexgettextprogram when extracting messagestrings. Thegettext keyword is not used

directly, because of the substitution mechanismwrapped around it.

i18n error_intlio_comment_append fatal_intlreport_error verbose_intlreport_error gram_errorrpt_value_error

In general, theetc/Howto.cookfi le causes themessages to be extracted intoi18n-tmp/*.poforchecking during the build.

16.3. Translators Welcome

If youare able to translate the error messages intoanother language, please contact Peter Miller<[email protected]> and he will tell youhow it is done. (Actually, he’ll point you to thispart of the User Guide. :-)

To translate the error messages, look up the two-letter abbreviation(http://www.w3.org/WAI/ER/IG/ert/iso639.htm)of the language you are going to translate theerror messages to. The rest of these instructionswill call it xx.

In the source tree, you will see a directory calledlib/en/LC_MESSAGESwhich contains some.pofi les. Theseare the text form of the message cata-logues. You can view them with a simple texteditor.

Create a new directory for your translations, andcopy the English messages into it.

mkdir lib/ xx/LC_MESSAGEScp lib/en/LC_MESSAGES/*.po \

lib/ xx/LC_MESSAGES

Now you need to edit each of thelib/ xx/LC_MESSAGES/*.po fi les, replacingthemsgstr strings with suitable translations.Leave themsgid strings and the commentsuntranslated. Theseare text files, you can editthem with a simple text editor. GNU Emacs has aPO mode to make this easier.

The GNU Gettext (http://www.gnu.org/direc-tory/gettext.html) sources have fairly good docu-mentation (http://www.gnu.org/manual/get-text/index.html) about this process.

If you want to test your translations, you need to"compile" the text into the binary form used bythegettext() system call. This is done usingthemsgfmt(1) program from the GNU Gettextpackage. To see your new translations in action,you create a/usr/local/lib/aegis/ xx/LC_MES-

Page 126 (bl/lib/en/user-guide/cI.0.so) Peter Miller

Aegis UserGuide

SAGESdirectory and arrange for themsgfmt(1)output to be placed in it. Some of the messagesare hard to trigger, don’t expect complete test cov-erage.

There are almost 600 error messages. If you aver-age 1 message every 2 minutes, this is approxi-mately 20 hours work. TheGerman translation,for example, required around 12 hours.

When you are done translating, email the resultsto Peter Miller <[email protected]> andthey will be included in the next release of Aegis.

Peter Miller (bl/lib/en/user-guide/main.ms) Page 127

User Guide Aegis

.

Page cxxviii (bl/lib/en/user-guide/main.ms) Peter Miller