University of Waterloo

How does your software grow?Evolution and architectural change in open source software

Michael Godfrey

Software Architecture Group (SWAG)

University of Waterloo

Michael W. Godfrey ASERC workshop August 2001 How does your software grow? 2

What is software evolution?

“Evolution is what happens while you’re busy making other plans.”

• We distinguish between maintenance and evolution:– Maintenance is the planned set of tasks to effect changes.

– Evolution is what actually happens to the software.

• All I want to know is:How and why does software evolve?

Michael W. Godfrey ASERC workshop August 2001 How does your software grow? 4

Lehman’s examples

Michael W. Godfrey ASERC workshop August 2001 How does your software grow? 5

Growth of # of source files

0

1000

2000

3000

4000

5000

6000

Jan 1993 Jun 1994 Oct 1995 Mar 1997 Jul 1998 Dec 1999 Apr 2001

# o

f so

urc

e co

de

file

s (*

.[ch

] )

Development releases (1.1, 1.3, 2.1, 2.3)

Stable releases (1.0, 1.2, 2.0, 2.2)

Michael W. Godfrey ASERC workshop August 2001 How does your software grow? 8

Growth of Lines of Code (LOC)

0

500,000

1,000,000

1,500,000

2,000,000

2,500,000

Jan 1993 Jun 1994 Oct 1995 Mar 1997 Jul 1998 Dec 1999 Apr 2001

To

tal

LO

C

Total LOC ("wc -l") -- development releases

Total LOC ("wc -l") -- stable releases

Total LOC uncommented -- development releases

Total LOC uncommented -- stable releases

y = .21*x2 + 252*x + 90,055 r2=.997

Michael W. Godfrey ASERC workshop August 2001 How does your software grow? 10

Average/median .h file size

0

20

40

60

80

100

120

140

Jan 1993 Jun 1994 Oct 1995 Mar 1997 Jul 1998 Dec 1999 Apr 2001

Un

com

men

ted

LO

C

Average .h file size -- dev. releasesAverage .h file size -- stable releasesMedian .h file size -- dev. releasesMedian .h file size -- stable releases

Michael W. Godfrey ASERC workshop August 2001 How does your software grow? 12

SS LOC as percentage of total system

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

Jan 1993 Jun 1994 Oct 1995 Mar 1997 Jul 1998 Dec 1999 Apr 2001

Per

cen

tag

e o

f to

tal

sys

tem

un

co

mm

ente

d L

OC

driversarchincludenetfskernelmmipclibinit

Michael W. Godfrey ASERC workshop August 2001 How does your software grow? 13

SS LOC as percentage of total system (ignoring drivers)

0.0

5.0

10.0

15.0

20.0

25.0

30.0

Jan 1993 Jun 1994 Oct 1995 Mar 1997 Jul 1998 Dec 1999 Apr 2001

Per

cen

tag

e o

f to

tal

syst

em u

nco

mm

ente

d L

OC

archincludenetfskernelmmipclibinit

Michael W. Godfrey ASERC workshop August 2001 How does your software grow? 18

Growth of pine

0

50

100

150

200

250

300

350

Jan-93 Jun-94 Oct-95 Mar-97 Jul-98 Dec-99 Apr-01

# o

f M

od

ule

s

Michael W. Godfrey ASERC workshop August 2001 How does your software grow? 20

Growth of gcc/g++/egcs

0

100

200

300

400

500

600

700

800

900

1000

Aug-87 Dec-88 May-90 Sep-91 Jan-93 Jun-94 Oct-95 Mar-97 Jul-98 Dec-99 Apr-01

# o

f m

od

ule

s g++

gcc

egcs

Michael W. Godfrey ASERC workshop August 2001 How does your software grow? 21

Growth of vim (text editor)

0

20,000

40,000

60,000

80,000

100,000

120,000

140,000

160,000

May 1990 Sep 1991 Jan 1993 Jun 1994 Oct 1995 Mar 1997 Jul 1998 Dec 1999 Apr 2001

To

tal

LO

C

Total LOC ("wc -l")

Total LOC (ignoring comments and blank lines)

Michael W. Godfrey ASERC workshop August 2001 How does your software grow? 23

vim avg/median file size

0

100

200

300

400

500

600

700

800

900

1000

May 1990 Sep 1991 Jan 1993 Jun 1994 Oct 1995 Mar 1997 Jul 1998 Dec 1999 Apr 2001

Un

com

men

ted

LO

C

Average uncommented LOC per source fileMedian uncommented LOC per source file

Michael W. Godfrey ASERC workshop August 2001 How does your software grow? 24

vim’s architecture

Michael W. Godfrey ASERC workshop August 2001 How does your software grow? 25

Change patterns and evolutionary narratives• Phenomena observed in Linux evolution

– Bandwagon effect

– Contributed third party code

– “Mostly parallel” enables sustained growth

– Clone and hack

– Careful control of core code; more flexibility on contributed drivers, experimental features

Michael W. Godfrey ASERC workshop August 2001 How does your software grow? 26

Change patterns and evolutionary narratives• “Band-aid evolution” (just add a layer)

– quick way to add new functionality, esp. if system is not well understoode.g., Y2K fixing, adding portability, new features

• “Vestigial features”– design artifact persists after rationale dies

e.g., whale fin bone structure resembles hand

• “Adaptive radiation” [Lehman]

– when conditions permit, encourage wild variation for a while. – later, evaluate and let “best” ideas live on.

e.g., Linux kernel evolution

• “Convergent evolution”– compare similar systems to reference arch. (or to each other)

e.g., everyone grows an XML generator in response to market pressure

Michael W. Godfrey ASERC workshop August 2001 How does your software grow? 27

Open questions

• What are the recurring patterns and compelling metaphors of software evolution?

• Does software evolve in the same way as the natural world?– The Nature of Economies, by Jane Jacobs

• How to measure size?– How to correlate size and quality?

• How to measure change?– How to model architectural change?

• What is the predictive power of such models?– Do the “other phenomena” dominate?