Agent-based simulation of open source evolution

SOFTWARE PROCESS IMPROVEMENT AND PRACTICESoftw. Process Improve. Pract. 2006; 11: 423–434

Published online 28 June 2006 in Wiley InterScience(www.interscience.wiley.com) DOI: 10.1002/spip.280

Agent-based Simulationof Open Source Evolution

Research Section

Neil Smith1∗,† Andrea Capiluppi2 andJuan Fernandez-Ramil1

1 Computing Department, The Open University, Walton Hall, MiltonKeynes MK7 6AA, UK2 Department of Computing and Informatics, University of Lincoln,Brayford Pool, Lincoln LN6 7TS, UK

We present an agent-based simulation model developed to study how size, complexity andeffort relate to each other in the development of open source software (OSS). In the model, manydeveloper agents generate, extend, and re-factor code modules independently and in parallel.This accords with empirical observations of OSS development. To our knowledge, this is thefirst model of OSS evolution that includes the complexity of software modules as a limitingfactor in productivity, the fitness of the software to its requirements, and the motivation ofdevelopers.

Validation of the model was done by comparing the simulated results against four measuresof software evolution (system size, proportion of highly complex modules, level of complexitycontrol work, and distribution of changes) for four large OSS systems. The simulated resultsresembled the observed data, except for system size: three of the OSS systems showed alternatingpatterns of super-linear and sub-linear growth, while the simulations produced only super-linear growth. However, the fidelity of the model for the other measures suggests that developermotivation and the limiting effect of complexity on productivity have a significant effect onthe development of OSS systems and should be considered in any model of OSS development.Copyright 2006 John Wiley & Sons, Ltd.

KEY WORDS: simulation models; software process; open source software; software evolution; productivity; metrics; agent-basedsimulation

1. INTRODUCTION

The application of simulation models to softwareprocesses includes the support of decisions aboutresources and the impact of process improve-ments, such as the introduction of inspections.Simulation modelling can also help in evaluatingpossible explanations for empirical observations.

∗ Correspondence to: Neil Smith, Computing Department, TheOpen University, Walton Hall, Milton Keynes MK7 6AA, UK†E-mail: n.smith, @open.ac.uk

Copyright 2006 John Wiley & Sons, Ltd.

The vast majority of real world software applica-tions are evolved from existing versions, not createdfrom scratch, and a large amount of effort goesinto evolution, not into initial development (Som-merville 2001). There is now a body of knowledgeabout software evolution (e.g. Lehman and Belady1985, Rajlich and Bennett 2000, Aoyama 2002, Mad-havji et al. 2006) based on observations and on avariety of software process models (e.g. Lehmanet al. 2002, Smith et al. 2005). In spite of the advancesover the years, the management of software evo-lution is one of the most challenging softwareengineering problems. In order to make progress

Research Section N. Smith, A. Capiluppi and J. F. Ramil

in this area, we are interested in understandingthe drivers of software evolution, as a basis to thegeneration of theories of software evolution.

This is an important topic since softwareengineering lacks an empirically validated theory(Lehman 2000a). Even though this could be seen asa topic of interest to academics only, it can havea practical and useful output in terms of guid-ance and justification for good practice (Lehman2000b). Guidance to good practice is one of theroles of theory in other engineering disciplines. Forexample, one can use electromagnetic field the-ory to reduce electromagnetic interference betweennearby cables. In a similar way, one could considerBrooks’s law (Brooks 1995) that ‘adding people toa late project make it later’ to derive guidelinesfor managing a late project. Similarly, Lehman’sfirst law of software evolution, ‘Continual Change’(Lehman 1974), can justify the introduction of con-figuration and change management in the softwareprocess (Lehman 2000b). Owing to the humaninvolvement at all levels in the software process,we expect that the theories related to the softwareprocess will be more qualitative than the quantita-tive theories typical of physical systems. If softwareprocesses are subject to general influences that canbe modelled and captured in theories, then processsimulation modelling will play a role in evaluatingthe empirical support for them. The investigationof such theories should include not only the studyof empirical data in search of patterns but alsothe investigation of ways of abstracting and repre-senting any behavioural regularities found, and ofthe limits of such theories. For example, we wouldlike to know whether theories are able to explainand predict break points in otherwise apparentlyregular evolution.

This article reports our attempts to replicateempirical observations of size, complexity, re-factoring work, and distribution of changes ofa set of open source software (OSS) systems. Thefree availability of empirical data makes OSS anattractive topic for investigation. OSS developmentprojects are very modular: multiple developerswork in parallel on different parts of a softwaresystem with less influence from a formal planthan in proprietary software development. Thissuggests that an agent-based model of the softwaredevelopment process would be appropriate. Theuse of such a model was proposed some 10 yearsago (Lehman and Stenning 1996). However, to our

knowledge, the model presented here is the firstmodel of open source evolution that includes threesignificant factors: the complexity of the softwaremodules as a limiting factor in productivity, thefitness of the software to the requirements, and themotivation of developers. This is an advance over thework of other researchers (Madey et al. 2002, Robleset al. 2005). These OSS simulations have dealt withmainly two dimensions (system size and numberof developers) and how an increased number ofdevelopers produces more output (lines of codeand source files).

The structure of this article is as follows: Section 2provides the motivation for this research. Section 3introduces our agents-based model for simulatingan OSS environment, the assumptions on whichit is based, and the hypotheses of our research.Section 4 describes the empirical data and thesystem from which it is derived. Section 5 comparesthe simulation output with the empirical data.Section 6 describes related work in simulation ofOSS software evolution. Section 7 concludes thearticle and indicates topics for further work.

2. MOTIVATION AND BACKGROUND

Many simulations of software development arebased on the traditional approach to modelling com-plex systems. The system’s behaviour is abstractedinto a set of differential equations which define thestructure of the causal links between the state vari-ables in the model (Iwasaki and Simon 1986); thisdescribes the mechanisms that are believed to oper-ate in the referent system. The set of equations issolved to produce a behaviour, which is comparedto the empirical data. This approach has producedresults in the study of traditional software devel-opment (e.g. Abdel-Hamid and Madnick 1991) andhas produced predictions such as the diminishinggrowth rate implied by Lehman’s second law ofincreasing complexity (Lehman 1974) and observedin a number of proprietary software systems (Turski1996, FEAST 2001, Turski 2002). In our previouswork we explored the use of qualitative simulationto bridge the gap between high-level (qualitative)theories and empirical data (Ramil and Smith 2002,Smith et al. 2005).

Computing is a rapidly evolving discipline andthere is a need to compare the theories to newemergent forms of software development, such as

Copyright 2006 John Wiley & Sons, Ltd. Softw. Process Improve. Pract., 2006; 11: 423–434

424 DOI: 10.1002/spip

Research Section Simulation of Open Source Evolution

the OSS domain. This presents some challenges.Many OSS systems do not show the types ofevolutionary behaviour seen in the evolution oftraditional software (Godfrey and Tu 2001, Her-raiz 2005). About one-third of growing OSS systemsgrow at super-linear rates (seemingly contradictingLehman’s second and fifth laws) (Herraiz 2005).In OSS, the inclusion of more contributors leadsto more growth (also contradicting both Lehman’ssecond and fifth laws and Brooks’s law).

This different behaviour may be due to the dif-ferent architectural structure of OSS development:our hypothesis is that OSS development behavesdifferently because OSS systems are more modu-lar than proprietary software. We believe that thedistributed nature of OSS applies not only to thesoftware itself but also to the community evolv-ing it (also following Brooks’s observation that thearchitecture of a system reflects the structure ofthe organisation evolving it (Brooks 1995)). OSSevolution involves a community of individuals pro-viding their work mainly on a voluntary basis andwithout a strong centralised leadership (Raymond2001, Scacchi 2006). This invalidates one of theassumptions of our previous simulation models:the existence of a centralised management controlwhich will react against excessive complexity byassigning developers to complexity control work(Smith et al. 2005).

We propose that, while each module within anOSS system may be monolithic, and will behavein the manner described by Lehman, Belady, andBrooks, the overall modular and decentralisedarchitecture of the software and of the commu-nity evolving it will restrict the impact of softwaregrowth stagnation to small parts of the systemwhere they will not have a significant effect on theevolution of the whole. To investigate this hypoth-esis, we have developed an agent-based model(Rocha 2003) that captures the decentralised andmodular nature of OSS development. The nextsection describes the details of the model andSection 5 shows the simulation results generatedby it.

3. AGENT-BASED SIMULATION MODEL

Our simulation model is based on Lehman’s lawsof software evolution (Lehman 1974, Lehman andBelady 1985, Lehman 2000b), our own experience

observing OSS development, as well as on themodels developed by other researchers (Madeyet al. 2002, Antoniades et al. 2005, Robles et al.2005). However, to our knowledge, the modelpresented here is the first agent-based model ofopen source evolution that includes the complexityof the software modules as a limiting factor inproductivity, the fitness of the software to therequirements, and the motivation of developers.We believe that these are important factors thatneed to be studied.

Our motivation for developing this model lies inour understanding of the actions of individual OSSdevelopers (Mockus et al. 2002, Raymond 2001). OSSdevelopment is inherently decentralised and non-coercive: developers choose to become involvedin an OSS project and choose what aspects of theproject to work on. They generally have pride intheir work and take pains to ensure that it is easilymaintainable.

We used the NetLogo (2005) multi-agent sim-ulation tool to develop our model. We selectedNetLogo primarily because it is freely availableon the web and well documented and supported.In this tool, agents move around a virtual world,interacting with it and with other agents. Thereis no centralised control or co-ordination of theagents’ actions. Agents are responsible for main-taining their own state. The NetLogo virtual worldconsists of a grid of ‘patches’, each of which can havea state. Generally, agents have only local knowl-edge about their surroundings. Both agents andpatches are active agents in the simulation, per-forming actions and asking other agents to performother actions. Simulation proceeds by each agentand patch repeating its behaviour independently,often by following stochastic functions influencedby the agent’s state and local environment. Agentsperform their own actions asynchronously and asrapidly as they can. In an agent-based simula-tion, the overall behaviour of the system is anemergent property of the individual, independentinteractions of the agents. This approach differsfrom the traditional modelling approach wherethe state of the system is captured in a singleset of global state variables, such as stocks andflows.

In our model, patches represent modules ofsoftware source code and agents represent bothdevelopers and unfulfilled requirements. Figure 1shows a class diagram that illustrates the main


DOI: 10.1002/spip 425


Figure 1. UML diagram for the classes used in the NetLogo simulation engine

concepts of developers interacting in the system bycreating, modifying, and re-factoring modules. Thelevel of abstraction of the model is not determined apriori: a module could represent a single procedure,a file, a library, or some other modular part ofa software system. Each module records both itsfitness for purpose (gauged against a set of externalrequirements) and its complexity. The complexityof a module acts as an inhibitor to changes tothat module. To model the changes in externalrequirements and invalidation of assumptions thatare a driving force behind software evolution,patches have a stochastic process for modellingtheir decrease in fitness. Modules also have arandom chance to spawn a new requirement ina neighbouring, empty patch. Finally, moduleshave a chance to capture the attention of adeveloper passing through cyberspace, and so‘create’ a new developer agent in the model;this only happens if the module is interesting(i.e. its fitness is below the developer boredomthreshold; see below). In the model, each patchrepeatedly exhibits these behaviours, checking ifit should generate a new requirement, decreasein fitness, or capture the attention of a newdeveloper agent. These behaviours are shown inFigure 2.

In the model, software developers are representedby agents. These agents walk randomly aroundthe software system, changing the code as theygo. Agents have four behaviours, depending ontheir location. As with patches, each developerrepeatedly and asynchronously cycles throughthese behaviours, following the pseudo-code inFigure 2.

1. If a developer is on an unfulfilled require-ment, it creates a new module that fulfils thatrequirement, with a certain (low) fitness andcomplexity.

2. If a developer is on a module with highcomplexity and high fitness, it may attempt tore-factor that module. Re-factoring leaves themodule’s fitness unchanged, but reduces itscomplexity by a random amount.

3. If the developer is on a module that it is not re-factoring, it will attempt to develop the module;this increases both the module’s fitness andcomplexity by a random amount. It will alsoslightly reduce the fitness of adjacent modules,owing to the coupling between them. However,if the module is complex, the agent may not beable to improve the module, in which case themodule is left unchanged.

4. Finally, developers have a motivation factor(we call this boredom threshold in the model).If the fitness of the module they are on isabove this threshold, there is a chance that thedeveloper will find the project boring, and leave.Developers may also leave if they move outsidethe system.

Simulation starts with a single module. This bothspawns new requirements and attracts the attentionof developers. The developers will create modulesto fulfil the requirements and therefore enlarge theproject. As the project grows, more developers areattracted and more requirements are identified. Thecode of the full NetLogo model is freely availablefrom http://mcs.open.ac.uk/ns938/simulations/


426 DOI: 10.1002/spip


Code Patch[Reduce fitness][Create new developer here][Create a requirement on a neighbouring patch that has no code and no requirements]

DeveloperMove to a random neighbouring patchIf at a requirement Then

Remove the requirementCreate a new piece of code

Else If on a code patchIf [code is fit for function and complex] Then ; Refactor code

Preserve the code's fitness[Reduce the code's complexity]

Else ; Develop codeIf [code not too complex] Then

[Increase the code's fitness][Increase the code's complexity]

End IfEnd IfIf code fitness > boredom threshold Then

[Disappear][Disappear]

End IfEnd If

Figure 2. Pseudo-code of the model for the NetLogo engine. Terms in square brackets are stochastic operations: theresults of tests are random, though influenced by their parameters, and operations have a random effect

Table 1. Data sources availability for present study (N/A indicates that the source may be present, butits format and style did not permit its usage because essential fields were missing; ‘partial’ means thata CVS server is present, but it is not recording the complete evolution history of the OSS project, i.e.the project adopted a CVS at some point later)

Software system studied(URL of code repository)

ChangeLog CVS Number of releasesconsidered

Arla (www.stacken.kth.se/projekt/arla) Yes Yes 70Gaim (http://gaim.sourceforge.net) N/A Partial 100MPlayer (www.mplayerhq.hu) N/A 13. Partial 81Wine (www.winehq.com) Yes Partial 90

4. EMPIRICAL DATA

To validate the model, we compared the simulatedoutput to empirically observed behaviour. Theempirical data was derived from data in OSSrepositories. Previous research has shown that datasuch as change-log records, program headers, andconfiguration management offer a rich source toderive data for the study of software evolution(Capiluppi 2003, Fischer et al. 2003, Capiluppi et al.2004a,b, Mens et al. 2004, RELEASE 2005). For thisstudy, we selected four OSS systems which we

have examined in previous studies (Capiluppi et al.2004a,b). Table 1 indicates the data sources we usedto extract the empirical data used in this research.

We extracted several attributes for each softwaresystem, taking measurements over releases forsize in number of files, files handled (Lehmanand Belady 1985), average complexity (measuredusing the McCabe index (McCabe 1976)), andthe level of complexity control work (measuredas the proportion of files/functions that weresubject to a decrease in their complexity in twocontiguous releases). The data collection was aimed


DOI: 10.1002/spip 427


at measuring the systems’ size, complexity, amountof anti-regressive work, and distribution of touches:the purpose of this work is to characterise anOSS environment in order to compare it againstthe simulated results obtained from the modeldescribed above.

• Measurement of size of the system: The size wasevaluated using the number of source functions(as a surrogate for the systems’ growth). Whenrelease data was available, the code base and itssize at each release were evaluated using ourtools. We believe that an approach based onmeasuring the size of the released source codemore genuinely displays the overall evolutionof the system than measuring the size of thewhole repository. Releases contain those partsof code that were chosen to be included inwhat is normally called a stable configuration.We prefer releases for the study of size ofcode contained onto the whole CVS server,because they represent stable points. The right-most column in Table 1 indicates the numberof releases studied. The evaluation of the sizeachieved is displayed in a joint visualisationin Figure 3(a) for all the projects: the x-axisindicates the time of each release, while they-axis indicates the size achieved. Measuresare relative to the maximum values of bothmetrics, in order to allow the comparison ofsystems with largely different values in sizeachieved. As can be seen, only one case (Wine)exhibited a monotonically increasing trend;in the other systems at least one period ofstagnation was observed between two periodsof growth. Figure 3(b) shows one of the caseswith a discontinuous pattern of growth (Gaimsystem).

• Measurement of complexity: We looked at com-plexity at the level of granularity of functions.Within this level we consider the McCabe cyclo-matic number as a measure of complexity(McCabe 1976). Next, we define the highly com-plex subpart of the system as the set of highlycomplex functions. We use the accepted thresh-old value for the McCabe index of 15 in order todistinguish between less complex functions andmore complex ones (McCabe and Butler 1989).One of the proposed systems had already beenevaluated (Capiluppi et al. 2005) for the pur-pose of tracking the amount of highly complex

1(a) Relative growth of systems - joint view

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0

0.000 0.100 0.200 0.300 0.400 0.500 0.600 0.700 0.800

time (relative)

0.900 1.000

size

(re

lativ

e)

5500(b) Gaim - growth of system in number of functions

5000

4500

4000

3500

3000

2500

2000

1500

1000

500

0

03/99 12/99 08/00 04/01 12/01 09/02 05/03 01/04 09/04 05/05 02/06

time

func

tions

arla gaim mplayer wine

Figure 3. (a) Evolutionary pattern of size growth basedon a relative perspective (all the systems studied).(b) Evolutionary pattern of the growth of number offunctions for one system (Gaim)

functions: a similar approach has been used forthe other systems in this case study. This isshown through a boxplot (Box 1978) visualisa-tion, which displays the variation of the dataset(i.e. the percentage of the highly complex ele-ments in a specific system) along the numberof its releases. Figure 4(a) shows that, in all theanalysed systems, the highly complex subpartis never larger than 10% of the overall system.Figure 4(b) shows the overall trend of the highlycomplex functions for two systems: Gaim andMplayer.

• Measurement of the level of complexity controlwork: An adequate level of complexity control(also termed anti-regressive work (Lehman1974, Lehman and Belady 1985)) is widelyrecognised as an essential countermeasure tosoftware ageing (Parnas 1994) and to sustain theevolution of software (Ratzinger and Gall 2005).We measured the level of complexity controlwork by comparing every function betweentwo consecutive releases and by counting how


428 DOI: 10.1002/spip


0.0900(b)

(a)

0.0800

0.0700

0.0600

0.0500

0.0400

0.0300

0.0100

0.0000

0.0200num

ber

of fu

nctio

ns [%

]

Gaim MPlayer releases

1 5 9 13 18 23 28 33 43 48 53 58 63 68 73 78 83 88 93 9838

highly complex functions - Gaim and MPlayer

0.0814

0.0532

0.025Aria Gaim MPlayer Wine

Figure 4. (a) Boxplots showing the amount (as a percent-age of the overall size in functions) of highly complexfunctions in the studied systems. As seen, there is asmall variation along the releases for most of the softwaresystems. (b) Observed trends of the amount of highlycomplex functions in the releases of two systems (Gaimand MPlayer, right)

many of them experienced a reduction in theircyclomatic complexity. From a quantitativeanalysis, our results illustrate that there is acorrelation between the trend of the size growthand the amount of complexity control work:Figure 5(a–d) shows the trends for the studiedsystems.

• Distribution of changes: We term the number ofdifferent releases during which a file has beenmanipulated, via addition, removal, or modi-fication of code, as the release touches of thatfile. A single release touch indicates that the filehas never been modified after its creation. Themaximum possible number of release touchesfor a file is the number of releases. The quan-titative observation we drew from the chosencase studies, when analysing the distribution

4500

(a)

4000

3500

3000

2500

2000

1500

1000

500

0

900

800

700

600

500

400

300

200

100

0

Arla - growth of size and complexity control work

Nr. of functions Complexity control releases

11000

(c)

10000900080007000600050004000300020001000

0

1800

1600

1400

1200

1000

800

600

400

200

0

Mplayer - growth of size and complexity control

Nr. of functions Refactoring releases

5500(b)

45004000

5000

350030002500200015001000

5000

1500140013001200110010009008007006005004003002001000

Gaim - growth of size and complexity control

Nr. of functions Nr. of function releases

37500(d)

3500032500300002750025000225002000017500150001250010000

750050002500

0

1200011000100009000800070006000500040003000200010000

Functions Complexity control releases

Wine - growth of size and complexity control

Figure 5. Evolutionary pattern of size growth as com-pared to complexity control work for the followingsystems (Arla) (a), (Gaim) (b), (MPlayer) (c) and (Wine) (d)


DOI: 10.1002/spip 429


80

70

60

50

40

30

20

10

0element ID

Releases touches - Gaim

touc

hes

Figure 6. Distribution of release touches for one of thechosen systems (Gaim)

of release touches, resembled those alreadyachieved (Capiluppi et al. 2005): a small subsetof elements (files or functions/methods/classes)is touched a large number of times by devel-opers, while most of the elements receive few(if not none) touches (Figure 6). This behaviourcan be summarised by visualising, per eachsystem, its release-touches profile, or by giv-ing the skewness of the distribution (Table 2).The higher the skewness, the more asymmet-ric the distribution of the release touches in asystem.

5. RESULTS AND VALIDATION

We used the empirical data described above tocalibrate and validate our model. We did this byexploring the parameter space of the models, look-ing at the generated output, and comparing it to theempirical evidence from the four case studies. Thecomparison was done by examining four groups ofattributes: growth, complexity, complexity controlwork, and distribution of touches.

We exhaustively explored the parameter space ofthe model by determining a range of possible values

Table 2. Skewness factor for the distribution of all the systems(right). Highlighted in the table is the row corresponding to thesystem displayed in figure 6

System Skewness

Arla 4.55Gaim 3.33MPlayer 3.36Wine 2.73

for each of the nine controlling parameters in themodel. This resulted in 256 distinct combinationsof parameter values. The model was run for eachof these parameter combinations and the resultslogged.

The model seemed most sensitive to the value ofthe boredom threshold parameter, which controlswhen new developers join and leave the project (ifa module’s fitness is below the boredom threshold,it may attract new developers; if the fitness is abovethe boredom threshold, developers on that modulemay leave the project). If the boredom thresholdof developers was set very high, developers didnot leave the project when they encountered high-fitness modules. As these modules continued toattract new developers, the number of developersgrew extremely rapidly and soon swamped thedevelopment environment. In contrast, if the bore-dom threshold was very low, the evolution of thefirst few modules resulted in a system that wassufficiently fit for purpose that no new develop-ers were attracted; the original developers soonleft the project and were not replaced. Neither ofthese behaviours is at all similar to the empiricaldata, so the results from these simulation runs werediscarded.

The simulation results were also strongly affectedby the effectiveness of the re-factoring work doneby developers. When re-factoring was ineffectualor not attempted, both the average complexity ofthe system and the proportion of the highly com-plex subpart grew over time. This in turn led toan unusual behaviour for the average fitness of thesystem: it peaked in the early stages of developmentand gradually declined thereafter. This was due tothe large number of ‘old’ modules that had sufferedbitrot (so they became unfit for purpose) but couldnot be improved because of their complexity. Thisalso led to the anomalous situation of these modulesattracting many functionally ineffectual developers.

The rest of the runs yielded very similarbehaviours (up to linear scalings of the variousresults): the system size grew super-linearly, theproportion of complex modules remained constantand low, the portion of work assigned to complexitycontrol increased over time, and the distribution oftouches was very skewed.

• Growth patterns: Figure 3 shows the empiricalgrowth patterns. Figure 7(a) shows that thesimulation model is able to reproduce the


430 DOI: 10.1002/spip


550500450400350300250200150100500

sim

ulat

ed n

r of

ele

men

ts

simulated growth

simulated time

450400350300250200150100500

100%90%80%70%60%50%40%30%20%10%0%

elem

ents

simulated high complex subset of elements

elements highly complex simulated time

(a)

(b)

Figure 7. (a) Simulated outcome for the growth of size.(b) Simulated outcome for the high complexity elements:the primary y-axis shows the amount of global elementsin a system, while the secondary y-axis shows the highcomplex percentage of the same set

continuous growth pattern seen in Wine. Thesecond pattern (discontinuous growth) is notdirectly reproduced by the model, but it can begenerated by varying the controlling parametersduring simulation, such as adjusting the chanceof new developers appearing (thus changingthe number of developers) or changing thethreshold that determines when a moduleadequately meets its requirements (similar tothe idea of S-type programs defined by Lehmanand Belady (1985) ). In Section 7, we highlightan extension of the model for capturing thediscontinuous trends, based on the fulfilment ofthe initial requirements.

• Complexity patterns: As shown in Figure 4, allthe four systems studied present a similar trendwhen analysing the percentage of highly com-plex functions. The simulated data is similarto them (Figure 7(b)), as long as new moduleshave an initial complexity below the report-ing threshold. Both complex and non-complex

modules attract developers who perform somere-factoring work and hence keep the averagecomplexity constant. This is very similar to theempirical observations.

• Complexity control: In the four systems stud-ied the empirical observations show that thecumulative amount of complexity control (alsotermed re-factoring) work, i.e. the work aimedat decreasing the complexity of a software sys-tem, increases slowly but then closely follows thegrowth trend. That is, the complexity control ratemeets and even exceeds the functional growthrate. As visible in Figure 8(a), the simulationmodel is able to reproduce this pattern. Re-factoring work is taking an increasing amountof work, but this is sustained because in OSSthe effort comes from an unbounded pool ofdevelopers.

• Release touches: As shown in the empiricalobservations (Figure 6), the studied softwaresystems show a long-tailed distribution ofrelease touches. The model is able to simulatethis aspect. Albeit a qualitative resemblanceis obtained, the skewness factor is not very

100.00%90.00%80.00%70.00%60.00%50.00%40.00%30.00%20.00%10.00%

0.00%

45

5560

50

40353025201510

50

SizeRefactoring

Simulated refactoring work vs development

sim

ulat

ed to

uche

s

simulated release touches

simulated time

(a)

(b)

Figure 8. (a) Simulated outcome for the amount of com-plexity reduction work. (b) Release-touches-simulatedoutcome


DOI: 10.1002/spip 431


large, as we observed a value of 0.81 for thesimulated output. The highly skewed profile isreproduced by our simulation engine, as visiblein Figure 8(b).

6. RELATED WORK

The study of evolutionary trends in software evo-lution has been the focus of several quantitativesimulation models using system dynamics (e.g.Lehman et al. 2002). This modelling effort has beeninspired by observations of proprietary software.The OSS domain was originally studied using directvisualisation of trends based on quantitative metricdata extracted from OSS system (Capiluppi et al.2004a,b). A important study by Godfrey and Tu(2001) highlighted differences between the evolu-tion of Linux (a popular OSS operating system)and previously studied systems, particularly itsapparently super-linear growth. Our model pro-vides a possible explanation for such a super-lineargrowth.

Research efforts specifically involving simulationof some aspects of OSS development and evolutionhave taken place and this area of research isbecoming very popular, driven by the availability ofOSS data. Examples include Madey et al. (2002), whoused the existing SWARM agent-based simulationtools, and Dalle and David (2004), who builttheir own agent platform, SimCode. A dynamicsimulation of OSS processes is also described byAntoniades et al. (2005), where empirically observedpatterns of size growth and developers joining theproject are reproduced by a simulation. Robles et al.(2005) propose a biologically inspired simulation,where developers learn from other developersthrough observing changes in the source code,rather than explicitly communicating with eachother. This type of model has been used toinvestigate questions related to the amount of effortallocated to OSS projects and whether a significantattraction of new developers can be achieved inthe evolution of the project. Their research sharesour focus on product characteristics (e.g. size andcomplexity) and on evolution. However, to ourknowledge, the model presented here is the firstmodel of open source evolution that includes thecomplexity of the software modules as a limitingfactor in productivity, the fitness of the software tothe requirements, and the motivation of developers.

7. CONCLUSIONS AND FURTHER WORK

This article presented an agent-based simulationmodel of OSS evolution. Our model, while simple,incorporates many of the features that may explainthe differences between OSS and proprietary devel-opment (Godfrey and Tu 2001, Herraiz 2005). Wefound that the model was able to replicate theobserved patterns in three of the four areas exam-ined (complexity, complexity control, distributionof changes) in the four systems studied. In one area,system growth, the model was only able to repli-cate the continuous super-linear growth patternseen in one of the four systems studied. Discon-tinuous behaviour can be artificially generated bychanging parameters during a simulation. How-ever, this is not satisfying as an explanation andmore work is needed. Having said that, the modelpresented here appears to provide an explanationfor the unbounded growth trends (Godfrey and Tu2001, Herraiz 2005) observed in some OSS software.By itself, this is an important contribution.

We included three novel factors in our model:the complexity of software modules as a limitingfactor in productivity, the fitness of the softwareto its requirements, and the motivation of devel-opers. All three of these factors are required forthe model to produce plausible results. If the fit-ness of modules and the interests of developersare misaligned, the model quickly deviates fromempirical observations. If the modules created areperfectly fit for purpose, developers have little todo and leave the project, resulting in a moribundproject. If the developers never leave the project,no matter how fit for purpose existing modules are,the number of developers grows excessively large.The ‘desire’ of developers in the model to re-factorcomplex code, which reduces the complexity of thatmodule and hence the system overall, leads to thehighly complex subpart of the model to remain ata constant and low level, in accordance with theempirical observations. Experiments where this re-factoring behaviour was prevented led to situationswhere the highly complex subpart grew much fasterthan the overall size of the system and the averagefitness of the system decreased over time. This indi-cates that re-factoring is a significant activity in OSSdevelopment, which allows the system to remain fitfor purpose and able to continuously grow. Theseresults indicate that the novel factors introduced in


432 DOI: 10.1002/spip


this model are required for faithful simulations ofOSS evolution.

The work reported in this article, particularlythe problem in simulating discontinuous trends,has lead to the view that a better model may bedeveloped by modelling the actions of core devel-opers. Observations of some OSS projects indicatethat they are led by core developers who performmost of the changes in the software while oth-ers make a much smaller number of contributions(Mockus et al. 2002). This suggests that we need toinclude mechanisms in the model that reflect therole of core developers in influencing the evolu-tion of some OSS systems. An alternative way ofexplaining the discontinuous trends of size growthcould be to make the distribution of requirementsless even with clusters of requirements representingnew functional areas for the project to incorporate.This would generate rapid growth while a new areais explored, followed by slow growth when thoserequirements are fulfilled.

ACKNOWLEDGEMENTS

Andrea Capiluppi acknowledges the Faculty ofMaths and Computing, The Open University, andin particular Drs Bashar Nuseibeh and Uwe Grimm,for financial support that made this work possible.Juan Fernandez-Ramil gratefully acknowledges theUK EPSRC for funding.

REFERENCES

Abdel-Hamid TK, Madnick SE. 1991. Software ProjectDynamics–An Integrated Approach. Prentice Hall:Englewood Cliffs, NJ, 263.

Antoniades P, Samoladas I, Stamelos I, Bleris GL. 2005.Dynamical simulation models of the open sourcedevelopment process. In Free/Open Source SoftwareDevelopment, Koch Stefan (ed.). Idea Group Publishing,Inc. Vienna, 2005, 328 pp.

Aoyama M. 2002. Metrics and analysis of softwarearchitecture evolution with discontinuity. In Proceedingsof the 5th International Workshop on Principles of SoftwareEvolution, IWPSE 2002, Orlando, FL, 103–107.

Box GEP, Hunter WG, Hunter JS, Hunter WG. 1978.Statistics for Experimenters. An Introduction to Design,Data Analysis and Model Building. Wiley InterScience, 1978,672 pp.

Brooks F. 1995. The Mythical Man-Month: Essays onSoftware Engineering. 20th Anniversary edn. Addison-Wesley, New York, NY, 1995, 322 pp.

Capiluppi A. 2003. Models for the evolution of OSprojects. In Proceedings of the ICSM, Amsterdam, 22–26September. 2003, pp 65–74.

Capiluppi A, Morisio M, Ramil JF. 2004a. Structuralevolution of an open source system: A case study.In Proceedings of the 12th International Workshop onprogram Comprehension(IWPC), Bari, Italy, June 24–26,2004, pp 172–182.

Capiluppi A, Morisio M, Ramil JF. 2004b. The evolutionof source folder structure in actively evolved opensource systems. In Proceedings of the 10th InternationalSymposium on Software Metrics, Chicago, IL, September11–17, pp 2–13.

Capiluppi A, Faria AE, Ramil JF. 2005. Exploring therelationship between cumulative change and complexityin an open source systems. In 9th European Conferenceon Software Maintenance and Reengineering (CSMR),Manchester, England, March 21–23, 2005.

Dalle JM, David PA. 2004. SimCode: Agent-based simu-lation modelling of open-source software development,Available online at http://opensource.mit.edu/papers/dalledavid2.pdf <as of Feb. 2005>.

FEAST. 2001. Feedback, Evolution And Software Technology.Department of Computing, Imperial College: London,England, http://www.doc.ic.ac.uk/∼mml/feast/ <as ofOct. 2005>.

Godfrey M, Tu Q. 2001. Growth, evolution and structuralchange in open source software. In Proceedings of the4th International Workshop on the Principles of SoftwareEvolution, Vienna, Austria, September. 10–11 2001.

Fischer M, Pinzger M, Gall H. 2003. Populating a releasehistory database from version control and bug trackingsystems. In Proceedings of the ICSM 2003, Amsterdam,22–26 September 23–32.

Herraiz I. 2005. The Evolution of Large Open Source Software.Computing Department, The Open University: MiltonKeynes, England. Research Seminar, 28 July 2005.

Iwasaki Y, Simon HA. 1986. Causality in devicebehaviour. Artificial Intelligence 26: 3–32.

Lehman MM. 1974. Programs, cities, students, limits togrowth?’’ Lecture, May 1974. In Imperial College of Scienceand Technology Inaugural Lecture Series, London Vol. 9,pp 211–229.

Lehman MM. 2000a. Approach to a theory ofsoftware process and software evolution, FEAST2000 Workshop. Imperial College: London, England,


DOI: 10.1002/spip 433


10–12 Jul. 2000, also as Research Report 2000/2,Department of Computing, Imperial College, Feb. 2000http://www.doc.ic.ac.uk/∼mml/feast2/papers/pdf/645.pdf <as of October 2005>.

Lehman MM. 2000b. Rules and tools for softwareevolution planning and management. In FEAST 2000Workshop. Imperial College: London, England, 10–12Jul. 2000, also as Technical Report, 2000/14 Nov.2000, Imperial College, Department of Computing,http://www.doc.ic.ac.uk/∼mml/feast2/papers/pdf/611 2.pdf <as of October 2005>.

Lehman MM, Stenning V. 1996. FEAST/1 Proposal Part2, Case for Support’’, EPSRC Proposal, ComputingDepartment. Imperial College London: London, Eng-land, http://www.doc.ic.ac.uk/∼mml/feast2/case96-2.html <as of Oct 2005>.

Lehman MM, Belady L. 1985. Program Evolution–Processesof Software Change. Academic Press: London, England,Available from links at: http://w3.umh.ac.be/evol/publications.html <as of October 2005>.

Lehman MM, Kahen G, Ramil JF. 2002. Behaviouralmodelling of long lived evolution processes–some issuesand an example. Journal of Software Maintenance andEvolution 14: 335–351.

Madhavji NH, Fernandez-Ramil J, Perry DE, (eds), 2006.Software Evolution and Feedback: Theory and Practice, Wiley,Chichester, UK.

Madey GR, Freeh VW, Tynan RO. 2002. Agent-basedmodeling of open source using swarm. In Proceedingsof the Americas Conference on Information Systems (AMCIS2002), Dallas, TX, August.

McCabe TJ. 1976. A complexity measure, 1976. IEEETransactions on Software Engineering SE-2: 308–320.

McCabe TJ, Butler CW. 1989. Design complexitymeasurement and testing. Communications of the ACM32(12): 1415–1425.

Mens T, Ramil JF, Godfrey M. 2004. Analyzing theevolution of large-scale software: Guest editorial. Journalon Software Maintenance and Evolution 16(6): 363–365.

Mockus A, Fielding RT, Herbsleb J. 2002. Two casestudies of open source software development: Aacheand Mozilla. ACM Transactions Software Engineering andMethodology 11(3): 309–346.

NetLogo. 2005. http://ccl.northwestern.edu/netlogo/<as of Oct 2005>.

Parnas DL. 1994. Software Aging. Proc. ICSE 16, May16–21, 279–287, Sorrento, Italy.

Rajlich VT, Bennett KH. 2000. A staged model for thesoftware life cycle. IEEE Computer 33(7): 66–71.

Ramil JF, Smith N. 2002. Qualitative simulation ofmodels of software evolution. Journal of Software Process:Improvement and Practice 7: 95–112.

Ratzinger J, Fischer M, Gall H. 2005. ImprovingEvolvability through Refactoring, Proc. InternationalWorkshop on Mining Software Repositories (MSR ’05),St. Louis, Missouri, May 2005, 69–73.

Raymond ES. 2001. The Cathedral and the Bazaar. O’ReillyMedia Inc, Sebastopol, CA, USA.

RELEASE. 2005. Research links to explore and advancesoftware evolution, http://labmol.di.fc.ul.pt/projects/release/ <as of Oct. 2005>.

Robles G, Merelo JJ, Gonzalez-Barahona JM. 2005. Self-organized development in libre software: a model basedon the stigmergy concept. In ProSim 2005, St Louis, MO,May 21–23 2005.

Rocha LM. 1999. Complex Systems Modeling: UsingMetaphors from Nature in Simulation and Sci-entific Models, Los Alamos National Laboratory:http://informatics.indiana.edu/rocha/complex/csm.html <as of Oct 2005>. November 1999.

Scacchi W. 2006. Understanding Open – Source SoftwareEvolution, in Software Evolution and Feedback: Theory andPractice, Wiley, Chichester, UK, pp. 181–205.

Smith N, Capiluppi A, Ramil JF. 2005. A study ofopen source software evolution data using qualitativesimulation. Software Process Improvement and Practice 10:287–300.

Sommerville I. 2001. Software Engineering, 6th edn.Addison-Wesley: Wokingham, England.

Turski WM. 1996. A reference model for the smoothgrowth of software systems. IEEE Transactions SoftwareEngineering 22(8): 599–600.

Turski WM. 2002. The reference model for smooth growthof software systems revisited. IEEE Transactions SoftwareEngineering 28(8): 814–815.


434 DOI: 10.1002/spip

Documents

Agent-based simulation of open source evolution