Classical Open Source Software Process Model

1

Classical Open Source Software Process Model

Alan Kelon Oliveira de Moraes [email protected]

Recife, November 27, 2006

2

3

Open-source software is not a new idea but only recently have technical and

market forces converged to draw it out of a niche role.

Eric S. Raymond

The Cathedral & the Bazaar, 1999

4

open source companies had raised

~$900 million from venture capitalists since 1999

[Rivilin 2005]

5

Summary

What is open source and why should you care about it?

Classical open source process modelRemarksReferences

6

Definitions and Motivations

What and why?

7

What is Open Source?

Open Source software is the process of systematically harnessing open development and decentralized peer review to lower costs and improve software quality

[Raymond 1999]

8

Basic idea behind open source

When programmers can read, redistribute, and modify the source code for a piece of software, the software evolves.

People improve it, people adapt it, people fix bugs.

[Open Source Initiative]

http://www.opensource.org

9

Open source license definition

Open source means more than source code availability: The source must be available for redistribution

without restriction and without charge The license must permit the creation of

modifications and derivative works Must allow those derivatives to be redistributed

under the same terms as the original work.

[OSI] [Perens 1999] [O’Reilly 1999]

10

How is open source different?

The term open source generally refers to software that differs from its commercially sponsored counterpart in three ways source code’s ready availability to prepare

derivative works collaborative production approach community governance of production

[West 2005]

11

It is still evolving...

The term “open source project” does not have a clearly defined stable and shared meaning comparable to “open source license”

The processes evaluated refer to “classical” open source projects

[O’Mahony & West 2005]

12

Why should you care?

[O'Reilly 1999]

It’s important not to reduce the open-source debate to a question of NT versus Linux, or Microsoft versus the rest of the world

Understand how the lessons of open source can be applied to software development across the board.

The open-source process reflects a powerful global trend toward networked collaboration. Printing press: spread of knowledge in Renaissance Open Source: large scale cooperative development

efforts today.

13

Open wallets for open source softwareIn 1999 and 2000, according to

VentureOne, venture capitalists invested $714 million in 71 open-source companies Turbolinux raised $95 million Linuxcare raised at least $80 million Most of those projects collapsed…

[Rivilin 2005]

14[Rivilin 2005]

Open wallets for open source software

Venture capitalists are embracing open-source technology companies again $149 millions to 20 business in 2004 At least three open-source start-ups raised $20 million

in April 2005

The big difference is the increased adoption of open-source software by corporate users today Red Hat had $125 million in revenue in 2004 and now

has a market capitalization around $2 billion.

15

New business models

JBoss's business model, built on selling support services, made sense [Rivilin 2005]

Open Source has effect on commoditization, system architecture and network effect, and the development practices associated with software as a service [O'Reilly 2005]

16

Classical Open Source Process Model

A brief survey

17

Hey, wait! What is a software process?

A particular method of doing something, generally involving a number of steps or operations [Webster dictionary]

The software process consists of the activities and associated information that are required to develop a software system [Sommerville 1996]

A process defines specifically who does what, when, and how [Fayad 1997]

Software process is a set of activities, methods, practices, and transformations which people use to develop and maintain software and the associated products [Paulk et al. 1993]

18

And what is a software process model?

While a process is a vehicle for doing a job, a process description is a specification of how the job is to be done [Osterweil 1987]

19

Abstract generic process models

Specification-based models Waterfall Incremental

Evolutionary development models Prototyping Spiral

[Sommerville 1996]

20

The Cathedral and the Bazaar

Development style of Linux Release early and often, and listen to your

customers Delegate everything you can Be open to the point of promiscuity Present a plausible promise

[Raymond 1999]

21

Brooks vs. Linus

Brooks’ Law: ‘‘Adding more programmers to a late project makes it later.’’

Linus’ Law: “Given enough eyeballs, all bugs are shallow”

[Raymond 1999]

22

A case study of open source software development: the Apache server

Apache Server Process: decision making-oriented process Emphasized decentralized workspaces and

asynchronous communication Email lists exclusively to communicate with

each other, and a minimal quorum voting system for resolving conflicts

[Mockus et al. 2000]

23


There is no explicit system-level design, or even detailed design

There is no project plan, schedule or list of deliverables In order to keep track of the project status, an agenda

file is stored in each product's repository, containing a list of high priority problems, open issues, and release plans.


24


There is no single development process, but these are developer's common actions: Discovering that a problem exists Determining whether a volunteer will work on it Identifying a solution Developing and testing the code within their local copy

of the source Presenting the code changes to the group for review Committing the code and documentation to the

repository.


25



Quantitative analysis based on Developer mailing lists, CVS commit logs and Problem reporting database From may February 1995 to May 1999

26


Almost 400 individuals contributing code The top 15 developers contributed more than

83% of the MRs and deltas, 88% of added lines and 91% of deleted lines.


27


Around 3060 different people submitted 3975 problem reports. The top 15 problem reporters (only 3 also in the

core) submitted only 213 or 5% of Prs Almost 2600 developers submitted one report,

306 submitted two, 85 submitted three The maximum number submitted by one person

was 32.


28


Time to resolve problem reports 50% of PRs are resolved within a day 75% within 42 days 90% within 140 days


29

Descriptive Process Model for Open Source Software Development

Three basic views for software process models [Humphrey 1989] The state view The organizational view The control view

[Johnson 2001]

30


State view Closed prototyping Iterative and incremental enhancement Concurrent development Large-scale peer review User-driven requirements

[Johnson 2001]

31

Descriptive Process Model forOpen Source Software Development

Organizational view Decentralized collaboration Trusted leadership Internal motivation Asynchronous communication

[Johnson 2001]

32


Control view Informal planning Tiered participation Modular design Ubiquitous tool support Shared information space

[Johnson 2001]

33

An overview of the Software Engineering Process in the Mozilla project

The importance of tools Bugzilla Tinderbox Bonsai LXR

[Reis and Fortes 2002]

34



Great deal to develop and to document the Mozilla software process

Rely heavily on communication tools such as mailing lists, irc, news, bugzilla

“Bug-driven” development {bugzilla} Requirements

A message thread is started on a public newsgroup regarding a change in functionality

When a person has a specific idea for a change a bug will often be directly filed with no newsgroup discussion

“You can get a feature in if you’re willing to write code for it” -- Boris Zbarsky, Mozilla developer

Code review and inspection {bonsai} Nightly build {tinderbox}

35



Effective version controlA well-defined protocol for integrating

source code changesA high degree of accountability for people

who integrate codeHigh modularityGood communication channelsDocumentation not always up to date

36

Understanding the Requirements for Developing Open Source Software Systems

Traditional SE: Elicitation Analysis

Specification and modeling

Validation Communicating and

managing

Open Source Post-hoc assertion Reading, sense-making,

accountability Continually emerging

webs of discourse Condensing and

hardening discourse Global access to

discourse

[Scacchi 2002]

37

Free Software development lifecycle

[Senyard and Michlmayr 2004]

38

Bazaar style development lifecycle

[Senyard and Michlmayr 2004]

39

Quality assurance under the open source development model

User participation in open source projects is high up to 20% of the changes for almost 50% of the projects discovering 20–40% of the faults in 20% of the projects.

Change management processes and tools are at the cutting edge of large-scale collaborative software development

Documentation is not a high priority for most projects: ‘‘TODO’’ and install guide

Little investment in utilizing testing techniques and tools, relying mainly in community testing

[Zhao and Elbaum 2003]

40

Open source software maintenance process framework

[Koponen and Hotti 2005]

41

Open source software maintenance process framework

[Koponen and Hotti 2005]

42

Concluding remarks

Past, present and future

43

Past

Hacker community activitySelf-contained moviment

44

Present

Open source gets momentumDeveloping F/OSS is different than

software engineering [Scacchi] not better, not worse, but different and new more social, more accessible

F/OSS systems don’t need and probably won’t benefit from classic software engineering [Scacchi]

45

Abstract generic process models

Specification-based models Waterfall Incremental

Evolutionary development models Prototyping Spiral Open Source*

[Sommerville 1996]

46

So an open source process model is…

Evolutionary/maintenance-oriented development model Not just a technical development process Collaborative effort Based on community government User-driven requirements Modular architecture/design without UML stuff Heavily dependent on communication tools because it is

distributed, asynchronous and discontinuous Indeed it is not a traditional software engineering

process

47

Future

Open source is still in its infancy today OSS projects need to be more professionalOpen Source Software Process

Do not to move into Process Paralysis [Yourdon 1987]

48http://www.fabricadesol.com/in953

49[Neus et al. 2005]

50[Neus et al. 2005]

51

Something else?

52

References

O’Mahony, S. and West, J. 2005. What makes a project open source? Migrating from organic to synthetic communities. Academy of Management conference, Technology and Innovation Management division, Honolulu, August 2005.

O'Reilly, T. 1999. Lessons from open-source software development. Commun. ACM 42, 4 (Apr. 1999), 32-37

O'Reilly, T. 2005. The Open Source Paradigm Shift. In: Feller, J., Fitzgerald, B., Hissam, S. A. and Lakhani. K. R. eds. Perspectives on Free and Open Source Software. The MIT Press, 461-482. Also available in http://tim.oreilly.com/articles/paradigmshift_0504.html since June 2004

Open Source Initiative. http://www.opensource.org Perens, B. .1999. The Open Source Definition. In: DiBona, C., Ockman, S., and

Stone, M. eds. Open Sources: Voices from the Open Source Revolution. O'Reilly and Associates, 171-188.

Raymond, E. S. “The Cathedral & the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary”. O'Reilly and Associates, 1999. Also available in http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/ since May 1997

Rivilin, G. 2005. Open Wallets for Open-Source Software. The New York Times, April 27, 2005.

West, J., Jury, A., Sengupta, S., and Penberthy, S. 2005. Bookshelf. IEEE Softw. 22, 4 (Jul. 2005), 114-118.

53

References

Johnson, K. A. “Descriptive Process Model for Open-Source Software Development”, Master Thesis, Univ. Calgary, Alberta, 2001.

Koponen, T. and Hotti, V. 2005. Open source software maintenance process framework, ACM SIGSOFT Software Engineering Notes, v.30 n.4, July 2005

Mockus, A., Fielding, R. T., and Herbsleb, J. 2000. A case study of open source software development: the Apache server. In Proceedings of the 22nd international Conference on Software Engineering ( Limerick, Ireland, June 04 - 11, 2000). ICSE '00. ACM Press, New York, NY, 263-272.

Reis, C. and Fortes, R. P. M. 2002. An overview of the Software Engineering Process in the Mozilla project. In Proceedings of Workshop on Open Source Software Development, Newcastle, 155-175

Scacchi, W. 2002. Understanding the Requirements for Developing Open Source Software Systems. In IEE Proceedings Software, volume 148, number 1, pp. 24-39.

Senyard, A. and Michlmayr, M. 2004. How to Have a Successful Free Software Project. In Proceedings of the 11th Asia-Pacific Software Engineering Conference (Apsec'04) - Volume 00 (November 30 - December 03, 2004). APSEC. IEEE Computer Society, Washington, DC, 84-91

Zhao, L. and Elbaum, S. 2003. Quality assurance under the open source development model. J. Syst. Softw. 66, 1 (Apr. 2003), 65-75.

54

References

Neus, A., Scherf, P. and Subjects, R. 2005. Opening minds: Cultural change with the introduction of open-source collaboration methods. IBM Journal of Research and Development, Volume 44, Number 2, Page 215.

Sommerville, I. 1996. Software process models. ACM Comput. Surv. 28, 1 (Mar. 1996), 269-271.

Humphrey, W. S. 1989 Managing the Software Process. Addison-Wesley Longman Publishing Co., Inc.

Yourdon E. A Game Plan for Technology Transfer, Tutorial: Software Engineering Project Management. R.H. Thayer, ed. Computer Society Press, 1987

Paulk, M.C. et al., “Key Practices of the Capability Maturity Model, Version 1.1,” technical report CMU/SEI-93-TR-025, Software Engineering Institute, Carnegie Mellon University, 1993.

Fayad, M. E. 1997. Software development process: a necessary evil. Commun. ACM 40, 9 (Sep. 1997), 101-103.

Osterweil, L. 1987. Software processes are software too. In Proceedings of the 9th international Conference on Software Engineering (Monterey, California, United States). International Conference on Software Engineering. IEEE Computer Society Press, Los Alamitos, CA, 2-13.

Documents

Classical Open Source Software Process Model