1

Systems Development Methodologies

- set i forhold til projektledelse

Agile Project Management:Jeff Sutherland

http://jeffsutherland.com/papers/OTUG2003/Scrum_Theory_files/frame.htm

Projektledelse:Erik Staunstrup

2

Agile Project Management With SCRUM: Theory and Practice

Agile Project Management With SCRUM: Theory and Practice

Jeff Sutherland, Ph.D.Chief Technology Officer

jeff.sutherland@computer.org

http://jeffsutherland.com

Jeff Sutherland, Ph.D.Chief Technology Officer

jeff.sutherland@computer.org

http://jeffsutherland.com

3

Complex Adaptive Systems (cas) Interacting agents respond to stimuli. Stimulus-response behavior is defined in terms of rules. Agents adapt by changing rules as experience accumulates. Agents aggregate into meta-agents whose behavior is emergent. How can a collection of dumb things emerge smart system behavior?

Maamar, Zakaria and Sutherland, Jeff (2000) Toward Intelligent Business Objects: Focusing on Techniques to Enhance Business Objects that Exhibit Goal-Oriented Behaviors. Communications of the ACM 40:10:99-101.

Frozen

Chaos Fragmentation

cas Self Organization

Web services?

1998 Agents 1995 Components 1993 Business Objects

1980 Classes 1970 Procedures

4

Enterprise Systems are cas Business entities are examples of complex

adaptive systems. Modification time is on the order of months

or years, roughly time required to change software.

Automating business processes renders parts of the business in software.

Business systems have severely constrained rule sets, making ideal test bed for cas concepts. 

5

Change is Imperative:Wasserman's 7 Factors Driving Change1. Criticality of time to market 2. Shift in computing economics 3. Powerful desktop computers 4. Extensive networks and the Web 5. Growing availability of object technology 6. WIMP (windows, icons, menus, pointers) 7. Unpredictability of the waterfall model of

software development

6

"The Waterfall Methodology!" "Why Are Systems Late, Over Budget,

Wrong?" (Paul Bassett) Analysis Paralysis static modeling overused

specs are stale baked Design-from-Scratch no generic models

no standard architectures Large Project Teams User Intermediaries No Early Warning Signals

7

History of Iterative and Incremental Development (IID)(1)

1956 – Benington’s stagewise model USAF SAGE System

1957 – IBM Service Bureau Corp, Project Mercury, IBM Federal Systems Devision Gerry Weinberg

1960 – Weinberg teaching IID at IBM Systems Research Institute 1969 - Earliest published reference to IID: Robert Glass. Elementary Level Discussion of

Compiler/Interpreter Writing. ACM Computing Surveys, Mar 1969 Larman, Craig and Basili, Vic. A History of Iterative and Incremental Development. IEEE Computer, June 2003 (in press)

8

History of Iterative and Incremental Development (IID)(2) 1971 – IBM Federal Systems Division

Mills, Harlan. Top-down programming in Large Systems. In Debugging Techniques in Large Systems. Prentice Hall, 1971

1972 – TRW uses IID on $100M Army Site Defense software 1975 – First original paper devoted to IID

Gasili, Vic and Turner, Albert. Iterative Enhancement: A Practical Technique for Software Development. IEEE Transactions on Software Engineering. Dec 1975.

1977-1980 – IBM FSD builds NASA Space Shuttle software in 17 iterations over 31 months, averaging 8 weeks per iteration

Madden and Rone. Design, Development, Integration: Space Shuttle Flight Software. Communications of the ACM, Sept 1984.

Larman, Craig and Basili, Vic. A History of Iterative and Incremental Development. IEEE Computer, June 2003 (in press)

9

History of Iterative and Incremental Development (IID)(3) 1985 – Barry Boehm’s Spiral Model

Boehm, Barry. A Spiral Model of Software Development and Enhancement. Proceedings of an International Workshop on Software Process and Software Environments. March, 1985

1986 – Brooks, Fred. No Silver Bullet. IEEE Computer, April 1987 Nothing … has so radically changed my own practice, or its effectiveness [as

incremental development]. 1994 – First SCRUM at Easel Corporation 1994 – DOD must manage programs using iterative development

Report of the Defense Science Board Task Force on Acquiring Defense Software Commercially. June 1994.

1995 – Microsoft IID published McCarthy, Jim. Dynamics of Software Development. Microsoft Press, 1995.

1996 – Kruchten. A Rational Development Process. Crosstalk. July. Origins of RUP Larman, Craig and Basili, Vic. A History of Iterative and

Incremental Development. IEEE Computer, June 2003 (in press)

10

History of Iterative and Incremental Development (IID)(4) 1996 – Kent Beck Chrysler Project

Origin of XP 1996 – Larman meets with principal author of DD-STD-2167

David Maibor expressed regret for the creation of the waterfall-based standard. He had not learned of incremental development at the time and based his advice on textbooks and consultants advocating the waterfall method. With the hindsight of iterative experience, he would recommend IID.

1997 – Coad and DeLuca rescue Singapore project Origin of Feature-Driven Development

1998 – Standish Group CHAOS Project Top reason for massive project failures was waterfall methods. “Research also

indicates that smaller time frames, with delivery of software components early and often, will increase success rate.

1999 – Publication of extensive DOD failures Out of a total cost of $37B for the sample set, 75% of projects failed or were

never used, and only 2% were used without extensive modification. Jarzombek. The 5th Annual JAWS S3 Proceedings, 1999. Larman, Craig and Basili, Vic. A History of Iterative and Incremental Development. IEEE Computer, June 2003 (in press)

11

History of Iterative and Incremental Development (IID)(5)

2001 – 17 process expert “anarchists” meet at Snow Bird Agile Manifesto initiated 100s of books and papers on

agile development 2001 – MacCormack’s study of key success

factors MacCormack, Alan. Product-Develoment Practices that

Work. MIT Sloan Management Review 42:2, 2001.

Larman, Craig and Basili, Vic. A History of Iterative and Incremental Development. IEEE Computer, June 2003 (in press)

12

Manifesto for Agile Software Development

We are uncovering better ways of developing software by doing it and helping others do it.

Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

13

MacCormack Process Evolution

Waterfall mode – sequential process maintains a document trail

Rapid-Prototyping Model – disposable prototype helps establish customer preference

Spiral Model – series of prototypes identifies major risks Incremental or Staged Delivery Model – system is

delivered to customer in chunks Evolutionary Delivery Model – iterative approach in

which customers test an actual version of the software

MacCormack, Alan. Product-Development Practices That Work: How Internet Companies Build Software. MIT Sloan Management Review 42:2:75-84, Winter 2001.

14

MacCormack Success Factors Early release of evolving product design to

customers. Daily incorporation of new software code

and rapid feedback on design changes A team with broad-based experience in shipping multiple projects Major investment in design of product architecture

MacCormack, Alan. Product-Development Practices That Work: How Internet Companies Build Software. MIT Sloan Management Review 42:2:75-84, Winter 2001

15

SCRUM Origins: Takeuchi and Nonaka Lessons from Fuji-Xerox, Canon, Honda, NEC, Epson, Brother, 3M, Xerox, HP Old model – Relay Race (type A)

– Speed and flexibility not adequate in today’s market New model – Rugby (type C)

Takeuchi, Hirotaka and Nonaka, Ikujiro. 1986. The new new product development game. Harvard Business Review 64:1:137-146 (Jan/Feb), reprint no. 86116.

16

Takeuchi and Nonaka 6 Success Factors

1. Built-in instability 2. Self-organizing project teams 3. Overlapping development phases 4. “Multilearning” 5. Subtle control 6. Organizational transfer of learning

“These characteristics are like pieces of a jigsaw puzzle. Each element, by itself, does not bring about speed and flexibility. But taken as a whole, the characteristics can product a powerful new set of dynamics that will make a difference.”

17

Factor 1: Built-in instability

Top management kicks off development process by signaling broad goal. Project team is offered extremely challenging goals with wide

measure of freedom. Example: Fuji-Xerox gave FX-3500 project team two years

to come up with a copier that cut costs in half Top management creates an element of tension in the project

team through challenging requirements with wide freedom to achieve strategic objective.

Honda Executive: “It’s like putting the team members on the second floor, removing the ladder, and telling them to jump or else. I believe creativity is born by pushing people against the wall and pressuring them almost to the extreme.”

18

Factor 2: Self-organizing project teams A project team takes on a self-organizing character as it is

driven to a state of “zero information” – where prior knowledge does not apply.

Left to stew, the process begins to create its own dynamic order.

The project team begins to operate like a start-up company.

A group possesses a self-organizing capability when it exhibits three conditions:

1. Autonomy Conditions will be2. Self-transcendence uncovered in3. Cross-fertilization the next slides

At some point, the team begins to create its own concept.

19

Condition 1: Autonomy Headquarters involvement is limited to providing

guidance, money, and moral support at the outset.

On a day to day basis, management seldom intervenes and the team is free to set its own direction. In a way, top management acts as a venture capitalist

“We open our purse and keep our mouth closed.” Example: IBM development of personal computer Example: Honda City project team, average age

27, “Develop the kind of car that the youth segment would like to drive.”

20

Condition 2: Self-transcendence The project teams appear to be absorbed

in the never-ending quest for “the limit.” They elevate their goals through the development process. By pursuing what appear to be contradictory goals, they devise ways to override the status quo and make the big discovery.

Example: Canon AE-1 team

21

Condition 3: Cross-fertilization Team with wide variety of specializations, thought

processes, and behavior patterns carries out new product development.

Working in one large room is best (Fuji-Xerox). “When all team members

are in one room, others

information becomes yours

without even trying.”

Radcliffe Rugby Football Club

22

Factor 3: Overlapping Development Phases

Self-organizing character of the team produces unique dynamic or rhythm

Sashimi system – Fuji Xerox Rugby system – Honda Hard merits (demerits)

Speed and flexibility (watch out for muck and mall) Soft merits Share responsibility and cooperation Stimulates involvement and commitment Sharpens a problem-solving focus Develops initiative and diversified skills Heightens sensitivity to market conditions

23

Factor 4: Multilearning

Learning by doing in two dimensions Across organization

Across specialty Enhanced learning opportunities 15% of time devoted to “dreams” – 3M Peer pressure to study Send team to Europe to look around – Honda

Bring in top academics and consultants – HP Everyone learns multiple skills

24

Factor 5: Subtle Control

Management establishes checkpoints Prevents instability, ambiguity, and tension from

turning into chaos Emphasis on self-control, control by peer

pressure, control by love = “subtle control” Management responsible for: Selecting team members for balanced team Creating an open working environment Encouraging engineers to go out in the field Establishing rewards based on group performance Tolerating and anticipating mistakes Encouraging suppliers to become self-organizing

25

Factor 6: Organizational Transfer of Learning

Transfer knowledge outside group Scatter successful team to new projects

Institutionalize practice (monthly demos at IDX) Consciously pursue unlearning Next generation must be 40% better Cut product cycle by 80% Scrap old parts, processes, tools

26

Challenges and Opportunities

Winding the Rubber Band Principle: Broad mandate and demanding goals create tension.

Anti-Waterfall Principle: Operational decisions are made incrementally. Strategic decisions delayed to last moment.

Push/Pull Principle: Differentiation in concept phase, integration dominates in implementation phase

Spread the Wealth Principle: Non-experts take on new tasks.

Cuckoo Principle: Successful SCRUMs become company models (or they can get crushed because they are different). Control Anti-Pattern: Seniority based companies have difficult

time. But in times of desperation, SCRUMs are easily created.

27

Team Size: Development Effort in Months

The smaller the better. 491 medium sized projects with 35,000-95,000 SLOC (source lines of

code)

Putnam, Lawrence H. and Myers, Ware. Familiar Metrics Management: Small is Beautiful--Once Again. IT Metrics Strategis IV:8:12-16, Cutter Information Corp., August 1998.

28

"The Waterfall Methodology!"

"Why Are Systems Late, Over Budget, Wrong?"

Analysis Paralysis – static modeling overused specs are stale baked Design-from-Scratch – no generic models no standard architectures Large Project Teams User Intermediaries No Early Warning Signals

Bassett, Paul G. Framing Software Reuse: Lessons from the Real World. Yourdon Press Computing Series, 1997.

29

Spiral Methodology

Barry Boehm introduced the Spiral Methodology to "fix" problems with the Waterfall Methodology. This is the most commonly used variant of the Waterfall today.

The Spiral methodology "peels the onion", progressing through "layers" of the development process. A prototype lets users determine if the project is on track, should be sent back to prior phases, or should be ended. However, the phases and phase processes are still linear.

Boehm, B.W. A Spiral Model of Software Development and Enhancement. Proceedings of an International Workshop on Software Process and Software Environments, Coto de Caza, Trabuco Canyon, California, March 27-29, 1985. Boehm, Barry. A Spiral Model of Software Development and Enhancement.  ACM SIGSOFT Software Engineering Notes, August 1986. Boehm, Barry. A Spiral Model of Software Development and Enhancement.  IEEE Computer, vol.21, #5, May 1988, pp 61-72.

30

Iterative Methology

The Iterative methodology improves on the Spiral methodology. Each iteration consists of all of the standard Waterfall phases, but each iteration only addresses one subsystem. Further iterations can add resources to the project while ramping up the speed of delivery. Improves cost control, reduces risk, ensures delivery of (sub)systems, and improves overall flexibility. Still assumes that the underlying development processes are defined and linear.

31

SCRUM Methodology

The first and last phases (Planning and Closure) consist of defined processes. The Sprint phase is an empirical process. It is treated as a black box that requires external controls. Sprints are nonlinear and flexible. Sprints are used to evolve the final product. The project is open to the environment until the Closure phase. The deliverable can be changed at any time. The deliverable is determined during the project based on the environment.

32

Methodology Comparison

33

Risk with Current Methodologies

Any methodology is better than nothing.

Current approaches rests on the fallacy that the development processes are defined, predictable processes.

They lack flexibility needed to cope with the unpredictable results and respond to a complex environment.

Schwaber, Ken. SCRUM Development Process. Business Object Design and Implementation (Eds. Jeff Sutherland et al.). London: Springer-Verlag, 1997.

34

SCRUM Lowers Risk

Development teams need to operate adaptively within a complex environment using imprecise processes. SCRUM can accelerate closure by inducing the phenomenon known as "punctuated equilibrium" seen in the evolution of biological species.

Levy, Steven. Artificial Life: A Report from the Frontier Where Computers Meet Biology. Vintage Books, 1993. Lewin, Roger. Complexity: Life at the Edge of Chaos. Collier Books, 1994.

35

Direktør

Udlvikl.chef Salgschef Prod.chef

Styregruppe

Projekt-gruppeMedarb. Medarb.Medarb. Medarb.

Projektorganisation Basisorganisation

Skillelinie mellem basis- og projektorganisation

Projektorganisation

36

Fordeling af arbejdsopgaver

Projektgruppens opgaver Styregruppens opgaver

-gennemføre analyser-udarbejde løsningsforslag-opstille planer for aktiviteterne-afprøve de foreslåede løsninger-implementer og evaluere den/de valgte løsninger-dokumenter resultaterne-udarbejde rapporter til styregruppen

-formulere baggrunden for projektet-formulere opgavebeskrivelse, tidsramme og ressourceramme-opstille mål og strategier for projektet-sørge for de nødvendige ressourcer-overvåge og evaluere projektforløbet-deltage i valget af løsning

37

Projektet som et systemProjektorganisation Basisorganisation

Skillelinie mellem basis- og projektorganisation

Projektkultur

Projektledelse

Projektstyring

Ledelse

Kultur

Styring

Værdiorientering

Resultatorientering

38

Projektets subsystemerProjektorganisation

Projektkultur

Projektledelse- ledelsesform- involvering- samarbejde

Projekt-styring

Værdiorientering

Resultatorientering

39

Systemudviklings Metodologierifht risiko og ressourcebevidsthed

Figur 2.2: Sammenhængen mellem ressourcebevidsthed og risikomomentKilde: Delvis baseret på Karvø, Michael og Pedersen, Lars Bo: Projektledelse i tværfaglige teams, Schultz, 1993

Ressourcebevidsthed

Risikomoment

Høj

Lav

Lavt Højt

Vandfaldsmodel

Spiralmodel

Agile Project ManagementSCRUM

Iterativ model

Rapid Prototyping

Eksperimentel

40

Systemudviklings Metodologierifht resultat og værdiorientering

Resultatorientering

Værdiorientering

Høj

Lav

Lavt Højt

Vandfaldsmodel

Spiralmodel

Agile Project ManagementSCRUM

Iterativ model

Rapid Prototyping

Eksperimentel

41

Resultat- og værdiorientering

Værdiorientering

Vandfald

Resultat

Spiral/Iterativ

Resultat ogVærdi

Agile Project ManagementSCRUM

Værdi og Resultat

Eksperimentel

Værdi

Resultatorientering

42

Sammenligning af metodologierne

SystemudviklingsMetodologier

Højesteprioritet iprojektet

Opgaven(målet)

Risiko-moment

Ressource-bevidsthed

Orientering

Vandfald(slide 28)

Resultatet Klar ogafgrænset

Lavt Lav Resultat

Rapid Prototyping

Spiral (slide 29)

Iterativ (slide 30)

Resultatetogprocessen

Koncepteter kendt (igrovetræk)

Middel Middel Resultat (og værdi)

Agile ProjectManagementSCRUM(slide 31)

Resultat,proces ogStrategisk vigtig

Retningener kendt

Højt Høj Værdi (og resultat)

Eksperimentel Strategiskvigtighed

Retningener kendt

Højt Lav Værdi

43

Situationsbestemt ledelse

Værdiorientering Medarbejdernes indflydelse

ResultatorienteringLederens styring

Autoritær Demokratisk Laissez Faire

Lederen træffer beslutninger og meddeler dem

Lederen præsenterer forslag og opfordre til spørgsmål

Lederen fastsætter rammer og lader gruppen selv bestemme

Lederen lader gruppen råde frit inden for rammer, han selv er blevet pålagt

Medarbejdernes videnGeneralister Specialister

OpgavenStruktureret UstruktureretÈn metode Mange metoder

TidKnap Tilstrækkelig

RessourcerFå Rigelige

44

Systemudviklings Metodologier ogSituationsbestemt ledelse

Autoritær Demokratisk Laissez Faire

Lederen træffer beslutninger og meddeler dem

Lederen præsenterer forslagog opfordre til spørgsmål

Lederen fastsætter rammer oglader gruppen selv bestemme

Lederen lader gruppen rådefrit inden for rammer, hanselv er blevet pålagt

Medarbejdernes videnGeneralister Specialister

OpgavenStruktureret UstruktureretÈn metode Mange metoder

TidKnap Tilstrækkelig

RessourcerFå Rigelige

Vandfald Spiral / Iterativ Agile ProjectManagementSCRUM

Eksperimentel

45

Selvstændighedsbarrieren!

Simple og sikre Komplekse og Nye og usikrearbejdsopgaver udfordrende arbejdsopgaver arbejdsopgaver

Projektlederens styring

Projektdeltagernes indflydelse

Almindelige samarbejds- og ledelsesformer

Nye samarbejds- og ledelsesformer

Selvorganisering

Alm. Teamarbejde

Individuelt arbejde

Usikkerhed iArbejds-opgaverne

Selvstændigheds-barriere

Samarbejds- ogledelsesformer

Sam-arbejde

46

Systemudviklings Metodologierne ogselvstændighed / kreativitet

Simple og sikre Komplekse og Nye og usikrearbejdsopgaver udfordrende arbejdsopgaver arbejdsopgaver

Projektlederens styring

Projektdeltagernes indflydelse

Almindelige samarbejds- og ledelsesformer

Nye samarbejds- og ledelsesformer

SCRUMSprints

SpiralIterativScrum Plan/Clos

Vandfald

Usikkerhed iArbejds-opgaverne

Selvstændigheds-barriere

Samarbejds- ogledelsesformer

Sam-arbejde