1 / 23

CS 709B Advanced Software Project Management and Development

Software Estimation - II

Based on Chapter 4 of the book [McConnell 2006] Based on Chapter 4 of the book [McConnell 2006] Steve McConnell, Software Estimation: Demystifying the

Black Art, Microsoft Press, 2006

May 1, 2012May 1, 2012

2 / 23

Outline

Where Does Estimation Error Come From? Sources of Estimation Uncertainty The Cone of Uncertainty Chaotic Development Process Unstable Requirements Omitted Activities Unfounded Optimism Subjectivity and Bias Unwarranted Precision

3 / 23

Introduction

Estimation in general presents challenges U of Washington CSE Department’s building: months

late and $20.5 million over budget Seattle Mariners’ new baseball stadium was

estimated in 1995 to cost $250 million. It was finally completed in 1999 and cost $517 million

Examples in software Irish Personnel, Payroll and Related Systems

(PPARS) canceled after its overran its 8.8 million Euro budget by 140 million Euro!

FBI’s Virtual Case File shelved in 2005 after costing $170 million and delivering 1/10 of capabilities.

4 / 23

Sources of Estimation Uncertainty

Four generic sources Inaccurate information about the project Inaccurate information about organization’s capabilities Too much chaos in the project (moving target) Inaccuracies arising from the estimation error itself

Software development is a process of gradual refinement. The many ways a software could ultimately take shape will produce widely different combinations of cost, schedule, and feature set.

5 / 23

The Cone of UncertaintySoftware development means making literally thousands of decisions. Uncertainty in software estimates results from uncertainty on how these decisions will be made.

6 / 23

The Cone of Uncertainty

Estimation accuracy increases as the project Estimation accuracy increases as the project progresses, usually from +/-4x to +/-1.25x in about progresses, usually from +/-4x to +/-1.25x in about 30% of time30% of time

7 / 23

The Cone of Uncertainty Can you beat the Can you beat the

cone? It’s very cone? It’s very hard, as it hard, as it represents represents best-best-case accuracycase accuracy. . It’s only possible It’s only possible to be luckier. The to be luckier. The main issue is main issue is project project variabilityvariability, , and the cone and the cone narrows only if narrows only if you remove you remove sources of sources of variability.variability.

8 / 23

The Cone of Uncertainty You must force the Cone to narrow by removing You must force the Cone to narrow by removing

sources of variability from your projectsources of variability from your project

9 / 23

The Cone of Uncertainty One way of dealing with too narrow estimate ranges is to use One way of dealing with too narrow estimate ranges is to use

predefined multipliers applied to “most likely” estimates; predefined multipliers applied to “most likely” estimates; another is to use a “know-how-much” and “know-how-another is to use a “know-how-much” and “know-how-uncertain” strategyuncertain” strategy

10 / 23

The Cone of Uncertainty

The Cone and the Commitment Effective organizations delay their commitments until they

have done the work to force the Cone to narrow

The Cone and the Iterative Development It’s impractical and almost impossible not to have iterations There is a mini Cone in each iteration Short iterations will narrow this mini Cone early Another approach is to have (more) iterations after

narrowing the Cone. Also, it’s possible to plan for some unexpected requirements

(“positive variability”)

11 / 23

Project Chaos

Common examples of sources of chaos include: Requirements inadequately investigated Lack of end-user involvement in requirements Poor designs Poor coding practices Incomplete or unskilled project planning Inexperienced personnel Prima donna team members Abandoning planning under pressure Lack of automated source code control

These induce variability and need be fixed with improved process control

12 / 23

Unstable Requirements

Specific challenges raised by unstable requirements: They don’t allow the Cone to narrow Requirements changes are often not tracked, and the

cost and schedule are not re-estimated

Stabilizing requirements is more effective than improving estimation techniques

Also, consider applying development techniques for high-volatile environments (XP, Scrum, DSDM, etc.)

13 / 23

Unstable Requirements It’s useful to incorporate allowances for requirements growth. It’s useful to incorporate allowances for requirements growth.

NASA SEL plans for 40% requirements increase. COCOMO NASA SEL plans for 40% requirements increase. COCOMO incorporates a similar concept (“requirements breakage”).incorporates a similar concept (“requirements breakage”).

14 / 23

Omitted Activities

Errors also arise from estimation practicesErrors also arise from estimation practices A common source of errors is forgetting to A common source of errors is forgetting to

include necessary tasksinclude necessary tasks Omitted work includes: Omitted work includes:

Missing requirementsMissing requirements Missing software development activitiesMissing software development activities Missing non-software development activitiesMissing non-software development activities

15 / 23

Omitted Activities Missing requirementsMissing requirements Missing software development activitiesMissing software development activities Missing non-software development activitiesMissing non-software development activities

16 / 23

Omitted Activities Missing requirements examples are shown belowMissing requirements examples are shown below You must include estimates for stated requirements, implied You must include estimates for stated requirements, implied

requirements, and non-functional requirementsrequirements, and non-functional requirements

17 / 23

Omitted Activities

18 / 23

Omitted Activities

Examples of non-software development activities Examples of non-software development activities that tend to be omittedthat tend to be omitted

19 / 23

Unfounded Optimism [the collusion of optimists]

Developers underestimate their work by 20% to Developers underestimate their work by 20% to 30% [Cusumano and Shelby 1995, 300 software 30% [Cusumano and Shelby 1995, 300 software projects]projects]

Optimism applies to management as well, with a Optimism applies to management as well, with a “fantasy factor” of about 1.33 [Boehm 1981, DoD, “fantasy factor” of about 1.33 [Boehm 1981, DoD, 100 schedule estimates]100 schedule estimates]

Variations:Variations: We’ll be more productive in this project than in the last oneWe’ll be more productive in this project than in the last one A lot of things went wrong with the last project. This will A lot of things went wrong with the last project. This will

not happen with the current project.not happen with the current project. We are past the hard climbing of a steep learning curve We are past the hard climbing of a steep learning curve

20 / 23

Subjectivity and Bias Bias – tendency to “fudge an estimation”Bias – tendency to “fudge an estimation” Subjectivity – human judgment influenced by experienceSubjectivity – human judgment influenced by experience Too many “control knobs” are likely to introduce subjectivity.Too many “control knobs” are likely to introduce subjectivity.

21 / 23

Subjectivity and Bias

Smaller number of adjustment factors (“knobs”) Smaller number of adjustment factors (“knobs”) >>> smaller variation in estimates>>> smaller variation in estimates

22 / 23

“Off-the-Cuff” Estimates

It’s highly recommended to avoid “off-the-cuff” It’s highly recommended to avoid “off-the-cuff” estimatesestimates

23 / 23

Unwarranted Precision Accuracy Accuracy refers to how close to the real value a number is.refers to how close to the real value a number is.

PrecisionPrecision refers merely to how exact a number is (how many refers merely to how exact a number is (how many significant digits).significant digits).