Software Technology Support Center BE AMERICA’S BEST The Problems in Software Estimating Dr....

BE AMERICA’S BEST

Software Technology Support Center

The Problems in Software

Estimating

Dr. Randall JensenSoftware Technology Support Center

Hill AFB, UT

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Outline

Evolution of software estimating models

Software problems

Management is the issue

Data collection concerns

New models?

It’s all about models – “Existence is elsewhere”

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Project Uncertainty Principle

O G D E N A I R L O G I S T I C S C E N T E R

Software model genealogy

NordonIBM 1970

Putnam/SLIMUS Army/GE/QSM 1976

DotyRADC 1977

JS1,2,3CEI 1980

SEER-SEMGAI 1989

Seer II / SageSEI 1995

COCOMOTRW 1981

COCOMO IIUSC 1995

Validation

SeerHAC 1979

PRICE-SPrice 1977

TRUE-SPrice 2003

REVICUSAF 1991

Management

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Sage/SEER-SEM - Normal form

where K = life cycle cost (person years), Td = development time (years)

slocdtee TKCS

3dT

KD

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Common estimate dilemma

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Software problems are not new

Unreliable

Late delivery

Modification costs prohibitive

Impossible to maintain

Inadequate performance

Product exceeds budget costs

1968 CS Conference, Munich, Germany

KEY

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Historic note:

“More software projects have gone awry for lack of calendar time than for all other causes combined…”

F. P. Brooks, Jr., Mythical Man Month

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Common technology claim

If you use (new technology),Productivity will improve by an order of magnitude

AndDefects will reduce to zero

Note: None of the history-based estimating models (tools) support this claim, not even close.

Neither does history.

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

There is always HOPE

1960 1970 1980 1990

100

90

80

70

60

50

Ada

OOD

Structured Analysis

Structured DesignProcess Maturity

PWB

Structured Programming

3rd Generation Languages

PRODUCTIVITY

lppm

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

3 Dimensions of development

PROJECT

PROCESSPEOPLE

1975

1985

1995

1938

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Impact of management on cost

“Poor management can increase software costs more rapidly than any other factor…

Despite this variation, COCOMO does not include a factor for management quality, but instead provides estimates which assume that the project will be well managed [italics mine, RWJ]”

Boehm, B. W., Software Engineering Economics, (Englewood Cliffs, NJ: Prentice-Hall, Inc.), 1981, pg. 486

Note: On a scale of 0 to 10 well managed must rank approximately 3.4?

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Where would you focus effort?

3

11

17

64

TOOLS

PEOPLE

SYSTEMS

MANAGEMENT

Source: G. Weinberg, Quality Software Management, Vol. 3

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Biased research?

Tools People Systems Managers0

10

20

30

40

50

60

70

SEI Papers

Pe

rce

nt

Source: G. Weinberg, Quality Software Management, Vol. 3

O G D E N A I R L O G I S T I C S C E N T E R

Cost driver relative impact

Target ExperienceMultiple Sites

Develop Sys ExperienceMultiple Organizations

Resource LocationDevelopment Tools

Modern PracticesApplication Experience

Practices ExperienceStaff Capability

0 0.5 1 1.5 2 2.5

Relative Development Cost Impact

It takes people to implement processes and use technology.

April 21st, 2009

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

BIG productivity drivers

Analyst capability Management style Motivation Use of team methods Working environment Problem solving skills

Programmer capabilityApplication domain experience Use of modern practicesAutomated tool support

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Capability shift (environment)

Basic Technology Constant

1980 2005

5500 750065006200 85008000

8635(1990)

O G D E N A I R L O G I S T I C S C E N T E R

Historic data observations

Models are made from historic data More data is better Adequate historic data takes time to collect (Lots of time)

Repository vs Database Repository = a place where things may be stored for safekeeping

Data stored without concern for a formal definition of the data placed in the repository

SRDR is defined as a financial report that happens to contain technical information

Database = a collection of data arranged for ease and speed of retrieval

Requires consistency of entries Each data field must satisfy common definition

DCARC is a repository as was SMC database Data definitions?

BE AMERICA’S BEST 18

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Problems?

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Popular estimating solution

Let’s create a new software model

Models are out of date

Models are inaccurate

Too difficult to use

Oh, really! Are you sure?

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

New estimating models?

Lack of confidence in existing tools (current issues) Optimistic estimates Unacceptable estimates

Aging tools (?) Culture is constant New models require validation

New models -- No silver bullets Quality data (if, where, proprietary) New models require validation

New (or better) estimators? Experience Skill Integrity

O G D E N A I R L O G I S T I C S C E N T E R

EstimatingPrediction is very hard, especially when it’s about the

future