“Budgeting of IT-projects: Standards and best practices for cost and schedule planning.” Galorath Incorporated Daniel D. Galorath blog:

“Budgeting of IT-projects: Standards and best practices for cost and schedule planning.”Galorath Incorporated

Daniel D. Galorathblog: www.galorath.com/wp

Copyright Galorath Incorporated 2013

•Cutter Consortium Software Project Survey:• 62% overran original schedule by more than 50%

• 64% more than 50% over budget

• 70% had critical product quality defects after release

• Standish Group CHAOS Report• 46% challenged

• 19% failed

• 35% successful

~$875 billion spent on IT~$300 billion spent on IT projects ~$57 billion wasted annually

ROI of better planning huge

IT Failures Are Pervasive: And Even Successful Projects May Not Be

Some suggest 80% of systems NEVER produce a positive ROI

An Estimate Defined

• An estimate is the most knowledgeable statement you can make at a particular point in time regarding:• Effort / Cost

• Schedule

• Staffing

• Risk

• Reliability

• Estimates more precise with progress

• A WELL FORMED ESTIMATE IS A DISTRIBUTION

© 2013 Copyright Galorath Incorporated 3

Human Nature: Humans Are Optimists

HBR Article explains this Phenomenon:

• Humans seem hardwired to be optimists

• We routinely exaggerate benefits and discount costs

Delusions of Success: How Optimism Undermines Executives' Decisions (Source: HBR Articles | Dan Lovallo, Daniel Kahneman | Jul 01, 2003)

4

Solution - Temper with “outside view”:Past Measurement Results, traditional forecasting,

and statistical parametrics can help

Don’t remove optimism, but balance optimism and realism

http://hbr.org/search/Dan%20Lovallo/

http://hbr.org/search/Dan%20Lovallo/

http://hbr.org/search/Daniel%20Kahneman/

http://hbr.org/search/Daniel%20Kahneman/

Estimation Methods 1 of 2

Method Description Advantages Limitations

Guessing Off the cuff estimatesQuick Can obtain any answer desired

No Basis or substantiationNo ProcessUsually Wrong

AnalogyCompare project with past similar projects.

Estimates are based on actual experience.

Truly similar projects must exist.

Expert Judgment

Consult with one or more experts.

Little or no historical data is needed; good for new or unique projects.

Experts may be biased; may not be consistent.

Top Down Estimation

Hierarchical decomposition into progressively smaller components to size software components

Provides an estimate linked to requirements and allows common libraries to size lower level components.

Need valid requirements. Difficult to track architecture; engineering bias may lead to underestimation.

Bottom UpUse a WBS and identify every task, then sum these

Forces thinking about the work

Can be optimistic with items missing


Estimation Methods 2 of 2

Model Category Description Advantages Limitations

Design To Cost

Uses expert judgment to determine how much functionality can be provided for given budget.

Easy to get under stakeholder number

Little or no engineering basis.

Simple CER’s

Equation with one or more unknowns that provides cost / schedule estimate

Some basis in data

Simple relationships may not tell the whole storyHistorical data may not tell the whole story

Comprehensive Parametric Models

Perform overall estimate using design parameters and mathematical algorithms.

Models are usually fast and easy to use, and useful early in a program; they are also objective and repeatable.

Models can be inaccurate if not properly calibrated and validated; historical data may not be relevant to new programs; optimism in parameters may lead to underestimation.


Reasons To For Viable / Repeatable Estimates (Adapted from CEBok)

To support good decisions To schedule work

To determine how long the project should take and its cost

To determine whether the project is worth doing

To develop cash flow needs

To determine how well the project is progressing

To develop time-phased budgets and establish the project baseline


Best Practice: Estimation Process: Consistent Processes = Reliable Estimates

1. Establish Estimate Scope

2. Establish Technical Baseline, Ground

Rules, Assumptions

3. Collect Data

4. Estimate and Validate Software Size

5. Prepare Baseline

Estimates

7. Quantify Risks and Risk Analysis

6. Review, Verify and Validate

Estimate

8. Generate a Project Plan

9. Document Estimate and Lessons

Learned

10. Track Project Throughout

Development

Estimate When Project Changes Or Information Is Available• Traditional Estimate Phases

• During Feasibility

• At Concept

• After Requirements

• After Design

• After Drops if Incremental

• Agile Estimate Phases

• At Requirements (Use Cases, User Stories, etc.)

• Before Each Release

0

200

400

600

800

1000

1200

1400

1600

Feasibility Concept Requirements Design-Code-Test

Delivery

Eff

ort

Phase

Estimate Convergence

Most

Least

Likely

The Development Method Is Part Of The Solution Not The Problem

Estimates typically become more accurate and less uncertain as the project progresses…


Example: Project Cost Alone Is not The Cost of IT Failure (Source: HBR)

• Case Study: Levi Strauss

• $5m ERP deployment contracted

• Risks seemed small

• Difficulty interfacing with customers systems

• Had to shut down production

• Unable to fill orders for 3 weeks

• $192.5M charge against earnings on a $5m IT project failure


“IT projects touch so many aspects of organization they pose a new singular risk”

http://hbr.org/2011/09/why-your-it-project-may-be-riskier-than-you-think/ar/1

© 2013 Galorath Incorporated

Manual Estimates: Human Reasons For Error (Metrics Can Help)

• Manual Task estimates yield SIGNIFICANT error

• Desire for “credibility” motivates overestimate behavior (80% probability?)– So must spend all the time to be “reliable”

– Better approach: force 50% probability & have “buffer” for overruns

• Technical pride sometimes causes underestimates


Best Practice: Estimate Entire Total Ownership Cost

• Software Development (SEER-SEM)

• Software Maintenance (SEER-SEM)

• IT Infrastructure (SEER-IT)

• IT Services (SEER-IT)



Best Practice: Understand & Manage Project Realities

For a given Size, Complexity and Technology

Impo

ssib

le

Minimum TimeTo Complete

(Effort Increases

to Reduce Schedule)

Work ExpandsTo Fill Time

(Effort Increases

due to lack of pressure)

Inefficient

Minimum Time

Optimal Effort(Lower Effort

for Longer Schedule)

Calendar Time

Effo

rt M

onth

s

Effort Increasedue to Longer

Schedule


© 2013 Galorath Incorporated14

Best Practice: Every Estimate needs to be Substantiated & Benchmarked

SEER-SEM Estimate

Your Data Regression Trend Line

Galorath Benchmark Trend Line

Your History Data

Why Should We Care: Variances can identify estimation issues. Benchmarking can be path to improvement


Best Practice Probability based Estimates

15

What is likely to happen

Feel lucky?

Firm Fixed Price?

Understand the risk before you commit!


Best Practice: Understand What Drives Productivity

People Process

TechnologyEffective

Technology


Best Practice: Consider Total Cost of Ownership, Not Just Development

Why Should We Care: Impacts ROI & development decisions can impact maintenance costs.

Staff Vs Maintenance Rigor

0500

100015002000250030003500

1 7 13 19 25 31 37 43 49 55 61 67 73 79 85

Time

sta

ff h

ou

rs p

er

mo

nth

develop

Rigor vhi+

Rigor nom

Rigor vlo


Best Practice: Multi-Dimensional Progress Tracking (Earned Value)

Track defect discovery and removal rates

against expected rates

Increased defect

reporting rate shows a

worsening trend

• Heath and Status Indicator shows status and trends from the previous snapshot

• Thresholds are user definable

Why Should We Care: Catching deviations from plan early allows adjustment or replanning

© 2013 Galorath Incorporated © 2013 Copyright Galorath Incorporated 19

Best Practice: Relative Estimates For Early Estimates

Why Should We Care: Generate viable estimates even before any requirements analysis. Provide crosschecks & portfolio analysis

• Generate Sizing even before details are known based on relative analysis

• Access corporate knowledge from historical database

• Use domain specific analogies

• Configurable for Different Needs - estimate any type of variable

Best Practice: Establish A Project Repository


Conclusions

• Viable Estimates are critical to project success

• With proper estimation processes and tools estimation should SAVE FAR MORE THAN THEY COST

• Knowledge is power… Making the appropriate decisions can have a huge return


Copyright Galorath Incorporated 2013

Backup Slides

Mythical Man-Month Rules (Source: MIT/ Brooks)• “Cost varies as product of people and months,

progress does not.”

• “Hence the man-month as a unit for measuring the size of job is a dangerous and deceptive myth”

• The myth of additional manpower (Brooks Law)

• “Adding manpower to a late project makes it later”

• Communication & Training Cost

• Brooks original formula: n(n-1)/2

• 12 month project schedule• 1 person: 12 months

• 2 persons = 7 months (2 man-months extra)

• 3 persons = 5 months (3 man-months extra

Models Like SEER Refined The Mathematics… The Phenomena Continues

High Quality Estimate Characteristics• Accurate (within a viable range)

• Comprehensive (includes all costs or identifies exclusions)

• Replicable and Auditable (not just guessss)

• Traceable (to the source of the requirement)

• Credible (Believable)

• Timely (Available when needed)

24

25

Estimate Model Components and Characteristics (adapted from CEBOK)

• Inputs• Technical and programmatic parameters

• Applicable rates

• Detail (Model structure)• Varying level of detail depending on need

• Applies parametric, analogy, or build-up techniques to inputs

• Outputs• Costs, & cost risk

• Schedule & schedule risk

Methodology OutputsInputs

Data

Groundrules & Assumptions

Frederick Brooks Classic Paper “No Silver Bullets”• “There is no single development, in either

technology or management technique, which by itself promises even one order-of magnitude improvement within a decade in productivity, in reliability, in simplicity.”

• -- Fred Brooks, 1986

• i.e. There is no magical cure for the “software crisis”

• Not software measurement

• Not better tools

• Not ---- (fill in the blank… the current great hope)

Key Components Of A Software Project That Uses COTS

• COTS Software:• Purchased functionality

• Direct Cost component of

COTS integration

• COTS Cognition: • Required functionality within

the COTS software that must be understood

• “Glue” Code:• Code written to bind COTS to

developmental software

• Development effort must be captured

Customization

DevelopmentalSoftware

COTS Software

“COTS Cognition”

Glue

27


Process For Combining Estimation, Planning & Control, Measurement & Analysis

28

PrepareEstimate

Baseline ApprovedEstimate

CollectIn Progress

Data

SnapshotPoint in Time• Progress

• Effort• Schedule

• Size Growth• Defect Rates

EffortProgress

ScheduleProgress

Size Growth

Defect Insert/Remove

Progress

Multi-DimensionEarned Value


Packaged Applications Are Costly To Organizations

• “Commercial application program or collection of programs developed to meet the needs of a variety of users, rather than custom designed for a specific organization”

• Many are enterprise applications

• Often allows / requires customization

• Examples: SAP; Rational PPM, SEER for Software; Microsoft Excel, CA Clarity, Oracle Business Suite

Why should we care: Packages sometimes comprise solutions for parts of complicated systems and can be trouble

"One-third [of the budget] has to go to testing. Don’t ever short change testing. Everyone always underestimates it, and says it’s the last thing to worry about. Don’t do that!“

- Jim Larson, consultant for communications solutions provider


Understand Project Total Ownership Costs Up Front

Staff Vs Maintenance Rigor

0500

100015002000250030003500

1 7 13 19 25 31 37 43 49 55 61 67 73 79 85

Time

sta

ff h

ou

rs p

er

mo

nth

develop

Rigor vhi+

Rigor nom

Rigor vlo

• Most Projects Spend Low During Maintenance

Documents

“Budgeting of IT-projects: Standards and best practices for cost and schedule planning.” Galorath Incorporated Daniel D. Galorath blog: