New York Galorath DCG Event Oct 2011 Combined Slides

7/28/2019 New York Galorath DCG Event Oct 2011 Combined Slides

1/105

Galorath / DCG New York Event October 2011

Dan Galorath

[email protected]

Copyright 2011 Galorath Incorporated


2/105

Ite

m

Who What First

AnswerBG Introductions, logistics, etcBG Format o f meet ingDG Opening remarks, personal perspective on

software estimation, give a few examples of Galorath clients where implementing theestimation tool has paid off (15 minutes max, fewslides). Mention Ford, Euroclear

MH Opening remarks, personal perspective onsoftware estimation, give a few examples of DCGclients where implementing the centralizedestimation has paid off (15 minutes max, fewslides). Mention Ford, Euroclear

BG Questions sent in before the meeting (if any) DG + MHrespond as

pre-assignedby BG

BG Any Questions from the audience? DG + MHrespond as

requested byBG

BG If other questions exhausted and time remains,start working through these questions (DG & MHcan have prepared slides to help with question if needed):

BG About 20 minutes before end: How is it you

typically engage with a new customer and whatdoes that look like?

Important of change managementImportance of process review and changeQuick Start AssessmentCall to Action: stay and sign up

MH then DG

BG Why is software estimation so difficult and so ofteninaccurate?

MH

BG How can we make software estimation easier andmore accurate?

DG

BG What stops people from implementing softwareestimation best practices?

MH

BG What are some ways to get around the challenge s? DG

BG We get our subject-matter experts to do ourestimates surely they know best what it willtake?

MH

BG We are moving towards agile development so wewont need software estimating any more, will w e?

DG

BG Does Experience really matter in Estimation whenyou go Statistical?

MH

BG I have heard that a point estimate is a bad thing?Is that true and if so why?

DG

BG What are the best inputs to use for softwaredevelopment estimation? Function points?Something else?

MH thenhandoff toDG to talk

about what

SEER canhandle

BG We have our own estimation parameters built intoan excel spreadsheet that our SMEs use. Is thatOK?

DG

2011 Copyright Galorath Incorporated 2

Opening Remarks:


3/105

Full program Opening Remarks: Item Who What First Answer 1. BG Introductions, logistics, etc 2. BG Format of meeting 3. DG Opening remarks, personal perspective on software estimation, give a few examples of Galorath clients where implementing the

estimation tool has paid off (15 minutes max, few slides). Mention Ford, Euroclear

4. MH Opening remarks, personal perspective on software estimation, give a few examples of DCG clients where implementing the centralizedestimation has paid off (15 minutes max, few slides). Mention Ford, Euroclear

5. BG Questions sent in before the meeting (if any) DG + MH respond as pre-assigned by BG 6. BG Any Questions from the audience? DG + MH respond as requested by BG

7. BG If other questions exhausted and time remains, start working through these questions (DG & MH can have prepared slides to help withquestion if needed): a. BG About 20 minutes before end: How is it you typically engage with a new customer and what does that look like? Important of change management Importance of process review and change Quick Start Assessment Call to Action: stay and sign up MH then DG b. BG Why is software estimation so difficult and so often inaccurate? MH c. BG How can we make software estimation easier and more accurate? DG d. BG What stops people from implementing software estimation best practices? MH e. BG What are some ways to get around the challenges? DG

f. BG We get our subject-matter experts to do our estimates surely they know best what it will take? MH g. BG We are moving towards agile development so we wont need software estimating any more, will we? DG h. BG Does Experience really matter in Estimation when you go Statistical? MH i. BG I have heard that a point estimate is a bad thing? Is that true and if so why? DG j. BG What are the best inputs to use for software development estimation? Function points? Something else? MH then handoff

to DG to talk about what SEER can handle

k. BG We have our own estimation parameters built into an excel spreadsheet that our SMEs use. Is that OK? DG l. BG What is the ROI for buying a software estimation tool? DG m. BG What differentiates the different software estimating tools on the market? MH n. BG Is it better to have a centralized estimating team or to give everyone access to the tools? MH o. p. q. r. s.



4/105

3. DG Opening remarks

How can we make software estimation easier andmore accurate?

2011 Copyright Galorath Incorporated


5/105

Some of the most important businessdecisions about a project are made at the

time of minimum knowledge and maximum uncertainty .


6/105

Estimation Is Critical

Estimating is critical for all kinds of systems Yet many treat is as a second rate process

Everyone estimates. Just most get it wrong anddont have a process

Having a repeatable estimation process is critical toboth estimating AND to successful projects

Estimation and measurement go hand in hand



7/105

Introduction: Software ProjectsMust Return Business Value

Economics is primarily a science of choice, andsoftware economics should provide methods foranalyzing the choices software projects must make.

Leon Levy Software projects should be based on choices that

provide the maximum business value to theorganization Eli Goldratt

Some say business value is not our problem While others generally need to perform benefit analysis

We need to start building systems that optimize the

business Make IT part of the solution



8/105

An Estimate Defined

An e s t i m a t e is the most knowledgeable statement youcan make a t a p a r t i c u l a r p o i n t i n t i m e regarding:

Effort / Cost

Schedule Staffing

Risk

Reliability

Estimates more precise with progress A WELL FORMED ESTIMATE IS A

DISTRIBUTION



9/105

What can a parametric model tellyou?

9 9

What islikely tohappen

Feel lucky?

Firm FixedPrice?

Understand the risk before you commit!


10/105

The Gap Between IT & Management Needs (Source: Fraunhofer)


Financial

Customer Business Process

Innovation / Growth

Vision & Strategy

Goals

Data CollectionMeasurementMetrics

AnswersQuestionsGoal Attainment

Software Development & Maintenance

6

GQM

PSM

CoBIT

Business/OrganizationLevel

ITGovernance

ITGovernance

Customer

Focus


11/105

Who Uses SEER?


12/105

How?We created a global I.T. estimation service

We combined software function point & estimation services in one place.We combined hardware and software estimates into one processWe created a global community of estimators (Europe, AP, North America)We create estimates based on a common toolset calledSEER .

28-Oct-11

Slide 12

The complexities of rolling out an IT Estimation ServiceCopyright 2010 Ford Motor Company. All rights reserved.

Software Estimation

Hardware Estimation

Development

Cost(SEER-SEM)

InfrastructureCost

(SEER-IT)

Overall projectCost

Request for Estimate


13/105

When? Types of Estimate.

28-Oct-11

Slide 13


From the Project Apollo:Portfolio Management

Release2 Module 2.3IT TCODefine Target

IT Vision andRoadmap

Set budgetallocationguidelines

Monitor App licati on

Portfolio ID &

Assess

RebalancePortfolio to

Create aMulti-Year ITPlan

PrioritizeCandidateand PlannedPrograms to

Optimize

Value

RefineBusiness

Case / ID&

Assess

Monitoringand Tracking Analysis Analysis

Update Application Ass et Bas e DesignDesignBuildBuildImplementImplement

KOKO

Identify Ideas andSelect Opportunities

ProgramPrioritization

AndSelection

Cycle Planning

Program/Project Execution

Define Business StrategyDefine IT Strategy

E5 E4

E1

Planning Estimates (likely requested by Consultancy group or Planners)E1 Initial estimate for budgeting. A finger in the air view of project costs.E2 Program/project funding estimate conducted. A refined estimate, used to reserve funding for a project.

E3 Program estimate used for Program approval, developed after detailed work has been completed to develop firm ben-efits and business case ( Long L ead Programs)

Project Estimates (likely requested by PM or lead BA)E4 Project deliverables estimate completed at the end of Analysis/Elaboration, when requirements are firmE5 Estimate/Function Point count based on completed deliverables. Feeds back into knowledge base.

Estimation

Points

Opportunityis created

and logged

Develop InitialBusiness Case for

Candidates

Projectis createdE3

E2

E1E2E3

E4E5


14/105

Why bother estimating? Who wins?Consultancy Group :

Consistent, reliable estimates will help you allocate budget efficiently.

Cycle Planning: what can be delivered within your budget constraintsSingle process (which this is already in place in the U.S.)

General Management:It is a key tool for IT Demand Management managing supply vs demandIt will enable us to analyse make vs. buy decisions.It drives Function Point counting, which in turn enables us to measure:

Quality (project defect rates), development costs per FP, support costs etc

All:It will reduce the reliance on SMEs for estimating project costs

It reduces complexity with no need to engage hw/sw/Architects separatelyIt is a no-cost service estimation team costs are not charged to your project.We all benefit from the experience of years of software development experience -which should improve your success rates on complex projects.It documents assumptions and allows re-use of estimates when project scopechanges, when a project gets deferred etc.

28-Oct-11

Slide 14



15/105

15

Fo r int e rna l use

6. Step3: Budget estimation & SEER => assessment of ITproductivity

As of J anuary 2010:

ITPAO started to c onduct Function Pointcounting a t 3 different stages during the projec tlife cycle

At BC : independent budget assessmentAt end DSG: monitor evolution of

functionalitiesAt project end: historical DB build-up

SEER-SEM usage is extended to those steps:At end DSG: with the intention to measure the

impact of functional changes on budget (incase of off-shored p rojec ts)

At project end: with the intention to furthertarget budget estimation of future projectsba sed on some key characteristics (calibration)Function Point counting results shared with PLs

FPA/SEER rolled-out through project life cycle withthe objective to assess development productivity:

FPA/SEER usage along the life cycle of the

project will allow to identify patterns per projecttypes thus allowing for a more detailed monitoringof the projec ts.

Post-launch, a comprehensive FPA is performedto doc ument the main characteristics of theproject, build-up a historical database and mostimportantly, assess the factors of changeimpacting the p rojec t; input to improve ourdevelopment.

Overall, data collected and analysed will allowto better anticipate issues per project type andtherefore contain/ mitigate risks i.e. better controlproject budget spend.

Post-ProjectDeployTestBuildDesignInitiate (BC)Initiate (Pre-BC)Feasibilitystudy

Main objective : budgetestimation independent

assessment

Main obj ective :productivity analysis &estimation assessment

Main objecti ve : scopebaselining i n terms of FPs

Function point calculationbased on DetailedFunctional Design

Function point calculationbased on High Level

Functional Design

Function pointcalculation based on

final Detailed FunctionalDesign

2 years ago 1 year ago Today Future


16/105

16

Fo r int e rna l use

7. Step4: Budget estimation & SEER => support tooutsourcing contracts discussions

As of Q3 2010:

FPA / SEER will also be used to validate the budget based on FP assessmentprovided by the outsourcers in the c ontext of our outsourcing strategy: at the initiation of the projec t at the end of the design phase.

It provides an independent benchmark and adds transparency to thenegotiation process.

Post-ProjectDeployTestBuildDesignInitiate (BC)Initiate (Pre-BC)Feasibilitystudy

Main objective : budgetestimation independent

assessment

Main objective :productivity analysis &estimation assessment

Main objective : scopebaselining i n terms of FPs

Function point calculationbased on DetailedFunctional Design

Function point calculationbased on High Level

Functional Design

Function pointcalculation based on

final Detailed FunctionalDesign

Independent benchmarkused to support

the negotiation process

2 years ago 1 year ago Today Future


17/105

Beyond Estimation

Metrics Analysis & Benchmarking Integration to Project Planning& Portfolio Systems

Project Monitoring & Control

AND MO REEnterprise Shared Database


18/105

e. What are some ways to get aroundthe challenges? DG



19/105

19

10 Step Software Estimation Process:Consistent Processes = Reliable Estimates

1. EstablishEstimate Scope

2. Es tab lis h Tec hn ic alBaseline, GroundRules, Assump tions

3. Collect Data

4. Estimate and Validate

Software Size

5. Prepare

BaselineEstimates

7. Quantify Risks andRisk Analysis

6. Review, Verifyand ValidateEstimate

8. Generate aProject Plan

9. Document Estimateand LessonsLearned

10. Track ProjectThroughout

Development


20/105

High Level Overview

Determine & Quantify

Benefits overtime

Determinetotal

ownershipcosts against

schedule

Determine if project

worthwhile onits own or as

part of aportfolio



21/105

g. We are moving towards agiledevelopment so we wont need softwareestimating any more, will we? DG



22/105

An Agile Approach to Planning

From: Agile Estimating and Planning by Mike Cohn1. Rapidly and reliably generating useful new capabilities

& knowledge

2. Flow of new capabilities & knowledge guides the work3. Plan for learning more than what end product is

4. Traditional projects emulate a 10K race You know where the finish line is

Try to get there as fast as possible

5. Agile projects are like a timed race see how far you can run in sixty minutes!


23/105

Why Agile Needs Estimation & PlanningAs Much or More Than Traditional

The 90% complete syndrome on Agile With just a little more time we can do so much more

When do you stop?

When do you kill an agile project that is failing withouteffort / schedule commitments

Cant do business case analysis if we dont know howlong & how much

Are you sure the project is agile & not justunplanned?

Lowest level feature planning by itself can makeprojects take longer

Velocity can be useful but may not forecast actualproject completion



24/105

Software & IT Systems Are AboutBusiness Value

Stakeholder Needs

Agile


25/105

I have heard that a point estimate is a badthing? Is that true and if so why? DG



26/105

Human Nature Optimism in

EstimatesH B R A r t i c l e e x p l a i n s t h i s P h e n o m e n o n :

Problem: Humans seem hardwired to be optimists We routinely exaggerate benefits and discount costs

Delusions of Success: How Optimism UnderminesExecutives' Decisions (Source: HBR Articles | DanLovallo , Daniel Kahneman | Jul 01, 2003)

26

Solution: Temper with outside view

Viable estimation is temperingPast Measurement Results t raditional forecasting w/ statisticalParametrics Can help

Dont remove optimism, but balance optimism & realism


27/105

Contractors At Least Level 3 Would Be

Acceptable

Level0

Informal or noestimating

Manual effortestimatingwithout a

process

Manual effortestimatingwithout a

process

Level1

Direct TaskEstimationDirect TaskEstimation SpreadsheetsSpreadsheets

Ad HocProcess

Level2

FormalSizing(e.g.

functionpoints)

DirectTask

Estimation

DirectTask

Estimation

Simple model(Size *

Productivity)or informal

SEER Use

Simple model(Size *

Productivity)or informal

SEER Use

Somemeasureme

nt & analysis

InformalProcess

Level3

FormalSizing

RobustParametricestimation

(SEER)

RobustParametricestimation

(SEER)

Estimate vs.actual capture

FormalizedMultipleEstimateProcess

Rigorousmeasurement

& analysis

Parametricplanning &

Control

RiskManagement

Repeatableprocess

Level4

Formal sizing Repeatableprocess

Robustparametricestimating

(SEER)

Robustparametricestimating

(SEER)

Rigorousmeasurement

& analysis

Parametricestimation

with tracking& control

RiskManagement

Processimprovement

via lessonslearned

Level5

Formal sizing Repeatable

process

Robustparametric

estimating(SEER)

Robustparametric

estimating(SEER)

Rigorousmeasurement

& analysis

Parametricestimation

with tracking& control

Risk

Management

Continuousprocess

improvement

Why should we care? Maturity is related to estimate viabilityBetter estimation process more likely to be successful in

execution


28/105

j. BG What are the best inputs to

use for software development estimation?Function points? Something else? MHthen handoff to DG to talk about whatSEER can handle



29/105

SEER Handles Any Sizing FromStandard To Organization Specific



30/105

SEER Provides Numerous Size Measures ANDAllows Organization Specific



31/105


32/105

COTS Estimation (ExistingComponents)

32


33/105

Galoraths - Function Based Sizing

33

i f C d C A f


34/105

Lines of Code Can Be Accurate If Definitions Are Consistent

34


35/105

K. We have our own estimationparameters built into an excel spreadsheetthat our SMEs use. Is that OK? DG



36/105

Basic Estimation Tribes (Adapted fromDCG/Galorath Webinar)

1. Napkins Ad-hoc, hero driven estimators past successes arelegendary napkin scribbling taken as gospel

2. Guts Feeling /experience oriented project managersyears of experience, good and badtrusted regardless of actual results

(tenure benefits of being a survivor)3. Spreadsheets Former Napkins and Guts translating tribal

knowledge onto spreadsheets Bestows mathematical accuracy and empirical integrity on home-

grown estimation algorithms

Tribal estimation knowledge can and does work However, it comes with high risk and cost

Rarely repeatable

Consistency is sporadic

Heroic energy is kept in reserve used to mitigate risk

Knowledge almost never institutionalized outside of personalknowledge and desktop PC files

Knowledge lost when heroes retire or leave


B l i g R & S h d l I


37/105

Balancing Resources & Schedule IsA Science

For a given Size, Complexity and TechnologyMinimum Time

To Complete(Effort Increases

to Reduce Schedule )

Work ExpandsTo Fill Time

(Effort Increases

due to lack of pressure )

Minimum Time

Optimal Effort(Lower Effortfor LongerSchedule)

Calendar Time

E f f o r t

M o n

t h s

Effort Increasedue to Longer Schedule

S ft E ti ti B i M d l &


38/105

Software Estimation Basic Model & Associated Metrics

38

Start Finish

E f f o r t K

S i z e S t R e u s e D I T

S i z e S e ( w o r k u n i t s )

S i z e ( Ef f e c t i v e Se

& T o t a l St )

D e f e c t s C o u n t ( Q i Q r )

Ca l e n d a r T i m e

O n - g o i n g I t e r a t i o n s o f Effort ( A C W P o r Sp e n t )

Progress ( B CW P o r Ea r n e d Va l u e ) Defects ( Q i Q r )

Growth ( S g )

SoftwareDevelopment

Process

S t a f f a &

C o n s t r a i n t s

Ef f e c t i v e T e ch n o l o g y Ct e

People Process

Technology

S t a k e h o l d e r R e q u i r e m e n t

s D e l i v e r e d So f t w a r e

Effective complexity

D

Maintenance/

Block ChangeDevelopment

Process

D e v e lo p m e n t L e g a c y ,

M a i n t e n a n c e S p e c i f i c s & C o n s t r a i n t s

a n d / o r B l o c k Ch a n g e s

A s R e d e v e l o p m e n t


39/105

Poor Estimates Effect on Projects

Inaccurate estimates can reduce project success: Poor implementations

Critical processes dont scale

Emergency staffing

Cost overruns caused by underestimating project needs

Scope creep Forever changing project goals

Frustration Customer dissatisfaction

Cost overruns and missed schedules

Project Failures

Important project business decisions made early withminimum knowledge & maximum uncertainty

39

Why should we care: Poor estimates and plans area root cause of program failure

Impact of Individual Personnel and


40/105

Impact of Individual Personnel andTeam


122%100%

75%55%

0%

20%

40%

60%80%

100%

120%

140%

Fully Engaged Engaged Somewhat Engaged Disengaged

P r o

d u c

t i v i t

y

L e v e

l

Individual Productivity By Level of Engagement

Personnel Differences Make Big Project


41/105

Personnel Differences Make Big ProjectDifferences (SEER Uses Team Effectiveness)

Level Characteristics Impact Actions Possible

FullyEngaged

Passionate about work & organization Will do whatever it takes to deliver results Feels like a true owner Delivers consistent, high quality results Finds innovative solutions to the toughest

problems Seen as role model & leader

122% Manager Action Cankeep about 100%fully engaged

Engaged Focused on their deliverables, project, andindividual responsibilities

Delivers good, solid performance Always does their fair share of work Works well with others and will help out

when asked

100% 1 IN 5 CANINCREASE TO FULLYENGAGED

SomewhatEngaged

Selective about where they put theirenergy

Spends a lot of time doing things that arenot helping customers or the organization

Delivers when they have to, or when youare watching

Does what it takes to get by

75% 1 IN 2 CANINCREASE TOENGAGED

Disengaged Only works when they have to, they reallywant to be doing something else

Results are NOT meeting the standard Has a We They perspective Negative about the organization, as well as

in their interactions with co-workers and

customers Master at distracting others

55% 1 IN 10 CANBECOME ENGAGED



42/105

L. What is the ROI for buying a softwareestimation tool? DG


Estimate Total Ownership Costs;


43/105

Estimate Total Ownership Costs;60+% Can Be Infrastructure & Services

Software Development

Software Maintenance

IT Infrastructure IT Services


Impact of Individual Personnel and


44/105

Impact of Individual Personnel andTeam


122%100%

75%55%

0%

20%

40%

60%80%

100%

120%

140%

Fully Engaged Engaged Somewhat Engaged Disengaged

P r o

d u c

t i v i t y

L e v e

l

Individual Productivity By Level of Engagement

Personnel Differences Make Big Project


45/105

Personnel Differences Make Big ProjectDifferences (SEER Uses Team Effectiveness)

Level Characteristics Impact Actions Possible

FullyEngaged

Passionate about work & organization Will do whatever it takes to deliver results Feels like a true owner Delivers consistent, high quality results Finds innovative solutions to the toughest

problems Seen as role model & leader

122% Manager Action Cankeep about 100%fully engaged

Engaged Focused on their deliverables, project, andindividual responsibilities

Delivers good, solid performance Always does their fair share of work Works well with others and will help out

when asked

100% 1 IN 5 CANINCREASE TO FULLYENGAGED

SomewhatEngaged

Selective about where they put theirenergy

Spends a lot of time doing things that arenot helping customers or the organization

Delivers when they have to, or when youare watching

Does what it takes to get by

75% 1 IN 2 CANINCREASE TOENGAGED

Disengaged Only works when they have to, they reallywant to be doing something else

Results are NOT meeting the standard Has a We They perspective Negative about the organization, as well as

in their interactions with co-workers and

customers Master at distracting others

55% 1 IN 10 CANBECOME ENGAGED


anua st mates: uman easons orE (S f A l i & M


46/105

Error (Software Analysis & MeasurementCan Help)

Desire for credibility motivatesoverestimate behavior (80%probability?) So must spend all the time to be reliable

Better approach force 50% probability &

have buffer for overruns Technical pride causes underestimates Buy-in causes underestimates

46

An ROI Analysis of A New System:


47/105

y yShould We Fund This?


Can we do better? Will stakeholders tolerate a

loss for 3 years? What is the risk?

Example of Analysis of Estimation


48/105

p yProcess & Tools Analysis


Human Nature Optimism In


49/105

Human Nature Optimism In

SoftwareH B R A r t i c l e e x p l a i n s t h i s P h e n o m e n o n :

Problem: Humans seem hardwired to be optimists We routinely exaggerate benefits and discount costs

Delusions of Success: How Optimism UnderminesExecutives' Decisions (Source: HBR Articles | DanLovallo , Daniel Kahneman | Jul 01, 2003)

49

Solution: Temper with outside view

Software Analysis & Measurement can temper

Viable estimation can temper

Dont remove optimism, but balance optimism & realism

Software Projects Must Return


50/105

Business Value Economics is primarily a science of choice

software economics should provide methods for analyzingthe choices software projects must make. Leon Levy

Base choices on those providing the maximum businessvalue to the organization Eli Goldratt

Measurement and its uses such as estimating and defectanalysis help this science of choice

Some say business value is not our problem

While others generally need to perform benefit analysis We need to build systems that optimize the business

Make IT part of the solutionIf IT & measurement dont generate sufficient profitmoney will go elsewhere

Looking At Maintenance and


51/105

Technical Debt


Conclusions


52/105

Conclusions

Viable estimation process & tools yield significant ROI Rigor in the process helps make the best decisions Viable estimation reduces the surprises management

so dislikes Standardized measures and tools provide:

Ability to benchmark

Should cost analysis both internally and with vendors Ability to quantify internal & external improvements



53/105

DCG - Galorath Estimation Workshop

October 27, 2011

Why Do We Care About


54/105

Copyright 2010. David Consulting Group

. . . because your customers pay the bills.

. . . because selling a current customer somethingnew costs a lot less then getting a newcustomer.

. . . because in the long run ALL customers canchange who are servicing their needs.

. . . because happy customers promote yourorganization.

External Customer Satisfaction?

Why Do We Care About


55/105


. . . because your customers pay the bills whetherdirectly or through intra-department

relationships.. . . because in the long run internal customers can

change who are servicing their needs.

. . . because we all are supposed to be working forthe same organization.

. . . because happy customers lead to lessinfighting and quieter relationships thereforehigher value.

Internal Customer Satisfaction?


56/105


Do internal and external customers have

the same expectations for: Dates? Cost? Profit Margins? Resource Utilization? Quality?

Differing Expectations?

Do we need to predict how wewill perform on these topics?

How Are Customer Satisfaction and


57/105


Ask yourself if success is a result of planning orluck?

Estimation Related

If success is going to be more than luck DCGsuggests the following steps:

1. Estimate performance

2. Plan to deliver3. Monitor and measure performance4. Perform a retrospective when done

Customer Satisfaction is a suitcase that when unpackeddescribes many of the components that must be estimated,planned, monitored and measured.

B d i E i i Pl i


58/105


Budgeting v Estimation v Planning

Budgeting

Defines how much we have to spend and influencesscope

Tends to ignore the cone of uncertainty

Estimation

Rough or approximate size extent or nature

Focused by the cone of uncertainty

Planning

Definition of tasks and allocation of resources

Focused on the narrow part of the cone of uncertainty

Pl i O i d Ti i


59/105


Planning Onion and Timing

Budgeting

Estimating

Planning

Mike Cohns Planning Onion

Analogy

Parametric - Light

IntegrationPlanningStandup Meetings


60/105

When Is An Estimate A Commitment


61/105


When Is An Estimate A Commitment

An estimate stated as a single number is a

COMMITMENT if there is no understanding thatan estimate is a prediction therefore will vary.

Estimates are:

Price Commitment


62/105

Example Implementations


63/105


Ford, Detroi t Used FP Counters from DCG since 2005 for project metric

normalization Acquired SEER for Software Realized that FP counts are an excellent input to SEER Converted DCG consultants to central team for project

sizing/estimating

Euroclear Use Galorath to do estimates

Galorath use DCG to do FP counts

Example Implementations


64/105


DIFFICULT & INACCURATE?Estimating

So Why Doesnt This Stuff Always Work?


65/105


So Why Doesn t This Stuff Always Work?

Consistency of

Method

Self Knowledge

Uncertainty

Effective Estimating


66/105


Effective Estimating

The Software Engineering Institute (SEI)requirements for good estimating: Corporate historical database Structured processes for estimating product

size and reuse Mechanisms for extrapolating benchmark

characteristics of past projects Audit trails Integrity in dealing with dictated costs and

schedules

Data collection and feedback processesfoster correct data interpretation

What Defines Good


67/105


What Defines Good

Hitting the midpoint of the original range? Making your stakeholders happy? Assuring a correct profit margin?

A good estimate is an estimate that provides aclear enough view of the project reality to allowthe project leadership to make good decisionsabout how to control the project so that it hits itstarget. McConnell

Impact of Under and Over Estimating


68/105


Impact of Under and Over Estimating

Under and Over Estimation

100%100%

ParkinsonsLaw

PlanningErrors,Compression,

Defects,IncreasedRisk

NegativeImpact

Make It Smaller


69/105


Make It Smaller

Pressure comes from many angles. Internalpressure comes from natural optimism and aneed to please.External comes from business pressures.

EstimateWith most pressure beingfocused on reducing theit is not surprising that mostestimates are low.

Why Estimate Better


70/105


W y

Improved ability to provide status Higher quality (better behavior) Better coordination between teams Better long term budgeting

Early warning risk information Indicates we have mismatches between targets,estimate and commitments

Better than guessing

Accuracy v Precision


71/105


y

Accuracy describes the nearness of a measurement to thestandard or true value.

Precision describes how exact the number is.

Pi = 3 accurate to one digit but not precise Pi = 3.37882 precise to 6 digits but not accurate Pi = 3.14159 accurate and precise to 6 digits.

Wikipedia


72/105

Cone of Uncertainty / Estimation Funnel


73/105


y

+Estimate too

high 1.5x

2x

1,25x

.8x.67x

.5x

*Probability

.25x

4x

P1 P2 P3 P4 P5

Actual-

Estimate toolow

EstimationFunnel

Strategy

Knowledge transforms your ability to accurately

predict and control projects .


74/105


WHAT STOPS PEOPLE?Estimating


75/105

Estimation Flavors


76/105


Parametric

Decomposition/Recomposition J udgment

Hybrid

Methods

McConnells Count, Compute, Judge Advice


77/105


Count if possible Must be available earlier rather than later

Must be correlated to the scope of what you are estimating Compute when done counting

Convert what you have counted into an estimate using history . .. form somewhere

History has to relate to what you count

J udge as a last resort Only when you dont have data (pressure will be downward away

from reality)

Judgment


78/105


Estimation Methods based on expert opinions afterconsideration or deliberation Structured based defined methods (Delphi) Unstructured

Viability is very dependent on the expertise of those

involved. It is usually recommended that estimates aredeveloped by the people doing the work Effectiveness and accuracy are enhanced when multiple

estimates are developed and synthesized Common estimate weighting plans include 1/3/2 or 1/4/1

(H, M, L)


79/105


80/105


DOES EXPERIENCE REALLYMATTER?

Estimating

Historical Data


81/105


Yours, Ours, Mine? Industry, Organization, Project/Program

Historical data is important because:Past predicts the future!

Use the data to calibrate and refine based onperformance magnitude of relative error

Estimation Flavors


82/105


Parametric

Decomposition/Recomposition J udgment

Hybrid

Methods


83/105


BEST INPUTS?Estimating

A Basic Estimating Model


84/105


PROJECTSIZE XX

RISKFACTORSPROJECTCOMPLEXITY

DEFINITION CAPABILITY ESTIMATE

S c h

e d ul e

Effort CostsREQUIREMENT

Estimating Influences


85/105


PROJECTSIZE XX

RISKFACTORSPROJECTCOMPLEXITY

DEFINITION CAPABILITY ESTIMATE

S c h

e d ul e

Effort CostsREQUIREMENT

Basics: Inputs and Outputs


86/105


Inputs Sized Requirements Skills People Equipment

Estimate Outputs Effort Cost

All data should be leveraged when planning

Planning Outputs Schedules Resource Allocation

Basic Estimation Model


87/105


Functional Size Function Points Use Case Points Story Points Tee Shirt Sizing

Behavioral

Attributes Process Disciplines Sourcing Model Skills

Technical

Complexity Security Languages Architecture

Productivity and DurationSignatures

What How With What

Why Is Size Important


88/105


Team Size

RiskManagementImportance

Size: Internal v External Definitions


89/105


Organizational Specific

DefinitionsIndustry Defined

Modules,

Use Cases,Test Cases

Story

Points

Lines of

Code

Use Case

Points

Cosmic,

NESMA FPIFPUG Function PointsMark II

Size: Accuracy


90/105


Less Accurate More Accurate

Hours,Days

StoryPoints

Lines of Code

Use CasePoints

COSMICNESMA FP

IFPUG Function PointsMark II


91/105

Complexity


92/105


Language Architecture Technologies Logical and Mathematical Algorithms

Data Relationships Code Structure Performance Memory

Economies and Diseconomies of Scale


93/105


Diseconomies Team Size Project Size Process Maturity Architecture Team Cohesion

Economies Product Complexity Personal Capabilities

Diseconomies become morepronounced as projects get bigger.


94/105

Deciding Estimates based on riskMonte Carlo Analysis


95/105


Monte Carlo methods are a class of computational algorithms thatrely on repeated random sampling to compute their results.

Monte Carlo methods are often used in simulating physical and

mathematical systems. Monte Carlo simulation methods are especially useful in studyingsystems with a large number of coupled degrees of freedom

Monte Carlo methods are useful for modeling phenomena withsignificant uncertainty in inputs, such as the calculation of risk inbusiness.

It is a widely successful method in risk analysis when compared withalternative methods or human intuition.

Deciding Estimates based on riskMonte Carlo Method(s)


96/105


There is no single Monte Carlo method; instead, the term describesa large and widely-used class of approaches. However, theseapproaches tend to follow a particular pattern:

Define a domain of possible inputs. Generate inputs randomly from the domain using a certain specified probability

distribution. Perform a deterministic computation using the inputs. Aggregate the results of the individual computations into the final result.

For our purposes, we can consider two methods for Define adomain of possible inputs for our software estimates: User selects a range of possible values for an input parameter (explicitly or by

comparison) and specifies a likely distribution within that range (may default touniform) e.g. size may be uniformly distributed between 90FP and 110FP

(based on a count of 100FP) User specifies or system defaults to a distribution around a single selected value

(e.g. Size is normally distributed around 100FP)


97/105

Deciding Estimates based on riskMonte Carlo Method - Example

I l M C l h d ld d l


98/105


In a real Monte Carlo method, a program would use many random valuesbut for our purposes, lets just take a few values within a uniformlydistributed range that we are going to select.

Imagine that we have some prior experience estimating apple picking andwe have a sense that there will be some variation in our input values e.g.

Estimate Min Max

Number of Apple Trees 20 20 20

Number of Apples per Tree 100 20 150 Time to pick one basket 10 5 12

Number of Apples per basket 40 38 43

If we imagine that, with a uniform probability distribution for ease of manualcalculation, any of thesee values is equally likely, then there are 3 4 (= 81)possible values for total picking time. We can plot these.

Monte Carlo Method ExampleBasic Plot of all 81 points - doesnt help much

1000.00


99/105


0.00

100.00

200.00

300.00

400.00

500.00

600.00

700.00

800.00

900.00

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 73 75 77 79 81

Monte Carlo Method ExampleStill happy with 500 minutes?


100/105


0

2

4

6

8

10

12

14

16

Time 0-99 Time 100-199Time 200-299Time 300-399Time 400-499Time 500-599Time 600-699Time 700-799Time 800-899Time 900-999

Probability Distribution

Monte Carlo Method ExampleHow about now? Is 65% confidence enough?

Cumulative Probability


101/105


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Time 0-99 Time 100-199

Time 200-299

Time 300-399

Time 400-499

Time 500-599

Time 600-699

Time 700-799

Time 800-899

Time 900-999

Cumulative Probability


102/105


AGILE?Integrating Estimation Techniques: Hybrid Agile Estimation


103/105

Translating Size Into An Estimate


104/105


Input: SizedBacklog Disaggregationand Resize

Planning Poker

(factor inattributes andtechnical )

And /or

EstimationWorksheet

Sprint Planning(Productivity or

Commitment

Driven)

Quick Overview Of Sprint Planning Session

ProductOwnerPrioritizes


105/105


Product Owner Prioritizes

Teams Sequence

Apply Estimates

Team Commits

Documents

New York Galorath DCG Event Oct 2011 Combined Slides