Upload
kaelem
View
29
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Developing An Early Project Cost Estimate: An Applied Case Study From Project Concept through Contract Award . Linda Esker Kathleen Dangle Fraunhofer Center Maryland. Project Overview The Early Project Cost Estimate Estimate Phases Challenges Strategy - PowerPoint PPT Presentation
Citation preview
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
Developing An Early Project Cost Estimate:
An Applied Case Study From Project Concept through Contract Award
Linda EskerKathleen Dangle
Fraunhofer Center Maryland
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
2
Topics Covered
• Project Overview
• The Early Project Cost Estimate
• Estimate Phases– Challenges– Strategy– Approach: Methodology and
Models Used– Decisions to go forward
• Summary / Lessons Learned
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
3
Project Overview
• A large Ground Space Communication System
• COTS-intensive– For both HW and SW
• 5-year project duration
• Many areas highly scientific– Need appropriate technical expertise as well as those who
understand what is involved in the cost estimate, especially SW– Would require a blended estimation team
• Development of the estimate began in the earliest initial concept phase
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering 4
Cost Estimate Began Before Typical 1st Milestone
RFP
Life-Cycle Phases
Reviews
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
5
The Early Project Cost Estimate
• Cost estimate developed by the project to estimate government and contractor project costs for budget/funding approval
• Activity played a significant role in the development of the project– Value of the activity included not just the resulting estimate, but also drove
understanding, evolution, and “selling” of the project concept
• State of the practice for early project estimates:– Typically use general, high-level, “personal” heuristics/ rules of thumb by
subject matter experts
– Not able to effectively use estimation tools because details required as inputs are not known early in the project
– Process followed not as disciplined or structured (or repeatable) as expected in estimates during later phases of project lifecycle
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
6
The Early Project Cost Estimate (cont.)• The good news: With government emphasis on
better project estimates, projects now need a basis of estimate and confidence levels
• The bad news: Solid estimates face challenges that make use of estimation tools an uphill battle
• Politics — Organizational budgets have a life of their own, are highly competitive, and defy comprehension for those not in the political know
• Personal opinion and aggressive (or naive) optimism, “This should be easy…”
• Distrust of what they do not understand
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
7
Early Project Estimate Phases
“In the Beginning…”
•Forming the concept
“Let there be light…”
•Maturing the project’s architecture and requirements
•Defining a project lifecycle & WBS
•Defining the estimation process and models
“And it was good…”
• Iterating on the architecture, schedule, models, and estimate
•Evaluating options and “what ifs”
1 2 3 4
Lessons Learned
Lessons Learned
Lessons Learned
“And it was better…”
•Formalizing Estimate Confidence
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
• Started with:– Top-down approach– Fully understanding of the scope – Understanding SW and HW requirements of the project– Examined use of a commercial estimation tool
• Found it was difficult to use at this very early stage– Needed more details than what was available
– Input was constantly changing as project investigated technologies and needs and tool data cumbersome to update
• Management did not trust estimation tool results– Wanted greater insight into and control on how estimate was determined
– Wanted costs broken down into their areas of interest
Early Project Estimate Phase 1
8
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
Early Project Estimate Phase 1
• Concept studies performed to develop notional architecture
• Estimate used: • General parametric models
• Expert judgment
• COCOMO
• Spreadsheets
• HW: Developed MEL (Master Equip. List)
• SW: Used analogies/LOC• Percentages used to estimate many
areas:• Management
• Contingency, reserve & inflation
• Spread of labor, HW, SW by fiscal year
• Other technical unknowns
9
Approach
Challenge:Create initial estimate with
minimum information
Strategy:Formulate
concept
Lessons Learned
Initial informal “off-the-cuff” estimates can derail a project from the start
Do not count on Mgmt. acceptance of tool estimate results
Phase Start: Blank page; immaturerequirements; forming team; gatheringhistorical data; top-down approach
Decision:Next use
bottoms-up approach for
greater accuracy
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
• Enhanced the Estimate with some structure: a notional architecture, schedule, and bottoms up-analysis
• Stabilized requirements to refine the HW and SW estimate
• COCOMO schedule and effort distributions allowed us to distribute costs over the timeframe to allocate budget to fiscal years
• COSYSMO could now be used to validate engineering SME estimateo Needed to use requirement expansion factors
• Project still going through definition and needed to adjust for changes
Early Project Estimate Phase 2
10
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
Early Project Estimate Phase 2
• Migrated approach from parametric overlay to bottoms up
• Established physical notional architecture to organize costs
– Approach helped drive engineer thought process
– Estimate used:
– Expert judgment
– Analogous estimation - some historical data
– COCOMO for SW effort & duration
– % still used to estimate
– Management (based historical data)
– Contingency, reserve, inflation, & unknowns
– Spread of labor, HW, SW by fiscal year
11
Approach
Challenge:Establish
sound basis for good estimate
Strategy:Use bottoms- up approach; define models;HW/SW by arch
Lessons Learned
Hybrid of bottoms-up & parametric modeling works best
Everyone understood what was known (justifiable) and unknown
Phase Start: Immature requirements; notional architecture beginning to evolve
Decisions:Pursue 2
independent paths (dev., deploy.); focus
on things missing
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
Estimation Model
Iterative Development of the Estimate
12
WBS(By Notional Architecture)
RESOURCESHardware– Antenna
o Antenna part 1o Antenna part 2
– Computerso Server o Workstation
• • •
Software– Software item 1– Software item 2
• • •
Engineering tasksIntegration TasksTransition tasks
COCOMO Model Effort,
Duration
Items,Costs
SLOC
Effort
Notional System Architecture
Project Lifecycle Schedule
Master Resource Sheet of
• Hardware• Software• Engineering Effort• Integration Effort• Transition Effort
$0
$10,000,000
$20,000,000
$30,000,000
$40,000,000
$50,000,000
$60,000,000
$70,000,000
Labor, Material, Cost Phasing
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
• Areas of Focus– Identifying a good system model on which to base confidence in
the estimate—having a solid Basis of Estimate (BOE)
– Ensuring Implementation schedule is realistic
– Understanding changing expectations and being flexible to deal with change
► Filling voids in analogous systems by using SW SMEs to provide sizing information
► Using a notional system to obtain sufficient information for an early project estimate and trying to avoid over-engineering the perfect system
• Key Accomplishments– Met goals of project: Early Project Estimate for RFP; presentation
to HQ; understandable/supportable basis for budget
Early Project Estimate Phase 3
13
– SW:
– HW:
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
Early Project Estimate Phase 3
• Incorporated inputs from Trade Studies and another independent cost estimate
• Aligned cost structure, schedule, WBS, and notional architecture
– PM worked with all technical teams to understand basis of estimate (BOE)
– Loaded Resources by skill level & labor rates– Spread labor, HW, SW costs by fiscal year based
on schedule– Only PM remained %– Enhanced estimate model to allow:
• Many different views (architecture elements, high cost drivers)
• Investigation of options ("what if" a ground station were eliminated?)
14
Approach
Challenges:Honing the estimate;
Strategy:Use 2 separateTeams (dev. &
deploy.); merge results
Lessons Learned
BOE review sessions need a strong driver
The better the estimate fidelity the more useful for “holes” & “what ifs”
Working as a team creates buy-in and project team is much smarter
Phase Start: Matured requirements & team; near complete notional architecture
Decisions:Ready to go;
freeze estimate for RFP
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
Early Project Estimate Phase 4
• Performed Risk Cost Analysis• Focused on high cost drivers
– Optimistic– Most likely – Pessimistic estimate
• Used triangular distribution and Monte Carlo to simulate confidence ranges
• Compared dispersions with expected ranges
• Detailed resource items enabled analysis of procurement long lead items & phasing
15
Approach
Challenge:Understand
Confidence in Estimate
Strategy:Examine Estimate
Cost Risks
Lessons Learned
A solid early Project cost estimate enables developing confidence levels and understanding where the estimate falls within the levels before contract start
Phase Start: RFP Released; Project estimate frozen; fewer distractions
Decisions:Project has
confidence with expected
range
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering 16
COCOMO Lessons Learned
• At this phase, the summary COCOMO tables are very helpful. However,
– They do not give consistent information
– Percentages do not add up to 100%
– Total effort has two different values
• This makes them appear to be unreliable
– Had to make adjustments
Module Name SampleTotal Size 37975Total Effort 230.308313
Overall Schedule (%)Schedule (Months) Effort (%) Effort Staff
Plans And Requirements 22.53% 14.763379 7.00% 16.121582 1.091998Product Design 27.27% 17.864799 17.00% 39.152413 2.191595Programming 42.93% 28.130345 54.20% 124.82886 4.437516Integration and Test 29.80% 19.524292 28.80% 66.327036 3.397155TOTALS 122.53% 80.282815 107.00% 246.4299 11.118264
EFFORTPlans and
RequirementsProduct Design Programming
Integration and Test TOTALS
Requirements Analysis 7.211762 4.894052 4.993155 1.658176 18.757145Product Design 2.842752 16.052489 9.986309 3.316352 32.197902Programming 0.929637 5.337729 70.528308 26.220951 103.016625Test Planning 0.666338 2.401298 7.031867 2.078163 12.177666Verification and Validation 1.230594 2.988584 10.776733 18.63815 33.634061Project Office 1.972248 3.810935 7.323452 4.554541 17.661176CM/QA 0.462173 0.926657 7.947597 5.217811 14.554238Manuals 0.806079 2.740669 6.241443 4.642893 14.431084TOTALS 16.121583 39.152413 124.828864 66.327037 246.429897
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering 17
COCOMO Lessons Learned
• Modular approach of COCOMO products is ideal
• Project had definite need for a COCOTS-like tool, but – Seemed too immature
– Decisions on COTS were not controlled by government
• Overlap of COSYSMO and COCOMO II makes management wary of using them together– Need some better quantification of the overlap in the
estimate
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
Start early: Plan for iterations of the estimate– The project estimate will iterate through phases
and different estimation techniques as more information is learned
– The estimation process can aid in the maturation of the project concept, requirements, and schedule
Summary / Lessons Learned
18
Be Adaptable/Flexible: Standard tools cannot handle all of management’s needs
– Early on there is not a clear definition/agreement on how to proceed and implement the project; need to change and adapt
– Be sure your cost estimation tools are flexible• Need to handle what is known about the project at early and also later phases
of the evolution• COTS may not match all your needs—a hybrid approach works well
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
Be Mindful of Management: Need to work with high-level management and keep them in the loop
– Early “off-the-cuff” promises can derail a thorough estimation process
• Don’t promise too much too soon—early promises cannot be rescinded
Summary/Lessons Learned
19
– It is imperative that the higher management teams have faith in the estimation work
• You need to thoroughly understand (and justify) how a tool’s model handles all aspects of the estimate
– Even if not accepted for an end result, use tools to also give a sanity check for any ad hoc estimation practices
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
WBS is Critical: Projects need a WBS that can help them collect data and learn
– Lobby for better WBS to facilitate data and information collection when necessary
Summary / Lessons Learned
20
– Current WBS templates/guidelines make it difficult to extract software from other parts of the system development work
• Project was afraid to specify that WBS differentiate between software and other activities
• Would love to see this addressed as a SW best practices
© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering
For more information contact us at:
Linda [email protected]
Kathleen Dangle [email protected]
21
Questions