U.S. Army PEO STRI Cost Management Implementation · U.S. Army PEO STRI Cost Management...

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 1 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence

U.S. Army PEO STRI Cost Management

Implementation Anthony A DeMarco

PRICE Systems, LLC

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 2

Outline

Who - PEO STRI Overview – Dr. James T. Blake, SES, Program Executive Officer

Why - Improving Software Cost Estimates – Mr. James A. Golden, Associate Chief Financial Manager – Cost

How – Cost Management Implementation Details – Anthony A DeMarco, President, PRICE Systems

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 3

Washington

Dr. James T. Blake, SES

Program Executive Officer

for

Simulation, Training, and Instrumentation

United States Army

24 October 2013

Approved for public release; distribution is unlimited.

Vision: To be the Nation’s Acquisition and

Contracting Center of Excellence for Training, Testing, and Simulation Solutions.

Mission: Acquire and Sustain Training, Testing,

and Simulation Solutions in Support of the Nation.

Strategic Goals

as of OCT 2012

1.0 - Provide STRI capabilities

to support Decisive Action

Operations

2.0 - Ensure capabilities meet the needs of the

Integrated Training and T&E

Environments

3.0 - Develop a workforce with

expert level acquisition,

technical, and leadership skills

4.0 - Promote and sustain

excellence in the execution of contracting

activities

6 Approved for public release; distribution is unlimited.

7

Simulation Is Safe, Saves Time, Money, Fuel and the Environment

Safety – less training injuries

Minor Fuel Needs

No Ammo Required

Limits Environmental Impact

▫ Air Pollution

▫ Noise Pollution

▫ Terrain Damage

Inexpensive to Operate

Simulation Enhances Live Training Approved for public release; distribution is unlimited.

• Army Training Inventory

• 335,000 training devices

located at over 600 sites

worldwide

• Sustained by 7,000 people

• Valued at $3.9B

• Life-Cycle Management Plan is

required to support training

systems

• MDEP WCLS is PEO STRI’s

sustainment budget line for

Army accepted TADSS

• Field OPS operates 24/7 around

the world – “the sun never sets

on their support”

Life-Cycle Support

Iraq – 0 (As of 31 Aug 13)

Afghanistan – 759

Kuwait – 63

• 850 Field OPS Contractor personnel deployed in SWA at over 25 locations.

• Since March 2003, we have supported over 2,126,000 individual soldier training events in

SWA.

Joint Forces: Army, Navy, Marine Corps, Air Force and SOCOM.

11+ Coalition Nations including the United Kingdom, Australia, Italy, Norway, Poland, France,

Netherlands, Lithuania, Canada, New Zealand and Germany.

More than 560,000 Iraq Army and Afghanistan National Army soldiers trained to date.

Approved for public release; distribution is unlimited.

Foreign Military Sales

52 Active Country

Programs

NZ

Open Cases 204

Open Requisitions 1528

FY13 $ Value $ 451M

Total Value $2.11B

Foreign Military Sales - Across the World

Approved for public release; distribution is unlimited.

Mr. James A. Golden Associate Chief Financial Manager – Cost

Program Executive Office for

Simulation, Training, and Instrumentation

24 October 2013

Approved for public release; distribution is unlimited.

11

Approved for public release; distribution is unlimited.

• Problem: Software development cost estimates may be

overstated by 30-56%, negatively impacting ability to provide

timely and affordable capabilities

• Goal: Significantly reduce growth factors in software cost

estimates:

• Improve AFFORDABILITY of software-intensive programs.

• Deliver more CAPABILITY; eliminate arbitrary growth factors.

• Decrease CYCLE TIME to develop software cost estimates

• DEFENDABLE and CREDIBLE software cost estimates

• Develop solution set using Lean Six Sigma (LSS) process by

initiating a LSS Black Belt project to improve software cost

estimates at PEO STRI

• In Scope

• PEO STRI ACAT II and III programs with a software

development requirement, including:

• All appropriations (RDTEA, OPA, OMA)

• All software development effort (e.g., P3I, PPSS, PDSS)

• Contractor software data

• PM software data

• PEO STRI software data collection process

• Out of Scope

• ACAT I and Non-ACAT programs

• PEO STRI management systems (e.g., CIO-developed, GFEBS)

12

Approved for public release; distribution is unlimited.

• Software Resources Report (SRR):

• SLOC count based on logical lines of code as defined by the

University of Southern California (USC) Code Counting Tool

• Skill hours (Resource Effort)

• Provided to the program office by contractors and

summarized into consolidated report

• Code counts will be automated using the USC Code Counting

Tool and a PRICE Systems provided Excel spreadsheet

• Collection of the appropriate amount of data necessary to

provide defendable cost estimates based on historical data

• Identify and validate a collection process / set of procedures

which minimizes burden / cost impact to the program and

contractors

13

Approved for public release; distribution is unlimited.

• Quantitative Benefits:

• Program cost savings of over $90M indicated by pilot

• Across six years – life cycle usually full operational capability +20

• Using only a 15% reduction in growth factor – actual reduction higher

• Pilot looked at only eight software-intensive programs

• Cost savings on program life cycle costs significantly reduced

• Cost savings can be reinvested in new technology and to reduce funding requests

• Revenue generation increases as PEO STRI lowers cost to take on more work

• Effective measurement of the productivity of contractors’ software development efforts for future pricing/cost estimates.

14

Approved for public release; distribution is unlimited.

• Qualitative Benefits:

• Compliance – valid and reasonable software cost estimate

IAW policy

• Quality – standardized methodology across domains

• Efficiency – faster cycle time to gather data and estimate

costs

• Credibility – defendable estimates based on historical data

• Accuracy – improved accuracy of cost estimates and funding

requests

• Improve contractor software data collection process

15

Approved for public release; distribution is unlimited.

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence

Cost Management Implementation Details

Anthony A DeMarco

PRICE Systems, LLC

Anthony.DeMarco@pricesystems.com

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 17

Data Driven Estimation

Not a new idea – just a new focus on what’s behind an estimate

More credible and lends a a sense of defendability to any estimate

“Data-driven” estimation IS…

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 18

Data Driven Estimation

Based on data extrapolation from analogous systems

Based on a model using analogous systems to determine model inputs

Based on a model calibrated to historical data

A “data-driven” estimate MAY BE…

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 19

Data Driven Estimation

Transparent as to what data was used

Obtained from a credible, identifiable source

Clear about how that data was used

A “data-driven” estimate MUST BE…

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 20

How do you achieve Data-driven Cost Management?

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 21

Steps 1 & 7 – Software Data Collection & Integration

Software data collection and analysis to support more defendable estimate

–Army Program Executive Office for Simulation, Training, and Instrumentation (PEO STRI)

–No data collection processes in place

– Pilot involves one large, complex program – two contractors – one does primarily development, one primarily integration

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 22

PEO STRI Data Collection

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 23

Software Pilot – Software Resource Report

Approved for public release, distribution is unlimited

Collection Tool

Unified Code Count (UCC)

Free code counting tool developed by University of Southern

California Center for Software Excellence (USC-CSE)

Implements popular code counting standards

Supports over 21 software languages and performs source

differencing

• Key to identify new, deleted and unchanged code base

Approved for public release, distribution is unlimited

Data Collection Process

What, when and how to collect

Build reporting frequency

Typically every 10-12 weeks

Three lane data collection

Capability Development

Integration and Test

Release

Allows for segregation of effort

in a way that would help

determine productivity levels

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 26

Step 2 – Data Mining & Analysis

Task minimized because we designed the form and the process together!

– Project Attributes measured to match TruePlanning software model cost drivers

– Effort collected by activities and capabilities to align to desired outcomes

–Normalization guidelines developed

–Documentation of these understandings and guidelines in a clear, concise, understandable and repeatable fashion.

–Automation to ease the collection process, ensures consistency in the process and facilitates the process of normalization

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 27

Data Collection – Keys to Success

Use of an Integrated Product Team (IPT)

Data Item Dictionary created to define…. – What data to collect

– A clear understanding of the data and potential uses

– How often to collect it

– How to align project specific data to more general categories (across multiple contractors)

PEO STRI

PRICE

Contractors

Lots of collaboration between PEO STRI personnel, contractors and the PRICE team resulting in…..

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 28

Data Collection – Keys to Success

Requirement added to Contract Deliverable Requirements List (CDRL) making data collection a contractor requirement

Creation of automated code counting tools by wrapping USC’s Code Counting tool with Excel to help ensure consistency of data

Software Resource Report (SRR) developed to align with output of code counting tool, automate mapping processes and align with potential uses

Lots of collaboration between PEO STRI personnel, contractors and the PRICE team resulting in…..

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 29

Step 3 – Turn data into information for estimating

Calibrate the inputs of interest in TruePlanning® using data from TrueFindings ™ – Organizational Productivity

– Functional Complexity by capability by language

Import the calibrated inputs into TrueFindings™ file

Drive inputs values using the “findings” from TrueFindings™ inside TruePlanning® to create defendable estimates.

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 30

Introducing TrueFindings find•ing /ˈfīndiNG/ Noun

A conclusion reached as a result of an inquiry, investigation, or trial.

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 31

Step 4 –Produce Estimates with Calibrated Data

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 32

Step 4 – Contour Projects to FY Funding

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 33

Step 5 – Map to Army Cost Budget Structure

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 34

PEO STRI is doing Data-driven Cost Management!

• Quantitative Benefits:

• Program cost savings of over $90M indicated by pilot

• Across six years – life cycle usually full operational capability +20

• Using only a 15% reduction in growth factor – actual reduction higher

• Pilot looked at only eight software-intensive programs

• Cost savings on program life cycle costs significantly reduced

• Cost savings can be reinvested in new technology and to reduce funding requests

• Revenue generation increases as PEO STRI lowers cost to take on more work

• Effective measurement of the productivity of contractors’ software development efforts for future pricing/cost estimates.

35

Approved for public release; distribution is unlimited.

• Qualitative Benefits:

• Compliance – valid and reasonable software cost estimate

IAW policy

• Quality – standardized methodology across domains

• Efficiency – faster cycle time to gather data and estimate

costs

• Credibility – defendable estimates based on historical data

• Accuracy – improved accuracy of cost estimates and funding

requests

• Improve contractor software data collection process

36

Approved for public release; distribution is unlimited.

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 37

Data Collection - Lessons Learned

Alleviate contractor concern about being ‘measured’ or misuse of data

Facilitate determination of data collection targets

Need buy in from the data collectors

Entire team needs to understand motivation for data collection

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 38

Data Collection - Lessons Learned

Creates consistency

Creates efficiency

Reduces impact on contractors

Promotes harmony

Only means to handle such large code counts / bases

Facilitates normalization and enforces normalization guidelines

Automation is key to success

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 39

Data Collection - Lessons Learned

Communication, teamwork and building trust are paramount

Data needs to be inspected for quality and ported into an environment for easy analysis.

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 40

Conclusions

The need for data driven estimation is driving many organizations in Aerospace and Defense and industry in general to look for new ways to collect and use data

Data driven estimation is really nothing new – we have been using data (real or experiential) to justify our estimates forever. The new requirement is transparency and openness about the data and how it is being used.

To support this effort organizations may need to establish formal processes for collecting their project data. – CDRL, Data Collection Form with Definitions SRR

– Standard Code Counting Tool USC Code Counter

– Standard Estimating Tool PRICE TruePlanning Software Model

When doing this it is important to understand the importance of – Communication

– Automation

– Flexibility

– Buy in across all the stakeholders

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 41

Vision: To be the Nation’s Acquisition and

Contracting Center of Excellence for Training, Testing, and Simulation Solutions.

Mission: Acquire and Sustain Training, Testing,

and Simulation Solutions in Support of the Nation.

Strategic Goals

as of OCT 2012

1.0 - Provide STRI capabilities

to support Decisive Action

Operations

2.0 - Ensure capabilities meet the needs of the

Integrated Training and T&E

Environments

3.0 - Develop a workforce with

expert level acquisition,

technical, and leadership skills

4.0 - Promote and sustain

excellence in the execution of contracting

activities

41 Approved for public release; distribution is unlimited.

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 42

Questions

Anthony.DeMarco@pricesystems.com