U.S. Army PEO STRI Cost Management Implementation · U.S. Army PEO STRI Cost Management Implementation Anthony A DeMarco ... Allows for segregation of effort in a way that would help

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 1 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence

U.S. Army PEO STRI Cost Management

Implementation Anthony A DeMarco

PRICE Systems, LLC

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 2

Outline

Who - PEO STRI Overview – Dr. James T. Blake, SES, Program Executive Officer

Why - Improving Software Cost Estimates – Mr. James A. Golden, Associate Chief Financial Manager – Cost

How – Cost Management Implementation Details – Anthony A DeMarco, President, PRICE Systems


Washington

Dr. James T. Blake, SES

Program Executive Officer

for

Simulation, Training, and Instrumentation

United States Army

24 October 2013

Approved for public release; distribution is unlimited.

Vision: To be the Nation’s Acquisition and

Contracting Center of Excellence for Training, Testing, and Simulation Solutions.

Mission: Acquire and Sustain Training, Testing,

and Simulation Solutions in Support of the Nation.

Strategic Goals

as of OCT 2012

1.0 - Provide STRI capabilities

to support Decisive Action

Operations

2.0 - Ensure capabilities meet the needs of the

Integrated Training and T&E

Environments

3.0 - Develop a workforce with

expert level acquisition,

technical, and leadership skills

4.0 - Promote and sustain

excellence in the execution of contracting

activities

6 Approved for public release; distribution is unlimited.

7

Simulation Is Safe, Saves Time, Money, Fuel and the Environment

Safety – less training injuries

Minor Fuel Needs

No Ammo Required

Limits Environmental Impact

▫ Air Pollution

▫ Noise Pollution

▫ Terrain Damage

Inexpensive to Operate

Simulation Enhances Live Training Approved for public release; distribution is unlimited.

• Army Training Inventory

• 335,000 training devices

located at over 600 sites

worldwide

• Sustained by 7,000 people

• Valued at $3.9B

• Life-Cycle Management Plan is

required to support training

systems

• MDEP WCLS is PEO STRI’s

sustainment budget line for

Army accepted TADSS

• Field OPS operates 24/7 around

the world – “the sun never sets

on their support”

Life-Cycle Support

Iraq – 0 (As of 31 Aug 13)

Afghanistan – 759

Kuwait – 63

• 850 Field OPS Contractor personnel deployed in SWA at over 25 locations.

• Since March 2003, we have supported over 2,126,000 individual soldier training events in

SWA.

Joint Forces: Army, Navy, Marine Corps, Air Force and SOCOM.

11+ Coalition Nations including the United Kingdom, Australia, Italy, Norway, Poland, France,

Netherlands, Lithuania, Canada, New Zealand and Germany.

More than 560,000 Iraq Army and Afghanistan National Army soldiers trained to date.


Foreign Military Sales

52 Active Country

Programs

NZ

Open Cases 204

Open Requisitions 1528

FY13 $ Value $ 451M

Total Value $2.11B

Foreign Military Sales - Across the World


Mr. James A. Golden Associate Chief Financial Manager – Cost

Program Executive Office for

Simulation, Training, and Instrumentation

24 October 2013


11


• Problem: Software development cost estimates may be

overstated by 30-56%, negatively impacting ability to provide

timely and affordable capabilities

• Goal: Significantly reduce growth factors in software cost

estimates:

• Improve AFFORDABILITY of software-intensive programs.

• Deliver more CAPABILITY; eliminate arbitrary growth factors.

• Decrease CYCLE TIME to develop software cost estimates

• DEFENDABLE and CREDIBLE software cost estimates

• Develop solution set using Lean Six Sigma (LSS) process by

initiating a LSS Black Belt project to improve software cost

estimates at PEO STRI

• In Scope

• PEO STRI ACAT II and III programs with a software

development requirement, including:

• All appropriations (RDTEA, OPA, OMA)

• All software development effort (e.g., P3I, PPSS, PDSS)

• Contractor software data

• PM software data

• PEO STRI software data collection process

• Out of Scope

• ACAT I and Non-ACAT programs

• PEO STRI management systems (e.g., CIO-developed, GFEBS)

12


• Software Resources Report (SRR):

• SLOC count based on logical lines of code as defined by the

University of Southern California (USC) Code Counting Tool

• Skill hours (Resource Effort)

• Provided to the program office by contractors and

summarized into consolidated report

• Code counts will be automated using the USC Code Counting

Tool and a PRICE Systems provided Excel spreadsheet

• Collection of the appropriate amount of data necessary to

provide defendable cost estimates based on historical data

• Identify and validate a collection process / set of procedures

which minimizes burden / cost impact to the program and

contractors

13


• Quantitative Benefits:

• Program cost savings of over $90M indicated by pilot

• Across six years – life cycle usually full operational capability +20

• Using only a 15% reduction in growth factor – actual reduction higher

• Pilot looked at only eight software-intensive programs

• Cost savings on program life cycle costs significantly reduced

• Cost savings can be reinvested in new technology and to reduce funding requests

• Revenue generation increases as PEO STRI lowers cost to take on more work

• Effective measurement of the productivity of contractors’ software development efforts for future pricing/cost estimates.

14


• Qualitative Benefits:

• Compliance – valid and reasonable software cost estimate

IAW policy

• Quality – standardized methodology across domains

• Efficiency – faster cycle time to gather data and estimate

costs

• Credibility – defendable estimates based on historical data

• Accuracy – improved accuracy of cost estimates and funding

requests

• Improve contractor software data collection process

15


© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence

Cost Management Implementation Details

Anthony A DeMarco

PRICE Systems, LLC

[email protected]

mailto:[email protected]


Data Driven Estimation

Not a new idea – just a new focus on what’s behind an estimate

More credible and lends a a sense of defendability to any estimate

“Data-driven” estimation IS…



Based on data extrapolation from analogous systems

Based on a model using analogous systems to determine model inputs

Based on a model calibrated to historical data

A “data-driven” estimate MAY BE…



Transparent as to what data was used

Obtained from a credible, identifiable source

Clear about how that data was used

A “data-driven” estimate MUST BE…


How do you achieve Data-driven Cost Management?


Steps 1 & 7 – Software Data Collection & Integration

Software data collection and analysis to support more defendable estimate

–Army Program Executive Office for Simulation, Training, and Instrumentation (PEO STRI)

–No data collection processes in place

– Pilot involves one large, complex program – two contractors – one does primarily development, one primarily integration


PEO STRI Data Collection


Software Pilot – Software Resource Report

Approved for public release, distribution is unlimited

Collection Tool

Unified Code Count (UCC)

Free code counting tool developed by University of Southern

California Center for Software Excellence (USC-CSE)

Implements popular code counting standards

Supports over 21 software languages and performs source

differencing

• Key to identify new, deleted and unchanged code base

http://www.google.com/url?sa=i&rct=j&q=code counting&source=images&cd=&cad=rja&docid=a_dwVNsndy22qM&tbnid=L6FQoH9momGjBM:&ved=0CAUQjRw&url=http://drupalscout.com/knowledge-base/counting-lines-code&ei=nesxUsKzMaj54AP6lYDQBw&bvm=bv.52109249,d.dmg&psig=AFQjCNG6arQCJfrq-xGnrUtOaELVSzEpow&ust=1379089641319950

Approved for public release, distribution is unlimited

Data Collection Process

What, when and how to collect

Build reporting frequency

Typically every 10-12 weeks

Three lane data collection

Capability Development

Integration and Test

Release

Allows for segregation of effort

in a way that would help

determine productivity levels


Step 2 – Data Mining & Analysis

Task minimized because we designed the form and the process together!

– Project Attributes measured to match TruePlanning software model cost drivers

– Effort collected by activities and capabilities to align to desired outcomes

–Normalization guidelines developed

–Documentation of these understandings and guidelines in a clear, concise, understandable and repeatable fashion.

–Automation to ease the collection process, ensures consistency in the process and facilitates the process of normalization


Data Collection – Keys to Success

Use of an Integrated Product Team (IPT)

Data Item Dictionary created to define…. – What data to collect

– A clear understanding of the data and potential uses

– How often to collect it

– How to align project specific data to more general categories (across multiple contractors)

PEO STRI

PRICE

Contractors

Lots of collaboration between PEO STRI personnel, contractors and the PRICE team resulting in…..


Data Collection – Keys to Success

Requirement added to Contract Deliverable Requirements List (CDRL) making data collection a contractor requirement

Creation of automated code counting tools by wrapping USC’s Code Counting tool with Excel to help ensure consistency of data

Software Resource Report (SRR) developed to align with output of code counting tool, automate mapping processes and align with potential uses

Lots of collaboration between PEO STRI personnel, contractors and the PRICE team resulting in…..


Step 3 – Turn data into information for estimating

Calibrate the inputs of interest in TruePlanning® using data from TrueFindings ™ – Organizational Productivity

– Functional Complexity by capability by language

Import the calibrated inputs into TrueFindings™ file

Drive inputs values using the “findings” from TrueFindings™ inside TruePlanning® to create defendable estimates.


Introducing TrueFindings find•ing /ˈfīndiNG/ Noun

A conclusion reached as a result of an inquiry, investigation, or trial.


Step 4 –Produce Estimates with Calibrated Data


Step 4 – Contour Projects to FY Funding


Step 5 – Map to Army Cost Budget Structure


PEO STRI is doing Data-driven Cost Management!

• Quantitative Benefits:

• Program cost savings of over $90M indicated by pilot

• Across six years – life cycle usually full operational capability +20

• Using only a 15% reduction in growth factor – actual reduction higher

• Pilot looked at only eight software-intensive programs

• Cost savings on program life cycle costs significantly reduced

• Cost savings can be reinvested in new technology and to reduce funding requests

• Revenue generation increases as PEO STRI lowers cost to take on more work

• Effective measurement of the productivity of contractors’ software development efforts for future pricing/cost estimates.

35


• Qualitative Benefits:

• Compliance – valid and reasonable software cost estimate

IAW policy

• Quality – standardized methodology across domains

• Efficiency – faster cycle time to gather data and estimate

costs

• Credibility – defendable estimates based on historical data

• Accuracy – improved accuracy of cost estimates and funding

requests

• Improve contractor software data collection process

36



Data Collection - Lessons Learned

Alleviate contractor concern about being ‘measured’ or misuse of data

Facilitate determination of data collection targets

Need buy in from the data collectors

Entire team needs to understand motivation for data collection



Creates consistency

Creates efficiency

Reduces impact on contractors

Promotes harmony

Only means to handle such large code counts / bases

Facilitates normalization and enforces normalization guidelines

Automation is key to success



Communication, teamwork and building trust are paramount

Data needs to be inspected for quality and ported into an environment for easy analysis.


Conclusions

The need for data driven estimation is driving many organizations in Aerospace and Defense and industry in general to look for new ways to collect and use data

Data driven estimation is really nothing new – we have been using data (real or experiential) to justify our estimates forever. The new requirement is transparency and openness about the data and how it is being used.

To support this effort organizations may need to establish formal processes for collecting their project data. – CDRL, Data Collection Form with Definitions SRR

– Standard Code Counting Tool USC Code Counter

– Standard Estimating Tool PRICE TruePlanning Software Model

When doing this it is important to understand the importance of – Communication

– Automation

– Flexibility

– Buy in across all the stakeholders


Vision: To be the Nation’s Acquisition and

Contracting Center of Excellence for Training, Testing, and Simulation Solutions.

Mission: Acquire and Sustain Training, Testing,

and Simulation Solutions in Support of the Nation.

Strategic Goals

as of OCT 2012

1.0 - Provide STRI capabilities

to support Decisive Action

Operations

2.0 - Ensure capabilities meet the needs of the

Integrated Training and T&E

Environments

3.0 - Develop a workforce with

expert level acquisition,

technical, and leadership skills

4.0 - Promote and sustain

excellence in the execution of contracting

activities

41 Approved for public release; distribution is unlimited.


Questions

[email protected]

mailto:[email protected]

Documents

U.S. Army PEO STRI Cost Management Implementation · U.S. Army PEO STRI Cost Management Implementation Anthony A DeMarco ... Allows for segregation of effort in a way that would help