Analytical Tools for Affordability Analysis

David Tate Cost Analysis and Research Division

Institute for Defense Analyses

Report Documentation Page Form ApprovedOMB No. 0704-0188

Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, ArlingtonVA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if itdoes not display a currently valid OMB control number.

1. REPORT DATE MAY 2015 2. REPORT TYPE

3. DATES COVERED 00-00-2015 to 00-00-2015

4. TITLE AND SUBTITLE Analytical Tools for Affordability Analysis

5a. CONTRACT NUMBER

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S) 5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Institute for Defense Analyses,Cost Analysis and Research Division,4850Mark Center Drive,Alexandria,VA,22311-1882

8. PERFORMING ORGANIZATIONREPORT NUMBER

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)

11. SPONSOR/MONITOR’S REPORT NUMBER(S)

12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited

13. SUPPLEMENTARY NOTES Presented at the 12th Annual Acquisition Research Symposium held May 13-14, 2015 in Monterey, CA.

14. ABSTRACT

15. SUBJECT TERMS

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT Same as

Report (SAR)

18. NUMBEROF PAGES

24

19a. NAME OFRESPONSIBLE PERSON

a. REPORT unclassified

b. ABSTRACT unclassified

c. THIS PAGE unclassified

Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18

What is “Affordability Analysis”?

Starting with 2010’s “Better Buying Power” memorandum, OSD has issued policy requiring acquisition programs to present affordability analyses at Milestone reviews This requirement is now part of Department

of Defense Instruction 5000.02 The Defense Acquisition Guidebook (DAG)

was updated in July 2013 to reflect the new requirement and provide guidance

5/19/2015 1

Affordability Analysis Content

Affordability Analysis shows each program’s planned development and production costs over time, in the context of The cost and schedules of the other programs in

the relevant acquisition portfolio The projected available funding over the life

cycles of those programs

This task is assigned to Service leadership Not the program’s responsibility Should reflect Service long-term planning

5/19/2015 2

CY14$B

0

50,000

100,000

150,000

200,000

250,000

300,000

FY 83 FY 93 FY 03 FY 13 FY 23 FY 33 FY 43

CY $

MILPER

O&M

Procurement

RDT&E

POM14-18

TOA

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

Light Tactical Vehicle (Not JLTV) Protection and OtherMedium Tactical Vehicles Heavy Tactical VehiclesJoint Light Tactical Vehicle (JLTV)

162

442

2,200

884

1,479

1,917

4562,200

2,1352,112

$B POM14–18

Procurement + RDT&E Portfolio$B POM14-18

…FY43

All Portfolios Under Component TOA

F Transportation (Procurement + RDT&E) TWV O&M Requirements

BY12

TWV Portion of O&M

Procurement + RDT&E Portfolios

New System Total

Lifecycle Costs (total

reserved profile)

Recommended Submission Formats (DAG)

5/19/2015 3

What Tools do Affordability Analysts Need?

5/19/2015 4

Reconcile inconsistent submissions Predict annual costs for alternative plans Estimate the consequences of various

possible funding levels Assess affordability risk For the portfolio For each program

Reconciliation of Inconsistencies

Which programs are in the portfolio? How much total funding is available? How much of that total will each program get

year by year? How many units will that buy?

5/19/2015 5

If Service plans or estimates have changed, need to be able to propagate

those changes to other portfolios as well

Estimated Cost of Alternative Plans

If the current plan is …then what would the annual costs be if instead we do This is a hard problem.

5/19/2015 6

Fiscal Year 2015 2016 2017 2018 2019 2020 2021Quantity 20 40 50 50 50 50 20Cost 1502.8 2331.1 2581.0 2403.9 2291.9 2210.7 883.8

Fiscal Year 2015 2016 2017 2018 2019 2020 2021 2022Quantity 10 30 40 40 40 40 40 40Cost ? ? ? ? ? ? ? ?

Why Is It Hard?

Cost progress (aka “learning curves”) Fixed costs at contractor and program levels Nonrecurring and non-end-item costs Production rate effects and incentives Causal ambiguity in historical data Schedule changes cause cost changes Cost changes also cause schedule changes Technical / management issues can cause both

5/19/2015 7

Competing Theories and Models

Fixed/Variable apportioning (e.g., Balut et al.) Plant capacity varies with workload Program share of fixed costs is proportional to

variable costs, some of which have learning

Cobb-Douglas production function (Womer) Unit cost as a function of learning and rate

Learning with forgetting (Benkard) Learning depreciates over time

Discretionary capital investment (Rogerson) 5/19/2015 8

Estimated Impact of Change in Budget

If there isn’t enough money in the budget to do what we had planned, what happens? Programs stretch – lower production rates If necessary, some may be canceled

In order to predict the impact of a given schedule, we need a heuristic that can estimate how the portfolio manager would react to the new budget Requires costing ability described above Should also work for unexpected surplus funds

5/19/2015 9

Affordability Risk Assessment

Affordability is often treated as a yes/no question, but reality is messier Cost estimates are uncertain Program outcomes are uncertain Budgets are uncertain Service priorities change over time New programs start

The question of interest is not “Is this program affordable?”, but rather “What is likely to happen if this is the plan?”

5/19/2015 10

Risk Assessment Support

Sensitivity analysis Vary one input at a time, see what happens Does not directly answer “What is likely?”

Monte Carlo estimation Vary all uncertain inputs according to user-

specified probability distributions Analyze the distribution of outcomes Requires credible driving distributions for many

parameters and program characteristics

5/19/2015 11

Analyst Support Tool Notional Design

Organize programs into portfolios Multiple alternative ways to partition the world

Coordinate across multiple data sources SAR / DAES / PB / POM Individual program affordability analysis

submissions Perform what-if and sensitivity analyses Alternative schedules Alternative budgets Revised cost estimates New programs

5/19/2015 12

Current Status: Software

APASS: the Acquisition Portfolio Affordability Support System Web application SQL Server database Migrating to D3 graphics from Google API

Data from multiple (conflicting) sources, organized by portfolio sets for analysis at the portfolio level To date, MDAP and pre-MDAP data only

5/19/2015 13

A Portfolio Set

5/19/2015 14

IDA I

Acquisition Portfolio Analysis Support System

Home Army Equ1pment

Dollars

O Then-Year Dollars

@Base Year 2013

Show

0 By Budget Category

@Equipment Portfolio Set

0 Aifordabihty Portfolio Set

Categories

rlRDT&E

l'liProcurement

E:Jo&M

D MILCON

Years to Show

Start Year: Earlrest 199~

End year:

Latest 2034

Max Dollars

Dshow Actuals

Source SAR 2013-12-25 lr

Serv1ce: Army

Portfolio Set: Equipment

Source: SAR 2013-12-25

Chart Type ~

$8,000

$6,000 :;:, ... ~

ol!1 8 ~ $4,000 0

~ 1.1..

~ ~

$2,000

$0

Workspace Projections ~

Total FY2013 Dollars

"' ... .,., co ..... 00 0>

~ ~ ~ ~ ~ !§ Fa

C Reload

• Soldier and Squad

• Mission Command

• Indirect Fires

• Aviation

• Air and Missile Defense

~ ~ ~ g) ~ ~ co ..... 00

~ § fj <'V "' ~ ~ f? ~ ~ ~ 2 2 <'V <'V <'V <'V <'V <'V <'V <'V <'V <'V

Fiscal Year

A Portfolio with Budget

5/19/2015 15

IDA I

Acquisition Portfolio Analysis Support System

Dollars

0 Then-Year Dollars

@Base Year· 2013

Show

0 By Budget Category

@Equipment Portfolio Set

0 Affordat>ility Portfolio Set

Categories

i!:JRDT&E

12JProcurement

Co&M

DMILCON

Years to Show

Start Year:

Eartrest 199~

End year

Latest 2026

Max Dollars.

rlshOw Actuals

Include

12lBudget

li"JFit to budget

Source· SAR 2013-12-25 •

Serv1ce: Army

Portfolio Set: Equ1pment

Portfolio: Av1at1on

Source: SAR 2013-12-25

Chart Type •

$5,000

$3,750 ::; .... "' <s "" 0 Q

~ $2,500 0

~ u_ 0;

~ $1 ,250

$0

Workspace Projections •

Total FY2013 Dollars

"' ~ II) <o "- <0 0)

~ ~ ~ ~ ~ ~ ~

~Reload

• Budget

• UH-80M Black Hawk

• MQ-1 C Gray Eagle

• CH-47F

• AH-84E RemanufacturE

• AH-84E New Build

0

~ "' ~ ~ II) <o "- <0 RJ 0 ;; "' f;J ~ ~ ~ ~ ~ ~ ~ & & ('(J ('(J ('(J ~ ('(J ('(J ('(J ('(J ('(J ~ ('(J ~ ('(J f?

Fiscal Year

Results of Fitting to Budget

5/19/2015 16

IDA I

FY2013 Procurement Dollars

$5,000

~ $3,750 ~

t? Jg 0 Q 1: (!)

E $2,500 ~ l>

ct (\') ....... C>

~ $1 ,250 ~

$0

Budget

FY 2030:

$1500M

~ ~ ~ ~ ~ ~ ~ m o ~ ~ ~

~2? ~ ~fJ~~~RfJRfJ Fiscal Year

• Budget

• UH-60M Black Hawk

MQ-1 C Gray Eagle

• CH-47F

AH-64E RemanufacturE

AH-64E New Build

Summary

We are developing software tools to support Affordability Analysis (and oversight of Affordability Analysis)

The current focus is on near-term ability to view and compare disparate data sources and alternative scenarios Spot discrepancies Produce reconciled views Provide “what if?” assessment of alternatives

Secondary focus on estimating the impact on portfolios of alternative budgets

5/19/2015 17

BACKUP

5/19/2015 18

What is “Affordability”?

Since the late 1990s, the military services have all spent large sums of money on programs that did not deliver their intended military capability Many of these programs spent billions and

delivered nothing at all

5/19/2015 19

“The purpose of Affordability Analysis is to avoid starting or continuing programs that cannot be produced and supported within reasonable expectations for future budgets.” DoDI 5000.02, Enclosure 8, “Affordability Analysis and Investment Constraints” (2015)

DAG Format 1 – TOA Top Line & Color of Money

5/19/2015 20

CY14$B

0

50,000

100,000

150,000

200,000

250,000

300,000

FY 83 FY 93 FY 03 FY 13 FY 23 FY 33 FY 43

CY $

MILPER

O&M

Procurement

RDT&E

POM14-18

TOA

All Portfolios Under Component TOA

DAG Format 2 – Service Portfolios

5/19/2015 21

Procurement + RDT&E Portfolio$B POM14-18

…FY43

Procurement + RDT&E Portfolios

DAG Format 4 – Portfolio Detail

5/19/2015 22

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

Light Tactical Vehicle (Not JLTV) Protection and OtherMedium Tactical Vehicles Heavy Tactical VehiclesJoint Light Tactical Vehicle (JLTV)

162

442

2,200

884

1,479

1,917

4562,200

2,1352,112

$B POM14–18

F Transportation (Procurement + RDT&E)

New System Total

Lifecycle Costs (total

reserved profile)

Benkard Formulation

5/19/2015 23

1

11

average unit cost in year production quantity in year theoretical first unit cost learning rate parameter annual forgetting rate production rate parameter

n

n

nn k

n n kk

C nL nT

C T L Lβ

γ

βδγ

δ −

=

≡≡≡≡≡≡

= ∑