Historical Trend Analysis (HTA) Analysed
SCAF April 2010
Dale ShermonSenior Executive Consultant
PRICE SystemsHook
[email protected]+44 (0)1256 760012
Copyright PRICE Systems 2010
Objective
The objective of the study is to stimulate discussion amongst the cost community as to the usage of Historical Trend Analysis (HTA), a common term which is used, but has not matured in many ways
Copyright PRICE Systems 2010
Agenda
Introduction
Three approaches to Historical Trend Analysis;1. Complexity versus time
2. Production Cost versus time
3. Multi-variable versus time
Analysis of accuracy
Case study
Summary
Copyright PRICE Systems 2010
Background - SDR
1.29 On the basis of experience in the United Kingdom and internationally, if we continue to search for a technological edge, including improved protection for our personnel, we can expect the cost of successive generations of equipment to continue to rise at above the rate of inflation.
1.31 These are enduring trends, and other advanced militaries around the world face the same challenge. Historically, rising unit costs have been offset by increases in capability and changes in the nature of the threat which have led us to reduce numbers of both personnel and platforms. But there are limits to how far capability improvements or efficiency can compensate for numbers.
Source: Adaptability and Partnership: Issues for the Strategic Defence ReviewISBN 978-0-10-177942-5
Copyright PRICE Systems 2010
Background – Acquisition Reform
1.7 Around 98% of our major projects deliver the operational performance needed at the front line. But they also tend to increase in cost – by an average of 2.8% each year. And they suffer delay, averaging 5.9 months (though some for reasons beyond MOD’s control).
2.4 Part of the problem of cost growth we have faced has come from not fully understanding at the outset what the costs and risks of a project might be.
Source: The Defence Strategy for Acquisition ReformISBN 978-0-10-177942-3
Copyright PRICE Systems 2010
Reform Strategy
Source: The Defence Strategy for Acquisition ReformISBN 978-0-10-177942-3
Copyright PRICE Systems 2010
Agenda
Introduction
Three approaches to Historical Trend Analysis;1. Complexity versus time
2. Production Cost versus time
3. Multi-variable versus time
Analysis of accuracy
Case study
Summary
Copyright PRICE Systems 2010
1. Complexity versus Time
Source: “Cost Complexity Forecasting Historical Trends of Major Systems” 1990 Varese ITALY - D Webb
Copyright PRICE Systems 2010
Complexity (MCPLX) is derived through a parametric cost model.Complexity ≈ Normalised Cost Density
= Technology and Productivity index
1. PRICE Systems KnowledgeManager
Copyright PRICE Systems 2010
1. Not all complexity trends are upwards
Commercial aircraft
y = -0.0514x + 110.04R2 = 0.6493
y = 0.0025x + 2.8405R2 = 0.0009
y = -0.0184x + 44.393R2 = 0.0303
0
1
2
3
4
5
6
7
8
9
10
1980 1985 1990 1995 2000 2005 2010
Normalised Cost Density
In-S
ervi
ce D
ate
Narrowbody WidebodyWidebody high capacity Linear (Widebody)Linear (Narrowbody) Linear (Widebody high capacity)
Copyright PRICE Systems 2010
1. Productivity metrics
y = -0.0329x + 74.538
y = -0.0141x + 36.701
6
6.5
7
7.5
8
8.5
9
9.5
1975 1980 1985 1990 1995 2000 2005 2010
AIRBUSBoeingLinear (AIRBUS)Linear (Boeing)
Copyright PRICE Systems 2010
Strong competition leading to productivity can balance technology gains
Assuming: Constant Operating Environment (Industry)
Time
Cost Lower complexity Higher complexity
+
-Technology Maturity Dates
Time Now
1. Normalisation - Technology Maturity model- effect of technology selection
Mature technology usage results in lower cost
Immature technology usage results in higher cost
Copyright PRICE Systems 2010
Assuming: Constant Operating Environment (Industry)
Time
Cost Lower complexity Higher complexity
+
-
Time Now
1. Normalisation - Technology Maturity model- demand for latest technology
• Cheaper but less capable technology• Not desirable for Defence
Copyright PRICE Systems 2010
2. Production Cost over time
“Source book of Defence Equipment costs” by Philip G. PughISBN 978-0-9556258-0-0
Copyright PRICE Systems 2010
2. Production Cost versus Time
Copyright PRICE Systems 2010
“Source book of Defence Equipment costs” by Philip G. PughISBN 978-0-9556258-0-0
Log-linear axis and exponential curves plotted, but linear cost growth documented
Is the total System subjected to technology cost growth?
Copyright PRICE Systems 2010
HMS StevenstoneIn Service Date 1943
7,000 tonnes of technology growth?
Type 45In Service Date 2009
Weight
T1 Cost US$2006
High
High
Low
Low
= Historical Normalised Data
3. Multi-variable analysis- Weight Cost Driver
Source: Systems Cost EngineeringISBN 978-0-566-08861-2
Copyright PRICE Systems 2010
} Positive Residual
} Negative Residual
1940 1950 1960 1970 1980 1990 2000 2010
Residual US$
2006 ec
3. Multi-variable analysis- Technology Cost Driver
Source: Systems Cost EngineeringISBN 978-0-566-08861-2
Copyright PRICE Systems 2010
+
-
Weight
T1 Cost US$2006
4,000 kg
Cost of 1950 system
Cost of earlier system
Cost of 1970 system
3. Multi-variable analysis- Combined weight and technology drivers
Source: Systems Cost EngineeringISBN 978-0-566-08861-2
Copyright PRICE Systems 2010
3. Aircraft Carriers Multi-variable analysisTone versus Weight (Tonnes) y = 561.9985x1.3374
R2 = 0.7343
$0$200,000,000$400,000,000$600,000,000$800,000,000
$1,000,000,000$1,200,000,000$1,400,000,000$1,600,000,000$1,800,000,000$2,000,000,000
0 10,000 20,000 30,000 40,000 50,000 60,000 70,000
Weight (Tonnes)
Rec
urrin
g T
one
US
$
Residual versus ISD y = 33,635.79x - 551,844,424.48R2 = 0.48
-$400,000,000
-$200,000,000
$0
$200,000,000
$400,000,000
$600,000,000
$800,000,000
$1,000,000,000
1900 1913 1927 1941 1954 1968 1982 1995
In Service Date
Res
idua
l US
$
Copyright PRICE Systems 2010
Agenda
Introduction
Three approaches to Historical Trend Analysis;1. Complexity versus time
2. Production Cost versus time
3. Multi-variable versus time
Analysis of accuracy
Case study
Summary
Copyright PRICE Systems 2010
Test Data – Fighter Aircraft
Meteor Mk1 Thunderjet F-84B Meteor Mk3 Meteor Mk4Sabre F-86A Sabre F-86DScorpion F-89D Cougar F-9F Delta Dagger F-102AThunderchief F-105BVoodoo F-101A Fury F-1Super Sabre F-100A Super Sabre F-100C Starfighter F-104A Super Sabre F-100DHunter F1 Hunter F4 HunterF6 Swift F1/ F2 Swift F5Javelin F(AW) Mk2 Swift F7Thunderchief F-105DVoodoo F-101C Delta DartF-106A/B Lightning F1Phantom F4-A JavelinF(AW) Mk9
Phantom F4-B Freedom Fighter F-5A Freedom Fighter F-5B Lightning F2Phantom F4-C LightningF3 Phantom F4-DPhantom F4-E Delta Dagger F-102A LightningF6 Phantom F4-FPhantom F4-J Tiger II F-5E Freedom Fighter F-5FFalcon F-16A/D TornadoF2 Eagle F-15A/D EagleF-15E Mirage 2000Mirage 2000-5 TornadoF3 Saab JAS-39C RafaleRafale EurofighterTyphoon Raptor F/A-22A
Copyright PRICE Systems 2010
Accuracy measure
Normalise the cost and technical data– Currency, economics, quantity
– Metric (kg) versus Imperial (lbs)
Subject the data to the trend process
Predict the historical projects based upon the trend predictions
Compare the coefficient of determination (R2) for the prediction versus the historical data
Copyright PRICE Systems 2010
1. Complexity over time
Copyright PRICE Systems 2010
R² = 0.9757
-100,000.00
-
100,000.00
200,000.00
300,000.00
400,000.00
500,000.00
600,000.00
- 100,000.00 200,000.00 300,000.00 400,000.00
Pred
icte
d co
st
Historical cost
Accuracy
UPC in National Currency and Dec 2001 ec
Linear (UPC in National Currency and Dec 2001 ec)
2. Production Cost ($/Kg) versus Time
Copyright PRICE Systems 2010
R² = 0.8824
-20,000,000.00
-
20,000,000.00
40,000,000.00
60,000,000.00
80,000,000.00
100,000,000.00
120,000,000.00
140,000,000.00
160,000,000.00
180,000,000.00
200,000,000.00
$0 $50,000,000 $100,000,000 $150,000,000 $200,000,000 $250,000,000
Pred
ictio
n (U
S $)
Historical Costs (US $)
Accuracy
US$ 2006 UPC Linear (US$ 2006 UPC)
3. Multi-variable versus Time
Copyright PRICE Systems 2010
R² = 0.2311
R² = 0.8758
R² = 0.9052
-$50,000,000.00
$0.00
$50,000,000.00
$100,000,000.00
$150,000,000.00
$200,000,000.00
$250,000,000.00
$0 $50,000,000 $100,000,000 $150,000,000 $200,000,000 $250,000,000
True
Con
cept
s pre
dict
ion
(US
$)
Historical Costs (US $)
Accuracy
T1 based on weight growth curve Core + Techniology
Core + Techniology + Performance Linear (T1 based on weight growth curve)
Linear (Core + Techniology) Linear (Core + Techniology + Performance)
Agenda
Introduction
Three approaches to Historical Trend Analysis;1. Complexity versus time
2. Production Cost versus time
3. Multi-variable versus time
Analysis of accuracy
Case study
Summary
Copyright PRICE Systems 2010
Case Study
Originally conducted in 2001 at the request of the UK MOD
Presented at International Society of Parametric Analysts (ISPA) conference 2003
Won best applications paper award
Study of the historical trend of fighter aircraft complexities
Prediction of the Lightning II F35 aircraft production cost
Copyright PRICE Systems 2010
Copyright PRICE Systems 2010
Copyright PRICE Systems 2010
Updated F35 Assumptions
Empty weight = 13,300 Kg
Length = 15.67m
Height = 4.33m
Wingspan = 10.7m
Crew = 1
Total Installed Power (kw) = 11,472
Production Quantity = 3,181
In-Service Date = 2012
Copyright PRICE Systems 2010
http://www.lockheedmartin.com/data/assets/aeronautics/mediacenter/mediakits/f35/F-35FastFacts01142010.pdfhttp://en.wikipedia.org/wiki/F-35_Lightning_II
1. Complexity versus time
Copyright PRICE Systems 2010
y = 0.0593x - 109.62
0
1
2
3
4
5
6
7
8
9
10
1940 1950 1960 1970 1980 1990 2000 2010
Com
plex
ity
In-Service Date
Flighter Aircraft
Complexity = 0.0593 x 2012 – 109.62= 9.691
1. Complexity versus time
Copyright PRICE Systems 2010
$132m at April 2010 ec
2. Production Cost versus Time
Copyright PRICE Systems 2010
y = 151.6188x - 295,621.6034R² = 0.7987
-$2,000.00
$0.00
$2,000.00
$4,000.00
$6,000.00
$8,000.00
$10,000.00
$12,000.00
1940 1950 1960 1970 1980 1990 2000 2010
UPC
/Kg
In Service date (ISD)
UPC/Kg = 151.6188 x 2012 – 295,621.6034= $ 9,435/kg
UPC = 9,435 X 13,300Kg= $ 125m at 2006 ec or $ 144m at 2010 ec
3. Multi-variable versus time - TrueConcepts
Copyright PRICE Systems 2010
$135m at April 2010 ec
Agenda
Introduction
Three approaches to Historical Trend Analysis;1. Complexity versus time
2. Production Cost versus time
3. Multi-variable versus time
Analysis of accuracy
Case study
Summary
Copyright PRICE Systems 2010
Conclusion – HTA Accuracy
Copyright PRICE Systems 2010
HTA Accuracy (R2) for Fighter
Aircraft
Complexity versus time 0.976
Production Cost versus time 0.882
Multi-variable versus time 0.905
Conclusion – Case Study Prediction
Copyright PRICE Systems 2010
Case Study – F35 Cost in 2010 US$
Complexity versus time 132m
Production Cost versus time 144m
Multi-variable versus time 135m
GAO- March 2010
Copyright PRICE Systems 2010
Conclusion – Average Cost Growth
Copyright PRICE Systems 2010
Case Study – F35 Average annual increase
(%)
Complexity versus time 4.6%
Production Cost versus time 1.9%
Multi-variable versus time 2.1%
Summary
Three different approaches to Historical Trend Analysis (HTA) have been discussed and analysed
All three Historical Trend Analysis (HTA) have a high degree of accuracy
Utilising a Parametric model is the most complete approach to normalising raw cost data
More systems (Sea, Land) need to be analysed, rather than just aircraft
Case Study predicts the F35 UPC to be $132m to $144m
Cost Growth for fighter aircraft is shown to be between 1.9% to 4.6% above inflation
Copyright PRICE Systems 2010
Historical Trend Analysis (HTA) Analysed
SCAF April 2010
Dale ShermonSenior Executive Consultant
PRICE SystemsHook
[email protected]+44 (0)1256 760012
Copyright PRICE Systems 2010