Helsinki University of Technology Systems Analysis Laboratory

Portfolio Analysis in thePortfolio Analysis in theCost-Efficiency Evaluation of Cost-Efficiency Evaluation of

Weapon SystemsWeapon Systems

7 July 2009

23rd European Conference on Operational Research in Bonn

Jussi Kangaspunta, Juuso Liesiö and Ahti Salo

Systems Analysis Laboratory

Helsinki University of Technology

P.O. Box 1100, 02015 TKK, Finland

firstname.lastname@tkk.fi

Helsinki University of Technology Systems Analysis Laboratory

Challenges in the Evaluation of Weapon SystemsChallenges in the Evaluation of Weapon Systems

Cost-efficiency of weapon systems depends on both impacts and costs

Several impact dimensions must be accounted for– Enemy and own casualties, mission success probability, combat duration etc.

Impacts depend on the context– Mission (attack/defense), weather conditions, enemy strategies etc.

There are strong interactions among weapon systems– How should joint impacts be attributed to constituent systems ?

– Earlier research mainly focused on individual systems

Impacts are often non-linear– 16 artillery guns may not be twice as effective as 8 guns

2Kangaspunta, Liesiö and Salo: Portfolio Analysis in the Cost-Efficiency Evaluation of Weapon Systems

Helsinki University of Technology Systems Analysis Laboratory

Impact Assessment ModelImpact Assessment Model

Estimates from an independent combat simulator of Defense Forces– Operating situation with pre-specified enemy, terrain, mission and strategies

– Some of own forces kept at a constant level but others are varied

– Numerous simulations with different portfolios of selected weapon systems

– Simulation results could be extended by appropriate mathematical methods (e.g. interpolation)

Kangaspunta, Liesiö and Salo: Portfolio Analysis in the Cost-Efficiency Evaluation of Weapon Systems 3

Criterion 1

Criterion 2

Criterion n

Overall impact of the portfolio

Impact model

...

Combat simulator

Operating

situation

Enem

yOwn forces

portfolio

Helsinki University of Technology Systems Analysis Laboratory

Modeling of Weapon SystemsModeling of Weapon Systems

Weapon system portfolio= number of different weapon systems

= number of weapon systems of the jth type in portfolio

= cost of portfolio

Feasible portfolios satisfy all relevant constraints– E.g. budget constraints, logical constraints (incompatibilities etc.)

Impact assessment criteria– Portfolios evaluated with regard to different impact criteria (enemy casualties, own casualties

etc.)

– Overall impacts approximated by an additive value function

Kangaspunta, Liesiö and Salo: Portfolio Analysis in the Cost-Efficiency Evaluation of Weapon Systems 4

Helsinki University of Technology Systems Analysis Laboratory

Incomplete Information and DominanceIncomplete Information and Dominance

Instead of point-estimate criterion weights, a set of feasible weights

– E.g. rank-ordering for criterion importance

Portfolio x1 dominates x2 if it has greater or equal overall impact for all feasible weights

Kangaspunta, Liesiö and Salo: Portfolio Analysis in the Cost-Efficiency Evaluation of Weapon Systems 5

Helsinki University of Technology Systems Analysis Laboratory

Cost-Efficient PortfoliosCost-Efficient Portfolios

Feasible portfolios that are not dominated by any less or equally expensive portfolio

Kangaspunta, Liesiö and Salo: Portfolio Analysis in the Cost-Efficiency Evaluation of Weapon Systems 6

Helsinki University of Technology Systems Analysis Laboratory

Numerical Example Based on Realistic DataNumerical Example Based on Realistic Data

Three weapon systems

– Only unit costs

Three impact criteria measuring different types of enemy casualties

Incomplete information on the value (i.e. relevance) of the impacts

Analysis of different budget levels with a focus on cost-efficient portfolios

Kangaspunta, Liesiö and Salo: Portfolio Analysis in the Cost-Efficiency Evaluation of Weapon Systems 7

}|{ 2130 wwwSwS ww

}1,0{}8,,1,0{}24,,1,0{ 321 xxx

3

1

)(j

jjxcxC

Helsinki University of Technology Systems Analysis Laboratory

Simulated and Interpolated Impact FunctionsSimulated and Interpolated Impact Functions

8Kangaspunta, Liesiö and Salo: Portfolio Analysis in the Cost-Efficiency Evaluation of Weapon Systems

Helsinki University of Technology Systems Analysis Laboratory

Impacts of Weapon System PortfoliosImpacts of Weapon System Portfolios

9Kangaspunta, Liesiö and Salo: Portfolio Analysis in the Cost-Efficiency Evaluation of Weapon Systems

Helsinki University of Technology Systems Analysis Laboratory

Composition of Cost-Efficient PortfoliosComposition of Cost-Efficient Portfolios

10Kangaspunta, Liesiö and Salo: Portfolio Analysis in the Cost-Efficiency Evaluation of Weapon Systems

Helsinki University of Technology Systems Analysis Laboratory

ConclusionsConclusions

Portfolio approach is necessitated by strong interactions

→ Evaluation of individual weapon systems makes little sense

These interactions are captured by the combat simulator results

Multi-criteria model aggregates several impact dimensions– Contextual importance of impacts captured through incomplete information

Cost-efficiency depends on both impacts and costs

→ Focus on the computation of cost-efficient portfolios

Kangaspunta, Liesiö and Salo: Portfolio Analysis in the Cost-Efficiency Evaluation of Weapon Systems 11

Helsinki University of Technology Systems Analysis Laboratory

ReferencesReferences

Brown, G.G., Dell, R.F., Newman, A.M. (2004). Optimizing Military Capital Planning, Interfaces Vol. 34, No. 6, pp. 415-425.

Bunn, D.W., Salo, A.A. (1993). Forecasting with Scenarios, European Journal of Operational Research, Vol. 68, pp. 291-303.

Fox, P. (1965). A Theory of Cost-Effectiveness for Military Systems Analysis, Operations Research, Vol. 13, No. 2, pp. 191-201.

Liesiö, J., Mild, P., Salo, A. (2007) Preference Programming for Robust Portfolio Modeling and Project Selection,European Journal of Operational Research, Vol. 181., No. 3., pp. 1488-1505.

Liesiö, J., Mild, P., Salo, A. (2008) Robust Portfolio Modeling with Incomplete Cost Information and Project Interdependencies, European Journal of Operational Research, Vol. 190, pp. 679-695.

Stafira, S., Parnell, G., Moore, J., (1997). A Methodology for Evaluating Military Systems in a Counterproliferation Role, Management Science, Vol. 43, No. 10, pp. 1420-1430.

Parnell, G., et. al. (1998). Foundations 2025: A Value Model for Evaluating Future Air and Space Forces,Management Science, Vol. 44, No. 10, pp. 1336-1350.

Kangaspunta, Liesiö and Salo: Portfolio Analysis in the Cost-Efficiency Evaluation of Weapon Systems 12

Helsinki University of Technology Systems Analysis Laboratory

Questions and Comments ?Questions and Comments ?

Kangaspunta, Liesiö and Salo: Portfolio Analysis in the Cost-Efficiency Evaluation of Weapon Systems 13

Helsinki University of Technology Systems Analysis Laboratory

Extensions and Further ResearchExtensions and Further Research

Considering multiple operating situations

– Cost-efficiency is highly context dependent

– Can be integrated to model for instance using probabilities

– Risk and/or robustness measures for portfolios can be formed

Complementing simulation data with expert evaluations

– Simulations can be augmented with judgmental expert evaluations of impacts

– Experimental design of simulations and/or expert evaluations

Considering cost-efficiency using core indices

– “What proportion of evaluations supports that a given portfolio is cost-efficient?”

– “What proportion of possible operating situations supports that a given portfolio is cost-efficient?”

14Kangaspunta, Liesiö and Salo: Portfolio Analysis in the Cost-Efficiency Evaluation of Weapon Systems

Helsinki University of Technology Systems Analysis Laboratory

Multiple Operating SituationsMultiple Operating Situations

15Kangaspunta, Liesiö and Salo: Portfolio Analysis in the Cost-Efficiency Evaluation of Weapon Systems

...

Overall expected impact of portfolio

Weapon system

portfolio

p2

pm

p1

...

Helsinki University of Technology Systems Analysis Laboratory

CostCost-Efficiency Using Core Indices 1/2-Efficiency Using Core Indices 1/2

16Kangaspunta, Liesiö and Salo: Portfolio Analysis in the Cost-Efficiency Evaluation of Weapon Systems

0 2 4 6 8 10 12 14 16 18 20 22 24

0

1

2

3

4

5

6

7

8

Expert 1

x1

x 2

0 2 4 6 8 10 12 14 16 18 20 22 24

0

1

2

3

4

5

6

7

8

Expert 2

x1

x 2

Helsinki University of Technology Systems Analysis Laboratory

CostCost-Efficiency Using Core Indices 2/2-Efficiency Using Core Indices 2/2

17Kangaspunta, Liesiö and Salo: Portfolio Analysis in the Cost-Efficiency Evaluation of Weapon Systems

0 2 4 6 8 10 12 14 16 18 20 22 24

0

1

2

3

4

5

6

7

8

Expert 1 & Expert 2

x1

x 2