1. Helsinki University of Technology Systems Analysis Laboratory Portfolio and Scenario Analysis in the Cost-Effectiveness Evaluation of Weapon Systems Jussi Kangaspunta, Ahti Salo and Juuso Liesiö Systems Analysis Laboratory Helsinki University of Technology P.O. Box 1100, 02015 TKK, Finland http://www.sal.tkk.fi firstname.lastname@tkk.fi

2. Helsinki University of Technology Systems Analysis Laboratory 2 Contents n Finnish Defense Forces n Challenges in the evaluation of weapon systems n Multi-criteria portfolio model for weapon systems n Numerical example and future research n Conclusions

3. Helsinki University of Technology Systems Analysis Laboratory 3 Finnish Defense Forces n Key statistics –Annual budget ~ 2.3€ (~$2.8) billion –About 1.3% of GNP (in USA ~4.5%) –Peacetime strength »13,000 regulars »27,000 conscripts »30,000 reservists trained annually –Wartime strength 430,000 »Population of Finland ~5.2 million n Tasks –Territorial surveillance –Safeguarding territorial integrity –Defense of national sovereignty in all situations

4. Helsinki University of Technology Systems Analysis Laboratory 4 Challenges in the evaluation of weapon systems n Several impact dimensions must be accounted for –E.g. enemy and own casualties, mission success probability n Impacts depend on the context –Mission (attack/defence), weather conditions, enemy strategies etc. n There are strong interactions among systems –How can joint impacts be best attributed to constituent systems? –Yet earlier work mainly focused on individual systems n Impacts are often very non-linear –16 artillery guns may not be twice as effective as 8 guns

5. Helsinki University of Technology Systems Analysis Laboratory 5 Modelling of weapon systems n Weapon system portfolio – = number of different weapon systems – = number of weapon systems of the j th type in portfolio x – = cost of portfolio x –Feasible portfolios satisfy all relevant constraints »E.g. budget constraints C(x) ≤ B, logical constraints (incompatibilities etc.) n Impact assessment criteria –Portfolios evaluated with regard to different impact criteria »Enemy casualties, own casualties etc. –Overall impacts approximated by an additive value function

6. Helsinki University of Technology Systems Analysis Laboratory 6 n Estimates from ground battle simulator of Defense Forces –Battle scenario with pre-specified enemy, terrain and mission –Numbers of own weapon systems varied according to an experimental design –Numerous simulations with different portfolios of selected weapon systems –Simulation results extended by interpolation Impact assessment model Criterion 1 Criterion 2 Criterion n Overall impact of the portfolio Impact model... Battle simulator Scenario Enemy Own weapon system

7. Helsinki University of Technology Systems Analysis Laboratory 7 n Feasible weight set –E.g. rank-ordering for criterion importance n Portfolio x’ dominates x if it has greater or equal overall impact for all feasible weights Incomplete information and dominance w 1 =1 w 2 =0 w 1 =0 w 2 =1 w 1 =.5 w 2 =.5 V2V2 V1V1 two criteria; w 1 ≥w 2 V(x’,w) V(x,w)

8. Helsinki University of Technology Systems Analysis Laboratory 8 n Feasible portfolios that are not dominated by any less or equally expensive portfolio Cost-efficient portfolios V1V1 V2V2 COST Cost-efficient portfolios Inefficient portfolios

9. Helsinki University of Technology Systems Analysis Laboratory 9 Numerical example based on realistic data n Three weapon systems –Additive costs n Three impact criteria for different types of enemy casualties n Incomplete information on the value (i.e relevance) of the impacts n Analysis at different budget levels with the aim of identifying cost-efficient portfolios

10. Helsinki University of Technology Systems Analysis Laboratory 10 Simulated and interpolated impact functions x 3 =0 x 3 =1

11. Helsinki University of Technology Systems Analysis Laboratory 11 Impacts of weapon system portfolios Cost-efficient portfolios ~25% Inefficient portfolios ~75%

12. Helsinki University of Technology Systems Analysis Laboratory 12 Composition of cost-efficient portfolios (1/2) Cost-efficient portfolios

13. Helsinki University of Technology Systems Analysis Laboratory 13 Composition of cost-efficient portfolios (2/2) Cost-efficient portfolios Inefficient portfolios x 3 =1 x 3 =0

14. Helsinki University of Technology Systems Analysis Laboratory 14 Extensions and future research n Complementing simulation data with expert evaluations –Simulations can be augmented with judgmental expert evaluations of impacts –This helps overcome the ”curse of dimensionality” with more weapon systems –Experimental design of simulations and/or expert evaluations n Considering multiple battle scenarios –Cost-efficiency is highly context dependent  many scenarios are needed for comprehensiveness –These can be integrated with the MAVT model using probabilities –Risk and/or robustness measures for weapon portfolios can also be formed

15. Helsinki University of Technology Systems Analysis Laboratory 15 Multiple battle scenarios Overall expected value of the portfolio Optimization Weapon system portfolio 1... p1p1 p2p2 pmpm 2 m

16. Helsinki University of Technology Systems Analysis Laboratory 16 Conclusions n Portfolio approach is necessitated by strong interactions  Evaluation of individual weapon systems makes little sense n These interactions are captured by the battle simulator n Multi-criteria model aggregates several impact dimensions –Contextual importance of impacts captured through incomplete information n Cost-efficiency depends on both impacts and costs  Focus on the computation of cost-efficient portfolios

17. Helsinki University of Technology Systems Analysis Laboratory 17 References »Liesiö, J., Mild, P., Salo, A. (2007) Preference Programming for Robust Portfolio Modelling and Project Selection, European Journal of Operational Research, forthcoming »Liesiö, J., Mild, P., Salo, A. (2007) Robust Portfolio Modeling with Incomplete Cost and Budget Information, European Journal of Operational Research, forthcoming. »Stafira, S., Parnell, G., Moore, J., (1997). A Methodology for Evaluating Military Systems in a Counterproliferation Role, Management Science, Vol. 43, No. 10, pp. 1420-1430. »Parnell, G., et. al. (1998). Foundations 2025: A Value Model for Evaluating Future Air and Space Forces, Management Science, Vol. 44, No. 10, pp. 1336-1350.

18. Helsinki University of Technology Systems Analysis Laboratory 18 Questions and comments?