1. S ystems Analysis Laboratory Helsinki University of Technology Games and Bayesian Networks in Air Combat Simulation Analysis M.Sc. Jirka Poropudas and Dr.Tech. Kai Virtanen Systems Analysis Laboratory Helsinki University of Technology jirka.poropudas@hut.fi, kai.virtanen@hut.fi

2. S ystems Analysis Laboratory Helsinki University of TechnologyOutline Air combat (AC) simulation Games in validation and optimization – Estimation of games from simulation data – Analysis of estimated games Dynamic Bayesian networks (DBNs)‏ – Estimation of DBNs from simulation data – Analysis of estimated DBNs Conclusions

3. S ystems Analysis Laboratory Helsinki University of Technology Air Combat Simulation Commonly used models based on discrete event simulation Most cost-efficient and flexible method Objectives for AC simulation studies: Acquire information on systems performance Compare tactics and hardware configurations Increase understanding of AC and its progress

4. S ystems Analysis Laboratory Helsinki University of Technology Discrete Event Simulation Model Simulation input Aircraft and hardware configurations Tactics Decision making parameters Simulation output Number of kills and losses Aircraft trajectories AC events etc. Decision making logic Aircraft, weapons and hardware models Stochastic elements Validation of the model? Optimization of output? Evolution of simulation?

5. S ystems Analysis Laboratory Helsinki University of Technology Existing Approaches to Simulation Analysis Simulation metamodels – Mappings from simulation input to output -Response surface methods, regression models, neural networks Validation methods – Real data, expert knowledge, statistical methods, sensitivity analysis Simulation-optimization methods – Ranking and selection, stochastic gradient approximation, metaheuristics, sample path optimization

6. S ystems Analysis Laboratory Helsinki University of Technology Limitations of Existing Approaches Existing approaches are one-sided – Action of the adversary is not taken into account – Two-sided setting studied with games Existing approaches are static – AC is turned into a static event – Time evolution studied with dynamic Bayesian networks

7. S ystems Analysis Laboratory Helsinki University of Technology Games from Simulation Data Definition of scenario –Aircraft, weapons, sensory and other systems –Initial geometry –Objectives = Measures of effectiveness (MOEs)‏ –Available tactics and systems = Tactical alternatives Simulation of the scenario –Input: tactical alternatives –Output: MOE estimates Games estimated from the simulation data Games used for validation and/or optimization

8. S ystems Analysis Laboratory Helsinki University of Technology Estimation of Games RED 0.0040.036 -0.833 x3x3 0.023 0.013 -0.811 x2x2 0.885 0.855 -0.077 x1x1 y3y3 y2y2 y1y1 BLUE III Ix3x3 Ix2x2 IV IIx1x1 y3y3 y2y2 y1y1 RED, min BLUE, max MOE estimatesPayoff Discrete tactical alternatives x and y Analysis of variance Simulation Discrete decision variables x and y Game

9. S ystems Analysis Laboratory Helsinki University of Technology Estimation of Games MOE estimates Payoff Continuous tactical alternatives x and y Simulation Continuous decision variables x and y Game Blue x Red y MOE estimate Blue x Red y Payoff Regression analysis Experimental design

10. S ystems Analysis Laboratory Helsinki University of Technology Analysis of Games Validation: Confirming that the simulation model performs as intended – Comparison of the scenario and properties of the game – Symmetry, dependence between decision variables and payoffs, best responses and Nash equilibria Optimization: Comparison of effectiveness of tactical alternatives – Different payoffs, best responses and Nash equilibria, dominance between alternatives, max-min solutions

11. S ystems Analysis Laboratory Helsinki University of Technology Example: Missile Support Time Game Phase 1: Support Relay radar information on the adversary to the missile Phase 2: Extrapolation Phase 3: Locked Symmetric one-on-one scenario Tactical alternatives: Support times x and y Objective => MOE: combination of kill probabilities Simulation using X-Brawler

12. S ystems Analysis Laboratory Helsinki University of Technology Game Payoffs Regression models for kill probabilities: Probability of Blue killProbability of Red kill Blue’s support time x Red’s support time y Payoff: Weighted sum of kill probabilities Blue: w B *Blue kill prob. + (1-w B )*Red kill prob. Red: w R *Red kill prob. + (1-w R )*Blue kill prob. Weights = Measure of aggressiveness

13. S ystems Analysis Laboratory Helsinki University of Technology Best Responses Best response = Optimal support time against a given support time of the adversary Best responses with different weights Nash equilibria: Intersections of the best responses W R =0 W R =0.5 W R =0.25 W R =0.75 W B =0.75 W B =0.5 W B =0.25 W B =0 Blue’s support time x Red’s support time y

14. S ystems Analysis Laboratory Helsinki University of Technology Analysis of Game Symmetry – Symmetric kill probabilities and best responses Dependency – Increasing support times => Increase of kill probabililties Different payoffs – Increasing aggressiveness (higher values of w B and w R ) => Longer support times Best responses & Nash equilibria – Increasing aggressiveness (higher values of w B and w R ) => Longer support times

15. S ystems Analysis Laboratory Helsinki University of Technology DBNs from Simulation Data Definition of simulation state –Aircraft, weapons, sensory and other systems Simulation of the scenario –Input: tactical alternatives –Output: simulation state at all times DBNs estimated from the simulation data –Network structure –Network parameters DBNs used to analyze evolution of AC –Probabilities of AC states at time t –What if -analysis

16. S ystems Analysis Laboratory Helsinki University of Technology Definition of State of AC 1 vs. 1 AC Blue and Red B t and R t = AC state at time t State variable values “Phases” of simulated pilots – Part of the decision making model – Determine behavior and phase transitions for individual pilots – Answer the question ”What is the pilot doing at time t?” Example of AC phases in X-Brawler simulation model

17. S ystems Analysis Laboratory Helsinki University of Technology Dynamic Bayesian Network for AC Dynamic Bayesian network – Nodes = variables – Arcs = dependencies Dependence between variables described by – Network structure – Conditional probability tables Time instant t presented by single time slice Outcome O t depends on B t and R t time slice

18. S ystems Analysis Laboratory Helsinki University of Technology Dynamic Bayesian Network Fitted to Simulation Data Basic structure of DBN is assumed Additional arcs added to improve fit Probability tables estimated from simulation data

19. S ystems Analysis Laboratory Helsinki University of Technology Continuous probability curves estimated from simulation data DBN model re-produces probabilities at discrete times DBN gives compact and efficient model for the progress of AC Evolution of AC

20. S ystems Analysis Laboratory Helsinki University of Technology What If -Analysis Evidence on state of AC fed to DBN For example, blue is engaged within visual range combat at time 125 s – How does this affect the progress of AC? – Or AC outcome? DBN allows fast and efficient updating of probability distributions – More efficient what-if analysis No need for repeated re-screening simulation data

21. S ystems Analysis Laboratory Helsinki University of TechnologyConclusions New approaches for AC simulation analysis – Two-sided and dynamic setting – Simulation data represented in informative and compact form Game models used for validation and optimization Dynamic Bayesian networks used for analyzing the evolution of AC Future research: – Combination of the approaches => Influence diagram games

22. S ystems Analysis Laboratory Helsinki University of TechnologyReferences »Anon. 2002. The X-Brawler air combat simulator management summary. Vienna, VA, USA: L-3 Communications Analytics Corporation. »Gibbons, R. 1992. A Primer in Game Theory. Financial Times Prenctice Hall. »Feuchter, C.A. 2000. Air force analyst’s handbook: on understanding the nature of analysis. Kirtland, NM. USA: Office of Aerospace Studies, Air Force Material Command. »Jensen, F.V. 2001. Bayesian networks and decision graphs (Information Science and Statistics). Secaucus, NJ, USA: Springer-Verlag New York, Inc. »Law, A.M. and W.D. Kelton. 2000. Simulation modelling and analysis. New York, NY, USA: McGraw-Hill Higher Education. »Poropudas, J. and K. Virtanen. 2007. Analyzing Air Combat Simulation Results with Dynamic Bayesian Networks. Proceedings of the 2007 Winter Simulation Conference. »Poropudas, J. and K. Virtanen. 2008. Game Theoretic Approach to Air Combat Simulation Model. Submitted for publication. »Virtanen, K., T. Raivio, and R.P. Hämäläinen. 1999. Decision theoretical approach to pilot simulation. Journal of Aircraft 26 (4):632-641.