1 S ystems Analysis Laboratory Helsinki University of Technology Kai Virtanen, Janne Karelahti, Tuomas Raivio, and Raimo P. Hämäläinen Systems Analysis Laboratory Helsinki University of Technology A Multistage Influence Diagram Game for Maneuvering Decisions in Air Combat

2 S ystems Analysis Laboratory Helsinki University of Technology Maneuvering decisions in one-on-one air combat Outcome depends on all the maneuvers of both players  Dynamic game problem Objective Find the best maneuvering sequences with respect to the overall goals of a pilot! - Preference model - Uncertainties - Behavior of the adversary - Dynamic decision environment t=  t t=0  t=  t 

3 S ystems Analysis Laboratory Helsinki University of Technology Influence diagram (Howard, Matheson 1984) Directed acyclic graphs Describes the major factors of a decision problem Offers several possibilities for quantitative analysis Decision Chance Deterministic Informational arc Conditional arc Alternatives available to DM Random variables Deterministic variables Time precedence Probabilistic or functional dependence Utility A utility function Conditional arc

4 S ystems Analysis Laboratory Helsinki University of Technology Influence diagram (continued) State of the world is described by attributes States are associated with –Utility –Probability Utility is a commensurable measure for goodness of attributes Results include probability distributions over utility Decisions based on utility distributions Information gathering and updating using Bayesian reasoning

5 S ystems Analysis Laboratory Helsinki University of Technology Decision theoretical maneuvering models Single stage influence diagram (Virtanen et al. 1999): –Short-sighted decision making Multistage influence diagram (Virtanen et al. 2004): –Long-sighted decision making –Preference optimal flight path against a given trajectory Single stage influence diagram game (Virtanen et al. 2003): –Short-sighted decision making –Components representing the behavior of the adversary New multistage influence diagram game model: Long-sighted decision making Components representing the behavior of the adversary Solution with a moving horizon control approach

6 S ystems Analysis Laboratory Helsinki University of Technology Influence diagram for a single maneuvering decision Adversary’s State Maneuver Present Threat Situation Assessment State Combat State Threat Situation Assessment Situation Evaluation Present State Adversary's Present State Present Combat State Measurement Present Measurement Adversary's Maneuver

7 S ystems Analysis Laboratory Helsinki University of Technology Multistage influence diagram air combat game Goals of the players: 1. Avoid being captured by the adversary 2. Capture the adversary Four possible outcomes Evolution of the players’ states described by a set of differential equations, a point mass model Evolution of the probabilities described by Bayes’ theorem Black White

8 S ystems Analysis Laboratory Helsinki University of Technology Graphical representation of the game Black’s viewpoint White's viewpoint Combat state Situation Evaluation at t-1 Situation Evaluation at t Situation Evaluation Cumulative expected utility Situation Evaluation at t+1 Situation Evaluation at t-2 stage t-1stage t

9 S ystems Analysis Laboratory Helsinki University of Technology Threat situation assessment Infers the threat situation from the viewpoint of a single player Discrete random variable, four outcomes: –Neutral –Advantage –Disadvantage –Mutual disadvantage Probabilities are updated with Bayes’ theorem: Pposterior( outcome | combat state) ∞ Pprior( outcome ) X Plikelihood( combat state | outcome ) Each outcome leads to a specific goal described with a utility function Sketch of geometry

10 S ystems Analysis Laboratory Helsinki University of Technology Players’ states at stage t Truncated influence diagram game lasting stages t, t+  t,…, t+K  t Game optimal control sequences over stages t, t+  t, …, t+K  t Implement the controls of stage t Players’ states at stage t+  t t:=t+  t Moving horizon control approach Dynamic programming Terminate? K  t = length of the planning horizon Resulting game optimal controls -the cumulative expected utility is maximized -approximative feedback Nash equilibrium

11 S ystems Analysis Laboratory Helsinki University of Technology Numerical example Black initially pursuing White White’s aircraft more agile White wins Look-ahead strategies: –one-step, solid lines, payoffs: White/Black = 1.21 –two-step, dashed lines, payoffs: White/Black = 1.25 x-range, km y-range, km altitude, km White Black

12 S ystems Analysis Laboratory Helsinki University of Technology Threat probability distributions Black White time, sec. Probability

13 S ystems Analysis Laboratory Helsinki University of Technology Effects of the likelihood functions Threat probability rate of change defined by the likelihood functions Steep likelihood functions: –Evolution of threat probabilities is sensitive to certain changes in combat state => Outcomes are distinguished sharply White, steep likelihoods Black, flat likelihoods

14 S ystems Analysis Laboratory Helsinki University of Technology Conclusions The multistage influence diagram game: –Models preferences under uncertainty and multiple competing objectives in one-on-one air combat –Takes into account Rational behavior of the adversary Dynamics of flight and decision making The moving horizon control approach: –Game optimal control sequences w.r.t. the preference model of the players Utilization: –Air combat simulators, a good computer guided aircraft –Contributions to the existing air combat game formulations: New way to treat uncertainties in air combat modeling Roles of the players are varied dynamically

15 S ystems Analysis Laboratory Helsinki University of Technology References Howard, R.A., and Matheson, J.E., “Influence Diagrams,” The Principles and Applications of Decision Analysis, Vol. 2, edited by R.A. Howard and J.E. Matheson, Strategic Decision Group, Palo Alto, CA, Virtanen, K., Raivio, T., and Hämäläinen, R.P., “Decision Theoretical Approach to Pilot Simulation,” Journal of Aircraft, Vol. 36, No. 4, Virtanen, K., Raivio, T., and Hämäläinen, R.P., “Influence Diagram Modeling of Decision Making in a Dynamic Game Setting,” Proceedings of the 1st Bayesian Modeling Applications Workshop of the 19th Conference on Uncertainty in Artificial Intelligence, Virtanen, K., Raivio, T., and Hämäläinen, R.P., “Modeling Pilot's Sequential Maneuvering Decisions by a Multistage Influence Diagram,” Journal of Guidance, Control, and Dynamics, Vol. 27, No. 4, Kai’s dissertation available at