1. A Multistage Influence Diagram Game for Maneuvering Decisions in Air Combat
Kai Virtanen, Janne Karelahti,
Tuomas Raivio, and Raimo P. Hämäläinen
Systems Analysis Laboratory
Helsinki University of Technology

2. Maneuvering decisions in one-on-one air combat
t=Dt
t=0
Outcome depends on all the maneuvers of both players Þ Dynamic game problem
Objective
Find the best maneuvering sequences with respect to the overall goals of a pilot!
- Preference model
- Uncertainties
- Behavior of the adversary
- Dynamic decision environment

3. Influence diagram (Howard, Matheson 1984)
Directed acyclic graphs
Describes the major factors of a decision problem
Offers several possibilities for quantitative analysis
Time
precedence
Informational arc
Alternatives
available to DM
Decision
Random
variables
Conditional arc
Chance
Probabilistic
or functional
dependence
Deterministic
variables
Conditional arc
Deterministic
Conditional arc
A utility
function
Utility

4. Influence diagram (continued)
State of the world is described by attributes
States are associated with
Utility
Probability
Utility is a commensurable measure for goodness of attributes
Results include probability distributions over utility
Decisions based on utility distributions
Information gathering and updating using Bayesian reasoning

5. Decision theoretical maneuvering models
Single stage influence diagram (Virtanen et al. 1999):
Short-sighted decision making
Multistage influence diagram (Virtanen et al. 2004):
Long-sighted decision making
Preference optimal flight path against a given trajectory
Single stage influence diagram game (Virtanen et al. 2003):
Components representing the behavior of the adversary
New multistage influence diagram game model:
Solution with a moving horizon control approach

6. Influence diagram for a single maneuvering decision
Adversary's
Present
State
Adversary's
Maneuver
Adversary’s
State
Measurement
Present
Combat State
Present
Measurement
Combat
State
Situation
Evaluation
Present
State
Maneuver
State
Present
Threat Situation
Assessment
Threat Situation
Assessment

7. Multistage influence diagram air combat game
White
Black
Goals of the players:
1. Avoid being captured by the adversary
2. Capture the adversary
Four possible outcomes
Evolution of the players’ states described by a set of differential equations, a point mass model
Evolution of the probabilities described by Bayes’ theorem

8. Graphical representation of the game
Black’s
viewpoint
Combat state
White's
viewpoint
stage t-1
stage t
Situation
Evaluation
at t-1
at t
Cumulative
expected utility
at t+1
at t-2

9. Threat situation assessment
Infers the threat situation from the viewpoint of a single player
Discrete random variable, four outcomes:
Neutral
Advantage
Disadvantage
Mutual disadvantage
Probabilities are updated with Bayes’ theorem:
Pposterior( outcome | combat state) ∞
Pprior( outcome ) X Plikelihood( combat state | outcome )
Each outcome leads to a specific goal described with a utility function
Sketch of geometry

10. Moving horizon control approach
Players’ states
at stage t
Truncated
influence diagram game
lasting stages t, t+Dt,…, t+KDt
Dynamic
programming
KDt = length of the planning horizon
Game optimal
control sequences over
stages t, t+Dt, …, t+KDt
Terminate?
t:=t+Dt
Resulting game optimal controls
the cumulative expected
utility is maximized
approximative feedback
Nash equilibrium
Players’ states
at stage t+Dt
Implement the
controls of stage t

11. Numerical example Black initially pursuing White
White’s aircraft more agile
White wins
Look-ahead strategies:
one-step, solid lines, payoffs: White/Black = 1.21
two-step, dashed lines, payoffs: White/Black = 1.25
altitude, km
White
Black
y-range, km
x-range, km

12. Threat probability distributions
White
Black
Probability
Probability
time, sec.
time, sec.

13. Effects of the likelihood functions
Threat probability rate of change defined by the likelihood functions
Steep likelihood functions:
Evolution of threat probabilities is sensitive to certain changes in combat state => Outcomes are distinguished sharply
White, steep likelihoods
Black, flat likelihoods

14. Conclusions The multistage influence diagram game:
Models preferences under uncertainty and multiple competing objectives in one-on-one air combat
Takes into account
Rational behavior of the adversary
Dynamics of flight and decision making
The moving horizon control approach:
Game optimal control sequences w.r.t. the preference model of the players
Utilization:
Air combat simulators, a good computer guided aircraft
Contributions to the existing air combat game formulations:
New way to treat uncertainties in air combat modeling
Roles of the players are varied dynamically

15. References
Howard, R.A., and Matheson, J.E., “Influence Diagrams,” The Principles and Applications of Decision Analysis, Vol. 2, edited by R.A. Howard and J.E. Matheson, Strategic Decision Group, Palo Alto, CA, 1984.
Virtanen, K., Raivio, T., and Hämäläinen, R.P., “Decision Theoretical Approach to Pilot Simulation,” Journal of Aircraft, Vol. 36, No. 4, 1999.
Virtanen, K., Raivio, T., and Hämäläinen, R.P., “Influence Diagram Modeling of Decision Making in a Dynamic Game Setting,” Proceedings of the 1st Bayesian Modeling Applications Workshop of the 19th Conference on Uncertainty in Artificial Intelligence, 2003.
Virtanen, K., Raivio, T., and Hämäläinen, R.P., “Modeling Pilot's Sequential Maneuvering Decisions by a Multistage Influence Diagram,” Journal of Guidance, Control, and Dynamics, Vol. 27, No. 4, 2004.
Kai’s dissertation available at