1. Multi-Agent Situation Awareness Error Evolution in Accident Risk Modelling
Sybert Stroeve, Henk Blom, Marco van der Park
National Aerospace Laboratory NLR, Amsterdam
5th USA/Europe ATM R&D Seminar, Budapest, June 2003
Contributing studies:
European Community supported study OPAL (Optimisation Platform for Airports, including Land-side),
NASA Safety Modelling: coupling with a human operator model Air-MIDAS,
TOPAZ.

2. Contents Problem background Situation awareness: definition & model
Situation awareness in accident risk modelling
Initial results
Conclusions

3. Problem background Accident risk modelling for ATM Human operators
Technical systems
Procedures
Nominal and non-nominal conditions
Situation awareness errors represent important class of hazards
Situation awareness error evolution due to multi-agent interactions
Application example: active runway crossing operation
Context: accident risk modelling in air traffic management.
Mathematical models of air traffic operations including models of human operators, technical systems and associated procedures in nominal and non-nominal situations.
Within the field of accident risk assessment it is well recognized that situation awareness and the lack of SA or errors in SA are important contributing factors to accidents.
As you are aware, in air traffic operations there typicaly are many interactions between human operators and technical systems. Consider, for instance, operations on or near the airfield. In such operations with multi-agent interactions, a minor situation awareness error may amplify to significant differences in the SA of agents due to the multiple interactions.
Application example

4. Contents Problem background Situation awareness: definition & model
Situation awareness in accident risk modelling
Initial results
Conclusions

5. Situation awareness in the literature
Situation awareness (Endsley, 1995):
The perception of elements in the environment within a volume of time and space
The comprehension of their meaning
The projection of their status in the near future
Errors in situation awareness at each of these levels (Endsley, 1995)
Following the definition of Endsley, situation awareness is a dynamic state of knowledge by a human, which discerns three levels:
1, 2, 3
Endsley discusses situation awareness errors at each of these levels.
At the first level, a person may wrongly or not perceive task-relevant information. This may, e.g., depend on signal characteristics and perception strategies.
At the second level, a person may wrongly interpret perceived information. This may, e.g. depend on the mis-use of proper mental models of the environment.
At the third level, a person may wrongly predict a future status. For instance, due to a wrong mental model or memory limitations.

6. Situation awareness in a multi-agent environment
Situation awareness error evolution due to intra-agent interaction
Definition of agent: entity which may possess situation awareness of the environment
An agent may be:
Human operator
Technical system(s)
As discussed previously, for complex operations also errors or discrepancies in situation awareness which have evolved due to interactions between agents are important.
Here, an agent is defined as an entity which may possess situation awareness of the environment. As such, it may be a human operator or a technical system. The environment of each agent simply is the whole set of agents.

7. Multi-agent representation of application example
For the active runway crossing example, this diagram shows the agents and their potential interactions. Nine of the agents are human operators and five are technical systems.

8. Situation awareness vector
In the model, the situation awareness of each is represented by a vector which contain the following elements:
- the identity of agents, e.g., the awareness of a pilot concerning the call-sign of a nearby aircraft;
- continuous states of agents, e.g., the awareness by a controller concerning the position and velocity of an aircraft;
- modes of agents, e.g., the awareness of an air traffic controller of the mode of an alert;
- intents of agents, e.g., expectations by a controller regarding a mode of an aircraft when it will pass a way-point.
Mathematical details
You may refer to the paper for more details.
Situation awareness at time t of agent j about agent i

9. Examples of situation awareness updating processes
Observation
state agent 1
SA agent 2
Communication
SA agent 1
SA agent 2
Reasoning
In addition to the definition of the elements of the situation awareness vector, SA modelling should consider how and when the SA may be updated. Here there is a close connection with other elements of cognitive performance models.
Regarding the how question, we consider that the SA may be updated
1) by observation of the current state of another agent,
2) by communication of the current SA of another agent, or 3) by reasoning on the basis of the own SA and decision rules of an agent.
Regarding the when question we consider a task scheduling process, which depends on the tasks of the agent.
Coupling with other elements of cognitive performance modelling.
SA agent
decision rules

10. Integration in human cognitive performance modelling
Contextual control mode model (Hollnagel, 1993)
opportunistic/tactical control modes
Multiple resources model (Wickens, 1992)
Human errror (e.g., Kirwan, 1994)
Human error recovery (Amalberti, 1997)

11. Contents Problem background Situation awareness: definition & model
Situation awareness in accident risk modelling
Initial results
Conclusions

12. Active runway crossing operation
Main human operators:
pilots
runway controller
Main technical systems:
R/T communication systems
active stopbar
stopbar violation alert
runway incursion alert

13. Collision risk tree (simplified)
After the accident risk model has been developed, a suitable risk decomposition has to be chosen. On the one hand, the risk decomposition is required to support efficient Monte carlo simulations of low probabilistic events. On the other hand, the risk decomposition supports insight in the main risk contributions.
For the accident risk model of the active runway crossing, a risk decomposition in 256 events was chosen. The present diagram shows a simplified decomposition in 8 events.
- The first level...
-The second level represents the intent SA of the PF of the crossing a/c, who may intend to either proceed on a runway crossing, or to proceed on a regular taxiway; this latter intend is an erroneous one.
- The third level represents the availability of the alerts of the runway controller. These alerts may be working or not.
- The fourth level represents the status of R/T communication systems. These may be working or not.

14. Contents Problem background Situation awareness: definition & model
Situation awareness in accident risk modelling
Initial results
Conclusions

16. Initial collision risk results
For all leaves of the accident risk tree the accident risk is evaluated using Monte Carlo simulations and event probabilities. Here are the results.

17. Initial collision risk results
Here is again the simplified accident risk tree.

18. Collision avoidance percentages of human operators

19. Contents Problem background Situation awareness: definition & model
Situation awareness in accident risk modelling
Initial results
Conclusions

20. Conclusions Mathematical situation awareness model
Represents situation awareness components of Endsley (1995)
In addition, it accounts for error evolution in a multi-agent environment including technical systems
Accident risk modelling
Technical systems & procedures & human operators
Dynamic evolution of nominal & non-nominal conditions
Results:
Accident risk
Safety critical conditions
Conflict resolution probabilities of human operators
Next step is bias and uncertainty assessment
The proposed situation awareness model can represent the situation awareness levels in the definitions of Endsley.
In addition, we proposed a framework for situation awareness error evolution in a multi-agent environment.
The SA model was integrated in an accident risk model which accounts for technical systems, human operators and associated procedures.
The accident risk model accounts for dynamic evolution of nominal as well as non-nominal conditions. The importance of this dynamic evolution was illustrated for the conflict resolution probabilities of the human operators.
Other results include the overall accident risk of the operation and an assessment of the safety criticality of various conditions.
In accident risk modelling as presented, there are many assumptions made. In a next step we plan to assess bias and uncertainty in the accident risk due to the model assumptions.
Thank you for your attention.