1. Management and Reduction of Human Error in ATM
Validation of the JANUS Technique:causal factors of human error in operational incidents
Julia Pounds Ph.D. and Anne Isaac Ph.D.
CAMI/FAA DAS/HUM
USA EUROCONTROL

2. Management and Reduction of Human Error in ATM
Objective
To develop a harmonised method providing a more consistent and effective approach for assessing human error causal factors in ATM incidents.

3. Two approaches
HFACS HERA

4. Harmonization Lighter treatment by HFACS
Addressed in both but to different degrees
Lighter treatment by HERA
Implied in HFACS
Addressed differently
Implied in HFACS

5. Harmonization
JANUS
HERA-JANUS

6. Error Details Perception and Vigilance Response Execution Memory
yes
Did the controller forget recent information/actions OR
forget future action/intentions
forget stored information?
Did the controller mis-see or mishear information OR
Was the information not detected or detected late? no
yes
Response Execution
Memory no
Did the controller immediately perform an action on the identified object. OR
Did the controller make an error in performing an action, or speech? no
yes
Did the controller misjudge information OR
Make an error in planning, problem solving or decision-making?
yes
Planning and Decision-making

7. The Contextual Conditions
More than 200 options under:
Pilot-ATCO Communication
Pilot Actions
Traffic & Airspace
Weather
Documentation & Procedures
Training & Experience
Workplace design & HMI
Environment
Personal Factors
Team Factors
Organisational Factors

8. Time-table

9. Validation Principles
Reliability and Objectivity
Content-Related Validity
Empirical Validity
Practicality and Usability
Face Validity and Acceptance

10. Beta-testing in Europe
From June 2001 to March 2002
Trained 48 investigators from Austria, France, Germany, Netherlands, Sweden, UK/Serco and Portugal
JANUS assessments used paper-based decision flow diagrams
Total incidents: 60 (mix of new incidents and re-analysis of previous reports)

11. Validation in Europe
Incident investigators reached reasonable levels of agreement
Error Type 78%
Error Detail 72%
Error Mechanism 66%
Information Processing 57%
Contextual Conditions 76%
Overall agreement of the group with each individual Incident Investigator 88%

12. Validation in Europe Comparison of HERA-JANUS to previous methods
86% - More detailed, objective, structured, and precise results
86% - More useful information and prompted looking in more detail at contextual conditions
66% - More confidence in investigation process
50% - Controllers accepted HERA-JANUS as better than previous method
100% -Investigators accepted HERA-JANUS as better than previous method

13. Beta-testing in the USA
From December 2001 to September 2002
Participation by 29 facilities with data collected from 12
ARTCCs – 6
TRACONS – 6
JANUS assessments made by CAMI researchers in parallel to but independent from current investigation process
JANUS interviews used PC-based tool that automated decision flow diagrams
Total incidents: 79 (new)

14. Validation in the USA
JANUS interview data collected from 79 primary controllers and 136 other controllers, supervisors, and managers
Inter-rater agreement with the primary controller ranged from 73% for other controllers (kappa=0.48), to 65% for facility managers (kappa=.47) on Perception and Vigilance factors
Per incident:
Primary controllers identified an average 3.6 psychological factors (Planning and Decision Making most frequently reported)
All participants identified an average 9.6 contextual factors (Traffic and Airspace most frequently reported)

15. Validation in the USA
48% of JANUS factors not found in current FAA report form ¨causal factors¨
Results from questionnaires showed positive support
Average number of causal factors with current FAA report form = 1.7
Average number of causal factors with JANUS = 13.2

16. Feedback
“The Technique takes an intensive look behind the incident and helps to eliminate the possible causes from the probable facts”
“It replaces the feeling of guessing with a structured approach”
“More detail, more informative”
“Pinpoints the causal factors”

17. Feedback
Conclusions
Does it work?
How well?
Better than current method?

18. Are there any questions?
Thank You
Are there any questions?