1. A Glimpse at Archie: The LOSA Archive
Ashleigh Merritt, Ph.D.
The University of Texas Human Factors Research Project (UT)
Third ICAO/IATA LOSA/TEM Conference
September 13-14, 2005, Kuala Lumpur

2. What is Archie? “Archie” is the LOSA Archive
A database of de-identified airline data which is maintained and updated by UT
Airlines that do a LOSA with The LOSA Collaborative contribute their data to Archie for research purposes

3. How big is Archie? As of October 2004, Archie had data from
15 airlines
3,309 flights
11,000+ threats
7,000+ errors
1,500+ undesired aircraft states

4. How old is Archie? Five LOSAs in five years to develop the methodology
Data in the Archive starts in 2000
Had to wait for Archie to grow before doing any industry-wide analyses.
If we think of Archie as a dog, he would be a young bloodhound

5. How big is Archie? As of August 2005, Archie has data from
20 airlines (including 3 repeat LOSAs)
4800 flights
4800 flight narratives written by trained observers
4800 ratings of TEM countermeasure performance
17,500+ threats
12,500+ errors
2,400+ undesired aircraft states

6. Archie’s Attributes 2004 Archie is loyal and faithful
Archie alerts of danger
Archie is a tracker
Archie plays well with others

7. 2005 Update on Archie
Archie has remained loyal and faithful to its aviation master
No airline has been publicly identified
No individual pilots can be or ever have been identified

8. Archie continues to play well with others
2004
Queries from / data-sharing with:
NOSS (ATC group)
IATA/ICAO (ITA)
Boeing
Airlines
ASAP
ATSB
2005

9. 2005: Archie is learning to track
Airline threats
ATC threats
Intentional Noncompliance

10. Airline threats: Simple Tracking
Airline threats include aircraft malfunctions, time pressure, errors/distractions/interruptions from Cabin, Dispatch, Maintenance, Ground, Ramp.
Airline threats occur on 2/3 of all flights, adding to overall workload.
Range: 55% - 81% across different airlines
Three-quarters of airline threats occur before the aircraft has even left the gate (predeparture).
TARGETS FOR ENHANCEMENT?

11. ATC threat chains: Complex tracking
10 LOSAs – 2426 Flights – 9450 threats
2350 ATC threats
= ¼ of all threats
= about 1 per flight
236 mismanaged ATC threats
= 30% of all mismanaged threats
10% of all ATC threats were mismanaged

12. Mismanaged ATC threats
2426 Flights – 9450 threats
Challenging clearances (53%)
236 mismanaged ATC threats
Runway Changes (14%)
Language difficulty (8%)
Similar call signs (5%)
ATC error (7%)

13. ATC threats -> Errors
Communication errors (27%) - misinterpret ATC instructions
2426 Flights – 9450 threats
280 errors
236 mismanaged ATC threats
Aircraft handling (24%) – unintentional speed, lateral, vertical deviations
Ground navigation errors (6%)
Cross-verification errors (7%)
Automation errors (20%) – wrong MCP/FCU altitude setting dialed

14. ATC threats -> Errors -> UAS
76 Communication errors
2426 Flights – 9450 threats 2
UAS
280 errors
236 mismanaged ATC threats
67 Aircraft handling errors 58
UAS
16 Ground navigation errors
19 Cross-verification errors
56 Automation errors 9
UAS 21
UAS 5
UAS

15. 236 mismanaged ATC threats ¾ occurred during descent/approach/land
ATC threats -> UAS
2426 Flights – 9450 threats
Lateral deviation %
Speed too high %
Vertical deviation %
Unstable Approach %
Continued Landing %
Incorrect Automation configuration %
Taxiway/runway incursion %
Speed too low %
Incorrect Aircraft configuration % Other %
280 errors
236 mismanaged ATC threats
106 UAS
Bottom line:
4% of flights had a UAS arising from an ATC threat that was mismanaged.
¾ occurred during descent/approach/land

16. Archie tracks intentional noncompliance
Across 22 LOSAs, the average is 40% of flights have one or more noncompliance errors
Range: 23% of flights at one airline to 90% (!!) at another
Most common noncompliance errors:
checklist performed from memory / nonstandard checklist use
failure to cross-verify MCP/FCU altitude alerter changes
PF makes own MCP/FCU changes

17. Noncompliance correlates with..
Based on 22 LOSAs (4800 flts), airlines that have more flights with intentional noncompliance errors also have more flights with:
Mismanaged threats (r = .7)
Mismanaged handling errors (r = .9)
Undesired aircraft states (r = .9)
Mismanaged UASs (r =.8)
Archie was shocked…

18. Archie’s Future
Archie is maturing and getting smarter – more LOSAs allow for more analyses at the airline level, which is where the greatest differences are emerging.
Soon a sub-Archive for regional airlines
Archie is getting involved in more industry projects
If LOSA and ASAP overlap, then focusing on LOSA targets should reduce ASAP incidents…
ATC is a big threat to pilots, are pilots a big threat to ATC?

19. Archie is recognizing his strength
LOSA is a macro approach for Safety Management Systems
LOSA detects “gross” effects – strengths and weaknesses in the system
Management uses the information
Self-corrections, training, procedural changes, …
If and when certain problems persist, micro approaches can investigate more closely…

20. The University of Texas
Human Factors Research Project