A Glimpse at Archie: The LOSA Archive Ashleigh Merritt, Ph.D. The University of Texas Human Factors Research Project (UT) Third ICAO/IATA LOSA/TEM Conference September 13-14, 2005, Kuala Lumpur

What is Archie? “Archie” is the LOSA Archive A database of de-identified airline data which is maintained and updated by UT Airlines that do a LOSA with The LOSA Collaborative contribute their data to Archie for research purposes

How big is Archie? As of October 2004, Archie had data from 15 airlines 3,309 flights 11,000+ threats 7,000+ errors 1,500+ undesired aircraft states

How old is Archie? Five LOSAs in five years to develop the methodology Data in the Archive starts in 2000 Had to wait for Archie to grow before doing any industry-wide analyses. If we think of Archie as a dog, he would be a young bloodhound

How big is Archie? As of August 2005, Archie has data from 20 airlines (including 3 repeat LOSAs) 4800 flights 4800 flight narratives written by trained observers 4800 ratings of TEM countermeasure performance 17,500+ threats 12,500+ errors 2,400+ undesired aircraft states

Archie’s Attributes 2004 Archie is loyal and faithful Archie alerts of danger Archie is a tracker Archie plays well with others

2005 Update on Archie Archie has remained loyal and faithful to its aviation master No airline has been publicly identified No individual pilots can be or ever have been identified

Archie continues to play well with others 2004 Queries from / data-sharing with: NOSS (ATC group) IATA/ICAO (ITA) Boeing Airlines ASAP ATSB 2005

2005: Archie is learning to track Airline threats ATC threats Intentional Noncompliance

Airline threats: Simple Tracking Airline threats include aircraft malfunctions, time pressure, errors/distractions/interruptions from Cabin, Dispatch, Maintenance, Ground, Ramp. Airline threats occur on 2/3 of all flights, adding to overall workload. Range: 55% - 81% across different airlines Three-quarters of airline threats occur before the aircraft has even left the gate (predeparture). TARGETS FOR ENHANCEMENT?

ATC threat chains: Complex tracking 10 LOSAs – 2426 Flights – 9450 threats 2350 ATC threats = ¼ of all threats = about 1 per flight 236 mismanaged ATC threats = 30% of all mismanaged threats 10% of all ATC threats were mismanaged

Mismanaged ATC threats 2426 Flights – 9450 threats Challenging clearances (53%) 236 mismanaged ATC threats Runway Changes (14%) Language difficulty (8%) Similar call signs (5%) ATC error (7%)

ATC threats -> Errors Communication errors (27%) - misinterpret ATC instructions 2426 Flights – 9450 threats 280 errors 236 mismanaged ATC threats Aircraft handling (24%) – unintentional speed, lateral, vertical deviations Ground navigation errors (6%) Cross-verification errors (7%) Automation errors (20%) – wrong MCP/FCU altitude setting dialed

ATC threats -> Errors -> UAS 76 Communication errors 2426 Flights – 9450 threats 2 UAS 280 errors 236 mismanaged ATC threats 67 Aircraft handling errors 58 UAS 16 Ground navigation errors 19 Cross-verification errors 56 Automation errors 9 UAS 21 UAS 5 UAS

236 mismanaged ATC threats ¾ occurred during descent/approach/land ATC threats -> UAS 2426 Flights – 9450 threats Lateral deviation 17% Speed too high 14% Vertical deviation 10% Unstable Approach 10% Continued Landing 9% Incorrect Automation configuration 9% Taxiway/runway incursion 6% Speed too low 6% Incorrect Aircraft configuration 5% Other 14% 280 errors 236 mismanaged ATC threats 106 UAS Bottom line: 4% of flights had a UAS arising from an ATC threat that was mismanaged. ¾ occurred during descent/approach/land

Archie tracks intentional noncompliance Across 22 LOSAs, the average is 40% of flights have one or more noncompliance errors Range: 23% of flights at one airline to 90% (!!) at another Most common noncompliance errors: checklist performed from memory / nonstandard checklist use failure to cross-verify MCP/FCU altitude alerter changes PF makes own MCP/FCU changes

Noncompliance correlates with.. Based on 22 LOSAs (4800 flts), airlines that have more flights with intentional noncompliance errors also have more flights with: Mismanaged threats (r = .7) Mismanaged handling errors (r = .9) Undesired aircraft states (r = .9) Mismanaged UASs (r =.8) Archie was shocked…

Archie’s Future Archie is maturing and getting smarter – more LOSAs allow for more analyses at the airline level, which is where the greatest differences are emerging. Soon a sub-Archive for regional airlines Archie is getting involved in more industry projects If LOSA and ASAP overlap, then focusing on LOSA targets should reduce ASAP incidents… ATC is a big threat to pilots, are pilots a big threat to ATC?

Archie is recognizing his strength LOSA is a macro approach for Safety Management Systems LOSA detects “gross” effects – strengths and weaknesses in the system Management uses the information Self-corrections, training, procedural changes, … If and when certain problems persist, micro approaches can investigate more closely…

The University of Texas Human Factors Research Project www.psy.utexas.edu/HumanFactors