1. Authors: Amy R. Pritchett, Scottie-Beth Fleming Presented by: Rachel A. Haga Cognitive Engineering Center, Georgia Tech 32 nd DASC, Oct. 9, 2013 Pilot Compliance to TCAS Resolution Advisories Work sponsored by the FAA, Tom McCloy as Technical Monitor

2. Outline 2 Background & Motivation Experimental Setup Observed Pilot Compliance Potential Interventions Conclusions

3. Potential Interventions Pilot Compliance Experimental SetupMotivation & Background Traffic alert and Collision Avoidance System (TCAS) o TCAS delivers a two stage advisory and vertical avoidance maneuver o Traffic Advisory (TA) - ‘Traffic Traffic’ o Resolution Advisory (RA) - ‘Climb Climb’ o Pilot is to follow an RA, even if it conflicts with ATC instructions, unless the pilot believes that safe flight would be jeopardized Federal Aviation Administration (2000). Introduction to TCAS II Version 7. Washington, D.C. 3 Traffic Situation Display Vertical Speed Indicator with Climb RA

4. 4 Motivation & Purpose Conclusions Potential Interventions Pilot Compliance Experimental SetupMotivation & Background For The Experiments TOPA observed compliance rates of Motivation : TOPA observed compliance rates of 41% to Climb RA’s and 59% to Descend RA’s Purpose : Study contributing factors and potential solutions to inform future FAA regulation For This Work Motivation: Have concluded four studies Purpose: Condensing the message into the essential ‘So What?’ Olson, W. and J. Olszta (2010). TCAS Operational Performance Assessment in the U.S. National Airspace. IEEE/AIAA Digital Avionics Systems Conference. Olszta, J., & Olson, W. (2011). Characterization and Analysis of Traffic Alert and Collision Avoidance Resolution Advisories Resulting for 500' and 1,000' Vertical Separation. Paper presented at the Ninth USA/Europe Air Traffic Management Research and Development Seminar (ATM 2011), Berlin, Germany.

5. The Experiment Setup

6. Conclusions Potential Interventions Pilot Compliance Motivation & BackgroundExperimental Setup Participants o Active air transport or regional pilots o Received TCAS training according to FAA training standards o 12 – 18 pilots per study Traffic events o Advisory type and aircraft trajectory o TA, Climb RA, Descend RA, Crossing Descend RA o RA caused by VFR Traffic o ATC information o Traffic Callout, Party-line Information, Conflicting ATC/TCAS Guidance o Traffic density o Heavy and Light Overview of Study 6

7. Conclusions Potential Interventions Pilot Compliance Motivation & BackgroundExperimental Setup Air Traffic Transcripts Charts & Checklists Audio Communications (Aviation Intercom) TCAS Alerts Simulation Architecture Eyetracker PartyLine First Officer Captain Experimenter/Instructor B747-400 Simulator (RFS) TSD PFD ND PFD ND Touch screen SideStick ATC->TSD VGA ATC Air Traffic Simulator (TGF) TCAS Logic Simulator Study in Integrated Flightdeck – ATC Environment Pritchett, A., Fleming, E., Cleveland, W., Zoetrum, J., Popescu, V., & Thakkar, D. (2012). Pilot Interaction with TCAS and Air Traffic Control. Paper presented at the 2nd International Conference on Application and Theory of Automation in Command and Control Systems (ATACCS), London. 7

8. Pilot Compliance

9. Conclusions Potential Interventions Pilot Compliance Motivation & BackgroundExperimental Setup Definition of Compliance 9 Vertical Speed Not in Compliance In Compliance time 5 seconds RA Climb 2 ½ sec TCAS weakens required vertical rate TCAS assumed ¼ g pull-up Clear of Conflict TCAS assumed constant vertical RA rate 10 seconds15 seconds TCAS RA Maneuver Pilot’s Response Fully Complied? No Percentage Compliance 60%

10. 10 Conclusions Potential Interventions Motivation & Background Overall Compliance Rate Experimental Setup Pilot Compliance 68% fully complied with TCAS RAs On average complied for 96.6% duration of a RA

11. 11 Conclusions Potential Interventions Motivation & Background How are pilot’s not complying? Experimental Setup Pilot Compliance Time to disconnect Mean time to disengage: 2.14 s No trend among outliers Tracking vertical speed No buffer = 62% compliance 3 fps buffer = 68% compliance 6% increase Time when pilot first achieved vertical rate

12. 12 Conclusions Potential Interventions Motivation & Background Operational Factors Impacting Compliance Experimental Setup Pilot Compliance RA Type ATC Communication Traffic Density Traffic Events

13. Potential Interventions 1.Traffic Situation Displays 2.Additional Training 3.Automatic RA

14. 14 ConclusionsMotivation & BackgroundExperimental Setup Pilot Compliance Potential Interventions Traffic Situation Displays: Current Traffic Advisory “Traffic Traffic” Resolution Advisory “Climb Climb”

15. 15 ConclusionsMotivation & BackgroundExperimental Setup Pilot Compliance Potential Interventions Traffic Situation Displays: TSD+ TSD+ Layout and Symbology

16. 16 ConclusionsMotivation & BackgroundExperimental Setup Pilot Compliance Potential Interventions Traffic Situation Displays: VSD VSD Layout and Symbology

17. 17 ConclusionsMotivation & BackgroundExperimental Setup Pilot Compliance Potential Interventions Traffic Situation Displays No significant impact on compliance

18. 18 ConclusionsMotivation & BackgroundExperimental Setup Pilot Compliance Potential Interventions Ground and Simulation Training Demonstration Based Training Event Based Training Approach to Training Design Complement of Two Methods +Computer-Based Training  Introduction to TCAS  Traffic Situation Display  TCAS Advising Logic  Traffic Advisories  Resolution Advisories  Example Timeline of RA Evolution +Mid-Training Quizzes +Presents traffic events that create the requirement to act +Builds context and complexity into each scenario as the flight progresses +Uses a more realistic training environment

19. 19 ConclusionsMotivation & BackgroundExperimental Setup Pilot Compliance Potential Interventions Ground and Simulation Training

20. 20 ConclusionsMotivation & BackgroundExperimental Setup Pilot Compliance Potential Interventions Automatic Resolution Advisory Response Pilots left autopilot engaged for 83% (53 of 64 RAs) of the runs Disengaged generally for Crossing Climb RA

21. Conclusions

22. Motivation Pilot compliance to TCAS is low Pilot Compliance In compliance for 96.6% of the duration of the RA, on average Fully complied for 68% of the runs Potential Interventions Changes to TSD had no significant impact Training program was effective Auto RA increased full compliance but greatly decreases percentage compliance for crossing climb 22 Motivation & BackgroundExperimental Setup Pilot Compliance Potential Interventions Conclusions

23. Questions?