1. When Things Go Wrong: Flight Crews and Emergencies Barbara Burian, Ph.D., FRAeS NASA Ames Research Center The Emergency and Abnormal Situations study was funded through NASA’s former Aviation Safety and Security Program. Human Systems Integration Division

2. Emergency and Abnormal Checklists NORMAL ITEM OVRD NOTES CHKL OVRD CHKL RESET NORMAL MENURESETSNON-NORMAL MENU FIRE ENG R Fire is detected in the right engine. RIGHT AUTOTHROTTLE ARM SWITCH........... OFF RIGHT THRUST LEVER......................CLOSE RIGHT FUEL CONTROL SWITCH.............CUTOFF RIGHT ENGINE FIRE SWITCH..................PULL If FIRE ENG R message remains displayed: 3 2 1 RIGHT ENGINE FIRE SWITCH............. ROTATE Rotate to the stop and hold for 1 second. Boeing 777 ECL

3. Checklists and Procedures Checklist DesignChecklist Content Human Systems Integration Division

4. Checklist Design Factors (Paper, Electronic, EFB) Physical Properties and Interface - size, weight, materials, integration w/displays Typography and use of Symbology - font, font size, boldface, intuitive symbology Layout, Format, Organization - visual look, arrangement, “white” space Purpose - fix, troubleshoot, manage situation, guide CRM Objective (of checklist item) - direct action, inform, assess, make decision Length and Workload - physical length, timing length, workload Nomenclature & Abbreviations - terms, labels, abbreviations Language, Grammar, & Wording - English?, verb tense, reading difficulty, clarity, orientation/perspective, directiveness Level of Detail - amount of information provided Engineering Completeness - all necessary steps included Engineering Coherence - order of steps/timing makes “sense” to aircraft Logical Coherence - order of actions makes sense to the pilot and make “sense” operationally Progression & Checklist Navigation - movement within & between checklists/manuals Access - finding correct checklist, prime real estate pgs.

5. Checklists and Procedures Context Checklist DesignChecklist Content Human Systems Integration Division

6. Pack Trip Human Systems Integration Division Master Caution Alert sounds Crew Identifies that Pack has tripped off Crew completes 4 step procedure Flight proceeds normally

7. ValueJet 592 - In-flight Fire Florida Everglades May 11, 1996 14:04:09 Takeoff 14:10:03 [chirping sound] “’bout to lose a bus” 14:10:15 “Got an electrical problem” 14:10:20 “We’re losing everything” 14:10:22 “We need to go back to Miami 14:10:25 “Fire, fire, fire, fire!” (from cabin) 14:10:32 “Uh, 592 needs immediate return to Miami” (to ATC) 14:13:43 CVR stops recording 14:14 ValuJet 592 disappears from radar

8. Requirements of the Emergency or Abnormal Situation Checklists Procedures Manuals Level of time criticality Degree of threat Degree of novelty, ambiguity, complexity Amount of increase in workload

9. Requirements of the Emergency or Abnormal Situation Operational Requirements Checklists Procedures Manuals SOPs Regulations Different phases of flight Weather, terrain, etc.

10. Requirements of the Emergency or Abnormal Situation Operational Requirements Aircraft and Systems Checklists Procedures Manuals Requirements of the specific malfunction History of false warnings Warning systems and alerts Appropriate level of automation Automated aircraft systems Flight protection envelopes Flight deck ergonomics

11. Requirements of the Emergency or Abnormal Situation Operational Requirements Aircraft and Systems Philosophies and Economic Constraints Checklists Procedures Manuals Manufacturer philosophies Company philosophies and policies Types of checklists available (electronic, paper) Philosophy of checklist use and functionality of electronic checklists Cost-benefit tradeoffs, e.g., diversions Updates and revisions

12. Human Performance Capabilities Checklists and Procedures Checklist DesignChecklist Content Context Human Systems Integration Division

13. Requirements of the Emergency or Abnormal Situation Operational Requirements Human Performance Capabilities and Limitations under High Stress and Workload Aircraft and Systems Philosophies and Economic Constraints Checklists Procedures Manuals Working, long term, and prospective memory Mental computations Judgment and decision making Visual and auditory processing Attention Dealing with interruptions & distractions Situational awareness Ability to shift mental sets Motor skills Affective responses to stress

14. Human Performance under Stress Human Systems Integration Division Well-learned motor skills remain robust and relatively unaffected by stress Our simulator training really paid off. This was my first engine shutdown in 20 years of flying and it felt like I had done it a thousand times before! (ASRS Report, Accession #466167)

15. Human Performance under Stress Human Systems Integration Division Tunneling narrowing of human attention restricts scanning of environmental cues narrow focus on most salient or threatening cues yields poor differential diagnosis of situation Working Memory capacity and length of time information can be held decreases when exceeded – difficulty performing mental calculations, problem solving, making sense of disparate pieces of information, shifting mental sets (concurrent task management) Tendency to Rush

16. Human Systems Integration Division Human Performance under Stress We did find communication difficult and the use of oxygen masks, intercom, trying to talk to ATC was a handful. At night made it that much harder to read/accomplish checklist items. Turning cockpit lights on sooner would have helped. (ASRS Report, Accession #472755)

17. Checklists and Procedures Checklist DesignChecklist Content Context How Checklists are Used Human Performance Capabilities

18. Example: Saudi Arabian Flight 163, August 19, 1980 Crew searched unsuccessfully for several minutes for a cargo fire checklist in the Abnormal section of the QRH. The checklist they were looking for, but never found, was in the Emergency section of the QRH instead. Human Systems Integration Division Checklists and Procedures: Context and Human Performance Considerations

19. Example: Six Different Checklist Titles for the Same Condition – B737 no flap, no slat All Flaps Up Landing Flaps – All Flap and LED Up Landing No Flap/No Slat Landing Symmetrical Non-Normal Trailing Edge Flaps or No Flaps No Trailing Edge Flaps Alternate Flaps Operation Human Systems Integration Division Checklists and Procedures: Context and Human Performance Considerations

20. Example: Pilots routinely make errors in correctly recalling “memory items” Air Carrier/Manuf. N of CL with MI Total N of MI Action Item MI Conditional MI Note MI Other MI A Classic23120932133 B Classic41513101 C Classic161127316212 D Classic51715200 Boeing Classic161137316222 E NG92017300 F NG31110100 G NG124537521 H NG104435522 Boeing NG181298319243 Boe. NG – Rev.13775210141 Checklists and Procedures: Context and Human Performance Considerations

21. Human Systems Integration Division Example: Applying multipliers to landing distances 1.7 1.55 1.8 1.35 1.15 Checklists and Procedures: Context and Human Performance Considerations

22. Example: Information missing from checklist – Valujet 558, Jan. 7, 1996 The missing information was included in the AOM expanded checklists but was never transferred to the QRH checklists. Pull cbs Reset cbs AOM Checklists and Procedures: Context and Human Performance Considerations

23. Human Systems Integration Division Example: Confusing and complex wording If Pack Fault due to low bleed air supply, then a bleed leak does not exist, and if WING ANTI-ICE is not required: If Pack Fault due to low bleed air supply, and if a bleed leak does not exist, and if WING ANTI-ICE is not required: Checklists and Procedures: Context and Human Performance Considerations

24. Human Systems Integration Division Example: Crew confusion – FedEx 1406, September 5, 1996 FE was confused by step 5 and did not complete steps 6 and 7 Items Pertaining to Adjusting Cabin Altitude or Flight Level Checklists and Procedures: Context and Human Performance Considerations

25. Start Example: Complex Navigation

26. Human Systems Integration Division Accomplishing the [immediate action] item for cockpit/cabin smoke on the ground in the XXX aircraft induces the abnormal procedure of equipment overheat due to the step of the turning off left and right recirculation fans, the left recirculation fan being the primary equipment cooling on the ground. (ASRS Report, Accession #473359) Checklists and Procedures: Context and Human Performance Considerations Example: Checklist actions cause another abnormal situation to occur

27. Human Systems Integration Division Hydraulic caution light illuminated while taxiing…I completed the QRH checklist…We rolled to a stop in the grass…A very poorly written QRH emergency checklist. CALLBACK: …The checklist is for use in- flight, not on the ground. (ASRS Report, Accession #437817) Checklists and Procedures: Context and Human Performance Considerations Example: Checklist actions not appropriate for situation

28. Human Systems Integration Division Example: Swissair 111, Sept. 2, 1998 Checklists and Procedures: Context and Human Performance Considerations

29. Human Systems Integration Division Example: Swissair 111, September 2, 1998 - continued If smoke/fumes are not eliminated, land at nearest suitable airport Time needed for completion of the two checklists: Approximately 30 minutes Time from first abnormal odor until Swissair 111 crashed in the ocean: 20 minutes, 40 seconds Checklists and Procedures: Context and Human Performance Considerations

30. Human Performance and Context in Emergency and Abnormal Checklists Human Systems Integration Division can be written to address the multiple contexts in which emergency and abnormal situations occur can be designed to accommodate many human performance limitations that occur under high stress and high workload The Good News: Emergency and Abnormal Checklists and Procedures

31. Human Systems Integration Division To Ensure that Context is Addressed: Addressing Context in Emergency and Abnormal Checklists Make sure checklists are designed for full range of the scenarios for which they will be used (e.g., pressurization: slow leak – to – explosive/rapid decompression) Make sure checklists are appropriate for all phases of flight and aircraft configurations for which they may be used (e.g., on ground, in-flight, throttles at idle) Make sure checklists address relevant environmental conditions (e.g., icing) and loss of pertinent equipment that has been MEL’ed. Conduct a realistic timing assessment of crucial checklists and procedures (length of completion). Conduct a realistic workload assessment of checklists and procedures, especially when conducted during various phases of flight.

32. Human Systems Integration Division To Ensure that Tunneling is Addressed: Draw crew attention to environmental or situational cues that support diagnosis Draw crew attention to environmental or situational cues that contraindicate diagnosis Cues specified must be ones that flight crews are able to assess Addressing Human Performance under Stress in Emergency and Abnormal Checklists

33. To Ensure that Working Memory Limitations are Addressed: Provide information or resources (e.g., EFB) to diminish/eliminate memory load and need to perform mental calculations Place remaining “memory items” on quick reference cards/QRH cover (paper and unannunciated electronic checklists) Integrate all needed information (tables, normal checklists) with emergency and abnormal checklists – “get in, stay in” Make sure all checklists are complete and ramification of crucial steps is provided (before the step is to be carried out) Provide “purpose of item” and “purpose of checklist” statements Provide information describing aircraft performance limitations Provide information describing remaining aircraft capabilities However, don’t go overboard with providing so much information that checklists take forever to go through/read Addressing Human Performance under Stress in Emergency and Abnormal Checklists

34. Barbara Burian, Ph.D. bburian@mail.arc.nasa.gov Emergency and Abnormal Situations Study http://human-factors.arc.nasa.gov/eas Human Systems Integration Division

35. The Context in Which Emergency and Abnormal Situations Occur JetBlue 292 Aloha 243 Air Canada 797 Explosive Decompression Misaligned Nosegear In-flight Fire

36. In-flight Smoke, Fire, Fumes Integrated Checklist Courtesy of United Airlines