1. Monitoring Normal Operations and the Line Operations Safety Audit (LOSA): The Perspective of ICAO Captain Dan Maurino Captain Dan Maurino Flight Safety and Human Factors – ICAO Royal Aeronautical Society Conference on Mitigating Human Error London, 15 October 2003

2. Systemdesign Baseline performance Baselineperformance Operationalperformance “ Practical Drift” Operationaldeployment When the Book Goes Out of the Window

3. Raiding the Lost Ark NaturalPerformance “Angel”Performance Regulator Nobody LineChecks LOSA

4. A Day at the Office A Day at the Office To achieve daily safety and production goals, flight crews must Threat Management Error Management Undesired Aircraft State Management 3 Manage aircraft deviations LOSA Indices Manage operational complexity 1 Manage their own errors 2

5.  Not flight crew errors  Not deficiencies in the system Factors external to the flight deck that increase complexity of flight operations Factors external to the flight deck that increase complexity of flight operations  Must be managed by the flight crew Threats = Red flags Threats = Red flags LOSA Index : Threats LOSA Index : Threats

6. Threats Are the Context Weather Maintenance GroundCrew Ground Crew CabinCrew Cabin Crew Passenger events ATC Terrain Similarcallsigns Similar call signs Timepressures Time pressures Heavytraffic Heavy traffic Unfamiliarairports Unfamiliar airports Automationevents Automation events Missedapproaches Missed approaches Flight diversions Systemmalfunctions Distractions

8. LOSA Index : [Expert] Errors LOSA Index : [Expert] Errors

10. LOSA Provides Data Only ProficiencyKnowledge CRM Skills Procedural Compliance Crew Strengths and Weaknesses Corporate Culture Airports / Navigational Aids Aircraft Design and Automation Standards / Maintenance / Training Crew Support Systemic Strengths and Weaknesses LOSA Raw Data LOSA Report

12. Targets for Improvement  Stabilized Approaches  Checklists  Procedural errors  Automation errors  ATC communications  Intentional non-compliance

13. SCP: Data Develops Into Information Re-measure ProficiencyKnowledge CRM Skills Procedural Compliance Crew Strengths and Weaknesses Corporate Culture Airports / Navigational Aids Aircraft Design and Automation Standards / Maintenance / Training Crew Support Systemic Strengths and Weaknesses Safety Change Process(SCP) LOSA Raw Data LOSA Report

14. Reduced 49% Reduced 59% Reduced 59%

15. LOSA 1996 vs. 2000 Checklist Errors 11% of the flights9% of the flights LOSA 2000 385 flights Missed items Wrong response to a challenge Wrong checklist performed Unintentional checklist omission Performed from memory Nonstandard phraseology Self initiated Not called “complete ” Intentional checklist omission Checklist Error Examples 33% of the flights Flights with protocol checklist errors 9% of the flights Flight with unintentional checklist errors LOSA 1996 389 flights LOSA 2000 indicated a significant drop in protocol checklist errorswhile unintentional checklist errors have remained unchanged LOSA 2000 indicated a significant drop in protocol checklist errors while unintentional checklist errors have remained unchanged

16. Threat & Error Management Model (Doc 9803) CrewError Crew Error Inconsequential ErrorInduced Incidentor Accident Error Induced Incident or Accident Threats ThreatManagement Threat Management ThreatInduced Incidentor Accident Threat Induced Incident or Accident UndesiredAircraft State Undesired Aircraft State CrewUndesired AircraftState Responses Crew Undesired Aircraft State Responses CrewError Responses Crew Error Responses

17. TEM Model as Licensing tool (ICAO) Licensing tool (ICAO) Training tool (Numerous airlines) Training tool (Numerous airlines) Safety management tool (IATA) Safety management tool (IATA) Research tool (Boeing) Research tool (Boeing) The Expanding Role of the TEM Model

18. TEM - Licensing Tool (ICAO FCLT/P) Performance criteria Competencies SKA’s Error Management Threat Management Undesired Aircraft State Management Follow SOPsFollow SOPs Perform BriefingPerform Briefing State plansState plans Assign workloadAssign workload Contingency planningContingency planning Monitor / Cross-checkMonitor / Cross-check Manage automationManage automation … … … CRM skills TEM principles

19. Managing the Red Flags System resources  GPWS  Automation  ATC  Dispatch  SOP’s  Checklists  Briefings  Training  …

20. Managing the Red Flags (cont.) The human contribution  Proficiency  Vigilance  Assertiveness  Monitoring  Decision making  Experience  Leadership  Discipline  …

21. Real World Human Performance Flapsomitted Effectivewarning Normaloperation Checklist works

22. Descent, Approach & Landing  TO TOD 10000 10000 ft. Configuration FAF FlareLanding Bottom Lines  Most errors (LOSA)  Most consequential errors (LOSA)  Most accidents worldwide (ALAR) Highest risk Highest risk phase of flight

23. Briefing the Approach  Crews who brief before TOD commit 1.6 times less errors during descent, approach and land than crews who brief after TOD  Landing runway changes occurred 8 percent of the time Training point: brief approach prior TOD Training point: brief approach prior TOD

24. TEM Model - Training Tool CRM, circa 2003 Beyond behavioural stereotypes Beyond behavioural stereotypes Operational performance Operational performance Develop and support threat and error management competencies Develop and support threat and error management competencies  The bottom line is safe and efficient operational performance, not CRM

25. SMS Data Triangle Normal Operations FDA Voluntary Reporting

26. A Casque Bleu Perspective  LOSA: a tool to capture systemic data  NOTECHS: a CRM evaluation template  LOSA: context and cognition  NOTECHS/CRM: behaviours  CRM: just one piece of the infinitely broader information picture developed from LOSA A situation, generated by misperceptions and misinformation, that must be put behind A situation, generated by misperceptions and misinformation, that must be put behind LOSA, CRM and NOTECHS