1. International Civil Aviation Organization Welcome to the ADREP/ECCAIRS End-user Course – SSP/SMS Module N° 1 Appreciation of ADREP and ECCAIRS Mexico City – 22-26 November 2010

2. 2 Content and Goal To provide an overview of what ECCAIRS can and cannot do for a Safety Manager ADREP/ECCAIRS End-user Course (Rev. #3)

3. 3 Outline History… Theory… glorious boring but essential theory – ADREP in Annex 13 – Data requirements for SSP Practice… now I get it! – Runway Excursions (for example)  From ALoS to Action Plans ADREP/ECCAIRS End-user Course (Rev. #3)

4. 4 ICAO – ADREP ADREP – Accident Incident Data Reporting System – ADREP 2000 Taxonomy (17 September 2010) A database of accidents and serious incidents reported to ICAO by States Maintained in the ISM Section Contains 38000 plus occurrences from 1970 to present ADREP/ECCAIRS End-user Course (Rev. #3)

5. 5 ICAO – Annex 13 and ADREP Annex 13 – Essential definitions AIG/08 – Divisional meeting ADREP/ECCAIRS End-user Course (Rev. #3) Annex 13 Annex 13 Annex 13 Annex 13 AIG08 Feedback AIG08 Feedback

6. 6 ICAO – Annex 13 and ADREP Initial notification Preliminary report within 30 days Data report after accident investigation for all aircraft above 2250 kg – Also for incidents investigated on aircraft above 5700 kg ADREP/ECCAIRS End-user Course (Rev. #3)

7. 7 Concept of safety (Doc 9859) Safety is the state in which the risk of harm to persons or property damage is reduced to, and maintained at or below, an acceptable level through a continuing process of hazard identification and risk management. ADREP/ECCAIRS End-user Course (Rev. #3)

8. 8 Navigational aids ADREP/ECCAIRS End-user Course (Rev. #3) Reactive method The reactive method responds to the events that already happened, such as incidents and accidents Reactive method The reactive method responds to the events that already happened, such as incidents and accidents Proactive method The proactive method looks actively for the identification of safety risks through the analysis of the organization’s activities Proactive method The proactive method looks actively for the identification of safety risks through the analysis of the organization’s activities Predictive method The predictive method captures system performance as it happens in real-time normal operations to identify potential future problems Predictive method The predictive method captures system performance as it happens in real-time normal operations to identify potential future problems

9. Strategies – Levels of intervention and tools Baseline performance System design Operational performance Practical drift HAZARDS Predictive Highly efficient Very efficient Efficient Insufficient Proactive Reactive FDA Direct observation system ASR Surveys Audits ASR MOR Accident and incident reports Safety management levels Desirable management levels 9ADREP/ECCAIRS End-user Course (Rev. #3)

10. 10 ICAO safety management SARPs – The big picture  Two audience groups – States – Service providers  Three distinct requirements – State safety programme (SSP) Acceptable level of safety (ALoS) – Safety management System (SMS) Safety performance of the SMS – Management accountability ADREP/ECCAIRS End-user Course (Rev. #3)

11. 11 Basic safety management SARPs– Part I States shall establish a State safety programme (SSP), in order to achieve an acceptable level of safety (ALoS) in civil aviation. The ICAO SSP framework (Module 4) ADREP/ECCAIRS End-user Course (Rev. #3)

12. 12 ICAO SSP framework ADREP/ECCAIRS End-user Course (Rev. #3) 1.State safety policy and objectives 1.1 State safety legislative framework 1.2 State safety responsibilities and accountabilities 1.3Accident and incident investigation 1.4 Enforcement policy 2.State safety risk management 2.1 Safety requirements for service providers SMS 2.2 Agreement on service providers safety performance 3.State safety assurance 3.1 Safety oversight 3.2 Safety data collection, analysis and exchange 3.3 Safety data driven targeting of oversight on areas of greater concern or need 4.State safety promotion 4.1 Internal training, communication and dissemination of safety information 4.2 External training, communication and dissemination of safety information

13. 13 3. State safety assurance 3.2 – Safety data collection, analysis and exchange The State has established mechanisms to ensure the capture and storage of data on hazards and safety risks at both an individual and aggregate State’s level. ADREP/ECCAIRS End-user Course (Rev. #3)

14. 14 3. State safety assurance 3.2 – Safety data collection, analysis and exchange The State has also established mechanisms to develop information from the stored data, and to actively exchange safety information with service providers and/or other States as appropriate. ADREP/ECCAIRS End-user Course (Rev. #3)

15. 15 3. State safety assurance 3.2 – Safety data collection, analysis and exchange Develop and establish mechanisms to ensure the capture and storage of data on hazards and safety risks  Confidential reporting systems Capture data on hazards and safety risks  State safety hazard database Store data on hazards and safety risks.  Means to extract information from the stored data. ADREP/ECCAIRS End-user Course (Rev. #3)

16. 16 3. State safety assurance 3.2 – Safety data collection, analysis and exchange... Develop and establish mechanisms...:  Determine what mechanism(s) will be used to collect hazards at an aggregate State level.  Exchange information and/or data with other sources of safety information. ADREP/ECCAIRS End-user Course (Rev. #3)

17. 17 Practical Example Runway Excursion Example – Based on actual data in the global database – Sample of an ALS – Sample of a ‘drill down’ ADREP/ECCAIRS End-user Course (Rev. #3)

18. 18 Introductory concepts – Selection of safety indicators The selection of appropriate safety indicators is an essential foundation for the development and implementation of ALoS The selection of safety indicators is a function of the detail to which the level of safety of the system is to be represented Meaningful safety indicators must be representative of the elements that characterise system safety – Accidents/accident rates – Incidents/incident rates – Level of regulatory compliance – … ADREP/ECCAIRS End-user Course (Rev. #3)

19. Runway Excursion – Safety Indicator Not Manageable Not Acceptable 19ADREP/ECCAIRS End-user Course (Rev. #3)

20. Runway Excursion – Safety Indicator 20ADREP/ECCAIRS End-user Course (Rev. #3)

21. Runway Excursion – Safety Indicator 21ADREP/ECCAIRS End-user Course (Rev. #3)

22. Runway Excursion – Safety Thingys Acceptable Level of Safety Current Level of Safety TRENDTREND Safety Target 22ADREP/ECCAIRS End-user Course (Rev. #3)

23. 23 Now let’s talk about performance What did the SSP course say about that? ADREP/ECCAIRS End-user Course (Rev. #3)

24. 24 Introductory concepts – A fundamental differentiation Safety performance measurement – Generally associated to an SMS – Quantification of the outcomes of selected low-level, low- consequence processes Number of FOD events per number of ramp operations Number of ground vehicle events in taxiways per number of airport operations … A measure of the actual performance of safety interventions and/or mitigation strategies, beyond accident rates and regulatory compliance ADREP/ECCAIRS End-user Course (Rev. #3)

25. 25 Now let’s talk about performance Oh yes now I remember….. But how does one identify the “selected low-level, low- consequence processes”? Rule 1 : its all risk based Rule 2 : its done intelligently : not spit out by a computer, but thought out by a group of intelligent people But risked based intelligent process is a very very cumbersome process… so we use software to support the ANALYTICAL PROCESS ADREP/ECCAIRS End-user Course (Rev. #3)

26. 26 The Analytical Process… Essential Behaviour to be considered for membership in the order of Data Analysts Scientific method: – Hypothesis, attempted destructions, followed by peer review – Re-creatable – Assumptions are expressed and evaluated Expert opinions: – Trends are determined by mathematicians – Conclusions are determined by Aviation experts ADREP/ECCAIRS End-user Course (Rev. #3)

27. n n n Occurrence Event / phase of flight Descriptive factor subject + modifier Org/person + EF subject + modifier Organization /person + EF subject (SHELL) + modifier Data that rolls up into the safety indicator Potential area for finding safety information The path to Intelligence – The Road Map : 27ADREP/ECCAIRS End-user Course (Rev. #3)

28. Explanatory factor Descriptive factor Event / phase of flight Analyses on occurrence categories (CFIT, RI, etc.) Analyses on symptoms Statistics n Occurrence Who, where, when ? What ? How ? WHY ? Analyses on causes (HF) Investigation, Data and Analyses 28ADREP/ECCAIRS End-user Course (Rev. #3)

29. Pilot / stress Preflight check / incomplete Fuel starvation / cruise Accident Fuel starvation example (1/2) Emergency landing Fuel gauges / breakdown Pilot / time pressure Occurrence class Occurrence category = FUEL Events Descriptive factors Explanatory factors 29ADREP/ECCAIRS End-user Course (Rev. #3)

30. 30 Fuel starvation example (2/2) ADREP/ECCAIRS End-user Course (Rev. #3) In ECCAIRS, use mouse right click to add, edit or remove items

31. The Path to Intelligence – What the Machine Says

32. The Path to Intelligence – What the Human Thinks These are outcomes that are removed from ‘hazards’ These are outcomes that are closer to ‘hazards’

33. The Path to Intelligence – What the Machine Does Next What’s the trend for collisions or ‘near’ collisions?

34. The Path to Intelligence – What the Machine Does Next And What’s the FOD situation? Values for Safety Indicators Basis for Action Plans 34ADREP/ECCAIRS End-user Course (Rev. #3)

35. 35 Draw Backs… ECCAIRS is not yet quite there… it cannot: – Explicitly Identify Hazards Being fixed by creating a separate Hazards database that is interoperable with ECCAIRS – Work in all ICAO languages English, French and Spanish done.. Russian and Chinese being worked on… Arabic not yet started – No other major draw backs… … so don’t send us home! ADREP/ECCAIRS End-user Course (Rev. #3)

36. 36 Module N° 1 Appreciation