1. 17-MAR-2010 Meeting SMS Goals Through AQP and Risk Management John Ross CEO OK3 Group Inc. The Spectrum Group Civil Aviation Team Associate

2. The Story Behind the OK3 Group The “Perfect Approach” to Safety through Risk Management and Integrated Processes

3.  An increase in accidents caused by human factors  Increased complexity of the operating environment  Increased sophistication of the aircraft and automation  Increased sophistication of training devices  Need for curriculum development and modifications through use of a systematic data collection process with validations  Need to integrate training and safety into operations throughout the Corporation AQP and Risk Management Developed in the Airline Industry in Response to:

4. Regulatory Landscape  AC 120-92 Introduction to Safety Management Systems  The FAA is transitioning the traditional oversight process from a quality control approach with principal emphasis on surveillance of compliance with technical standards to a systems approach that stresses the systemic nature of aviation businesses and the larger system as a whole

5. Regulatory Landscape …from a quality control approach…

6. Regulatory Landscape FAAAirline Objective: Public Safety Objective: Serve customer ProtectionProduction Surveillance Audits Approvals …from a quality control approach…

7. Regulatory Landscape …to a systems approach…

8. Regulatory Landscape FAAAirline Objective: Public Safety Objective: Serve customer Protection Surveillance Audits Approvals Production …to a systems approach…

9. Regulatory Landscape FAA Safety Management System Airline Objective: Public Safety Objective: Serve customer Protection Surveillance Audits Approvals Production …to a systems approach…

10. Regulatory Landscape FAA Safety Management System Airline Operating System Flight Ops OCC / Dispatch Maint & Insp Cabin Safety Gnd Handling Cargo Handling Training Objective: Public Safety Objective: Serve customer Protection Surveillance Audits Approvals Production …to a systems approach…

11. Regulatory Landscape FAA Safety Management System Airline Operating System Flight Ops OCC / Dispatch Maint & Insp Cabin Safety Gnd Handling Cargo Handling Training Objective: Public Safety Objective: Serve customer Protection Airline Safety Management System Surveillance Audits Approvals Production …to a systems approach…

12. Regulatory Landscape FAA Safety Management System Airline Operating System Flight Ops OCC / Dispatch Maint & Insp Cabin Safety Gnd Handling Cargo Handling Training Objective: Public Safety Objective: Serve customer Protection Airline Safety Management System Surveillance Audits Approvals Safety Assurance Risk Management Safety Promotion Production …to a systems approach…

13. Regulatory Landscape FAA Safety Management System Airline Operating System Flight Ops OCC / Dispatch Maint & Insp Cabin Safety Gnd Handling Cargo Handling Training Objective: Public Safety Objective: Serve customer Protection Airline Safety Management System Surveillance Audits Approvals Safety Assurance Risk Management Safety Promotion Audits Approvals Production …to a systems approach…

14. Regulatory Landscape (FAA SMS Model) FAA Safety Management System Airline Operating System Flight Ops OCC / Dispatch Maint & Insp Cabin Safety Gnd Handling Cargo Handling Training Objective: Public Safety Objective: Serve customer Protection Airline Safety Management System Surveillance (sampling) Audits Approvals Safety Assurance Risk Management Safety Promotion Production …to a systems approach…

15. SMS Mandate  Task analysis for operational personnel and training personnel is required  Training as a safety risk control will be documented, measured and continuously monitored  Training as a preventive or corrective action will be documented, measured and continuously monitored  Training processes will be audited internally  Training processes will be audited externally Source: AFS-230, WATS, 4-09

16. Now, How Does AQP Work?  Employs a systems approach to training  Builds Technical Skills Trains maneuvers to required proficiency  Builds Risk Management Skills Crews train and check in realistic scenarios  Collects and analyzes de-identified training data to monitor and improve the AQP  De-identified data monitors and improves the system  Identified data monitors and improves the individual and the instructors

17. Benefits of AQP  More Efficient Training content based upon actual needs identified by data collection and analysis  More Realistic and Responsive Emphasizes risk management from a human factors perspective Scenario driven, flexible curriculum; uses airline’s SOP’s Facilitative briefing & debriefing  Enhanced Training Quality Standardized training, from bottom up—good procedures are easier to standardize, I/E CQT and IRR lead to standardization, and standardization becomes the expectation on the line

18. AQP Data Analysis

20. Training Performance Data Investigate Evaluate Monitor Audit

21. AQP Data Analysis FAA AFS-230 Training Performance Data Investigate Evaluate Monitor Audit Monthly Data Submissions

22. AQP Data Analysis Training Performance Data Investigate Evaluate Monitor Audit Monthly Data Submissions Annual Report and Review FAA AFS-230

23. AQP Data Analysis Protection Production Training Performance Data Investigate Evaluate Monitor Audit Monthly Data Submissions Annual Report and Review …a systems approach… FAA AFS-230

24. Comparison - SMS and AQP SMS ModelAQP Model

25. Introduce AQP and gain buy-in and acceptance from…  Line Pilots and Line Flight Attendants Begin with straight forward training and evaluation  Instructors/ Check Airmen Begin with basic data collection processes  FAA Make optimum use of existing industry precedents Keep data submissions, updates, reports, etc. “clean” and on time  Leadership Develop a cost-effective program Get it right the first time Continual PROMOTION from executive leadership Secrets to AQP Success

26. Risk Management  Is Integral to the AQP philosophy  An outgrowth of AQP’s success resulted in the FAA’s Safety Management System (SMS)  Like AQP, SMS uses data collection, analysis and feedback for corrections (risk management) to the system  Tears down departmental silos creating “One Process, One Language”!  Is the vehicle to be used in creating and maintaining a Corporation’s Safety Culture  Just as it becomes ingrained into corporate culture, it can actually be carried over into one’s personal life

27. ABC ’S for Decision Making Analyze the task/situation Build Barriers to minimize risk In Risk Management, Barriers are GOOD! They PREVENT, TRAP and MITIGATE errors! Communicate Risks and Intentions S ense - Does this decision really make sense! With the ABC ’S, you have an easy to remember, simple to use model to aid in the decision making process, especially during TIME CRITICAL situations ABC’S

28. A Analyze the Situation  How is my Situational Awareness? Am I in the Green, Yellow, or Red?  What “Barriers” can I build that can keep me or get me back “In The Green”

29. A Analyze the Situation  How is my Situational Awareness? Am I in the Green, Yellow, or Red?  Am I task loaded? (If so, reduce tasks or increase time)  Are there Additive Conditions?  Are their Crew Factors that can negatively affect our safety?  What “Barriers” can I build that can keep me or get me back “In The Green”

30. B Build Barriers Policies and Procedures Flows and Checklists Briefings and External Resources Automation Flying Skills, Knowledge, Techniques, and Experience Barriers provide DEFENSES against ERRORS, and most importantly, CONSEQUENTIAL ERRORS! For Pilots, 5 Main Areas

31. C Communicate  Communicate RISKS and INTENTIONS  Remember……Aviate, Navigate, then Communicate  Communicate CLEARLY, realizing that in stressful situations, hearing patterns change and communication patterns MUST change

32. S Does this Decision Make Sense?  If it does, then: Do It Continue to analyze the results of the decision, and do ABC again, if necessary Debrief the results  90% of all learning occurs when events are de-briefed properly!

33. How About “LUCK”? Accident Incident Violation Error LUCK! Without a Risk Management Process….. your luck will eventually run out!!

34. Analyzing the Colgan Accident Sterile cockpit, crew rest, fatigue, training issues Checklist deficiencies or untimely use Non existent monitoring of automation, stick pusher recognition, stall horn Insufficient or poorly timed briefings. Poor monitoring skills Uncommanded flap retraction, poor stall recognition and recovery Policies, and Procedures, Flows and Checklists Briefings and Ext Resources Automation Knowledge, Skills, Techniques, and Experience Flows and Checklists

35. No Processes, No Luck! Hazards Threats MAJOR ACCIDENT! MAJOR FATALITIES! Errors

36. Building a Corporate Safety Culture Using the OK3 Group Model Adapt the model’s barriers SPECIFIC to each department

37. Build Barriers for Inflight Policies and Procedures (FAM) Flows/Checklists (Quick Reference Handbook ?) Briefings and Teamwork Preflight Equipment Checks Skills, Knowledge, and Experience Barriers provide DEFENSES against ERRORS, and most importantly, CONSEQUENTIAL ERRORS! For Flight Attendants, 5 Possible Categories

38. Adapt the model’s barriers Risk Management Training AQP Model Training System Building a Corporate Safety Culture Using the OK3 Group Model

39. Adapt the model’s barriers Risk Management Training AQP Model Training System Training and Operations SMS Building a Corporate Safety Culture Using the OK3 Group Model

40. End result? Adapt the model’s barriers Risk Management Training AQP Model Training System Training and Operations SMS Building a Corporate Safety Culture Using the OK3 Group Model

41. End result? “One Process, One Language” Adapt the model’s barriers Risk Management Training AQP Model Training System Training and Operations SMS Building a Corporate Safety Culture Using the OK3 Group Model

42. The OK3 Group Risk Management Model: The Foundation for “One Process, One Language”

43.  Thank you for your attention!  john.ross@ok3group.com Questions?