1. SMSPP Phase 0 Federal Aviation Administration 1 Phase 0 of the Safety Management Systems (SMS) Pilot Project Federal Aviation Administration The Beginning!

2. SMSPP Phase 0 Federal Aviation Administration 2 AGENDA 1.Introductions 5 min 2.Overview of SMS45 min 3.Break15 min 4.Use of the SMS Guidebooks10 min 5.Summary SMS Audit/Gap Analysis Tool10 min 6.Detailed SMS Audit/Gap AnalysisTools10 min 7.Pilot Project Implementation Plan15 min 8.Open Discussion40 min

3. SMSPP Phase 0 Federal Aviation Administration 3 INTRODUCTIONS

4. SMSPP Phase 0 Federal Aviation Administration 4 Overview of Safety Management Systems (SMS) Federal Aviation Administration

5. SMSPP Phase 0 Federal Aviation Administration 5 At this airline, safety is our first priority Really?

6. SMSPP Phase 0 Federal Aviation Administration 6 What is the primary objective of a business organization?

7. SMSPP Phase 0 Federal Aviation Administration 7 Safety Management – Rationale In order to achieve its production objectives, the management of any aviation organization requires the management of many business processes.In order to achieve its production objectives, the management of any aviation organization requires the management of many business processes. Managing safety is one such business process.Managing safety is one such business process. Safety management is a core business function just as financial management, HR management, etc.Safety management is a core business function just as financial management, HR management, etc. This brings about a potential dilemma for management.This brings about a potential dilemma for management.

8. SMSPP Phase 0 Federal Aviation Administration 8 The management dilemma Production Protection Resources Management levels

9. SMSPP Phase 0 Federal Aviation Administration 9 The management dilemma Management levels Production Protection Resources Catastrophe

10. SMSPP Phase 0 Federal Aviation Administration 10 The management dilemma Management levels Production Protection Bankruptcy Resources

11. SMSPP Phase 0 Federal Aviation Administration 11 Safety Space Production Protection Bankruptcy Catastrophe Source: James Reason

12. SMSPP Phase 0 Federal Aviation Administration 12 Safety Management – The response to the dilemma Safety issues are a byproduct of activities related to production/services delivery.Safety issues are a byproduct of activities related to production/services delivery. An analysis of an organization's resources and goals allows for a balanced and realistic allocation of resources between protection and production goals, which supports the needs of the organization.An analysis of an organization's resources and goals allows for a balanced and realistic allocation of resources between protection and production goals, which supports the needs of the organization. The product/service provided by any aviation organization must be delivered safely (i.e. protecting users and stakeholders).The product/service provided by any aviation organization must be delivered safely (i.e. protecting users and stakeholders).

13. SMSPP Phase 0 Federal Aviation Administration 13 What is safety? Safety is not equivalent to risk free (U.S. Supreme Court, 1980)Safety is not equivalent to risk free (U.S. Supreme Court, 1980) Carelessness and overconfidence are more dangerous than deliberately accepted risk (Wilbur Wright, 1901)Carelessness and overconfidence are more dangerous than deliberately accepted risk (Wilbur Wright, 1901) Practical safety is risk managementPractical safety is risk management

14. SMSPP Phase 0 Federal Aviation Administration 14 Mitigating to “Acceptable Risk” “Safety in a system may be defined as a quality of a system that allows the system to function under predetermined conditions with an acceptable minimum of accidental loss.” System Safety Engineering & Management, 2nd Edition, John Wiley & Sons 1990 ISBN 0471618160

15. SMSPP Phase 0 Federal Aviation Administration 15 Why SMS? An imperfect system Operational performance Operationaldeployment Systemdesign Baseline performance “Practical drift”

16. SMSPP Phase 0 Federal Aviation Administration 16 Risk Management is…. ….a continuous process ….a closed loop process ….cross discipline ….a continuous process ….a closed loop process ….cross discipline New Info? Tracking

17. SMSPP Phase 0 Federal Aviation Administration 17 A Risk Assessment Matrix Example Risk severity Risk probability Catastrophic A Major C Minor D Negligible E Hazardous BB Frequent 5 Occasional 4 Remote 3 Improbable 2 Extremely improbable 1 5A5B5C 5D5E 4A4B 4C4D4E 3A 3B3C3D 3E 2A 2B2C 2D2E 1A1B 1C1D1E

18. SMSPP Phase 0 Federal Aviation Administration 18 Risk Acceptance Risk is inherent in aviation operationsRisk is inherent in aviation operations Risk results from aspects of the environment and byproducts of operational activitiesRisk results from aspects of the environment and byproducts of operational activities Operator is responsible for risk management (Title 49 – FA Act)Operator is responsible for risk management (Title 49 – FA Act) A fundamental concept of risk management is acceptance of riskA fundamental concept of risk management is acceptance of risk Risk management is fundamental to the SMSRisk management is fundamental to the SMS

19. SMSPP Phase 0 Federal Aviation Administration 19 Risk Mgmt. Safety Assurance Policy: (Structure) Safety Promotion: (Culture) Four “Pillars” of SMS

20. SMSPP Phase 0 Federal Aviation Administration 20 SMS is the toolbox for… Policies & management practicesPolicies & management practices Risk Management processesRisk Management processes Safety Assurance processesSafety Assurance processes Confidential reporting processes/employee involvementConfidential reporting processes/employee involvement

21. SMSPP Phase 0 Federal Aviation Administration 21 What is the Regulator’s Duty? It is NOT safety achievement …or “management”It is NOT safety achievement …or “management” It IS safety “oversight”It IS safety “oversight”

22. SMSPP Phase 0 Federal Aviation Administration 22 What is the Provider’s Duty? Safety achievement is the responsibility of the “provider”Safety achievement is the responsibility of the “provider” The FAA’s resources are best applied in assisting / enabling safety management by the business or operation

23. SMSPP Phase 0 Federal Aviation Administration 23 Is ATOS the same as SMS? ATOS: An Oversight System used to fulfill FAA safety responsibilitiesATOS: An Oversight System used to fulfill FAA safety responsibilities SMS: A Management System used to fulfill operator safety responsibilitiesSMS: A Management System used to fulfill operator safety responsibilities

24. SMSPP Phase 0 Federal Aviation Administration 24 Process Operator’s Safety Management System (SMS-P) FAA’s Safety Management System (SMS-O) ATOS and SMS DA PA FAA (ATOS) Oversight Program Management (8 Modules) Air Carrier’s SRM/SA Outputs are Products / Services Inputs Objective: Serve customer requirements Objective: Manage safety actions as part of “business” processes Objective: Public Safety Direct sampling (e.g. surveillance) Protection Production SRMSRM SASA

25. SMSPP Phase 0 Federal Aviation Administration 25 SMS for “Providers” is … … a top-down program starting with “Top Management”

26. SMSPP Phase 0 Federal Aviation Administration 26 A SMS Organization … Develops a “Just Culture” or “Safety Culture” to: Capture the operational knowledge and experience of the employees Involve the employees in the safety achievement process

27. SMSPP Phase 0 Federal Aviation Administration 27 A Positive “Safety” Culture is… …An informed culture where People understand hazards and risks. Staff work continuously to identify and overcome threatsStaff work continuously to identify and overcome threats …A reporting culture where People are encouraged to voice safety concernsPeople are encouraged to voice safety concerns Those concerns are analyzed and appropriate action is takenThose concerns are analyzed and appropriate action is taken The workforce knows and agrees on what is acceptable and unacceptable (shared values)The workforce knows and agrees on what is acceptable and unacceptable (shared values) Dr. James Reason

28. SMSPP Phase 0 Federal Aviation Administration 28 …A learning culture where: People are encouraged to develop and apply their skills and knowledge to enhance organizational safetyPeople are encouraged to develop and apply their skills and knowledge to enhance organizational safety Management updates staff on safety issuesManagement updates staff on safety issues Safety reports are fed back to staff so everyone learnsSafety reports are fed back to staff so everyone learns …A just culture where: The workforce knows and agrees on what is acceptable and unacceptable (shared values) The workforce knows and agrees on what is acceptable and unacceptable (shared values) Errors are understood but willful violations are not tolerated Errors are understood but willful violations are not tolerated A Positive “Safety” Culture is…

29. SMSPP Phase 0 Federal Aviation Administration 29 What’s in a “Just Culture?” The immediate response is to find “what happened and why,” not “who to blame and punish”The immediate response is to find “what happened and why,” not “who to blame and punish” Acceptance that more can be learned through full reporting and detailed investigation than by blame and punishmentAcceptance that more can be learned through full reporting and detailed investigation than by blame and punishment Partnership in identifying hazards and root causesPartnership in identifying hazards and root causes

30. SMSPP Phase 0 Federal Aviation Administration 30 SMS …emphasizes Risk Management …It integrates safety with Line Management

31. SMSPP Phase 0 Federal Aviation Administration 31 The International Picture  Annex 11 - Air Traffic Services  Annex 14 - Aerodromes  Annex 6 - Operation of Aircraft  Annex 6 - Maintenance of Aircraft ICAO safety management requirements

32. SMSPP Phase 0 Federal Aviation Administration 32 ICAO Annex 6 Requirements “…a safety management system acceptable to the State of the Operator that, as a minimum:“…a safety management system acceptable to the State of the Operator that, as a minimum: identifies safety hazards;identifies safety hazards; ensures that remedial action necessary to maintain an acceptable level of safety is implemented; andensures that remedial action necessary to maintain an acceptable level of safety is implemented; and provides for continuous monitoring and regular assessment of the safety level achieved.”provides for continuous monitoring and regular assessment of the safety level achieved.” …and, aims to make continuous improvement to the overall level of safety (proposed amendment)…and, aims to make continuous improvement to the overall level of safety (proposed amendment)

33. SMSPP Phase 0 Federal Aviation Administration 33 Annex 6 requirements (cont.) “An accepted safety management system shall clearly define lines of safety accountability throughout the operator’s organization, including a direct accountability for safety on the part of senior management.”“An accepted safety management system shall clearly define lines of safety accountability throughout the operator’s organization, including a direct accountability for safety on the part of senior management.”

34. SMSPP Phase 0 Federal Aviation Administration 34 An Industry Answer? ICAO, FAA, overseas Regulators, Air Carriers, ALPA, IATA, IBAC common Launching a common idea and a new approach: SMSSMS

35. SMSPP Phase 0 Federal Aviation Administration 35 FAA Support of the SMS Concept U.S. Response to the Annex 6 proposal endorsed the SMS conceptU.S. Response to the Annex 6 proposal endorsed the SMS concept Prior U.S. implementation of system safety-based oversight systemsPrior U.S. implementation of system safety-based oversight systems ATOSATOS SASOSASO FAA (AVS) will also apply safety management concepts to oversight activitiesFAA (AVS) will also apply safety management concepts to oversight activities

36. SMSPP Phase 0 Federal Aviation Administration 36 The SMS Standard First FAA SMS standard was delivered on June 22, 2006 in Advisory Circular AC 120-92.First FAA SMS standard was delivered on June 22, 2006 in Advisory Circular AC 120-92. SMSs are currently voluntary in the United StatesSMSs are currently voluntary in the United States The standard is organized around the “four pillars”The standard is organized around the “four pillars” The standard is based on an extensive review of existing SMSs around the worldThe standard is based on an extensive review of existing SMSs around the world The format of the SMS standard is similar to that of the ISO standardsThe format of the SMS standard is similar to that of the ISO standards

37. SMSPP Phase 0 Federal Aviation Administration 37 Rulemaking Effort Part 121 Rulemaking Project Record (RPR) opened Nov. 2006Part 121 Rulemaking Project Record (RPR) opened Nov. 2006 Project cancelled Jan. 2008Project cancelled Jan. 2008 Project team commissioned to create integrated SMS Rule for all CFR partsProject team commissioned to create integrated SMS Rule for all CFR parts Objective: harmonization of requirementsObjective: harmonization of requirements Strategy is being formulatedStrategy is being formulated ARC is a possibilityARC is a possibility

38. SMSPP Phase 0 Federal Aviation Administration 38 Testimonials – “Why didn’t TC do this years ago?” (705 Operator)“Why didn’t TC do this years ago?” (705 Operator) “We are now much more aware of our safety responsibilities and liabilities.” (AMO)“We are now much more aware of our safety responsibilities and liabilities.” (AMO) “My employees are involved in developing the SMS processes. They tell me they feel safer at work.” (small AMO)“My employees are involved in developing the SMS processes. They tell me they feel safer at work.” (small AMO) “It is too early to realize all the benefits of SMS, but already employees and customers have an increased confidence in the company. (AMO)“It is too early to realize all the benefits of SMS, but already employees and customers have an increased confidence in the company. (AMO) “Already we are seeing the benefits of recording and tracking incidents and hazards” (FTU)“Already we are seeing the benefits of recording and tracking incidents and hazards” (FTU) “The idea of SMS is much scarier than the actuality.” (702,703 Helicopter Operator).“The idea of SMS is much scarier than the actuality.” (702,703 Helicopter Operator).

39. SMSPP Phase 0 Federal Aviation Administration 39 Others Air Transat initially saved over $1 million per month Air Transat initially saved over $1 million per month Skyservice saved $5 million in first year of SMS operation Skyservice saved $5 million in first year of SMS operation

40. SMSPP Phase 0 Federal Aviation Administration 40 BREAK

41. SMSPP Phase 0 Federal Aviation Administration 41 SMS Implementation and Tools Federal Aviation Administration

42. SMSPP Phase 0 Federal Aviation Administration 42 The purpose of the Guidebooks is to provide a framework and offer developmental guidance to aid organizations in defining their organizational safety structure. SMS Guidebooks

43. SMSPP Phase 0 Federal Aviation Administration 43 Format of the Guidebook (OBJ) are objectives. This information describes or outlines what you will be expected to develop in this specific area of your Safety Management System documentation, in order to meet an SMS standard.(OBJ) are objectives. This information describes or outlines what you will be expected to develop in this specific area of your Safety Management System documentation, in order to meet an SMS standard. (STD) are standards. This is the specific language of the standard that the organization will be held to in order to meet the statement of requirements conveyed in the policy/procedure section.(STD) are standards. This is the specific language of the standard that the organization will be held to in order to meet the statement of requirements conveyed in the policy/procedure section. (DG) is developmental guidance This provides instruction regarding how you can develop your policy, procedures, and controls to address each specific requirement of the Safety Management System.(DG) is developmental guidance This provides instruction regarding how you can develop your policy, procedures, and controls to address each specific requirement of the Safety Management System.

44. SMSPP Phase 0 Federal Aviation Administration 44 Example Section of the Guidebook 4.4 Safety Planning 4.4 Safety Planning (OBJ) The expectation of this section is that you describe your safety management plan to meet the objectives described by your above stated safety policy. (DG) Management has historically been defined as planning, organizing, directing, and controlling. Therefore a Safety Management System will start with a plan to meet the Safety Objectives. A plan should be set by Top Management that will direct and sequence the implementation of the Safety Management System. Planning at one level becomes direction at the next level. The first thing to plan is how much and on what time table the various portions of the Safety Management System will be created, who will be responsible for the overall system and the various portions of it. (STD) - FAA Advisory Circular AC 120-92 Appendix 1 4.4. Safety Planning The organization shall establish and maintain a safety management plan to meet the safety objectives described in its safety policy.

45. SMSPP Phase 0 Federal Aviation Administration 45 High level assessment toolHigh level assessment tool To be utilized for initial screening or audit summaryTo be utilized for initial screening or audit summary Questions at the objective levelQuestions at the objective level Evaluates the organization’s existing systems to determine conformance to the SMS standardsEvaluates the organization’s existing systems to determine conformance to the SMS standards Summary Audit/Gap Analysis Tool

46. SMSPP Phase 0 Federal Aviation Administration 46 Summary Audit/Gap Analysis Tool Evaluate Objectives utilizing assessment scales Measure the level of development for each objective 0 – No action 1 – Action initiated 2 – Implemented 3 – Integrated 4 – Evaluated and Sustained 5 – State of Art

47. SMSPP Phase 0 Federal Aviation Administration 47 Summary Audit/Gap Analysis Tool No action = 0 Action initiated=1 Implemented=2 Integrated=3 Evaluated and Sustained=4 State of Art=5

48. SMSPP Phase 0 Federal Aviation Administration 48 Audit/Gap Analysis Tools Detailed assessment of the organization’s existing systems to determine conformance to the SMS standardsDetailed assessment of the organization’s existing systems to determine conformance to the SMS standards Used to determine what needs to be developedUsed to determine what needs to be developed After implementation, to assess continued performance of the systemAfter implementation, to assess continued performance of the system Final gap analysis prior to implementation planning must be based on the full set of tools to ensure that all of the standard's requirements are included in the system designFinal gap analysis prior to implementation planning must be based on the full set of tools to ensure that all of the standard's requirements are included in the system design

49. SMSPP Phase 0 Federal Aviation Administration 49 Audit/Gap Analysis Tools System 8.0 of the Air Transportation Oversight System (ATOS): Includes the SMS pillars as subsystemsIncludes the SMS pillars as subsystems 8.1 Policy 8.2 Safety Risk Management 8.3 Safety Risk Assessment 8.4 Safety 13 Design Assessment Tools based on major sub-clauses of the Standard13 Design Assessment Tools based on major sub-clauses of the Standard

50. SMSPP Phase 0 Federal Aviation Administration 50 Audit/Gap Analysis Tools 8.1.1 Organizational Management 8.1.2 Documentation and Records Management 8.1.3 Emergency Preparedness and Response 8.2.1 System and Task Analysis 8.2.2 Hazard Identification 8.2.3 Safety Risk Analysis and Assessment 8.2.4 Safety Risk Controls 8.3.1 Data Acquisition 8.3.2 Data Analysis and System Assessment 8.3.3 Corrective and Preventive Action 8.4.1 Communication and Awareness 8.4.2 Personnel Qualifications and Training

51. SMSPP Phase 0 Federal Aviation Administration 51 Detailed Audit/Gap Analysis Tools Utilizes ATOS SAI format Has a stated functional objective - expectations and outcomes Questions in the procedures section of each element are based on specific requirements in the standard Requirements evaluated utilizing assessment scales and scales measure the level of conformance with each requirement 0 – No action 1 – Action initiated 2 – Implemented 3 – Integrated 4 – Evaluated and Sustained 5 – State of Art

52. SMSPP Phase 0 Federal Aviation Administration 52 Example: Design Assessment Tool

53. SMSPP Phase 0 Federal Aviation Administration 53 Example: Design Assessment Tool: Detail

54. SMSPP Phase 0 Federal Aviation Administration 54 Standard and Audit/Gap Analysis Tools

55. SMSPP Phase 0 Federal Aviation Administration 55 SMS Pilot Project A set of confirmatory trials that consist of voluntary SMS development and interface with oversight systemsA set of confirmatory trials that consist of voluntary SMS development and interface with oversight systems The purpose is to gather data to be used in further development of guidance material, implementation strategies, and oversight systemsThe purpose is to gather data to be used in further development of guidance material, implementation strategies, and oversight systems

56. SMSPP Phase 0 Federal Aviation Administration 56 Industry & FAA SMS Implementation Should follow a phased approachShould follow a phased approach The processes underlying the four pillars will be modularizedThe processes underlying the four pillars will be modularized “Growth” or “increasing maturity” will then be emphasized for each process and the system as a whole“Growth” or “increasing maturity” will then be emphasized for each process and the system as a whole Should follow a phased approachShould follow a phased approach The processes underlying the four pillars will be modularizedThe processes underlying the four pillars will be modularized “Growth” or “increasing maturity” will then be emphasized for each process and the system as a whole“Growth” or “increasing maturity” will then be emphasized for each process and the system as a whole

57. SMSPP Phase 0 Federal Aviation Administration 57 SMS Implementation Process 0 Orientation & Commitment 2 Reactive Processes 3 Proactive Processes 4 Continuous Improvement 1 Planning & Organization

58. SMSPP Phase 0 Federal Aviation Administration 58 Purpose and Methodology of SMS Phased implementation: a logical process for building the SMSPhased implementation: a logical process for building the SMS Management capability: Each step in the model will increase and strengthen management capabilityManagement capability: Each step in the model will increase and strengthen management capability Process based uponProcess based upon ICAO recommendations andICAO recommendations and Capability Maturity Model (CMM) conceptsCapability Maturity Model (CMM) concepts

59. SMSPP Phase 0 Federal Aviation Administration 59 Who’s Involved in Implementation? The OperatorThe Operator The Certificate Management Organization (CMO, FSDO, CHDO)The Certificate Management Organization (CMO, FSDO, CHDO) The Standardization and Assistance Team (SAT)The Standardization and Assistance Team (SAT)

60. SMSPP Phase 0 Federal Aviation Administration 60 Level Zero: Orientation and Commitment Outreach presentationsOutreach presentations Understanding what’s aheadUnderstanding what’s ahead Establishing the relationshipEstablishing the relationship Securing the commitment from top managementSecuring the commitment from top management

61. SMSPP Phase 0 Federal Aviation Administration 61 Level 1: Planning & Organization Charting the Course The Gap Analysis: Evaluating existing processes, programs, and practices for safety managementThe Gap Analysis: Evaluating existing processes, programs, and practices for safety management The Implementation PlanThe Implementation Plan Organizing for ImplementationOrganizing for Implementation

62. SMSPP Phase 0 Federal Aviation Administration 62 Level 1 - Meetings Session 1 (Today) SAT to Brief:SAT to Brief: SMS Process MapSMS Process Map SMS ObjectivesSMS Objectives Summary Audit/Gap Analysis ToolSummary Audit/Gap Analysis Tool Detailed Audit/Gap Analysis ToolsDetailed Audit/Gap Analysis Tools Jointly Perform Summary Gap AnalysisJointly Perform Summary Gap Analysis Schedule Operator’s Gap Analysis Out BriefSchedule Operator’s Gap Analysis Out Brief The operator may require 4 to 6 months to complete the detailed gap analysis. SAT and CMT will be available on call while the operator is performing this task.The operator may require 4 to 6 months to complete the detailed gap analysis. SAT and CMT will be available on call while the operator is performing this task.

63. SMSPP Phase 0 Federal Aviation Administration 63 Level 1 - Meetings Session 2 Operator to Brief:Operator to Brief: Overview of their Gap AnalysisOverview of their Gap Analysis Gaps found and how they may develop processes to close the gaps.Gaps found and how they may develop processes to close the gaps. Operator to Discuss DocumentationOperator to Discuss Documentation Operator to present objective evidence of conformityOperator to present objective evidence of conformity Operator to Identify Resource and Training NeedsOperator to Identify Resource and Training Needs SAT may provide additional training if necessary as a result of this discussion.SAT may provide additional training if necessary as a result of this discussion. SAT to Brief on Planning ProcessSAT to Brief on Planning Process Schedule for Operator’s SMS Implementation Plan Out BriefSchedule for Operator’s SMS Implementation Plan Out Brief The operator may require 2 to 4 months to complete the detailed implementation plan development.The operator may require 2 to 4 months to complete the detailed implementation plan development. Estimated time for this session is one day of presentation of gap analysis by the operator. Half to one day of SAT presentation and discussions of the planning processes.Estimated time for this session is one day of presentation of gap analysis by the operator. Half to one day of SAT presentation and discussions of the planning processes.

64. SMSPP Phase 0 Federal Aviation Administration 64 Level 1 - Meetings Session 3 Operator’s Presentation of Implementation PlanOperator’s Presentation of Implementation Plan Detailed presentation and discussion of their SMS implementation plan involving the operator, CMT and SATDetailed presentation and discussion of their SMS implementation plan involving the operator, CMT and SAT Consensus on PlanConsensus on Plan Agreement on the implementation plan among operator, CMT and SATAgreement on the implementation plan among operator, CMT and SAT Exit Level 1Exit Level 1 Agreement among operator, CMT and SAT that the participant is ready to move the next phase of the pilot project.Agreement among operator, CMT and SAT that the participant is ready to move the next phase of the pilot project. Estimated time for this session is one day of presentation by the operator.Estimated time for this session is one day of presentation by the operator.

65. SMSPP Phase 0 Federal Aviation Administration 65 Level 1 – Completion Criteria Identify and assign safety responsibilities to managers.Identify and assign safety responsibilities to managers. 1.1 Management commitment and responsibility (4.2B, 4.5A, 4.5B) 1.2 Safety responsibilities of managers (4.5D, 4.1) Identify the person responsible for implementing the SMS.Identify the person responsible for implementing the SMS. 1.3 Appointment of key safety personnel (4.5C) Describe the system.Describe the system. 1.4 SMS implementation plan (4.4, 4.1) Conduct a gap analysis of existing resources compared to SMS requirements.Conduct a gap analysis of existing resources compared to SMS requirements. 1.4 SMS implementation plan (4.4) Develop an SMS implementation plan.Develop an SMS implementation plan. 1.4 SMS implementation plan (4.4) Develop documentation relevant to safety policy and objectives.Develop documentation relevant to safety policy and objectives. 1.6 Documentation (4.9) Develop and establish means for safety communication.Develop and establish means for safety communication. 4.2 Safety communication (7.2) Communicate the SMS implementation to all employees.Communicate the SMS implementation to all employees. 4.2 Safety communication (7.2) Develop an initial training plan for all employees.Develop an initial training plan for all employees. 4.1 Training and education (7.4)

66. SMSPP Phase 0 Federal Aviation Administration 66 Level 1 Assessment Criteria The assessment will be accomplished using the Audit/Gap Analysis Tools below: 8.1.1. Organizational Management8.1.1. Organizational Management 8.1.2. Document and Records Management8.1.2. Document and Records Management 8.4.1. Communication and Awareness8.4.1. Communication and Awareness 8.4.2. Personnel Qualification and Training8.4.2. Personnel Qualification and Training All questions on these tools should achieve a score of at least 1, as defined on the next slide. All individual questions receiving less than a score of 1 will be reviewed by the Participant, the CMO and the SAT for creation of a mutually agreeable solution.

67. SMSPP Phase 0 Federal Aviation Administration 67 Level 1 Assessment Scale SAI Assessment Scale ScoreAssessmentWord Picture 0No ActionNo action has been taken on this requirement of the standard. 1Action InitiatedIdentifiable actions have been taken to satisfy this requirement of the standard. All responsibilities, processes and procedures are documented. However, the implementation of those defined actions is not complete. Example: 8.1.1 Section 2, Question 1. Does the operator have a control or controls in place to ensure that all SMS processes are documented, monitored, measured, and analyzed? To achieve a score of 1, the Participant must have policies and procedures defined and written on how the process will be monitored, measured and analyzed, but does not have to have taken any actions to actually do the monitoring, measuring or analysis.

68. SMSPP Phase 0 Federal Aviation Administration 68 Level 1 - Documents Management Commitment LetterManagement Commitment Letter Management responsibilities definedManagement responsibilities defined Safety Policy and ObjectivesSafety Policy and Objectives Safety responsibilities for managers definedSafety responsibilities for managers defined Results of Gap Analysis for the department/area under the Pilot ProgramResults of Gap Analysis for the department/area under the Pilot Program SMS Implementation Plan for the entire organizationSMS Implementation Plan for the entire organization All Employee Letter or equivalent advising employees of the SMS implementationAll Employee Letter or equivalent advising employees of the SMS implementation SMS Training Plan for all employeesSMS Training Plan for all employees

69. SMSPP Phase 0 Federal Aviation Administration 69 Level 1 - Expectations Development Partner –Sessions 1 Complete summary gap analysis –Session 2 Completed detailed gap analysis utilizing the detailed audit/gap analysis tools for Level 1 Brief gap analysis results Brief gaps and how they will be addressed Provide evidence of conformity –Session 3 Present implementation plan Complete the required documents Meet the requirements of the exit criteria for this phase Provide process inputs to the SAT

70. SMSPP Phase 0 Federal Aviation Administration 70 Level 1 - Expectations Oversight Organization –Review and utilize (if desired) audit/gap analysis tools –Attend audit/gap analysis meetings with the operator –Participate in meetings with the SAT and operator –Review the operators implementation plan and other documents –Discuss the requirements of the exit criteria for this phase with the operator and SAT –Provide input to the SAT regarding the SMS implementation, documents and audit tools

71. SMSPP Phase 0 Federal Aviation Administration 71 Level 1 - Expectations SAT – All Levels –Provide assistance, as requested, to the Development Partner and its Oversight Organization throughout the Pilot Project period. –Participate in meetings with the CMT and operator, if requested –Review the operators implementation plan and other documents –Discuss the requirements of the exit criteria for this phase with the operator and CMT –Receive Pilot Project inputs from the Development Partner and Oversight Organization –Report on summarized data gathered from the Pilot Project

72. SMSPP Phase 0 Federal Aviation Administration 72 Level 2: Reactive Processes Basic Risk Management Setting up the information infrastructure and processesSetting up the information infrastructure and processes Going after known problems: Reactive Hazard IdentificationGoing after known problems: Reactive Hazard Identification Designing and implementing risk controlsDesigning and implementing risk controls DocumentationDocumentation TrainingTraining General initial trainingGeneral initial training Specialist trainingSpecialist training

73. SMSPP Phase 0 Federal Aviation Administration 73 Level 2 – Completion Criteria Implement safety risk management (SRM) for reactive processes.Implement safety risk management (SRM) for reactive processes. 2.1 Hazard identification process (5.2) 2.2 Risk assessment and mitigation processes (5.3, 5.4, 5.5) 2.3 Internal safety investigations (6.3.5) Perform training relevant to SMS implementation plan and SRM components.Perform training relevant to SMS implementation plan and SRM components. 4.1 Training and education (7.4) Develop documentation relevant to SMS implementation plan and SRM components (reactive processes).Develop documentation relevant to SMS implementation plan and SRM components (reactive processes). 1.4 SMS implementation plan (4.4) 1.6 Documentation (4.9) Initiate a non-punitive voluntary employee reporting system. (6.3.6, 4.2B5)Initiate a non-punitive voluntary employee reporting system. (6.3.6, 4.2B5)

74. SMSPP Phase 0 Federal Aviation Administration 74 Level 2 - Documents Processes and procedures for operating the SMS to the level of assessment and actions reactivelyProcesses and procedures for operating the SMS to the level of assessment and actions reactively Voluntary employee reporting programVoluntary employee reporting program

75. SMSPP Phase 0 Federal Aviation Administration 75 Level 2 – Audit/Gap Analysis Tools 8.1.2 Document and Records Management8.1.2 Document and Records Management 8.1.3 Emergency Response8.1.3 Emergency Response 8.2.2 Hazard Identification8.2.2 Hazard Identification 8.2.3 Risk Analysis and Assessment8.2.3 Risk Analysis and Assessment 8.2.4 Risk Control8.2.4 Risk Control 8.3.1 Data Acquisition8.3.1 Data Acquisition 8.3.2 Data Analysis/System Assessment8.3.2 Data Analysis/System Assessment 8.3.3 Preventive/Corrective Action8.3.3 Preventive/Corrective Action 8.4.1 Communication and Awareness8.4.1 Communication and Awareness 8.4.2 Personnel Qualification and Training8.4.2 Personnel Qualification and Training

76. SMSPP Phase 0 Federal Aviation Administration 76 Level 3: Proactive Processes Looking Ahead System and task analysis of the operational (production) systemsSystem and task analysis of the operational (production) systems Proactive Hazard identificationProactive Hazard identification Updating:Updating: the risk controlsthe risk controls Updating the documentationUpdating the documentation Additional specialist trainingAdditional specialist training

77. SMSPP Phase 0 Federal Aviation Administration 77 Level 3 – Completion Criteria Implement safety risk management for proactive and predictive processes.Implement safety risk management for proactive and predictive processes. 2.1 Hazard identification process (5.2) 2.2 Risk assessment and mitigation processes (5.3, 5.4, 5.5) 2.3 Internal safety investigations (6.3.5) Perform training relevant to proactive and predictive processes.Perform training relevant to proactive and predictive processes. 4.1 Training and education (7.4) Develop documentation relevant to proactive and predictive processes.Develop documentation relevant to proactive and predictive processes. 1.4 SMS implementation plan (4.4) 1.6 Documentation (4.9) Perform a system and task analysis for one department (5.1)Perform a system and task analysis for one department (5.1) Incorporate hazards from system and task analysis into SRM (5.1 – 5.5)Incorporate hazards from system and task analysis into SRM (5.1 – 5.5) Develop policies and procedures for safety assurance (6.2)Develop policies and procedures for safety assurance (6.2) Information acquisition (6.3)Information acquisition (6.3) Analysis of data (6.4)Analysis of data (6.4) System assessment (6.5)System assessment (6.5) Preventive and corrective actions (6.6)Preventive and corrective actions (6.6) Management reviews (6.7)Management reviews (6.7)

78. SMSPP Phase 0 Federal Aviation Administration 78 Level 3 - Documents All processes and procedures for operating the SMS.All processes and procedures for operating the SMS. Safety assurance policy and procedures must be documented.Safety assurance policy and procedures must be documented.

79. SMSPP Phase 0 Federal Aviation Administration 79 Level 3 – Audit/Gap Analysis Tools 8.1.2 Document and Records Management8.1.2 Document and Records Management 8.2.1 System and Task Analysis8.2.1 System and Task Analysis 8.2.2 Hazard Identification8.2.2 Hazard Identification 8.2.3 Risk Analysis and Assessment8.2.3 Risk Analysis and Assessment 8.2.4 Risk Control8.2.4 Risk Control 8.3.1 Data Acquisition8.3.1 Data Acquisition 8.3.2 Data Analysis/System Assessment8.3.2 Data Analysis/System Assessment 8.3.3 Preventive/Corrective Action8.3.3 Preventive/Corrective Action 8.4.1 Communication and Awareness8.4.1 Communication and Awareness 8.4.2 Personnel Qualification and Training8.4.2 Personnel Qualification and Training

80. SMSPP Phase 0 Federal Aviation Administration 80 Level 4: Continuous Improvement Continued Assurance Mature safety assurance processMature safety assurance process Analysis of dataAnalysis of data System performance assessmentSystem performance assessment Corrective/Preventive action: Maintaining the controlsCorrective/Preventive action: Maintaining the controls Identifying new and emerging hazardsIdentifying new and emerging hazards Management Reviews: Formal involvementManagement Reviews: Formal involvement Documentation & Training as requiredDocumentation & Training as required

81. SMSPP Phase 0 Federal Aviation Administration 81 Level 4: Continuous Improvement This is the final stage of maturity of the SMSThis is the final stage of maturity of the SMS Continuous improvement is based on:Continuous improvement is based on: Continuous risk management, andContinuous risk management, and Continuous safety assuranceContinuous safety assurance

82. SMSPP Phase 0 Federal Aviation Administration 82 Level 4 – Completion Criteria Implement an operational safety assurance program (6.x)Implement an operational safety assurance program (6.x) 3.1 Safety performance monitoring and measurement (6.3.1 - 6.3.3) 3.2 The management of change (5B4, 5E) 3.3 Continuous improvement of the safety system (6.8, 4.2B2) Develop acceptable level(s) of safety (4.7)Develop acceptable level(s) of safety (4.7) Develop safety indicators and targets (4.7)Develop safety indicators and targets (4.7) Perform training relevant to operational safety assurance.Perform training relevant to operational safety assurance. 4.1 Training and education (7.4) Develop documentation relevant to operational safety assurance.Develop documentation relevant to operational safety assurance. 1.6 Documentation (4.9)

83. SMSPP Phase 0 Federal Aviation Administration 83 Level 4 - Documents  All SMS program documents that have been previously developed should be reviewed to ensure that any changes, amendments or maturity in the processes or procedures have been appropriately documented.

84. SMSPP Phase 0 Federal Aviation Administration 84 Level 4 – Audit/Gap Analysis Tools 8.1.1 Organizational Management8.1.1 Organizational Management 8.1.2 Document and Records Management8.1.2 Document and Records Management 8.3.1 Data Acquisition8.3.1 Data Acquisition 8.3.2 Data Analysis/System Assessment8.3.2 Data Analysis/System Assessment 8.4.2 Personnel Qualification and Training8.4.2 Personnel Qualification and Training

85. SMSPP Phase 0 Federal Aviation Administration 85 Interfaces and Integration Regulatory OversightRegulatory Oversight ATOSATOS Safety AssuranceSafety Assurance CASSCASS IEPIEP Employee ReportingEmployee Reporting ASRSASRS ASAPASAP Compliance and Enforcement: InterfacesCompliance and Enforcement: Interfaces VDRPVDRP EDTEDT

86. SMSPP Phase 0 Federal Aviation Administration 86 Voluntary Implementation of SMS Advisory Circular has been DraftedAdvisory Circular has been Drafted Process Based on AC 120-92Process Based on AC 120-92 Interim Solution while rulemaking is ongoingInterim Solution while rulemaking is ongoing Award SystemAward System Bronze: Maturity Level 2: Risk Management ProgramBronze: Maturity Level 2: Risk Management Program Silver: Maturity level 3: SMS (provisional)Silver: Maturity level 3: SMS (provisional) Gold: Maturity Level 4: SMSGold: Maturity Level 4: SMS

87. SMSPP Phase 0 Federal Aviation Administration 87 Draft AC 120-XX Contents Description of LevelsDescription of Levels Implementation ProcessImplementation Process Gap Analysis ToolGap Analysis Tool Functional ObjectivesFunctional Objectives Maturity RatingMaturity Rating Cross Reference to Detail Design Assessment ToolsCross Reference to Detail Design Assessment Tools Cross Reference to ICAO FrameworkCross Reference to ICAO Framework

88. SMSPP Phase 0 Federal Aviation Administration 88 SMS Pilot Projects 0 Orientation & Commitment 2 Reactive Processes 3 Proactive Processes 4 Continuous Improvement 1 Planning & Organization Phase 1: Readiness Phase 2: Experience

89. SMSPP Phase 0 Federal Aviation Administration 89 Summary Phased implementation of SMSPhased implementation of SMS  Four levels plus a “zero” level  Approximate three-year estimated time to complete Integration of existing programs encouragedIntegration of existing programs encouraged SMS Pilot Projects underway to refine:SMS Pilot Projects underway to refine:  Oversight practices and interfaces  Implementation strategies SMS gap analysis and system assessment tools providedSMS gap analysis and system assessment tools provided Additional guidance and advisory material being developedAdditional guidance and advisory material being developed

90. SMSPP Phase 0 Federal Aviation Administration 90 Are we done yet? Aviation systems are open systems that operate in a dynamic rapidly changing environmentAviation systems are open systems that operate in a dynamic rapidly changing environment Safety management is, therefore, never “done”Safety management is, therefore, never “done” Continuous improvement is based on continuous risk management and assuranceContinuous improvement is based on continuous risk management and assurance

91. SMSPP Phase 0 Federal Aviation Administration 91 CONTACT INFORMATION Dr. Donald N. Arendt, Manager Federal Aviation Administration AFS-940 Suite 131 45005 Aviation Drive Dulles, VA 20166-7537 (703) 661-0516 Don.Arendt@faa.gov Bill Jarrott Lead System Engineer The MITRE Corporation 7515 Colshire Drive McLean, VA 22102-7539 (703) 983-1842 bjarrott@mitre.org Keith D. DeBerry FAA, AFS-940 Suite 131 45005 Aviation Drive Dulles, VA 20166-7537 (571) 423-8720 Keith.D.DeBerry@faa.gov http://www.mitrecaasd.org/SMS/documents.html

92. SMSPP Phase 0 Federal Aviation Administration 92 OPEN DISCUSSION