GA JSC SAT and Working Group Processes GA JSC SAT and Working Group Processes Corey Stephens Co-Chair, GA JSC SAT GA JSC SAT Meeting March 22, 2011 Washington, DC

GA Joint Steering Committee Evolve GA JSC to a CAST like Model – Voluntary commitments – Consensus decision-making – Data driven risk management – Implementation-focused The GA JSC is a means to… Focus Limited Government/Industry Resources on Data Driven Risks and Solutions

What is CAST? Work began in 1997 after two significant accidents in 1996 (TWA 800 & ValueJet 592) CAST focus was set by: – White House Commission on Aviation Safety – The National Civil Aviation Review Commission (NCARC) Opportunity for industry and government to focus resources on one primary aviation safety initiative

What is CAST? Vision Key aviation stakeholders acting cooperatively to lead the world-wide aviation community to the highest levels of global commercial aviation safety by focusing on the right things. Mission Enable a continuous improvement framework built on monitoring the effectiveness of implemented actions and modifying actions to achieve the goal. Goal Reduce the US commercial aviation fatal accident rate 80% by 2007 and Maintain a continuous reduction in fatality risk in US and International commercial aviation beyond 2007.

CAST Safety Strategy Influence Safety Enhancements - Worldwide Data Analysis Set Safety Priorities Achieve consensus on priorities Integrate into existing work and distribute Implement Safety Enhancements - U.S. Agree on problems and interventions How CAST Works

CAST Safety Strategy Ongoing Accident/ Incident/Studies Incident Analysis Process Emerging/ Changing Risk Develop/Revise Enhancements & Metrics CAST-064 CAST Plan Performance To Plan Review Things to Watch Industry/ Government Action Safer System Information on System Performance Future Changes Analysis Process Develop/Revise Enhancements & Metrics Develop/Revise Enhancements & Metrics Master Contributing Factors \

NTSB Accident Incident Reports 21.3 Reports Airclaims data Implementation Strategy JSIT Historical Data Pareto Plots JSAT ASIAS data Causal Analysis Combined Threat Cause Accident CAST Safety Analysis Process Industry JSAT Intervention Strategy Coordinated Plan Measuring Progress to Goal Industry Government Safer Skies AvSP

Safety enhancement development Master safety plan Enhancement effectiveness Future areas of study Data analyses CAST Joint Safety Analysis Teams (JSAT) Joint Safety Implementation Teams (JSIT) Joint Implementation Measurement Data Analysis Team (JIMDAT) Commercial Aviation Safety Team (CAST)

CAST Accomplishments Forensic analysis of US and world accidents since 1987 (ongoing) Industry and government cooperative safety plan: – 72 Prioritized Safety Enhancements – 50 Complete and 22 underway – Projected 74% fatality risk reduction by 2020 Development of proactive analytic processes for incident data CAST was the recipient of the 2008 Collier Trophy For achieving an unprecedented safety level in U.S. commercial airline operations by reducing risk of a fatal airline accident by 83 percent, resulting in two consecutive years of no commercial scheduled airline fatalities

For this discussion… GA JSC = CAST Steering Committee = CAST ExCom Safety Analysis Team (SAT) = JIMDAT Working Groups (WGs) = JSAT/JSIT GA JSC Groups and their CAST Counterparts

Identify future areas of study/risk Charter safety studies Provide guidance and direction Draw data from various areas Develop a prioritized Safety Plan Develop metrics to measure effectiveness of safety solutions Data analyses Safety enhancement Mitigation development Strategic guidance Management/Approval of Safety Plan Provide direction Membership Outreach Provides linkage to ASIAS Steering Committee Co-Chairs: Bruce Landsberg (AOPA/ASF) Tony Fazio (FAA/AVP) Government - FAA (AFS, AIR, ATO & ARP) - NASA (Research) - NWS Industry - GAMA, EAA, NBAA, NATA, & SAMA Safety Analysis Team Co-chairs: Corey Stephens (FAA) Jens Hennig (GAMA) Members: FAA, NTSB, AOPA, FSF, UAA, CGAR, FAST, NAFI, Insurance, Academia, SAFE Working Groups (To include SMEs from various general aviation segments, depending on study) General Aviation Joint Steering Committee (GAJSC)

GA Safety Plan GAJSC Safety Strategy GAJSC GAJSC SAT WG  Time  Ongoing Fatal Accident Studies Accident Area Proposed Approves Priority / Assigns Resources to WG Detailed Accident Review and Propose Mitigations Approves Proposed Mitigations Amend Safety Plan Develops Detailed Implementation Plans (DIP) Approves DIP & Assigns Industry Government Responsibility Industry Action Government Action Develop, Revise & Monitoring of Metrics Monitor Effect Identification of System Changes General Aviation NAS Safety State Aviation System Establish SAT FAA NTSB Pilots Manufacturers Academia ASIAS Accident Selection NASA Evaluate Cost & Benefit Review Proposed Mitigations

GA JSC Working Group Process Step 1: Analysis

Typical CAST JSAT Membership ALPA/APA FAA (AIR, AFS, ASA, AAI, ATO) Airbus EASA ATA Transport Canada NASA Engine companies – (PW, GE, RR-Allison) Boeing RAA NACA AIA NATCA

GA JSC WG Process Charter Development Establish Team Select Data Set Review Data Identify Intervention Strategies Assign Standard Problem Statements Record Characteristics/ Indicators Develop Event Sequence Evaluate Intervention Effectiveness Prioritize Interventions Technical Review & Report Results Identify Problems (what/why) Global Review of Characteristics/ Indicators Evaluate Problem Importance

Developed Event Sequence Facts and data Pilot - controller voice events Missed calls Events that occurred or should have Time coded each event

Develop Problem Statements Problem statements –What went wrong –Deficiency definition –Potential reason –Something which happened or didn’t happen

Sample Standard Problem Statements CAST Examples 10 FLIGHTCREW – Failure of flight crew to follow established procedures (SOP) 39 AIRCRAFT EQUIPMENT – DESIGN NOT ERROR TOLERANT System design does not provide adequate redundancy to counteract errors or alerting of the effects of errors 44 FLIGHTCREW – Flight crew failure to recognize and correct unstable approach 100 REGULATORS – INSUFFICIENT AIR CARRIER OVERSIGHT. Insufficient regulatory oversight of air carrier operations including management and training practices

Identify Intervention Strategies Intervention strategies –Suggested solutions –Things to do to prevent or mitigate the problem –Etc.

Intervention Effectiveness Power – Effectiveness of a specific intervention in reducing the likelihood that a specific accident would have occurred (“Perfect World”) Confidence – Confidence that this specific intervention will have the desired effect Future Global Applicability – How well the intervention can be extrapolated to apply to a world- wide fleet in the future

Effectiveness Rating Scales POWER This scale is to be used to judge the effectiveness of a specific intervention in reducing the likelihood that a specific accident would have occurred had the intervention been in place and operating as intended. (“perfect world”) Hardly any effect Slightly effective Moderately effective Quite effective Highly effective CONFIDENCE This scale is to be used to define the level of confidence that you have that this specific intervention will have the desired effect. Hardly any confidence Slightly confident Moderately confident Quite confident Highly confident Not at all effective Completely effective Not at all confident Completely confident FUTURE GLOBAL APPLICABILITY This scale is to be used to estimate how well the intervention can be extrapolated to apply to a world-wide fleet in the future. (for example: how often the situation it addresses occurs in accident scenarios; whether its impact is on present and future operations (equippage, traffic, regulatory differences); and whether it is applicable across airlines/airplanes/regions. Hardly any applicable Slightly applicable Moderately applicable Quite applicable Highly applicable Not at all applicable Completely applicable

GA JSC Working Group Process Step 2: Implementation

GA JSC Feasibility Scales Technical Financial Operational Schedule Regulatory Sociological

GA JSC Safety Enhancements Develop Safety Enhancements from Interventions Collect detailed resource information Prepare Detailed Implementation Plans (DIP’s)

GA JSC WG Reports Standard Problem Statements Interventions Prioritized Recommendations Detailed Implementation Plans (DIPs)

What’s a DIP?

GA JSC SAT (Safety Analysis Team) Process Safety Plan Development

Develops a Prioritization Methodology (GA JSC SAT) Identifies the most effective solutions derived from all accident categories Considers effectiveness vs. resources Tests solutions against fatal and hull loss accidents Creates draft master strategic safety plan Identifies areas for future study/mitigation

Effectiveness that an intervention has for reducing the accident rate if incorporated Portion of world fleet with intervention implemented (),  Accident Risk Reduction = General Methodology for Calculating the Potential Benefit of a Safety Enhancing Intervention

Spreadsheet Example – Historical Airplane Accidents & Proposed Safety Enhancements – CAST Example

Basics of the Selection Spreadsheet Effectiveness Each safety enhancements is evaluated against each undesired condition in the set to determine how effective the enhancement would be at eliminating these conditions if the enhancement were put in place. Implementation Implementation level is based on the portion of the affected population with the enhancement incorporated or predicted to be incorporated by a future date. Severity Weighting To account for differences in severity or significance of the undesired conditions, a weighting value can be entered so that the relative risk of the undesired conditions is realized.

To account for differences in fatality risk associated with each accident in the data set, a severity value was applied. In this assessment, the severity value represented the portion of people onboard that perished in the given accident. Example: Comparison of two fatal accidents 757 CFIT accident, 98% perished. Weighting factor is Turbulence accident,.6% perished. Weighting factor is.006 Hypothetically assume an assessment showed that the chance of these accident occurring would have been reduced by proposed safety enhancements by 50%. The associated portion of fatality risk eliminated can be determined using the severity weighting factor as follows: 757 CFIT.98 x.5 = Turbulence,.006 x.5 =.003 Severity Weighting Overview CAST Example

Analysis Tool Output The spreadsheet output can be set up to show the effect that an individual safety enhancement, or group of safety enhancements have on reducing exposure to the undesired condition.The spreadsheet output can be set up to show the effect that an individual safety enhancement, or group of safety enhancements have on reducing exposure to the undesired condition. SE1SE2SE3SE1 & SE2 SE1 & SE3 Fatality Risk Reduction

Robust CAST Methodology Detailed event sequence - problem identification from worldwide accidents and incidents Broad-based teams (45-50 specialists /team) Over 450 problem statements (contributing factors) Over 900 interventions proposed Analyzed for effectiveness and synergy - CAST Safety Enhancements

CAST Process Led to Integrated Strategic Safety Plan Part 121 or equivalent passenger and cargo operations studied Current CAST plan: 72 Prioritized Safety Enhancements 50 Complete and 22 underway Projected 74% fatality risk reduction by 2020 Industry and Government implementing plan

Resource Cost Vs. Risk Reduction CAST Example APPROVED PLAN Completed + Plan (2007 Implementation Level) Completed + Plan (2020 Implementation Level) All JSIT Proposed Enhancements (2020 Implementation Level) Resource Cost ($ Millions) Risk Reduction Total Cost in $ (Millions) % 25% 50% 75% 100% Risk Eliminated by Safety Enhancements Completed $ $ $ $ $

Dollars/Flt. Cycle Part 121 Aviation Industry Cost Due to Fatal/Hull Loss Accidents Historical cost of accidents per flight cycle 74% Risk reduction Savings ~ $74/Flight Cycle Or ~ $814 Million Dollars/Year Cost of accident fatalities following implementation of the CAST 2020 levels 2020 Cost Savings CAST Example 2007

What the GA JSC can accomplish GA accident and incident data drives direction of GA JSC activities GA JSC to charge the SAT with chartering study groups on specific topics Working groups of SMEs formed to identify risks and develop mitigations Mitigations are assessed and prioritized A cooperative industry/government GA safety plan is developed and implemented

GA JSC SAT & WGs - Moving Forward History shows focused action and introduction of new capabilities have led to accident risk reductions Joint industry and government teams working together to a common goal can further enhance the safety of our very safe aviation system Full implementation will require a coordinated effort between industry and government The GA JSC is moving forward to meet the challenge