1. 1 © The MITRE Corporation This is the copyright work of The MITRE Corporation and was produced for the U.S. Government under Contract Number DTFA01-01-C-00001 and is subject to Federal Aviation Administration Acquisition Management System Clause 3.5-13, Rights in Data-General, Alt. III and Alt. IV (Oct. 1996). No other use other than that granted to the U.S. Government, or to those acting on behalf of the U.S. Government, under that Clause is authorized without the express written permission of The MITRE Corporation. For further information, please contact The MITRE Corporation, Contracts Office, 7515 Colshire Drive, McLean, VA 22102, (703) 883-6000 Commercial Aviation Accidents Before and During Alaska Capstone (ADS-B, FIS-B, Terrain Awareness, IFR Infrastructure) Fifth USA/EUROPE Seminar on ATM R&D 24 June 2003 Worth Kirkman

2. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 2 Capstone Joint initiative by FAA Alaska region and industry to improve safety and efficiency of commercial aviation ADS-B and related broadcast technologies –Traffic and terrain situational awareness –Flight information including weather in the cockpit –Surveillance and radar-like ATC services –Additional automated weather sites and GPS instrument approaches Capstone Phase 1: ground systems and avionics for commuter and air taxi aircraft (FAR part 135) in isolated southwest Alaska –Near Yukon and Kuskokwim rivers – the “YK delta”

3. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 3 Aviation in the YK Delta Usually the only means of transport Limited services and infrastructure –Unpaved airstrips –Mostly non-radar –Few weather sites Aircraft: single-engine or light twin Operations: mostly VFR Commercial accident rate 2.5 times the remainder of US

4. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 4 Safety Problems Targeted by Capstone Enroute controlled flight into terrain (CFIT) Mid-air collisions Inadequate flight information –Especially weather Inadequate support for IFR operations

5. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 5 Overview of Capstone Capabilities

6. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 6 Capstone’s Impact on Accidents: Overview of Approach Characterize accidents before and during Capstone Model the projected impact of Capstone –Different accidents types and causes –Assessed affect on historical accidents if Capstone had been in place –Progress toward putting Capstone in place –Changes is operations Compare projections against (and learn from) accidents that still occur Characterize significance of / confidence in any observed changes

7. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 7 Categorizing Accidents Analyze detailed narratives from the National Transportation Safety Board (NTSB) for each accident Apply a decision tree to categorize each accident –Based upon causal / contributing factors and phase of flight –Categories specialized toward understanding Capstone’s effectiveness Assess likelihood of Capstone preventing each historical accident –Rules-of-thumb for sub-categories based on expert judgment –Adjust for specifics of each accident scenario that would limit (or increase) effectiveness

8. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 8 Accidents and Fatal Accidents Before Capstone YK Delta / Part-135 / 1990-1999

9. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 9 Accident Categories Affected by Capstone, Potential Effectiveness if Fully Implemented Navigation –Enroute CFIT, most often in reduced visibility or ‘Flat Light” (Capstone potentially ~90% effective) –Terrain Clearance Floor (TCF) violation on approach/depart (Capstone Phase 1 not effective) –Mis-location of aircraft (moving Map potentially ~100% eff.) Traffic –Mid-Air collisions (Capstone potentially ~100% effective) Flight Information –Inadequate Weather data, especially icing, but also visibility* (Capstone effect expected but not quantifiable from scenarios) *Surface winds contributing to take-off and landing accidents are included in take-off and landing categories

10. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 10 Anticipating Accident Rates With Capstone Formulas for Projections - Overall and by Category Project numbers of accidents: Historical accidents X growth in operations Project reductions in navigation and traffic accidents from: % of Aircraft operations equipped X % of Other aircraft equipped (affects traffic only) X % of Pilots adequately trained X % of Historical accidents in preventable sub-categories X Assessed effectiveness for historical accident narratives Weather and other categories should benefit, but effectiveness could not be quantified from historical narratives

11. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 11 2000-2001 Progress on Implementation: Equipage and Pilot Training Training Effectiveness: ~ 50% Aircraft Days: 45% Operations: 50%

12. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 12 Anticipated vs. Observed Accidents YK Delta / Part-135 / 2000-2001

13. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 13 Assessing Significance and Confidence Accident counts in all categories are close to two-year standard deviations from previous years –Not statistically significant Further analyses: Compare accidents by Capstone- equipped vs. non-equipped aircraft –Distribution of accident types –Rate of total accidents per number of operations

14. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 14 Accidents by Equipped and Non-Equipped Aircraft YK Delta / Part-135 / 2000-2001 Not Capstone EquippedCapstone Equipped

15. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 15 Comparing Accident Rates Assess separation and stability of cumulative accident rates over time Challenge: poor data on operations  Operations at Bethel  Enplanements and fleet changes at Bethel  Capstone equipped / non-equipped ratio at Bethel (SVFR) ~Aviation fuel usage –Operations and enplanements at villages –Airframe hours of operation Seasonal (monthly) operations at Bethel averaged between 2000 & 2001; equipage ratios from SVFR

16. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 16 Non-Equipped vs. Capstone Equipped Accident Rates YK Delta / Part-135 / 2000-2001

17. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 17 Interim Assessments & Planned Extensions to the Analyses 2000-2001 is too soon for statistically significant changes in specific types of accidents or to accidents over-all - though indications appear “promising” 2002+ will provide longer basis for comparison and larger samples of accidents Plan to obtain better data on operations –Operations counts / miles / hours from operator records –Gather reported operations changes associated with Capstone that should guide additional data collection/analysis Examine trends –vs. nearby non-Capstone areas –vs. all of Alaska

18. 6/24/03 © 2003 The MITRE Corporation. All Rights Reserved. 18 Additional Details and Updates The Safety Impact of Capstone Phase 1 – An Interim Assessment of 2000-2001 (MPW0000150 Revision 2) –http://www.alaska.faa.gov/capstone/docs/mitre%20study.pdfhttp://www.alaska.faa.gov/capstone/docs/mitre%20study.pdf Updated analysis is in progress –Implementation Progress and Accidents reported through 2002 –Watch http://www.alaska.faa.gov/capstonehttp://www.alaska.faa.gov/capstone