1. Risk Assessment using Flight Data Analysis
Dr. Thom Mayer
Austin Digital Inc.
International Aviation Safety Conference
June 4, 2008

2. Risk Assessment using Flight Data Analysis
Measuring Accident Risk using Flight Data
based on work done 2005 to present
(No actual Airline results are shown. Any results shown are not real data.)
Future Directions for Analytical Risk Assessment
Suggestion of where to go next with the technique

3. FDM Provides the Ability to Measure every Flight
the mean and standard deviation of the distribution tell about “standard flight operations”
but to study safety risk we need to look at the tails of the distribution

4. Company Confidential - Do Not Distribute
For risk measurement purposes we need measures for closeness to each accident category
Controlled Flight into Terrain (CFIT) ………...………………………………
Landing Overrun ……………………………………………………………
Fuel Exhaustion ………………………………………………………………
Stall ………….…………………………………………………………………
Hard Landing …………………………………………………………………
Takeoff Runway Overrun ……………………………………………………
Landing Offside ………………………………………………………………
Landing Short ……….……………………………………………………
Runway Incursion ……….………………………………………………
Fuel Tank Explosion ……….……………………………………………
For some accident categories the measure is relatively straight-forward, but for other accident categories a good measure is more involved.
Risk Assessment – 2006
Company Confidential - Do Not Distribute

5. and the frequency and severity of “near misses”
Why does one believe that a flight that only came close to an accident represents risk? Because variations in other variables could have caused a different outcome. ( i.e. variations in winds, weight & balance, training, reaction times, …)
a “near miss”
accident
threshold
measure of closeness to accident category
distribution of how the event could have played out
So it is believable that there is a relationship between accident rates (i.e. risk)
and the frequency and severity of “near misses”

6. Our approach takes advantage of a large population of flight data (i.e. 107) and the multi-year accident rate statistics
tail of distribution measured
from large sample of flights
(actually it is sparse)
accident
threshold
measure of closeness to accident category
Theoretical distribution that one could measure if we had infinite data
actual
“rate”
we define a mathematical measure of the tail of the distribution and “calibrated” it using a large pool of flights and the accident rate statistics for that population.

7. This enables one to compare risk between sub-populations
< 300 % Remaining
# %
B737 no data no data B B A A
Airline
EMS Group
The “weight” of the tail of the entire population represents the accident rate statistics for that population.
The “weight” of the tail of a sub-population, compared to the “weight” of the tail for the entire population, give relative risk.

8. This Analysis is Performed for each Accident Category and each sub-Population Drill Drown
> 25 % Decel Needed
# % B B B A A
Airline
EMS Group
> 35 % Decel Needed B B B A A
Airline
EMS Group

9. The calculated risk is reported per sub-population (in this example the accident category is CFIT and the sub-populations are fleet type)

10. When the sub-populations are time intervals, the technique provides risk trending
This data has been corrected to remove variations resulting from the addition or removal of fleets to the FOQA program from one quarter to the next.

11. One can drill down into specific runways and fleet combinations (note that as you drill down, the error bars become larger)

12. Company Confidential - Do Not Distribute
Using this technique we generate an Annual Risk Assessment Report to Participating Airlines
Risk Assessment 2006
Risk Assessment – 2006
Company Confidential - Do Not Distribute

13. What Types of Accident Risk are Assessed for this Report?
Accident categories, representing 63% of American and European commercial jet passenger fleet fatalities were analyzed for this report.
Risk Assessment – 2006
Company Confidential - Do Not Distribute

14. Company Confidential - Do Not Distribute
This Report is the Second Step of the Annual Risk Assessment Cycle Use this report to set goals (perhaps the reduction of a specific accident risk for each fleet)
Risk Assessment – 2006
Company Confidential - Do Not Distribute

15. Company Confidential - Do Not Distribute
For each accident category, the report to the participating Airline includes a trending of risk for the group and the airline compared to the group, and a drill down of risk by fleet type and airport/runway.
Airline Risk Overview
Risk Assessment – 2006
Company Confidential - Do Not Distribute

16. What FDM capabilities do you need to support this type of Analysis?
Ability to generate sophisticated measures
e.g. CFIT or risk of running off end of runway
Ability to support very low false positive rates
if accident rate is 10-8 then “close call” rate might is 10-5, so false positives rate can’t be 10-3
Ability to handle large numbers of flights
We are getting close to 107 flights in the EMS group

17. Identical processing for all data
What implications does this have for Data Aggregation from multiple Airlines?
Identical processing for all data
Either the processing systems at the Airlines need to be identical, or aggregation needs to be at the flight data level
No “Cherry Picking” of data set
Participants need to provide complete data set. Filtering of “Incident” or “Sensitive” flights would invalidate the data set.
Large Data Sets Required
to get any resolution on drill-down