1. SBAS and GBAS Integrity for Non- Aviation Users: Moving Away from "Specific Risk" ION ITM 2011San Diego, CA. 25 January 2011 Sam Pullen, Todd Walter, and Per Enge Stanford University spullen@stanford.edu

2. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"2 Motivation (1): SBAS and GBAS for Non- Aviation Users Where augmentation signals can be received, SBAS and GBAS benefits are available to all users. However, integrity algorithms in airborne MOPS are designed to support specific aviation applications. –Resulting integrity protection levels are not well- suited for other classes of users Correcting this would increase the attractiveness of SBAS and GBAS to non-aviation transport users (auto, rail, marine) and others.

3. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"3 Motivation (2): Accuracy and Integrity Accuracy bounds (e.g., 95% vertical position error, or VPE) can be measured and modeled with high precision Integrity bounds (e.g., 10 -7 vertical protection level, or VPL) cannot be –Lack of sufficient measurements –Flaws in Gaussian extrapolations to low probabilities –Dependence on details of failure models and assumptions –Too little is known; too much is uncertain… Illustrative example – not to scale or direction 95% HPE HPL (per MOPS) HPL (non- aviation application)

4. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"4 WAAS VPE vs. VPL from FAA PAN Data (3 rd Qtr 2010: July – Sept.) Source: WAAS PAN Report #34, Oct. 2010. http://www.nstb.tc.faa.gov/ DisplayArchive.htm VPL (m) VPE (m) Max. VPE  7 m (at Barrow, AK) 95% VPE  1.2 m 99% VPE  1.6 m

5. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"5 WAAS Reference Station Classifications (for this study only) Figure source: FAA GNSS Press Kit http://preview.tinyurl.com/4ofdzz4 7 Inner Stations 13 Outer Stations 18 Remote Stations

6. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"6 Max. VPE and VPL from FAA PAN Data (1 Jan. 2004 – 30 Sept. 2010) Worst Case Between “Inner” and “Outer” WAAS Stations  “InOut” Set

7. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"7 Max. HPE and HPL from FAA PAN Data (1 Jan. 2004 – 30 Sept. 2010) Worst Case Between “Inner” and “Outer” WAAS Stations  “InOut” Set One unusual result: 12 m error at Cleveland in Spring 2005 (correct number?) As expected, both HPE and HPL are significantly lower than VPE and VPL.

8. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"8 Ratio of Max. VPL and Max. VPE from FAA PAN Data (“InOut” Station Set)

9. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"9 Ratio of Max. HPL and Max. HPE from FAA PAN Data (“InOut” Station Set) Unusual error at Cleveland (if correct) just barely exceeded by HPL.

10. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"10 How Many Samples Were Collected? 4.25  10 -9 sec (49,324.6 days) (105.04 years) All validated PAN data from 1 Jan. 2004 to 30 Sept. 2010 Assume data correlated over 30 sec 1.4  10 -8 independent samples Assume data correlated over 150 sec (~ one CAT I approach) 2.8  10 -7 independent samples Assume data correlated over 600 sec (10 min) 7.1  10 -6 independent samples

11. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"11 Average vs. Specific Risk Assessment Average Risk (my definition): the probability of unsafe conditions based upon the convolved (“averaged”) estimated probabilities of all unknown events. –Probabilistic Risk Analysis (PRA) is based on this procedure –Risk aversion and value of information (VOI) are applied to the outputs of PRA  integrity risk requirements, alert limits Specific Risk (my definition): the probability of unsafe conditions subject to the assumption that all (negative but credible) unknown events that could be known occur with a probability of one. –Evolved from pre-existing FAA and ICAO safety standards –Risk aversion and VOI and buried inside specific risk analysis –Results (risk and protection levels) are inconsistent with PRA

12. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"12 Simplified Example: Ionospheric Spatial Decorrelation (1) 20:15 UT21:00 UT Severe Ionospheric Storm Observed over CONUS on 20 November 2003

13. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"13 Simplified Example: Ionospheric Spatial Decorrelation (2) Using PRA, estimated “prior” probabilities of severe decorrelation are combined with the likelihood of SBAS or GBAS mitigation to derive resulting user risk. –Prior probabilities need not be known precisely –Benefits of improved mitigation (“better information”) appear naturally as lower integrity risk. Under FAA interpretation of Specific Risk, worst-case iono. delay gradient is “credible” and thus is assigned a probability of one. –Worst-case for GBAS (CAT I): an extremely large gradient that escapes detection by “matching speed” with ground station »This differs in real time for each site and GNSS geometry –Worst-case for SBAS (LPV): a very large gradient that is just small enough to avoid detection by master station

14. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"14 Simplified Example: Ionospheric Spatial Decorrelation (3) 051015202530354045 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 User Vertical Position Error (meters) PDF Worst-case error, or “MIEV”, is  41 m Most errors are exactly zero due to ground detection and exclusion, but all zero errors have been removed from the histogram. Simulated results for Memphis GBAS impacted by severe ionospheric gradient (RTCA 24-SV GPS, 6-km, User-to-ground separation, 1 and 2-SV impacts) Most plotted (non-zero) errors are below 10 m even under severe conditions.

15. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"15 Benefits of an “Average Risk” Approach (Potential SBAS PL Reduction) “Average risk” approach supports large reductions in HPL and VPL implied by WAAS PAN data, pending more complete database analysis. Use “full-scale” PRA to re-assess “rare-normal” and faulted errors. 2425262728293031323334 0 5 10 15 20 25 30 35 40 PAN Report Number VPL or HPL (meters) 95% VPL 95% HPL Adjusted VPL Adjusted HPL From reports since Jan. 2008 Max. 95% PLs among stations in CONUS (“InOut” set) Conservative reduction factors from PAN data: VPL / 4.0 HPL / 2.5

16. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"16 A Combined “Average/Specific” Risk Approach Depending on user and decision maker risk aversion, separate “average risk” and “specific risk” integrity requirements could be issued. –Both apply at all times  one or the other will tend to dominate for a particular application. For example: 10 -7 integrity risk per operation (“average”) plus requirement that a worst-case undetected condition cannot increase the total vehicle loss risk by more than a factor of 10. –For aircraft case, factor of 10 increase in total risk equates to specific risk requirement of 10 -5 per operation for nav. system (more strictly, 9  10 -6 ) –Specific factors for each vehicle and application would vary. –There is no “correct” degree of risk aversion.

17. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"17 Summary Existing integrity assurance procedures for SBAS and GBAS are unique to aviation and its history and may not be suitable for other users. SBAS (and GBAS) data analysis suggests that 10 -7 HPL and VPL can be greatly reduced if “average risk” approach is taken. –Examination of past data is useful, but more thorough PRA analysis should be conducted. If worst-case elements of risk assessment are still desired, an average/specific risk mixture can be used. –This flexible “mixture” capability should satisfy almost any level of user and decision maker risk aversion.

18. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"18 Backup Slides follow…

19. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"19 WAAS VPE from FAA PAN Data (3 rd Qtr 2010: July – Sept.) Source: WAAS PAN Report #34, Oct. 2010. http://www.nstb.tc.faa.gov/ DisplayArchive.htm Max. VPE  7 m at Barrow, AK VPE (m) No. of Samples Meas. from 37 WAAS stations

20. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"20 Example Error Table from PAN #34 (from PAN #34)

21. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"21 Max. VPE and VPL from WAAS PAN Data (1 Jan. 2004 – 30 Sept. 2010) (all numbers are in meters)

22. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"22 95% and Max. VPE from FAA PAN Data (1 Jan. 2004 – 30 Sept. 2010) 101520253035 0 5 10 15 20 25 30 35 40 45 50 Quarterly PAN Report Number (8 – 34) Vertical Position Error (meters) VAL for LPV Note: VPL always bounds VPE. Remote Stations Outer Stations Inner Stations 95 % VPE Max. VPE Severe iono. scintillation in Alaska in March and May 2007 (user receiver should prevent)

23. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"23 WAAS VPE vs. VPL in CONUS (2003 – 2006) ( from Wanner, et al, 2006) Vertical Position Error (meters) 99.99% VPE 99.9% VPE 95% VPE 99% VPE Mean VPE 1  VPE Ratios : 6.96.46.86.56.0 6.8

24. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"24 WAAS Max. VPE in CONUS (2003 – 2006) (from Wanner, et al, 2008)

25. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"25 An “Average Risk” Approach to SBAS (and GBAS) – word version Data imply an “average risk” equivalent VPL for WAAS ~ 4 – 5 times lower than current value. Re-assess “rare-normal” and faulted error models and data to build a “certifiable” safety case. –Multiple rare-normal (“fault-free”) models built from existing data to incorporate remaining uncertainty –All fault-mode analyses follow the same approach: Estimate prior fault probabilities and probability uncertainties. Simulate all significant variations of each fault type rather than “worst case” focus  convolve with prior dist. to estimate risk. –Faults whose impact is driven by worst-case scenarios (ionosphere, signal deformation) will become less important. –Multiple-fault scenarios neglected as too improbable may become more important, as probabilistic weighting of risk may show that fault-combination cases are non-negligible.

26. 25 January 2011Integrity for Non-Aviation Users: Moving Away from "Specific Risk"26 A Combined “Average/Specific” Risk Approach (1) Derived from FAA “Hazard Risk Model” (1) and Simplified Aircraft Accident Risk Breakdown (2) (1) FAA System Safety Handbook, 2008. http://www.faa.gov/library/manuals/aviation/risk_management/ss_handbook/http://www.faa.gov/library/manuals/aviation/risk_management/ss_handbook/ (2) R. Kelly and J. Davis, “Required Navigation Performance (RNP),” Navigation, Spring 1994. Catastrophic (Likely a/c hull loss) Hazardous (Risk of a/c loss; Severe loss of safety margin) Major (Slight risk of aircraft loss/pilot challenged) 10 -5 10 -7 10 -9 10 -6 Overall a/c loss prob. Loss prob. due to equipment failure ~ 10% ~ 1% (~ 100 systems) Loss prob. due to GNSS nav. failure ~ 1% 10 -7 10 -9