1. A Comparative Overview of the Protection Level Concept for Augmented GNSS and LORAN Stanford University GPS Laboratory Weekly Meeting 20 December 2002 Sam Pullen Stanford University spullen@relgyro.stanford.edu

2. 20 December 2002 Sam Pullen 2 Aviation Requirements Definitions ACCURACY: Measure of navigation output deviation from truth, usually expressed as 1  (68%) or 2  (95%) error limits. INTEGRITY: Ability of a system to provide timely warnings when the system should not be used for navigation. INTEGRITY RISK is the probability of an undetected hazardous navigation system anomaly. CONTINUITY: Likelihood that the navigation signal-in-space supports accuracy and integrity requirements for the duration of the intended operation. CONTINUITY RISK is the probability of a detected but unscheduled navigation interruption after initiation of approach. AVAILABILITY: Fraction of time navigation system is usable (as determined by compliance with accuracy, integrity, and continuity requirements) before approach is initiated.

3. 20 December 2002 Sam Pullen 3 Summary of Aviation Requirements Original Source: GPS Risk Assessment Study: Final Report. Johns Hopkins University Applied Physics Laboratory, VS-99-007, January 1999. http://www.jhuapl.edu/transportation/aviation/gps/ Being reconsi- dered by RTCA WAAS LAAS (LAAS satisfies WAAS ops., within VDB coverage) SPS/RAIM + INS

4. 20 December 2002 Sam Pullen 4 LPV (APV 1.5) 350 ft DH 50 m VAL, 40 m HAL Courtesy: FAA AND-730 Approach with Vertical Guidance (APV) CAT I CAT II CAT III 200ft DH 10m VAL 100ft DH 5.3m VAL 0~100ft DH 5.3m VAL DH: decision height VAL:vertical alert limit HAL: horizontal alert limit Requirement: More Accuracy, Tighter Bounds Benefit: Lower DH Precision Approach Alert Limits

5. 20 December 2002 Sam Pullen 5 Protection Level Objectives To establish integrity, augmented GNSS systems must provide means to validate in real time that integrity probabilities and alert limits are met This cannot be done offline or solely within GNSS augmentation systems because: –Achievable error bounds vary with GNSS SV geometry –Ground-based systems cannot know which SV’s a given user is tracking –Protecting all possible sets of SV’s in user position calculations is numerically difficult Protection level concept translates augmentation system integrity verification in range domain into user position bounds in position domain

6. 20 December 2002 Sam Pullen 6 Key Assumptions in Existing Protection Level Calculations Distributions of range and position-domain errors are assumed to be Gaussian in the tails –“K-values” used to convert one-sigma errors to rare-event errors are computed from the standard Normal distribution Under nominal conditions, error distributions have zero mean (for WAAS and LAAS) Under faulted conditions, a known bias (due to failure of a single SV or RR) is added to a zero-mean distribution with the same sigma Weighted-least-squares is used to translate range- domain errors into position domain –Broadcast sigmas are used in weighting matrix, but these are not the same as truly “nominal” sigmas

7. 20 December 2002 Sam Pullen 7 LAAS Protection Level Calculation (1) Protection levels represent upper confidence limits on position error (out to desired integrity risk probability): –H 0 case: –H 1 case: –Ephemeris: Nominal range error variance Geom. conversion: range to vertical position (~ VDOP) Nominal UCL multiplier (for Gaussian dist.) Vert. pos. error std. dev. under H 1 H 1 UCL multiplier (computed for Normal dist.) B-value conver- ted to Vertical position error (S index “3” = vertical axis) (nominal conditions) (single-reference- receiver fault) (single-satellite ephemeris fault)

8. 20 December 2002 Sam Pullen 8 Fault-mode VPL equations (VPL H1 and VPL e ) have the form: VPL fault  LAAS users compute VPL H0 (one equation), VPL H1 (one equation per SV), and VPL e (one equation per SV) in real-time –operation is aborted if maximum VPL over all equations exceeds VAL –absent a fault, VPL H0 is usually the largest Fault modes that do not have VPL’s must: –be detected and excluded such that VPL H0 bounds –residual probability that VPL H0 does not bound must fall within the “H2” (“not covered”) LAAS integrity sub-allocation LAAS Protection Level Calculation (2) Mean impact of fault on vertical position error Impact of nominal errors, de-weighted by prior probability of fault

9. 20 December 2002 Sam Pullen 9 Top-Level LAAS Signal-in-Space Fault Tree Loss of Integrity (LOI) Nominal conditions (bounded by PL H0 ) Single LGF receiver failure (bounded by PL H1 ) All other conditions (H2) 2  10 -7 per approach (Cat. I PA) 1.5  10 -7 2.5  10 -8 Single-SV failures All other failures (not bounded by any PL) 1.4  10 -7 1  10 -8 Ephemeris failures (bounded by PL e ) 2.3  10 -8 Other single-SV failures (not bounded by any PL) 1.17  10 -7 Allocations to be chosen by LGF manufacturer (not in MASPS or LGF Spec.)

10. 20 December 2002 Sam Pullen 10 WAAS Protection Level Calculation Message Types 2-6, 24Message Types 10 & 28 MOPS Definition Message Type 26 MOPS Definition User Supplied User Supplied This “VPL H0 ” is the only protection level defined for WAAS. Errors not bounded by it must be excluded within time to alert, or  must be increased until this VPL is a valid bound. Courtesy: Todd Walter, SU WAAS Lab

11. 20 December 2002 Sam Pullen 11 Top-Level WAAS Signal-in-Space Fault Tree Courtesy: Todd Walter, SU WAAS Lab 90% of total 10 -7 integrity risk req’t. falls within domain of “H0” (actually “H_all”) protection level calculation −Remaining 10% allocated to WAAS hardware faults not covered by PL −UDRE and GIVE set based on the maximum of bounding sigmas for nominal and faulted conditions (after SP monitoring) Fault cases not represented in tree must have negligible probability Hardware faults (not covered by PL) 1e-8 Based on maximum of nominal and faulted conditions

12. 20 December 2002 Sam Pullen 12 LORAN Horizontal Protection Level Provide user with a guarantee on position –Horizontal Protection Level > Horizontal Position Error  i is the standard deviation of the normal distribution that overbounds the randomly distributed errors  i an overbound for the correlated bias terms  i an overbound for the uncorrelated bias terms => Biases are to be treated as part of the nominal error distribution Courtesy: Sherman Lo, SU LORAN Project

13. 20 December 2002 Sam Pullen 13 LORAN Integrity Fault Tree Phase Error Cycle Error Courtesy: Sherman Lo, SU LORAN Project

14. 20 December 2002 Sam Pullen 14 Threshold and MDE Definitions Test Statistic Response (no. of sigmas) Failures causing test statistic to exceed Minimum Detectable Error (MDE) are mitigated such that both integrity and continuity requirements are met.

15. 20 December 2002 Sam Pullen 15 MDE Relationship to Range Domain Errors Courtesy: R. Eric Phelts, SU GPS Lab MDE in test domain corresponds to a given PRE in user range domain depending on differential impact of failure source If resulting PRE  MERR (required range error bound), system meets requirement with margin If not, MDE must be lowered (better test) or MERR increased (higher sigmas  loss of availability)

16. 20 December 2002 Sam Pullen 16 Reasons for Sigma Inflation We cannot prove that the tails of LAAS/WAAS error distributions are Gaussian –Theoretical error analyses suggest Gaussian (noise, diffuse multipath) or truncated (specular multipath) distributions, but analysis alone cannot be relied upon to validate a 10 -7 or lower probability. –Some degree of “mixing” is unavoidable in practice Error distribution mean, sigma, and correlation estimates have statistical noise due to limited number of independent samples. Inflating sigma inputs to PL is a convenient way to account for integrity monitor limitations when no PL is defined for a particular fault case.

17. 20 December 2002 Sam Pullen 17 Theoretical Impact of Sampling “Mixtures” on Tails of Gaussian Distributions Normalize by theoretical sigma Normalize by actual sigmas Normalize by imperfect sigmas

18. 20 December 2002 Sam Pullen 18 Error Estimates from LAAS Test Prototype (9.5 – 10.5 degree SV elevation angle bin) 70+ days of data: June 1999 – June 2000 200 seconds between samples Significant tail inflation observed Source: John Warburton, FAA Technical Center (ACT-360)

19. 20 December 2002 Sam Pullen 19 Error Estimates from LAAS Test Prototype (29.5 – 30.5 degree SV elevation angle bin) 70+ days of data: June 1999 – June 2000 200 seconds between samples Tail inflation is less pronounced, most likely due to reduced multipath variation within this bin (i.e., less “mixing”) Source: John Warburton, FAA Technical Center (ACT-360)

20. 20 December 2002 Sam Pullen 20 Potential for Excessive Conservatism Each error/anomaly source that contributes to sigmas in PL calculations has some degree of magnitude and/or distribution uncertainty Traditional approach of “upper bounding” each uncertainty element may lead to excessive conservatism in the final sigma once conservative sigmas for each error source are convolved Avoiding this by creating less conservative bounds on each sigma element means giving up on the idea of protection levels “proving” system safety Clear trade-off exists between degree of conservatism/“provability” and system availability, which has its own safety impact

21. 20 December 2002 Sam Pullen 21 Solution: “Keep Two Sets of Books” Uncertain Parameters Detailed Study and Probability Modeling TEP (primary due to engineer and DM acceptance) PRA/DA (backup – less detailed) Compare and Contrast Alert DM if Significant Discrepancy (Add detail and re- compare) Uncertainty Bounding Deterministic Assessment / Sensitivity Studies Optimal Action (risk avoidance within tech./cost/schedule constraints) DA Utility Modeling Probabilistic Risk Assessment Decision Tree Resolution  Optimal Action

22. 20 December 2002 Sam Pullen 22 WAAS Vertical Performance at Queens, NY WRS Site Note that VPL’s imply much larger errors than are actually observed – significant sigma inflation is evident. For Phase 1 WAAS, GIVE (Grid Ionosphere Vertical Error) is the dominant contributor to VPL.

23. 20 December 2002 Sam Pullen 23 Impact of Sigma Inflation on Category I LAAS Availability Category I PA Availability Simulation: 10 user locations (6 US, 4 Europe), 5 o mask angle Cycle through all 22-of-24 GPS SV Outage Cases (276) Service Availability Maximum Service Outage Maximum Service Outage (min) Normalized  Inflation Factor (1 = AD curve value) Best location Worst location Mean Normalized  Inflation Factor (1 = AD curve value) Availability Worst location Best location Mean 11.21.41.61.822.22.42.6 0.95 0.955 0.96 0.965 0.97 0.975 0.98 0.985 0.99 0.995 1 B3/B C3/B 11.21.41.61.822.22.42.6 0 20 40 60 80 100 120 140 160 180 200 B3/B C3/B Best location Worst location Mean

24. 20 December 2002 Sam Pullen 24 Summary Protection Levels provide the means for users to translate range-domain integrity assurance from WAAS/LAAS/etc. into real-time safety assessments Protection Levels are defined to bound errors due to nominal conditions and specific failure modes –Failure modes not covered by specific PL’s must be overbounded by nominal PL or assigned a separate P(HMI) allocation within system level fault tree Broadcast sigma inputs to PL’s are a key design parameter and will be conservative in practice Protection levels are very useful but should not be misconstrued as an inherent safety guarantee –PL’s are highly dependent on assumptions on inputs –Try to avoid excessive conservatism in pursuit of a “provable” overbound