1. Ionospheric Integrity Lessons from the WIPP Todd Walter Stanford University http://waas.stanford.edu Todd Walter Stanford University http://waas.stanford.edu

2. History  Ionospheric Storms and Disturbances Originally Tested Via Scenarios  Simulated disturbances added to simulated ionosphere  Generally, large geographic features were placed near center of network  Ionospheric Algorithm Originally Based on JPL GIM Code  Tuned to work on scenarios  Live data from WRSs not yet available

3. Ionospheric Models  Provides Truth  Good for initial algorithm validation  Very Smooth  Average TEC  Loses small-scale variations  Spatial gradients smoothed as well  Useful Tool Before Data Was Available  However Does Not Faithfully Represent Real-World Instantaneous Ionosphere

4. Example Scenario  From “Ionospheric Specification for the Wide Area Augmentation System (WAAS) Simulation Studies” by Steve Chavin, ION GPS-96  dTEC/dt = 0.74 TECU/min  Gradient = 0.085 TECU/km  Shell Height = 360 km

5. Problem  Algorithm Tuned to Work on Scenarios  All passed easily  Did not Work as Well on Real Data  Required extensive retuning  Simulated Ionosphere Did Not Faithfully Reproduce Real Ionosphere  Real disturbances worse than predicted  Real slant-to-vertical errors better than predicted  Failing Scenario Could Prove Loss of Integrity  However, Passing All Scenarios Would Not Demonstrate Positive Integrity  Worst-case scenario is algorithm dependent  Does not demonstrate probability of missed detection requirement is met

6. National Satellite Test-Bed  Early Prototyping  Dual-Frequency Survey Receivers  Single Threaded  Initiated in 1993  Full Deployment Started in 1996  Prototyping Occurred During Solar Minimum  No significant ionospheric disturbances observed  Caused Us to Become Overconfident  Performance Dominated by Receiver Artifacts  Reasonability checks instituted to mitigate these errors  Too aggressive, would remove much of solar max observed behavior

7. 11 Year Solar Cycle  Solar activity changes dramatically over an 11 year solar cycle  Ionosphere at the peak is much worse than at minimum  Most disturbances at peak and declining phase NSTB WIPP

8.  At the End of 1999 FAA Certification Required a Change in the Safety Analysis  Level D code not considered reliable  Threat models required for all monitors  Rigorous accounting for monitor observability  Certification of Ionospheric Algorithms Left to Ionospheric Experts  Experts Created Threat Models From Data  Reliable threat not hypothetical  Must protect against worst observed conditions  Must overbound historical observations  Must have a demonstratable probability of missed detection

9. Supertruth Data  25 WRS - 3 Threads Each - Carrier Leveled - Biases Removed - Voting to Remove Artifacts  Clean Reliable Data Collected at the Peak of the Solar Cycle  Contained Worst Observed Gradients (Temporal and Spatial over CONUS)  Most Severely Disturbed Days Formed the Basis for Threat Model  Ionospheric Disturbances Are Deterministic, but Sampled Randomly  Worst cases are sampled over time  Will appear in the data as they move w.r.t IPPs  Apply Data Deprivation to Model Effects of Poor Observability

10. Ionospheric Measurements

11. Storm Example

12. Differences in Vertical Delay  Difference in Vertical Delay vs. IPP Separation Distance for Two Days: Quiet Day July 2 nd 2000 Disturbed Day July 15 th 2000

13. CONUS Ionosphere Threat  Not Well-Modeled by Local Planar Fit  Ionosphere well-sampled 1  Ionosphere poorly sampled 2  Ionosphere Changes Over the Lifetime of the Correction  User Interpolation Introduces Error 1. “Robust Detection of Ionospheric Irregularities,” Walter et al. ION GPS 2000 2. “The WAAS Ionospheric Threat Model,” Sparks et al. Beacon Symposium 2001

14. Well-Sampled Ionosphere  Chi-Square Metric Acts as “Storm- Detector”  Test using small decorrelation value  Nominally ~ 35 cm one-sigma  Passing test accepts larger value  Typically ~ 85 cm one-sigma  Analytic Approach  Does not require data except as validation  Fully specified before the first storm data of April 2000

15. Under-Sampled Ionosphere  Purely an Empirical Threat Model  Worst Storm Data Used  IPPs Removed From Estimation to Simulate Poor Sampling  Three quadrant removal schemes  Storm detector used on remaining data  Threat Based on Worst Deviation for Given Set of Metrics  Threat Region Is 5x5 Degree Cell Centered on IGP

16. CORS Data

17. Goal of Sigma Undersampled  To protect against an unfortunate sampling of the ionosphere such that we fail to detect an existing disturbance  Presumes the ionosphere is non- uniform near the IGP, i.e. it is divided into at least two states: a quiet one that is sampled, and a disturbed one that is not

18. Goal of Data Deprivation  To divide the IPPs into two groups: one that samples a relatively quiet ionosphere and does not trip the chi-square, the other that samples ionosphere not well-modeled from the quiet points  Data deprivation is used to simulate conditions that were not actually experienced but may reasonably be experienced in the future  It allows us to investigate threats that occurred in well observed regions as though they had occurred in poorly observed regions  Want to have the quiet IPP distribution match those that may occur on operational system

19. Storm Days  Over the last 5 years, ~100 active days have been identified  ~50 affected or would have affected WAAS performance  ~45 supertruth files generated  Working on files for the lesser days  16 days affect our empirical threat model (serious effect)  All supertruth files available to international SBAS community

20. Temporal Threat Model  Also an Empirical Threat Model  Worst Storm Data Used  Storm detector used  Look to See Largest Change With Respect to Planar Estimate  Overbound of Worst Rate of Change Ever Observed  Correlation with Spatial Gradient  Errors are currently double counted

21.  No Storm Detector  > 3 m / min  > 6 m /2 min  > 7 m over 5 min  With Storm Detector  < 0.5 m / min  < 1.25 m /2 min  < 2.5 m over 10 min Temporal Threat Model

22. Post IOC Storms  October 29-31 and November 20, 2003 Were Some of the Worst Storms Ever Observed  Conservatism in GIVE Calculation Protected Users  No HMI observed at any location  None even close  However Worse Than Predicted  Still much uncertainty in ionosphere

23. Conclusions  FAA Certification Required All Users Bounded Under All Conditions  Ionospheric deviations are deterministic  Ionospheric deviations are observable  Threat Models Essential for Limiting Ionospheric Behavior  Each Monitor Must Account for the Limits of Its Observability  Approach is Very Conservative  We are still learning!