1. Presented to: By: Date: Federal Aviation Administration Wide Area Augmentation System (WAAS) Operations Team AJW-1921 Offline Monitoring B. J. Potter Brad Dworak Chad Sherrell March 7, 2011 WIPP

2. Offline Monitoring 2 Federal Aviation Administration 2011-03-07 Introduction This presentation covers the 4th Quarter of 2010 –(2010-10-01 – 2010-12-31) Next Steps –Analyze data for entire quarter –Transition all OLM analysis to SOS

3. Offline Monitoring 3 Federal Aviation Administration 2011-03-07 Clock Runoff Assertion The a priori probability of a GPS satellite failure resulting in a rapid change in the GPS clock correction is less than 1.0x10 -4 per satellite. Monitoring Approach –Events typically result in a fast correction that exceeds 256 meters –When this occurs, the satellite is set Do Not Use until the correction reaches a reasonable size –Events where the satellite is set Do Not Use from excessively large fast corrections while the satellite is healthy are recorded

4. Offline Monitoring 4 Federal Aviation Administration 2011-03-07 Clock Runoff No Clock Runoff Events between 2010-10-01 – 2010-12-31

5. Offline Monitoring 5 Federal Aviation Administration 2011-03-07 Ephemeris Assertion The CDF of GPS ephemeris errors in a Height, Cross-track, and Along-track (HCL) coordinate system is bounded by the CDF of a zero-mean Gaussian distribution along each axis whose standard deviations are  osp-ephh,  osp-ephc, and  osp-ephl. The probability that a satellite’s position error is not characterized by this a priori ephemeris model is less than 10-4 per hour. Monitoring Approach –Compare broadcast vs precise in HCL to ensure sigmas are less than 1m, 2.5m, 7.5m for Radial, Cross Track, and In Track

6. Offline Monitoring 6 Federal Aviation Administration 2011-03-07 Ephemeris – Radial PRN: 11 2010-10-15 00:00:00 -2.16619174736 PRN: 05 2010-10-23 23:45:00 -1.25456410331 PRN: 31 2010-11-20 03:45:00 1.44416876193 PRN: 29 2010-12-16 02:45:00 -1.28223089084 PRN: 25 2010-12-23 04:45:00 -1.40505207557 PRN: 25 2010-12-24 06:00:00 -1.29037947749

7. Offline Monitoring 7 Federal Aviation Administration 2011-03-07 Ephemeris – In Track PRN: 11 2010-10-15 04:00:00 14.0549074655 PRN: 25 2010-12-24 07:45:00 14.6629813657 PRN: 24 2010-12-30 01:30:00 -15.7248821707 PRN: 30 2010-12-30 04:15:00 8.86779042477 PRN: 27 2010-11-07 20:15:00 8.39537775327

8. Offline Monitoring 8 Federal Aviation Administration 2011-03-07 Ephemeris – Cross Track PRN: 17 2010-12-01 06:45:00 -2.73555408015 PRN: 21 2010-12-31 00:00:00 -2.57918952043

9. Offline Monitoring 9 Federal Aviation Administration 2011-03-07 RIC Outliers

10. Offline Monitoring 10 Federal Aviation Administration 2011-03-07 Ionospheric Threat Model Monitoring Assertion The values of and  iono adequately protect against worst case undersampled ionosphere over the life of any ionospheric correction message, when the storm detectors have not tripped. Monitoring Approach –Monitor for Chi^2 values greater than 1 in the four regions CONUS > 1% Alaska > 2% Caribbean > 10% Other > 3%

11. Offline Monitoring 11 Federal Aviation Administration 2011-03-07 Monitoring Regions

12. Offline Monitoring 12 Federal Aviation Administration 2011-03-07

13. Offline Monitoring 13 Federal Aviation Administration 2011-03-07

14. Offline Monitoring 14 Federal Aviation Administration 2011-03-07

15. Offline Monitoring 15 Federal Aviation Administration 2011-03-07 Total Chi 2 values ≥ 1 from all regions for 2010 at ZLA (35.07% zeros)

16. Offline Monitoring 16 Federal Aviation Administration 2011-03-07 2010 Total Chi2 Values Over 1

17. Offline Monitoring 17 Federal Aviation Administration 2011-03-07 Antenna Monitoring Assertion The position error (RSS) for each WAAS reference station antenna is 10cm or less when measured relative to the ITRF datum for any given epoch. (Mexico City is allowed 25cm). The ITRF datum version (realization) is the one consistent with WGS- 84 and also used for positions of the GPS Operational Control Segment monitoring stations.

18. Offline Monitoring 18 Federal Aviation Administration 2011-03-07 Purpose Accurate antenna positions needed to support DGPS applications Correct for Time Dependent Process –Tectonic Plate Movement –Subsidence Correct for Shift Events –Seismic –Maintenance WIPP Review for integrity issues –Greater than 10 cm WIPP should review –Greater than 25 cm WIPP must review –Special case for Mexico City (25 cm for review) Project the need for a WAAS Antenna Coordinate Update

19. Offline Monitoring 19 Federal Aviation Administration 2011-03-07 Survey Details Survey Date –2011-01-28 –Cross Compared Against –CSRS-PPP –WFO-R2 –Coordinates Projected to six months beyond WFO WFO Release 3 –2012-05-01

20. Offline Monitoring 20 Federal Aviation Administration 2011-03-07 Results Against CSRS-PPP –All sites less than 5 cm. Against WFO-R2 –All sites less than 5 cm.

21. Offline Monitoring 21 Federal Aviation Administration 2011-03-07 Code Carrier Coherence Assertion The a priori probability of a CCC failure is less than 1x10 -4 per set of satellites in view per hour for GPS satellites and 1.14x10 -4 for GEO satellites.

22. Offline Monitoring 22 Federal Aviation Administration 2011-03-07 CCC monitoring approach  Anik, Galaxy 15 and all GPS satellites are monitored for CCC trips for Q4 2010 (last data for CCC data for Galaxy 15 was on 2010-12-15).  AMR is not currently monitored (not used as ranging source, UDRE floor=50m)  All CCC monitor trips are investigated whenever a trip occurs to determine source of trip  Minimum data sources used in correlation and analysis: CCC test statistic UDRE threshold value CMCI measurements from NETS SQA WAAS Iono calculation L1/L5 Iono GUST calculation published planetary K p and A p values Chi 2 values

23. Offline Monitoring 23 Federal Aviation Administration 2011-03-07 Reported CCC trips for Q4 2010 Date GEO PRN C&V 2010-10-24 06:38:25 138 ZLA 2010-10-24 06:38:30 138 ZDC 2010-10-25 19:23:54 138 ZTL 2010-10-25 19:24:05 138 ZLA 2010-12-08 11:55:40 138 ZDC ZTL 2010-12-08 14:03:56 138 ZDC ZLA 2010-12-11 22:55:44 138 ZDC ZTL 2010-12-12 02:45:52 138 ZDC ZTL 2010-12-12 15:24:27 138 ZDC ZTL

24. Offline Monitoring 24 Federal Aviation Administration 2011-03-07 CCC plots

25. Offline Monitoring 25 Federal Aviation Administration 2011-03-07 CCC plots

26. Offline Monitoring 26 Federal Aviation Administration 2011-03-07 Signal Quality Monitor Assertion The a priori probability of a signal deformation (SD) failure is less than 2.4x10 -5 per set of satellites in view per hour for GPS or GEO satellites. The worst-case range errors due to nominal signal deformations are more than 25cm on any satellite signal relative to the other satellites in view. Monitoring Approach –All SQM Trips will be monitored for and investigated –Max and Median data for each metric will be plotted by Requested UDRE Monitoring for discrepancies between satellite Plots are for the first 4 days of every week for the entire quarter Plots were made using the tools from HMI Build 299

27. Offline Monitoring 27 Federal Aviation Administration 2011-03-07 SQM max plot

28. Offline Monitoring 28 Federal Aviation Administration 2011-03-07 GEO Signal Quality Assertion The WAAS SIS satisfies the requirements for code-carrier coherence and fractional coherence stated in sections 3.1.4.2 and 3.1.4.3 of the [draft] system specification FAA-E-2892c Monitoring Approach –Collect WAAS SIS data from each GEO using GUST receivers connected to dish antennas –Compute and plot the metrics outlined in sections 3.1.4.2 and 3.1.4.3 of FAA-E-2892c –Examine plots, tabulate max metric values and pass/fail states, analyze failures in further detail to identify possible causes

29. Offline Monitoring 29 Federal Aviation Administration 2011-03-07 29 Performance Summary L1 CCCL5 CCCCC short- term long- term short- term long- term short- term long- term PRN 135 CRW max mean PRN 138 CRE max mean regularly below spec limit near spec limit regularly above spec limit

30. Offline Monitoring 30 Federal Aviation Administration 2011-03-07 Summary Only issue: fly-by of CRW apparently affected measurements of both CRE and CRW Twice-daily elevated noise on some days When CRW most N and S; bigger effect as CRE, CRW got close Different sites saw effects of different magnitude Additional periods of noise on Nov 12 (closest approach) PR oscillation correction now included in processing Mitigates systematic error in GUST receiver L1 and L5 PR corrections mapped using GUST receiver and prototype SIGGEN in Zeta lab Applicable to any WAAS GUST/G-II receiver Dependent on PRN code, PR(t) and PR(t-1) Allows more accurate evaluation of received signal 30

31. Offline Monitoring 31 Federal Aviation Administration 2011-03-07 31 Example of Elevated Noise: 19 Nov 2010 Elevated noise at approx. 2:21 and 14:19 UTC (N and S extremes of CRW orbit) -- close to but not at zero Doppler at either OKC or LTN (or APC Primary, which showed little to no effect at either time this day) caused higher than normal max CCC values Effect worse when CRW was close to CRE (effect also seen on CRE) Since different sites show different effects, probably not on SIS; will monitor as CRW returns to nominal orbit slot OKC PRN 135LTN PRN 135

32. Offline Monitoring 32 Federal Aviation Administration 2011-03-07 32 Example of PR Correction Effect: 19 Nov 2010 PR oscillation correction generally benefits L1 more than L5 and PRN 138 more than PRN 135 (oscillation signatures have different magnitudes) CRE performance marginal without correction but well below spec limits with correction (nominally; not including CRW fly-by effects) Since oscillations are a systematic receiver effect, mitigation allows better evaluation of received signal OKC PRN 138 before correctionOKC PRN 138 after correction

33. Offline Monitoring 33 Federal Aviation Administration 2011-03-07 33 PRN 135 Short-term CCC Note: missing values indicate days with switchovers or incomplete data CRW passes CRE Nov 12

34. Offline Monitoring 34 Federal Aviation Administration 2011-03-07 34 PRN 135 Long-term CCC Note: missing values indicate days with switchovers or incomplete data

35. Offline Monitoring 35 Federal Aviation Administration 2011-03-07 35 PRN 135 Short-term CC Note: missing values indicate days with switchovers or incomplete data

36. Offline Monitoring 36 Federal Aviation Administration 2011-03-07 36 PRN 135 Long-term CC Note: missing values indicate days with switchovers or incomplete data

37. Offline Monitoring 37 Federal Aviation Administration 2011-03-07 37 PRN 138 Short-term CCC Note: missing values indicate days with switchovers or incomplete data

38. Offline Monitoring 38 Federal Aviation Administration 2011-03-07 38 PRN 138 Long-term CCC Note: missing values indicate days with switchovers or incomplete data

39. Offline Monitoring 39 Federal Aviation Administration 2011-03-07 39 PRN 138 Short-term CC Note: missing values indicate days with switchovers or incomplete data

40. Offline Monitoring 40 Federal Aviation Administration 2011-03-07 40 PRN 138 Long-term CC Note: missing values indicate days with switchovers or incomplete data

41. Offline Monitoring 41 Federal Aviation Administration 2011-03-07 Code Noise and Multipath (CNMP) Overbounding Assertion The Code Noise and Multipath (CNMP) error bound is sufficiently conservative such that the error in linear combinations of L1 and L2 measurements is overbounded by a Gaussian distribution with a sigma described by the Root Sum Square (RSS) of L1 and L2 CNMP error bounds except for biases, which are handled separately.3 Monitoring Approach Bounding for L1, IFPR, Delay Aggregate and WRE Slices All bounding failures analyzed in further detail

42. Offline Monitoring 42 Federal Aviation Administration 2011-03-07 Equations Used Cumulative distribution function (CDF): For examining the behavior at larger values of x: Pass is Δx > 0 for all |x|>0.25

43. Offline Monitoring 43 Federal Aviation Administration 2011-03-07 Aggregate Plot of CNMP Delay

44. Offline Monitoring 44 Federal Aviation Administration 2011-03-07 Aggregate Plot of CNMP IFPR

45. Offline Monitoring 45 Federal Aviation Administration 2011-03-07 Aggregate Plot of CNMP RDL1

46. Offline Monitoring 46 Federal Aviation Administration 2011-03-07 CNMP Tabular Results from Poor Performing WRE Slices Sliced by WRE Legend: - = passed X = did not pass WRE #, WRE Name L1IFPRDelay μσ|max| pass/ fail μσ|max| pass/ fail μσ|max| pass/ fail 29, Houston C0.0320.463.09-0.0580.423.31--0.0640.403.10- 44, Salt Lake C-0.0710.271.69--0.0260.992.15-0.0020.262.20- 07, Anchorage B-0.0960.332.39--0.0380.332.03-0.0120.332.18- 99, Goose Bay A0.0290.211.81-0.0400.171.34--0.0410.151.01- 79, Bethel B-0.0140.181.27--0.0330.221.83-0.0210.211.53- *This is a subset of sites as an example

47. Offline Monitoring 47 Federal Aviation Administration 2011-03-07 Summary Quarterly monitoring results continue to support specific assertions called for in the HMI document. All antenna positions are within 5 cm. The CCC Test Statistic for the GEOs is ????

48. Offline Monitoring 48 Federal Aviation Administration 2011-03-07 48 Offline Monitoring Document Report format is separated into 3 hierarchical reading levels: Level 1: Executive summary 2-3 page overview of the events Level 2: Main body ~30 pages of technical briefings, limited number of graphs Level 3: Materials and Methods Supplemental information, including: Additional Figures Details of the tool configuration (build no, flag settings, etc.) Data filenames and location (to possibly re-run in the future) OLM coding standards and guidelines First draft is scheduled to be released on March 31 st

49. Offline Monitoring 49 Federal Aviation Administration 2011-03-07 49 Offline Monitoring Data Types and Standards Standards: Slicing requirements – data from different sources are examined separately and not aggregated UDRE index PRN Binning requirements – different bin sizes are used for different analyses (0.01, 0.001, etc.) 4 File Formats: (1) Histogram files – histogram of raw counts of the metric (not probabilities), can be compiled together

50. Offline Monitoring 50 Federal Aviation Administration 2011-03-07 50 Offline Monitoring Data Types and Standards (2) Statistics files – each column in histogram file has a list (rows) of 15 descriptive statistics associated with it: Counts Mean Standard deviation Minimum, Maximum, Absolute maximum Sigma over-bound (zero centered), Sigma over-bound (mean centered) 1 st quartile, Median, 3 rd quartile Mean and standard deviation of absolute value RMS Variance (3) Time series files: variable data over time Time represented in WAAS time, UTC time (HHMMSS) and seconds into the day Files can be concatenated together to form multi-day sets (4) Quantity files: two-dimensional slices of any particular quantity (ex. UDREI/GPS PRN of |CCC metric|)