Presented to: By: Date: Federal Aviation Administration Wide Area Augmentation System (WAAS) Operations Team AJW-1921 Offline Monitoring B. J. Potter Brad Dworak Chad Sherrell March 7, 2011 WIPP

Offline Monitoring 2 Federal Aviation Administration Introduction This presentation covers the 4th Quarter of 2010 –( – ) Next Steps –Analyze data for entire quarter –Transition all OLM analysis to SOS

Offline Monitoring 3 Federal Aviation Administration Clock Runoff Assertion The a priori probability of a GPS satellite failure resulting in a rapid change in the GPS clock correction is less than 1.0x10 -4 per satellite. Monitoring Approach –Events typically result in a fast correction that exceeds 256 meters –When this occurs, the satellite is set Do Not Use until the correction reaches a reasonable size –Events where the satellite is set Do Not Use from excessively large fast corrections while the satellite is healthy are recorded

Offline Monitoring 4 Federal Aviation Administration Clock Runoff No Clock Runoff Events between –

Offline Monitoring 5 Federal Aviation Administration Ephemeris Assertion The CDF of GPS ephemeris errors in a Height, Cross-track, and Along-track (HCL) coordinate system is bounded by the CDF of a zero-mean Gaussian distribution along each axis whose standard deviations are  osp-ephh,  osp-ephc, and  osp-ephl. The probability that a satellite’s position error is not characterized by this a priori ephemeris model is less than 10-4 per hour. Monitoring Approach –Compare broadcast vs precise in HCL to ensure sigmas are less than 1m, 2.5m, 7.5m for Radial, Cross Track, and In Track

Offline Monitoring 6 Federal Aviation Administration Ephemeris – Radial PRN: :00: PRN: :45: PRN: :45: PRN: :45: PRN: :45: PRN: :00:

Offline Monitoring 7 Federal Aviation Administration Ephemeris – In Track PRN: :00: PRN: :45: PRN: :30: PRN: :15: PRN: :15:

Offline Monitoring 8 Federal Aviation Administration Ephemeris – Cross Track PRN: :45: PRN: :00:

Offline Monitoring 9 Federal Aviation Administration RIC Outliers

Offline Monitoring 10 Federal Aviation Administration Ionospheric Threat Model Monitoring Assertion The values of and  iono adequately protect against worst case undersampled ionosphere over the life of any ionospheric correction message, when the storm detectors have not tripped. Monitoring Approach –Monitor for Chi^2 values greater than 1 in the four regions CONUS > 1% Alaska > 2% Caribbean > 10% Other > 3%

Offline Monitoring 11 Federal Aviation Administration Monitoring Regions

Offline Monitoring 12 Federal Aviation Administration

Offline Monitoring 13 Federal Aviation Administration

Offline Monitoring 14 Federal Aviation Administration

Offline Monitoring 15 Federal Aviation Administration Total Chi 2 values ≥ 1 from all regions for 2010 at ZLA (35.07% zeros)

Offline Monitoring 16 Federal Aviation Administration Total Chi2 Values Over 1

Offline Monitoring 17 Federal Aviation Administration Antenna Monitoring Assertion The position error (RSS) for each WAAS reference station antenna is 10cm or less when measured relative to the ITRF datum for any given epoch. (Mexico City is allowed 25cm). The ITRF datum version (realization) is the one consistent with WGS- 84 and also used for positions of the GPS Operational Control Segment monitoring stations.

Offline Monitoring 18 Federal Aviation Administration Purpose Accurate antenna positions needed to support DGPS applications Correct for Time Dependent Process –Tectonic Plate Movement –Subsidence Correct for Shift Events –Seismic –Maintenance WIPP Review for integrity issues –Greater than 10 cm WIPP should review –Greater than 25 cm WIPP must review –Special case for Mexico City (25 cm for review) Project the need for a WAAS Antenna Coordinate Update

Offline Monitoring 19 Federal Aviation Administration Survey Details Survey Date – –Cross Compared Against –CSRS-PPP –WFO-R2 –Coordinates Projected to six months beyond WFO WFO Release 3 –

Offline Monitoring 20 Federal Aviation Administration Results Against CSRS-PPP –All sites less than 5 cm. Against WFO-R2 –All sites less than 5 cm.

Offline Monitoring 21 Federal Aviation Administration Code Carrier Coherence Assertion The a priori probability of a CCC failure is less than 1x10 -4 per set of satellites in view per hour for GPS satellites and 1.14x10 -4 for GEO satellites.

Offline Monitoring 22 Federal Aviation Administration CCC monitoring approach  Anik, Galaxy 15 and all GPS satellites are monitored for CCC trips for Q (last data for CCC data for Galaxy 15 was on ).  AMR is not currently monitored (not used as ranging source, UDRE floor=50m)  All CCC monitor trips are investigated whenever a trip occurs to determine source of trip  Minimum data sources used in correlation and analysis: CCC test statistic UDRE threshold value CMCI measurements from NETS SQA WAAS Iono calculation L1/L5 Iono GUST calculation published planetary K p and A p values Chi 2 values

Offline Monitoring 23 Federal Aviation Administration Reported CCC trips for Q Date GEO PRN C&V :38: ZLA :38: ZDC :23: ZTL :24: ZLA :55: ZDC ZTL :03: ZDC ZLA :55: ZDC ZTL :45: ZDC ZTL :24: ZDC ZTL

Offline Monitoring 24 Federal Aviation Administration CCC plots

Offline Monitoring 25 Federal Aviation Administration CCC plots

Offline Monitoring 26 Federal Aviation Administration Signal Quality Monitor Assertion The a priori probability of a signal deformation (SD) failure is less than 2.4x10 -5 per set of satellites in view per hour for GPS or GEO satellites. The worst-case range errors due to nominal signal deformations are more than 25cm on any satellite signal relative to the other satellites in view. Monitoring Approach –All SQM Trips will be monitored for and investigated –Max and Median data for each metric will be plotted by Requested UDRE Monitoring for discrepancies between satellite Plots are for the first 4 days of every week for the entire quarter Plots were made using the tools from HMI Build 299

Offline Monitoring 27 Federal Aviation Administration SQM max plot

Offline Monitoring 28 Federal Aviation Administration GEO Signal Quality Assertion The WAAS SIS satisfies the requirements for code-carrier coherence and fractional coherence stated in sections and of the [draft] system specification FAA-E-2892c Monitoring Approach –Collect WAAS SIS data from each GEO using GUST receivers connected to dish antennas –Compute and plot the metrics outlined in sections and of FAA-E-2892c –Examine plots, tabulate max metric values and pass/fail states, analyze failures in further detail to identify possible causes

Offline Monitoring 29 Federal Aviation Administration Performance Summary L1 CCCL5 CCCCC short- term long- term short- term long- term short- term long- term PRN 135 CRW max mean PRN 138 CRE max mean regularly below spec limit near spec limit regularly above spec limit

Offline Monitoring 30 Federal Aviation Administration Summary Only issue: fly-by of CRW apparently affected measurements of both CRE and CRW Twice-daily elevated noise on some days When CRW most N and S; bigger effect as CRE, CRW got close Different sites saw effects of different magnitude Additional periods of noise on Nov 12 (closest approach) PR oscillation correction now included in processing Mitigates systematic error in GUST receiver L1 and L5 PR corrections mapped using GUST receiver and prototype SIGGEN in Zeta lab Applicable to any WAAS GUST/G-II receiver Dependent on PRN code, PR(t) and PR(t-1) Allows more accurate evaluation of received signal 30

Offline Monitoring 31 Federal Aviation Administration Example of Elevated Noise: 19 Nov 2010 Elevated noise at approx. 2:21 and 14:19 UTC (N and S extremes of CRW orbit) -- close to but not at zero Doppler at either OKC or LTN (or APC Primary, which showed little to no effect at either time this day) caused higher than normal max CCC values Effect worse when CRW was close to CRE (effect also seen on CRE) Since different sites show different effects, probably not on SIS; will monitor as CRW returns to nominal orbit slot OKC PRN 135LTN PRN 135

Offline Monitoring 32 Federal Aviation Administration Example of PR Correction Effect: 19 Nov 2010 PR oscillation correction generally benefits L1 more than L5 and PRN 138 more than PRN 135 (oscillation signatures have different magnitudes) CRE performance marginal without correction but well below spec limits with correction (nominally; not including CRW fly-by effects) Since oscillations are a systematic receiver effect, mitigation allows better evaluation of received signal OKC PRN 138 before correctionOKC PRN 138 after correction

Offline Monitoring 33 Federal Aviation Administration PRN 135 Short-term CCC Note: missing values indicate days with switchovers or incomplete data CRW passes CRE Nov 12

Offline Monitoring 34 Federal Aviation Administration PRN 135 Long-term CCC Note: missing values indicate days with switchovers or incomplete data

Offline Monitoring 35 Federal Aviation Administration PRN 135 Short-term CC Note: missing values indicate days with switchovers or incomplete data

Offline Monitoring 36 Federal Aviation Administration PRN 135 Long-term CC Note: missing values indicate days with switchovers or incomplete data

Offline Monitoring 37 Federal Aviation Administration PRN 138 Short-term CCC Note: missing values indicate days with switchovers or incomplete data

Offline Monitoring 38 Federal Aviation Administration PRN 138 Long-term CCC Note: missing values indicate days with switchovers or incomplete data

Offline Monitoring 39 Federal Aviation Administration PRN 138 Short-term CC Note: missing values indicate days with switchovers or incomplete data

Offline Monitoring 40 Federal Aviation Administration PRN 138 Long-term CC Note: missing values indicate days with switchovers or incomplete data

Offline Monitoring 41 Federal Aviation Administration Code Noise and Multipath (CNMP) Overbounding Assertion The Code Noise and Multipath (CNMP) error bound is sufficiently conservative such that the error in linear combinations of L1 and L2 measurements is overbounded by a Gaussian distribution with a sigma described by the Root Sum Square (RSS) of L1 and L2 CNMP error bounds except for biases, which are handled separately.3 Monitoring Approach Bounding for L1, IFPR, Delay Aggregate and WRE Slices All bounding failures analyzed in further detail

Offline Monitoring 42 Federal Aviation Administration Equations Used Cumulative distribution function (CDF): For examining the behavior at larger values of x: Pass is Δx > 0 for all |x|>0.25

Offline Monitoring 43 Federal Aviation Administration Aggregate Plot of CNMP Delay

Offline Monitoring 44 Federal Aviation Administration Aggregate Plot of CNMP IFPR

Offline Monitoring 45 Federal Aviation Administration Aggregate Plot of CNMP RDL1

Offline Monitoring 46 Federal Aviation Administration CNMP Tabular Results from Poor Performing WRE Slices Sliced by WRE Legend: - = passed X = did not pass WRE #, WRE Name L1IFPRDelay μσ|max| pass/ fail μσ|max| pass/ fail μσ|max| pass/ fail 29, Houston C , Salt Lake C , Anchorage B , Goose Bay A , Bethel B *This is a subset of sites as an example

Offline Monitoring 47 Federal Aviation Administration Summary Quarterly monitoring results continue to support specific assertions called for in the HMI document. All antenna positions are within 5 cm. The CCC Test Statistic for the GEOs is ????

Offline Monitoring 48 Federal Aviation Administration Offline Monitoring Document Report format is separated into 3 hierarchical reading levels: Level 1: Executive summary 2-3 page overview of the events Level 2: Main body ~30 pages of technical briefings, limited number of graphs Level 3: Materials and Methods Supplemental information, including: Additional Figures Details of the tool configuration (build no, flag settings, etc.) Data filenames and location (to possibly re-run in the future) OLM coding standards and guidelines First draft is scheduled to be released on March 31 st

Offline Monitoring 49 Federal Aviation Administration Offline Monitoring Data Types and Standards Standards: Slicing requirements – data from different sources are examined separately and not aggregated UDRE index PRN Binning requirements – different bin sizes are used for different analyses (0.01, 0.001, etc.) 4 File Formats: (1) Histogram files – histogram of raw counts of the metric (not probabilities), can be compiled together

Offline Monitoring 50 Federal Aviation Administration Offline Monitoring Data Types and Standards (2) Statistics files – each column in histogram file has a list (rows) of 15 descriptive statistics associated with it: Counts Mean Standard deviation Minimum, Maximum, Absolute maximum Sigma over-bound (zero centered), Sigma over-bound (mean centered) 1 st quartile, Median, 3 rd quartile Mean and standard deviation of absolute value RMS Variance (3) Time series files: variable data over time Time represented in WAAS time, UTC time (HHMMSS) and seconds into the day Files can be concatenated together to form multi-day sets (4) Quantity files: two-dimensional slices of any particular quantity (ex. UDREI/GPS PRN of |CCC metric|)