in the EGNOS Safety Case Field Data in the EGNOS Safety Case Peter Niemann CGI IT UK Ltd 14 April 2016

Service EGNOS augments GPS single frequency L1 signal By providing On corrections and integrity information On GPS satellites (Position, Clock) and Ionospheric delays Across Europe Certified for aviation Source Eurocontrol

Service – Performance Objectives Service Levels Open Service Safety of Life NPA (non-precision approach) APV-I (approach with vertical guidance) LPV200 (Localiser Performance with Vertical Guidance) since 2015 But not APV-II Service Performance Integrity Continuity Availability Accuracy

Service – Performance Objectives OS NPA APV-I LPV200 Horizontal Alert Limit - 0.3 NM 40m Vertical Alert Limit n/a 50m 35m Integrity 1 - 10-7/ hour 1 - 2.10-7/ 150s Continuity 1 - 10-5/ hour 1 - 8.10-6/ 15s Time to Alarm 6s Availability 0.999 0.99 Horizontal Accuracy 3m 220m 16m Vertical Accuracy 4m 20m

Service – System Architecture Safety Assurance level DO-178B level B for Check Set C/D for other components source Navipedia (ESA)

Subsystems Acceptance Service – History 1994 Start of feasibility studies 2000 Start of subsystem development 2002 Start of system integration Jul 2005 Start of initial operations Oct 2009 Start of Open Service Mar 2011 Start of Safety of Life Service Release Subsystems Acceptance Live V2.2-ext 06/2008 (01/2009) V2.3.1p 09/2011 02/2012 V2.3.1i 05/2012 08/2012 V2.3.2 05/2013 11/2013 V2.4.1M 09/2014 06/2015

Development Challenges – Algorithms Algorithmic safety objective is simple: All physical information, even after EGNOS correction, comes with errors Service meets its safety requirement if communicated error bound is greater than actual error with sufficient probability EGNOS system must understand and overbound the tail end of the true error distribution

Development Challenges – Algorithms Contributors to the tail all along the signal path EGNOS External: GPS clock GPS signal characteristics GPS broadcast information: position and iono delay Space weather EGNOS Internal: Measurement error at RIMS due to equipment Measurement error at RIMS due to environment Modelling error at the CPF Network disruptions and corruptions Software errors (not included in error budgets) User Equipment/Application and User Environment Application error is subject to separate safety case

Development Challenges – GPS GPS Hardware evolution GPS on-board clock stability Signal characteristics, especially signal to noise Frequencies and modulation GPS Software generating GPS broadcast Types and rate of erroneous broadcasts may change EGNOS has no control over GPS upgrades, would not be consulted on hardware or software upgrades GPS provides some accuracy indications - but no safety information In a sense, one of the main reasons why EGNOS (and WAAS) exist!

Development Challenges – Space Weather Signal travels through Earth’s atmosphere, interacts Ionosphere (refraction of GPS signals in dispersive medium) Troposphere Physics of nominal conditions well understood and modelled Irregularities (“bubbles”) ionospheric scintillation

Development Challenges – Space Weather Geographic variability (geomagnetic equator and poles) Daily variability, cloud of ions dragged around by the sun Seasonal variability, strongest at equinox Directly linked to space weather and solar cycle source wikipedia Goal posts move over very long time frames Next big maximum may bring phenomena not seen since the start of EGNOS source NOAA

Development Challenges – Research GPS started in 1978 Initial EGNOS algorithms were defined in the 1990s, GNSS research at the time still a very young subject Ongoing research continues, highly relevant to EGNOS algorithms Ionosphere models Clocks Receiver tracking capabilities Research far from complete Space weather impact largely correlated at global level Unknowns at local level – poor prediction of local scintillation events Lack of observations in large areas Oceans, land mass outside Europe and North America

Development Challenges – Validation Cannot test all potential system states Cannot partition the system into testable sub-systems Cannot test tails directly – data sets would span decades! Test by injecting feared events, measurement errors of various types and magnitudes into scenario Evaluate detection capabilities (missed detection, false alarm) from system response to controlled conditions

Development Challenges – Validation But: Who determines the types of error to be tested, their distribution, the aggregation of system responses into missed detection and false alarm probabilities? The theory and the assumptions! Validation cannot be fully independent!

Qualification and Certification Approach System (including integrity performance) qualified using Synthetic scenarios representing the known error sources Agreed performance evaluation method determines performance figures Scenarios reflect set of fundamental assumptions Feared Events Error sources Error distributions Worst case environmental conditions System qualification campaign, collecting 4 to 6 weeks of live data, evaluated for actual performance ASQF (application specific qualification facility) based at AENA Service certification builds on Continuous performance monitoring by Operator (PACF Performance Assessment and Checkout Facility based at CNES, part of ESSP) Independent performance monitoring by EUROCONTROL EGNOS Data Collection Network (EDCN), using independent monitoring sites Parallel operation of new releases in test mode for several months

Field Data – Subsystem Level For Qualification Product Service History is admissible For assurance levels AL3 down Collection campaigns within the target environment and in target configuration only (not a problem given overall development timelines) PSH not used at AL2 / DO-178B level B - all level B software is qualified through full waterfall development evidence For Monitoring Operational records Error logs For all parts of the operational system Responsibility of operator ESSP Maintenance contracts to receive external product service history

Field Data – Service Level All system/service level data are archived All messages between subsystems (RIMS, CPF, NLES, CCF) Operational status of all subsystems Entire operational history available GEO broadcast archive is public (EMS) Offline performance analysis Using final IGS position information (cooperation of multiple independent computation centres) Evaluating all performance aspects: integrity, continuity, availability, accuracy At any user location within the EGNOS service area Analysis typically only a few days behind real time Note: Local environment degradation (multipath from aircraft itself) and user receiver (as any user application) not covered by EGNOS Safety Case

Field Data – Service Level Continuous Integrity Assessment Source: ESSP

Field Data – Service Level Availability and Vertical Protection Limit Source: ESSP

Field Data – Lessons Field data confirmed no Misleading Information events in user domain (ultimate safety objective) throughout SoL lifetime to date Field data highlighted Extremely rare Misleading Information events in pseudorange domain Numerous opportunities for continuity improvements Numerous opportunities for availability improvements Analysis of live scenario data has consistently provided better insights than the analysis of synthetic scenarios or theoretical analysis The rising solar activity between 2011 and 2014 altered the performance impact of some phenomena (including scintillation and strong ionospheric gradients): Effects within budget margins in 2011 degraded continuity in 2013! Extreme care needs to be taken if algorithmic performance credit is to be taken from earlier release or configuration – several examples of unexpected performance impact, field data evidence proved superior to expert analysis

Field Data – Impact

Conclusion EGNOS Safety Case consists of two elements Standard qualification evidence of subsystem production Specific evidence / processes to support safety case of EGNOS service EGNOS Field Data are essential to the EGNOS Safety Case An analytical proof of EGNOS safety is impossible due to complexity of the system and service Field data highlight gaps and invalid assumptions in analysis and higher level requirements Field data drive ongoing performance improvements Full performance analysis after the event is (relatively) straightforward EGNOS Field Data aren’t a guide to future performance Complexity of the system means that field data can never exhaustively test tails of distributions Variability (over decades) of environmental conditions Elements such as core GPS are outside our control and may change

Questions

Service – Regulatory Framework ICAO SARPS Annex 10 Vol 1 Single European Sky and Associated EU Regulations GPS ICD MOPS EGNOS MRD