1. in the EGNOS Safety Case
Field Data
in the EGNOS Safety Case
Peter Niemann
CGI IT UK Ltd
14 April 2016

2. Service EGNOS augments GPS single frequency L1 signal By providing On
corrections and
integrity information On
GPS satellites (Position, Clock) and
Ionospheric delays
Across Europe
Certified for aviation
Source Eurocontrol

3. Service – Performance Objectives
Service Levels
Open Service
Safety of Life
NPA (non-precision approach)
APV-I (approach with vertical guidance)
LPV200 (Localiser Performance with Vertical Guidance) since 2015
But not APV-II
Service Performance
Integrity
Continuity
Availability
Accuracy

4. Service – Performance ObjectivesOS
NPA
APV-I
LPV200
Horizontal Alert Limit -
0.3 NM
40m
Vertical Alert Limit
n/a
50m
35m
Integrity
/ hour
/ 150s
Continuity
/ hour
/ 15s
Time to Alarm 6s
Availability
0.999
0.99
Horizontal Accuracy 3m
220m
16m
Vertical Accuracy 4m
20m

5. Service – System Architecture
Safety Assurance level
DO-178B level B for Check Set
C/D for other components
source Navipedia (ESA)

6. Subsystems Acceptance
Service – History
Start of feasibility studies
Start of subsystem development
2002 Start of system integration
Jul Start of initial operations
Oct 2009 Start of Open Service
Mar 2011 Start of Safety of Life Service
Release
Subsystems Acceptance
Live
V2.2-ext
06/2008
(01/2009)
V2.3.1p
09/2011
02/2012
V2.3.1i
05/2012
08/2012
V2.3.2
05/2013
11/2013
V2.4.1M
09/2014
06/2015

7. Development Challenges – Algorithms
Algorithmic safety objective is simple:
All physical information, even after EGNOS correction, comes with errors
Service meets its safety requirement if communicated error bound is greater than actual error with sufficient probability
EGNOS system must understand and overbound the tail end of the true error distribution

8. Development Challenges – Algorithms
Contributors to the tail all along the signal path
EGNOS External:
GPS clock
GPS signal characteristics
GPS broadcast information: position and iono delay
Space weather
EGNOS Internal:
Measurement error at RIMS due to equipment
Measurement error at RIMS due to environment
Modelling error at the CPF
Network disruptions and corruptions
Software errors (not included in error budgets)
User Equipment/Application and User Environment
Application error is subject to separate safety case

9. Development Challenges – GPS
GPS Hardware evolution
GPS on-board clock stability
Signal characteristics, especially signal to noise
Frequencies and modulation
GPS Software generating GPS broadcast
Types and rate of erroneous broadcasts may change
EGNOS has no control over GPS upgrades, would not be consulted on hardware or software upgrades
GPS provides some accuracy indications - but no safety information In a sense, one of the main reasons why EGNOS (and WAAS) exist!

10. Development Challenges – Space Weather
Signal travels through Earth’s atmosphere, interacts
Ionosphere (refraction of GPS signals in dispersive medium)
Troposphere
Physics of nominal conditions well understood and modelled
Irregularities (“bubbles”) ionospheric scintillation

11. Development Challenges – Space Weather
Geographic variability (geomagnetic equator and poles)
Daily variability, cloud of ions dragged around by the sun
Seasonal variability, strongest at equinox
Directly linked to space weather and solar cycle
source wikipedia
Goal posts move over very long time frames
Next big maximum may bring phenomena not seen since the start of EGNOS
source NOAA

12. Development Challenges – Research
GPS started in 1978
Initial EGNOS algorithms were defined in the 1990s, GNSS research at the time still a very young subject
Ongoing research continues, highly relevant to EGNOS algorithms
Ionosphere models
Clocks
Receiver tracking capabilities
Research far from complete
Space weather impact largely correlated at global level
Unknowns at local level – poor prediction of local scintillation events
Lack of observations in large areas Oceans, land mass outside Europe and North America

13. Development Challenges – Validation
Cannot test all potential system states
Cannot partition the system into testable sub-systems
Cannot test tails directly – data sets would span decades!
Test by injecting feared events, measurement errors of various types and magnitudes into scenario
Evaluate detection capabilities (missed detection, false alarm) from system response to controlled conditions

14. Development Challenges – Validation
But: Who determines the types of error to be tested, their distribution, the aggregation of system responses into missed detection and false alarm probabilities? The theory and the assumptions!
Validation cannot be fully independent!

15. Qualification and Certification Approach
System (including integrity performance) qualified using
Synthetic scenarios representing the known error sources
Agreed performance evaluation method determines performance figures
Scenarios reflect set of fundamental assumptions
Feared Events
Error sources
Error distributions
Worst case environmental conditions
System qualification campaign, collecting 4 to 6 weeks of live data, evaluated for actual performance
ASQF (application specific qualification facility) based at AENA
Service certification builds on
Continuous performance monitoring by Operator (PACF Performance Assessment and Checkout Facility based at CNES, part of ESSP)
Independent performance monitoring by EUROCONTROL EGNOS Data Collection Network (EDCN), using independent monitoring sites
Parallel operation of new releases in test mode for several months

16. Field Data – Subsystem Level
For Qualification
Product Service History is admissible
For assurance levels AL3 down
Collection campaigns within the target environment and in target configuration only (not a problem given overall development timelines)
PSH not used at AL2 / DO-178B level B - all level B software is qualified through full waterfall development evidence
For Monitoring
Operational records
Error logs
For all parts of the operational system
Responsibility of operator ESSP
Maintenance contracts to receive external product service history

17. Field Data – Service Level
All system/service level data are archived
All messages between subsystems (RIMS, CPF, NLES, CCF)
Operational status of all subsystems
Entire operational history available
GEO broadcast archive is public (EMS)
Offline performance analysis
Using final IGS position information (cooperation of multiple independent computation centres)
Evaluating all performance aspects: integrity, continuity, availability, accuracy
At any user location within the EGNOS service area
Analysis typically only a few days behind real time
Note: Local environment degradation (multipath from aircraft itself) and user receiver (as any user application) not covered by EGNOS Safety Case

18. Field Data – Service Level
Continuous
Integrity
Assessment
Source: ESSP

19. Field Data – Service Level
Availability and Vertical Protection Limit
Source: ESSP

20. Field Data – Lessons
Field data confirmed no Misleading Information events in user domain (ultimate safety objective) throughout SoL lifetime to date
Field data highlighted
Extremely rare Misleading Information events in pseudorange domain
Numerous opportunities for continuity improvements
Numerous opportunities for availability improvements
Analysis of live scenario data has consistently provided better insights than the analysis of synthetic scenarios or theoretical analysis
The rising solar activity between 2011 and 2014 altered the performance impact of some phenomena (including scintillation and strong ionospheric gradients): Effects within budget margins in 2011 degraded continuity in 2013!
Extreme care needs to be taken if algorithmic performance credit is to be taken from earlier release or configuration – several examples of unexpected performance impact, field data evidence proved superior to expert analysis

21. Field Data – Impact

22. Conclusion EGNOS Safety Case consists of two elements
Standard qualification evidence of subsystem production
Specific evidence / processes to support safety case of EGNOS service
EGNOS Field Data are essential to the EGNOS Safety Case
An analytical proof of EGNOS safety is impossible due to complexity of the system and service
Field data highlight gaps and invalid assumptions in analysis and higher level requirements
Field data drive ongoing performance improvements
Full performance analysis after the event is (relatively) straightforward
EGNOS Field Data aren’t a guide to future performance
Complexity of the system means that field data can never exhaustively test tails of distributions
Variability (over decades) of environmental conditions
Elements such as core GPS are outside our control and may change

23. Questions

24. Service – Regulatory Framework
ICAO SARPS Annex 10 Vol 1
Single European Sky and Associated EU Regulations
GPS ICD
MOPS
EGNOS MRD