1. Safety Forum 2016 Brussels, 07 - 08 June 2016
Airborne and Ground Based Safety Nets make a positive difference in ATM Safety - Application in Real Time Operations & Operational Monitoring
Ground Based SAFNET for Operational Monitoring Identifying positive deviance - Safety performance monitoring
Safety Forum 2016
Brussels, June 2016
Tony Licu
Head of Safety Unit
EURCONTROL
Marco Ducci
Carlo Valbonesi
ASMT ECTL Ext. Operations Support
ENAV Safety Team
ASMT User
ENAV Spa
Dr. Frederic Lieutaud
ASMT Project Manager
EUROCONTROL

2. Automatic Safety Monitoring Tool Operational Expertise
Airborne
Safety Nets make positive difference in ATM safety
Ground Based
Real-time Operations
ASMT
Automatic Safety Monitoring Tool
Operational Expertise
Drawn attention
Drive analysis/questions
Interpret the results
Implement actions
Operational Monitoring
Best practices (Safety II)
Systemic issues (Safety I)
ASMT (focus the attention, ask good questions) + saggezza operativa

3. The Positive Deviance Approach
Identify meaningful Network Metrics and Indicators
Correlate metrics with ASMT to capture safety nets events
Identify Outliers
Identify Hotspots
“Positive” and “Negative”

4. Baseline performance
Each triangle represent a different waypoint of the Rome FIR
The analysis revealed that the number of STCA alerts is usually increasing as traffic load and airspace complexity increase.

5. Baseline performance
Dashed lines represent the average of traffic load and vertical movements measured for all waypoints
STCAs are generally more frequent around waypoints experiencing traffic load and vertical movements higher than the average. However, the identification of outliers, i.e. navigation points that showed a different behaviour with respect to the predicted one, represented the real added value. For example positive outliers are identified when “A navigation point is performing better, having less associated STCAs than the average of other navigation points with similar traffic load and vertical movements, i.e. it deviates positively from the baseline”.

6. Positive Outliers
Green triangles consist of waypoints that are associated to a lower number of STCA compared to the average of other waypoints with similar traffic and vertical movements. These are positive outliers with respect to both traffic load and vertical movements.

7. Baseline performance

8. Outliers by Traffic Load
Yellow triangles consist of waypoints with a high number of vertical movements but low traffic load that are associated to a higher number of STCA when compared to similar waypoints. These points are also negative outliers, but in relation to traffic load only.

9. Baseline performance

10. Outliers by Vertical movements
Black triangles consist of waypoints with high traffic load but a low number of vertical movements that are associated to a higher number of STCA when compared to similar waypoints. These points are also negative outliers, but in relation to vertical movements only.

11. Baseline performance

12. Negative Outliers
Red triangles consist of waypoints that are associated to a higher number of STCA compared to the average of other similar waypoints. These points are negative outliers.

13. Outliers overview Better than expected: which are the drivers?
Intervene on airspace design: reduce vert movements?
Worse than expected: which are the causes?
Intervene on traffic load: reduce traffic?
What can we learn from outliers? What are the strategies to exploit the information retrieved?

14. Positive Outliers
Green triangles consist of waypoints that are associated to a lower number of STCA compared to the average of other waypoints with similar traffic and vertical movements. These are positive outliers with respect to both traffic load and vertical movements.

15. Positive outliers – the Rome FIR case
Where are the positive outliers located in the Italian Airspace? Mainly at the borders of the FIR  improved coordination, better procedures for handover.

16. Analysis of outliers: what can we learn?
Better than expected: which are the drivers?
Specific procedures used to avoid conflicts in transition areas
Worse than expected: which are the causes?
A non-optimal tuning of the STCA?
Non-optimal ATC procedures?

17. Conclusion and what’s next?
The Positive Deviance Approach relies on operational data and use of ASMT to perform Outliers analysis and make difference in:
Understanding how operations work as a whole
Measuring and assessing the presence of safety, performing better or worse than the average:
capture best practices to reproduce (Safety II), as opposed to
identify systemic issues to be mitigated (Safety I)
Italian airspace STCA analysis is only an example - future studies in ATM safety using SAFNET data and ASMT in the pipeline:
Correlation of Safety Events (STCA/SMI/ACAS-RA) & Traffic Metrics
Analyses of Hotspot of false/nuisance SAFNET events for tuning & performance monitoring
Analyses of correlation of RIMCAS alerts and Go-around/Missed Approach (expanding on aerodrome design and procedures)