Safety Forum 2016 Brussels, 07 - 08 June 2016 Airborne and Ground Based Safety Nets make a positive difference in ATM Safety - Application in Real Time Operations & Operational Monitoring Ground Based SAFNET for Operational Monitoring Identifying positive deviance - Safety performance monitoring Safety Forum 2016 Brussels, 07 - 08 June 2016 Tony Licu Head of Safety Unit EURCONTROL Marco Ducci Carlo Valbonesi ASMT ECTL Ext. Operations Support ENAV Safety Team ASMT User ENAV Spa Dr. Frederic Lieutaud ASMT Project Manager EUROCONTROL

Automatic Safety Monitoring Tool Operational Expertise Airborne Safety Nets make positive difference in ATM safety Ground Based Real-time Operations ASMT Automatic Safety Monitoring Tool Operational Expertise Drawn attention Drive analysis/questions Interpret the results Implement actions Operational Monitoring Best practices (Safety II) Systemic issues (Safety I) ASMT (focus the attention, ask good questions) + saggezza operativa

The Positive Deviance Approach Identify meaningful Network Metrics and Indicators Correlate metrics with ASMT to capture safety nets events Identify Outliers Identify Hotspots “Positive” and “Negative”

Baseline performance Each triangle represent a different waypoint of the Rome FIR The analysis revealed that the number of STCA alerts is usually increasing as traffic load and airspace complexity increase.

Baseline performance Dashed lines represent the average of traffic load and vertical movements measured for all waypoints STCAs are generally more frequent around waypoints experiencing traffic load and vertical movements higher than the average. However, the identification of outliers, i.e. navigation points that showed a different behaviour with respect to the predicted one, represented the real added value. For example positive outliers are identified when “A navigation point is performing better, having less associated STCAs than the average of other navigation points with similar traffic load and vertical movements, i.e. it deviates positively from the baseline”.

Positive Outliers Green triangles consist of waypoints that are associated to a lower number of STCA compared to the average of other waypoints with similar traffic and vertical movements. These are positive outliers with respect to both traffic load and vertical movements.

Baseline performance

Outliers by Traffic Load Yellow triangles consist of waypoints with a high number of vertical movements but low traffic load that are associated to a higher number of STCA when compared to similar waypoints. These points are also negative outliers, but in relation to traffic load only.

Baseline performance

Outliers by Vertical movements Black triangles consist of waypoints with high traffic load but a low number of vertical movements that are associated to a higher number of STCA when compared to similar waypoints. These points are also negative outliers, but in relation to vertical movements only.

Baseline performance

Negative Outliers Red triangles consist of waypoints that are associated to a higher number of STCA compared to the average of other similar waypoints. These points are negative outliers.

Outliers overview Better than expected: which are the drivers? Intervene on airspace design: reduce vert movements? Worse than expected: which are the causes? Intervene on traffic load: reduce traffic? What can we learn from outliers? What are the strategies to exploit the information retrieved?

Positive Outliers Green triangles consist of waypoints that are associated to a lower number of STCA compared to the average of other waypoints with similar traffic and vertical movements. These are positive outliers with respect to both traffic load and vertical movements.

Positive outliers – the Rome FIR case Where are the positive outliers located in the Italian Airspace? Mainly at the borders of the FIR  improved coordination, better procedures for handover.

Analysis of outliers: what can we learn? Better than expected: which are the drivers? Specific procedures used to avoid conflicts in transition areas Worse than expected: which are the causes? A non-optimal tuning of the STCA? Non-optimal ATC procedures?

Conclusion and what’s next? The Positive Deviance Approach relies on operational data and use of ASMT to perform Outliers analysis and make difference in: Understanding how operations work as a whole Measuring and assessing the presence of safety, performing better or worse than the average: capture best practices to reproduce (Safety II), as opposed to identify systemic issues to be mitigated (Safety I) Italian airspace STCA analysis is only an example - future studies in ATM safety using SAFNET data and ASMT in the pipeline: Correlation of Safety Events (STCA/SMI/ACAS-RA) & Traffic Metrics Analyses of Hotspot of false/nuisance SAFNET events for tuning & performance monitoring Analyses of correlation of RIMCAS alerts and Go-around/Missed Approach (expanding on aerodrome design and procedures)