1 centre dÉtudes de la navigation aérienne COMMON METRICS FRAMEWORK FOR ATM PERFORMANCE ASSESSMENT AND MONITORING Almira Williams, CSSI Inc.

2 centre dÉtudes de la navigation aérienne Background Conclusions from ATM system performance R&D are often inadequate for policy decisions due to the lack of: Standards for quantitative analysis and clear terminology definition; Consistency and comparability of outcomes from different analyses; Links between quantitative analysis and corporate objectives. Performance assessments of the operational and/or future ATM concepts, systems and goals are often incomparable In 1995, FAA and EUROCONTROL initialized a co-operative R&D effort to address this and other similar problems Action Plans were developed, each addressing a specific area of interest, e.g. AP1–Airborne Separation Concepts, AP6–ATM Decision Support Tools, etc. In 2002, AP2–Air Traffic Operational Concepts, and AP5–Validation and Verification Strategy, combined their activities in searching for a more general and systematic approach Collaborative effort of CSSI, FAA, EuroControl, and CENA in support of the Action Plans 2 and 5

3 centre dÉtudes de la navigation aérienne Objectives Develop common ATM performance assessment and monitoring capability for: periodic evaluation of system performance, operational concept validation and/or comparison, and target setting; Support an understanding of terminology and performance metrics used by different organizations by: providing a common metrics taxonomy, organizing the existing ATM metrics to maintain the connection between the identical metrics, identify metrics that have identical definitions but different names, identify metrics that have identical names but different definitions Establish a top-down framework for ATM performance assessment and monitoring that roll-ups performance measures to organizational goals and expectations of various ATM stakeholders Founded on ISO principles for QOS in IT, and ICAO hierarchical structure of the performance based ATM Operational Concept

4 centre dÉtudes de la navigation aérienne Principal Considerations Common performance framework must facilitate the following: Evaluation of both overall system and individual aspects of performance; Evaluation of system-wide and regional performance; Evaluation within the real-world or a modeling environment; and Comparison of system performance before and after a change is implemented.

5 centre dÉtudes de la navigation aérienne Capacity Flight Efficiency Safety #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 #13 # Airspace Organization And Management Aerodrome Operations Demand And Capacity Balancing Traffic Synchronization Conflict Management Airspace User Operations ATM Delivery Management Before After Performance Based ATM Concept Performance Outcomes* Radar Graphs Level 1: Political and Socio-Economic Requirements Level 2: Required Total System Performance Level 3: Required Total System Performance Level 4: System Requirements Level 5: Standards and Specifications Safety Capacity metric indicator Cost-Efficiency Environment Security National Security Human Perform. Uniformity Quality indicator metric indicator ATM Performance Framework * ICAO Concepts of RASP/RTSP Metric Collection metric specification Classifications metric specification Metric Taxonomy * ISO Concept of QOS in IT Metric Attributes Performance Metrics and Indicators Specializations, Derivations, etc. entities processes/ operations events

6 centre dÉtudes de la navigation aérienne The Relationship Between the Metrics Taxonomy and the ATM Performance Framework Level 1 Level 2 Level 5 Level 4 Level 3 Safety ATM Security Cost Effectiveness Primary Characteristics Primary Classifications Specification ATM Performance Framework Metrics Taxonomy

7 centre dÉtudes de la navigation aérienne Example: Safety Primary Characteristic

8 centre dÉtudes de la navigation aérienne Common Dashboard

9 centre dÉtudes de la navigation aérienne Conclusions We developed guiding principles for elaborating a comprehensive ATM performance framework that is applicable to all phases of (gate-to-gate) flight and in a uniform manner across international civil aviation community. Proposed framework is consistent with the International Civil Aviation Organization (ICAO) Required ATM System Performance (RASP) and Required Total System Performance (RTSP) approach Proposed framework is based on the principal concepts for Quality of Service (QOS) framework developed by the International Organization for Standardization (ISO).

10 centre dÉtudes de la navigation aérienne Next Steps Expand and consolidate the performance example- safety, Extend the ATM Performance Framework to the remaining principal objectives, and Further efforts on the aggregation of lower level into higher level results using the cause-effect approach.