1. #1 TIM 9 : 15-17/05/02 DAY 1 B. Ng - Daily performance & charges D. Knorr - Work related to Free Flight and Operational Evolution Plan M. Hansen - How to evaluate post deployment E. de Muelenaere - CFMU's performance analysis DAY 2 M.Wambsganss - Post day analysis in traffic flow evaluation M. Bueno - Performance metrics used in Runway Capacity assessment S. Bradford – Weather factors in ATM performance assessment N. Durand – Arithmetic simulation and performance metrics X. Fron - PRC/PRU understanding of ATM performance and metrics issues TIM 9- Track 3 : Nine Presentations

2. #2 TIM 9 : 15-17/05/02 Track 3 purpose We analyze the current world performance and impact of operational / technology changes We provide a baseline against which future concepts will have to be compared We provide the basis for calibrating models (e.g. simulation)

3. #3 TIM 9 : 15-17/05/02 Relationships to Other Tracks Track 1 Track 1 and Track 3 must use the same metrics Track 3 provides reference/baseline Track 2 Track 2 uses track 3 to : –Calibrate models –Validate future against current performance Track 2 provides track 3 with : –Expected effects of proposed concepts –Context variables and Statistical results

4. #4 TIM 9 : 15-17/05/02 DATA Awareness of available operational data Shared knowledge of what data is being qrchived qnd where it resides. Good understanding of limitations in quality, reliability, and/or accuracy of the data Expand the data gathering effort to include items that provide predicted information. Infrastructure will have to support the ready accessibility to the data (e.g. protocol, standards, tools)

5. #5 TIM 9 : 15-17/05/02 METRICS Metrics shall be linked with goals The scope of what has to be covered by metrics shall be specified Metrics shall be available for Reference situations (i.e. ideal functioning to day of the system) A Critical review of existing metrics shall be conducted : are they smart enough, Exhaustive ? Comprehensive ? Need for more ? Limitations in associating metrics with individual or specific capabilities shall be explicit (metrics may not be additive, or contradictory) Metrics shall differentiate between local results versus systematic results, and identify the limits of aggregating ? Results shall be associated with the relevant explanatory metrics such as : tool usage ( e.g. number of inputs with respect to route shortening ), behavior ( e.g. slot compliance with respect to optimal delivery ), Causality (e.g. weather), Complexity)

6. #6 TIM 9 : 15-17/05/02 Need for a proper cooperation forum on « metrics » Informal methods of communication Formalized approach ? (see next slide) Shared Knowledge Points of Contact Tools Techniques analysis Data, metrics Better understanding definitions Assess benchmarking techniques and results Consider normalization efforts (through simulation, track 2 ?) Sharing with airlines Data Policy

7. #7 TIM 9 : 15-17/05/02 Proposed practical steps Compile list of POCs and make it available on the FAA Eurocontrol web site Action : everybody to mail to Wim Post before end of July Irvine, CA R&D Committee 20 october : possible proposal for redirection of AP6 ? Ensure internal FAA and Eurocontrol support for the proposal Draft revised ToRs Initiate a limited pilot activity to compare some specific aspects of US and Europe operation Advise the aviation community at the « Metrics » Workshop November Monterey California (Nextor)