E-OCVM (Version 2) Explained Episode 3 - CAATS II Final Dissemination Event Alistair Jackson EUROCONTROL Episode 3 Brussels, 13 & 14 Oct 2009

Episode 3 - CAATS II Final Dissemination Event 2 Operational Concept Validation What is it ? … and why do we need it ?  When we intend to develop and implement new operational systems we need evidence to support investment decisions.  We need to know that any new operational change: - will bring the performance benefits we require; - will be safe in operations; - Will be compatible with other system elements, and - will be usable and acceptable by the operational staff, pilots, controllers, etc., who will use it. Concept Validation is about identifying, collecting and structuring the necessary evidence.

Brussels, 13 & 14 Oct 2009 Episode 3 - CAATS II Final Dissemination Event 3 E-OCVM Version 2.0 What is the E-OCVM ?  The European Operational Concept Validation Methodology is a framework for R&D to establish a common approach to all projects contributing to the validation of operational concepts.  Mandated by Joint Programme Board (EC and EUROCONTROL) in January 2005 (V1.0).

Brussels, 13 & 14 Oct 2009 Episode 3 - CAATS II Final Dissemination Event 4 E-OCVM Version 2.0 Where did it come from ?  Initial collaborative programmes, like PHARE and ODID in 1990’s; need to improve the effectiveness of exercises but also the need to improve transfer from R&D to implementation.  Validation methodology projects like CAVA, MAEVA; Cooperation with the FAA (Operational Concept Validation Strategy Document – OCVSD); CAATS I.  Experience in large scale, multi-partner Concept Validation Studies – e.g. Gate-to-Gate.

Brussels, 13 & 14 Oct 2009 Episode 3 - CAATS II Final Dissemination Event 5 What does it consist of ? E-OCVM found the need for 3 ‘perspectives’ to capture different aspects of validation.  Concept Lifecycle Model (CLM) - Locates ‘Validation’ processes within the broader context of system development  Structured Planning Framework (SPF) - Provides a structure for the identification, planning, execution and analysis of appropriate validation exercises  Case Based Approach - Is concerned with capturing stakeholder expectations & concerns and ensuring that validation activities provide the necessary results in an appropriate form to address them.

Brussels, 13 & 14 Oct 2009 Episode 3 - CAATS II Final Dissemination Event 6 E-OCVM Scope Concept Lifecycle Model System development view inspired by NASA Technical Readiness Levels (TRL)s Gather and assess ATM Performance needs Scope Operational Concept and develop Validation Plans Iteratively develop and evaluate concept Build, consolidate and test Industrialisation and Approval Implementation V0 ATM Needs V1 Scope V2 Feasibility V3 Integration V4 Pre-Operational V5 Operational ? need

Brussels, 13 & 14 Oct 2009 Episode 3 - CAATS II Final Dissemination Event 7 Structured Planning Framework Step 0: State Concept and Assumptions Understanding the problem and the proposed solutions Step 1: Set Validation Strategy Identify stakeholders, their needs and interests Identify current and target levels of maturity Describe validation expectations & outline cases Identify Programme Validation Objectives in key performance areas Establish Validation requirements and draft validation strategy Select validation tools and techniques Define Validation Strategy Step 2: Determine the Exercise Needs Identify stakeholder’s acceptance criteria and performance requirements Identify project and objectives Refine validation strategy Identify Validation indicators and metrics Specify validation scenarios Produce validation exercise plan Prepare platform or facility Conduct pre-exercise testing and training Step 3: Conduct the exercise Including the assessment of unexpected behaviours Step 4: Determine the Results Perform analysis Prepare analysis contributions Prepare Validation report Step 5: Disseminate Information to Stakeholders Disseminate information to stakeholders and decision makers using case based approach where available Draw conclusions and decide on actions – feedback to validation strategy

Brussels, 13 & 14 Oct 2009 Episode 3 - CAATS II Final Dissemination Event 8 Case Based Approach Is about capturing and addressing stakeholder needs, concerns and priorities Ensuring that they are addressed during the validation phases (and after) through a structured process – the case Different cases for particular stakeholder concerns – safety; business case, environment, human factors, etc. Communicating the results in a way that is accessible and meaningful for the stakeholders

Brussels, 13 & 14 Oct 2009 Episode 3 - CAATS II Final Dissemination Event 9 Case Based Approach V1V2V3 Stakeholders ANSP Investors Operators Airspace-Users Regulators Technology Suppliers Others... Cases Business Environmental Technology Safety Others... V4 Stakeholder performance needs Barriers to Performance Improvements V0 V5 Information Requirements Results

Brussels, 13 & 14 Oct 2009 Episode 3 - CAATS II Final Dissemination Event 10 Setting the Scene