1. CARE/ASAS Validation Framework Ros Eveleigh Activity 2 Project Manager NATS

2. What is Validation ?  More than Simulation & Analysis “The process through which a desired level of confidence in the ability of a deliverable to operate in a real-life environment may be demonstrated against a pre-defined level of functionality, operability and performance.”

3. What is Validation “The process through which a desired level of confidence in the ability of a deliverable to operate in a real-life environment may be demonstrated against a pre-defined level of functionality, operability and performance.”  Process  Strategy  Guidance Materials

4. What is Validation “The process through which a desired level of confidence in the ability of a deliverable to operate in a real-life environment may be demonstrated against a pre-defined level of functionality, operability and performance.”  Real-life environment  Scenarios  Operationally realistic, relevant, testing

5. What is Validation “The process through which a desired level of confidence in the ability of a deliverable to operate in a real-life environment may be demonstrated against a pre-defined level of functionality, operability and performance.”  Functionality, operability & performance  Metrics  System performance, Human performance

6. Operations Idea Validation Strategy Survey

7. Validation Techniques  Analytical modelling  Fast-time simulation  Real-time techniques  Real-time simulation  Shadow mode  Operational trials  Survey

8. Analytical Modelling  “High-Level Fast-Time” techniques  Back of envelope  Excel spreadsheets  Mathematical models  Often specifically developed to solve a particular problem  Quick turn-around

9. Fast-Time Simulation  Relatively resource “light”  “limited” ATCO participation  small facilities  Models such as RAMS,TAAM, SIMMOD  Wide scope assessment  Time  Geographic extent  Relatively quick study time  Many scenarios

10.  Operator in the loop  Small scale and large scale  Resource intensive  ATCOs  pseudo-pilots  analysts  developers  engineers  Reduced scenario size  Raises awareness across ATC spectrum Real-Time Simulation

11.  Simulator installed and linked into ACC  Live inputs  No effect on aircraft behaviour  No control over scenarios explored  Soak testing  Full safety case not required Field Test & Shadow Mode

12. ASAS Categories  Airborne Traffic Situation Awareness Current ATC rules apply  Airborne Spacing Aircrew given more tasks  Airborne Separation Temporary delegation of separation to aircrew  Airborne Self-separation Full delegation of separation

13. Validation Framework Requirements  Generic  Across validation techniques  Across ASAS categories  Comprising  Scenario - ‘environment’ for validation  Metrics - what will be measured  Guidelines - assistance to validators and managers  To ensure  Consistency of validation  Comparability of validation

14. Primary Focus  Provide a structured approach  Ensure consistency of PROCESS  Guidelines  Scenario specification process  System performance metric structure  Human performance metric structure  Route map from EMERALD RTD Plan

15. Work Packages