International Civil Aviation Organization ICAO EUR/NAT OfficeCOG PERF TF Workshop1 COG Performance Task Force Workshop EASA Standardisation Inspections Verification of Safety KPIs Milen Dentchev, ATM/ANS Standardisation Manager Rome September 2011

ICAO EUR/NAT OfficeCOG PERF TF Workshop2 2 EASA Standardisation – Legal Basis Where Member States act as Competent Authority the EASA Basic Regulation establishes a mechanism to monitor the correct implementation of applicable regulations.  EASA Basic Regulation (EC) No. 216/2008: - Art. 24 (Monitoring the application of rules) - Art. 54 (Inspections of Member States)  Working methods regulated by Commission Regulation (EC) No 736/2006

ICAO EUR/NAT OfficeCOG PERF TF Workshop3 33 Head of Standardisation T. Mickler Head of Standardisation T. Mickler Air Operations G. Rebender Air Operations G. Rebender Accreditation J. Müller Accreditation J. Müller Flight Crew Licensing G. De Crescenzo Flight Crew Licensing G. De Crescenzo Airworthiness M. Kieft Airworthiness M. Kieft Deputy HoD M. Kompare Deputy HoD M. Kompare Secretariat Organisational Structure Development C. Trevisan Development C. Trevisan IAW CAW ANS M. Dentchev ANS M. Dentchev LIS FSTD MED ATM CNS AIS MET LIS CF NCF ACC OCC

ICAO EUR/NAT OfficeCOG PERF TF Workshop4 Standardisation Department (S.1) Tasks -Standardisation (“EASA Countries”) -Standardisation (“ECAA” and “WA” Countries) -International Standardisation (“Third Countries”) -Accreditation (NAAs working for EASA)

ICAO EUR/NAT OfficeCOG PERF TF Workshop5 Geographical Scope

ICAO EUR/NAT OfficeCOG PERF TF Workshop6 EASA Inspection process 6 Preliminary meeting 10 weeks min Preparatory phase Opening session Closure Phase Follow-up phase Final Report 16 weeks max On Site As necessary Closure Action Plan Finding Classification Committee Wrap-up meeting Reporting phase 12 weeks max Closing session As necessary Preliminary Report Visiting phase Notice of inspection Visit programme Supplementary Report

ICAO EUR/NAT OfficeCOG PERF TF Workshop7 Standardisation Strategy Successful standardisation cannot depend only on regulatory compliance verification – Performance verification is part of regulatory verification EASA’s strategy is based on three pillars: Regulatory Compliance verification Pro-active measures Regulatory feedback

ICAO EUR/NAT OfficeCOG PERF TF Workshop8 NSA Standardisation Forums NAA Partnership meetings NSA Standardisation Coordination meetings ANS National Coordination meetings Post ATM/ANS Standardisation visit meetings Internal EASA FCC meetings SES NCP

ICAO EUR/NAT OfficeCOG PERF TF Workshop9 9 Increase added value MoU between EU and ICAO – Addressing aviation safety, aviation security, air traffic management and environmental protection – New Working arrangements – Transparency through mutual exchange of safety information and access to databases – Reduce burden of multiple audits/inspections on Member States

ICAO EUR/NAT OfficeCOG PERF TF Workshop10 Second extension of EASA Regulation (EC) No 1108/2009 amending EASA Basic Regulation – EASA mandated to conduct ATM/ANS standardisation inspections – Regulation 736/2006 will be amended accordingly – Transition from EUROCONTROL ESIMS audits to EASA Standardisation inspections

ICAO EUR/NAT OfficeCOG PERF TF Workshop11 11 Scope of ANS according to ICAO Annex 2 - Rules of the Air Annex 3 - Meteorology Annex 4 - Charting Annex 5 - Units of measurements Annex 10 - Aeronautical telecommunications Annex 11 - Air Traffic Services Annex 12 - Search and Rescue Annex 13 - Accident & Incident Investigation Annex 15 - Aeronautical Information Services Annex 16 – Environmental protection PANEP meeting May 2011

ICAO EUR/NAT OfficeCOG PERF TF Workshop12 12 Scope of EASA ANS standardisation inspections Annex 1 - Personnel Licensing (ATCO + ATSEP) Annex 2 - Rules of the Air Annex 3 - Meteorology Annex 4 - Charting Annex 5 - Units of measurements Annex 10 - Aeronautical telecommunications Annex 11 - Air Traffic Services Annex 12 - Search and Rescue Annex 13 - Accident & Incident Investigation (SOR) Annex 15 - Aeronautical Information Services Annex 16 – Environmental protection PANEP meeting May 2011

ICAO EUR/NAT OfficeCOG PERF TF Workshop13 EASA’s ATM/ANS standardisation section: Checklists and pre-visit questionnaires in preparation Standardisation inspection annual programme ready NSAs invited to confirm or nominate National Coordinator NSAs have been invited to identify and nominate qualified inspectors to participate in EASA standardisation inspections 1 st training course for NAA inspectors in Jul, 2 nd in Sep, next in Jan 2012 (40 people trained) 6 EASA Team Leaders Q1 of 2012, 9 Team Leaders Q st inspection in Feb 2012 (1 TL + 4 TMs) Transition from ESIMS to EASA inspections

ICAO EUR/NAT OfficeCOG PERF TF Workshop14 Amendment of WA At the request of the European Commission, EASA has approached all WA countries (including all non EASA – EUROCONTROL countries) with a proposal to extend existing WAs to ATM/ANS. All ECAA States responded positively Non-ECAA States: Positive answers from Ukraine, Armenia, Georgia, Moldova, Monaco & San Marino No answer from Azerbaijan yet Turkey – no decision yet 14

ICAO EUR/NAT OfficeCOG PERF TF Workshop15 Verification of safety performance KPIs

ICAO EUR/NAT OfficeCOG PERF TF Workshop16 ADDITIONAL FACTORS Confidence Level: the concept Time Elapsed NAA CORE INDICATORS = Confidence Level Follow-up Indicators Authority Indicators Safety occ Indicators Activity Indicators General Indicators

ICAO EUR/NAT OfficeCOG PERF TF Workshop17 The model evaluates firstly two «core indicators»: Core indicators: the NAA and its follow-up Authority Complexity (structure, ROs,...) Funding / Salaries Staffing Stability Maturity Staff qualification and training Internal guidance External guidance Oversight methodology Management of findings Enforcement capabilities Reaction to AIBs' rec’s ICAO USOAP/CMA results Follow-up Reactivity Index Supplementary Reports Repetitive key findings This is, to a large extent, the current scope of our standardisation activities

ICAO EUR/NAT OfficeCOG PERF TF Workshop18 The model then takes into account the following indicators: Additional indicators Aircraft registered Organisations approved Personnel licences issued Traffic indicators Domain-specific indicators Accidents SAFA Country ratio Operators in "high SAFA“ list Immediate Safety Hazards Domain-specific indicators Activity Safety NAA Independence Safety culture Reporting culture Just culture Corruption index Financial indicators Major gov’t changes General

ICAO EUR/NAT OfficeCOG PERF TF Workshop19 Finally, the model considers the influence of time elapsed since the last inspection, as the reliability of the information on the NAA’s capability and performance taken at that stage will inevitably decrease with time: The influence of time Time elapsed Time elapsed Confidence level Threshold Inspection due! = Decrease with time

ICAO EUR/NAT OfficeCOG PERF TF Workshop Vulnerability Index - Confidence levels (CLs) are determined for each domain: - The lowest CL would highlight the «most vulnerable» area - A Vulnerability Index may therefore be defined as follows: VI =100 – min (CLs) Note: work in progress!