1. Briefing to ICAO TCB Seminar
13 January 2004
An Update
Of the ICAO Universal Safety Oversight Audit Programme

2. Briefing to ICAO TCB Seminar
13 January 2004
Outline
Background
Establishment of USOAP
Universal programme - principles and mandate
Programme activities and personnel
Audit Findings and implications
Future of the program
COSCAP-NA Briefing on USOAP 2

3. Safety Oversight - a definition
Safety oversight is a function by which States ensure the effective implementation:
Standards and Recommended Practices (SARPs)
The critical elements of a safety oversight system
Relevant safety practices and procedures
COSCAP-NA Briefing on USOAP

4. Briefing to ICAO TCB Seminar
13 January 2004
Why an ICAO programme?
Increased concern over the level of safety world-wide
Various reports in the early 1990s on the lack of Implementation of ICAO SARPs by States
Need to reduce accident rates to off-set the rapid increase in traffic
COSCAP-NA Briefing on USOAP

5. Directors General of Civil Aviation Conference
Montreal, November 1997.
Attended by DGCAs from 147 Contracting States and one non-Contracting State, and by 13 international organizations.
Objective: to develop a global strategy on safety oversight.
38 recommendations in all.
COSCAP-NA Briefing on USOAP

6. COSCAP-NA Briefing on USOAP
What is audited?
Compliance with:
Chicago Convention
State Regulations
Conformance with:
ICAO Standards
Adherence to:
Recommended practices
Related procedures
Guidance material
Relevant industry practices in general use
COSCAP-NA Briefing on USOAP

7. Audits and Audit Follow-ups Completed (as of 31 October 2003)
Briefing to ICAO TCB Seminar
13 January 2004
Audits and Audit Follow-ups Completed (as of 31 October 2003)
COSCAP-NA Briefing on USOAP

8. Analysis of the findings
Briefing to ICAO TCB Seminar
13 January 2004
Analysis of the findings
Audit Findings and Differences Database (AFDD)
Based on factual and recent data
Enables an accurate identification of deficiencies impacting on safety
Consistent with regional accident/
incident rates
Reliable safety tool, both for ICAO and its Contracting States
181 audit interim reports analyzed
Updated as audit follow-up validation missions are completed
COSCAP-NA Briefing on USOAP

9. USOAP – findings and analysis
Briefing to ICAO TCB Seminar
13 January 2004
USOAP – findings and analysis
Lack of effective implementation of the critical elements (%)
GLOBAL/181 – 28.8%
ESAF/21 – 40.2%
EUR/NAT/51 – 19.65%
APAC/33 – 24.9%
MID/17 – 30.52%
WACAF/21 – 49.37%
NACC/21 – 24.04%
SAM/13 – 23.24% 10 20 30 40 50 60 70
Primary
Aviation
Legislation
Specific
Operating
Regulations
CAA Structure &
Safety Oversight
Functions
Technical
Guidance
Material
Qualified
Technical
Personnel
Licensing &
Certification
Obligations
Continued
Surveillance
Obligations
Resolution
of Safety
Issues
COSCAP-NA Briefing on USOAP

10. Briefing to ICAO TCB Seminar
13 January 2004
What does this all mean?
What is the impact of the results of the audits on safety?
What can and should be done?
COSCAP-NA Briefing on USOAP

11. Parallel between findings and regional accident rates80
Audit Findings – Lack of Effective Implementation of SARPs, (178 audit reports)
Accident rates per departures - scheduled (International and Domestic, 1996) 70
Accident rates per departures - scheduled (International and Domestic, 2000) 60 50 40 30 20 10
Asia and Pacific
Region
Eastern and
Southern African
Region
European and
N. Atlantic
Region
Middle East
Region
N. / C. American
and Caribbean
Region
Southern American
Region
Western and
Central African
Region
COSCAP-NA Briefing on USOAP

12. Briefing to ICAO TCB Seminar
Critical Elements of a Safety Oversight System - Lack of Effective Implementation
Comparison of Audit and Audit Follow-up Results
13 January 2004
Global - Initial Audit = 30.3%
85 States - Initial Audit = 22.7%
85 States - Follow-up = 7.7%
Global - Revised = 21.4% 40 35 30 25 20 15 10 5
Primary
Aviation
Legislation
Specific
Operating
Regulations
CAA Structure
and Oversight
Functions
Technical
Guidance
Material
Qualified
Technical
Personnel
Licensing and
Certification
Obligations
Continued
Surveillance
Obligations
Resolution
of Safety
Issues
COSCAP-NA Briefing on USOAP

13. COSCAP-NA Briefing on USOAP

14. Future of the Programme
Briefing to ICAO TCB Seminar
13 January 2004
Future of the Programme
Complete the remaining follow-up missions to validate the implementation of States’ corrective action plans
Continue the implementation of the SOA Quality Management System
Expansion of the Programme
COSCAP-NA Briefing on USOAP

15. Expansion to Annexes 11, 13 and 14
Briefing to ICAO TCB Seminar
13 January 2004
Expansion to Annexes 11, 13 and 14
Expansion due to start in 2004
Provisions in Annexes 11 and 14 are highly interrelated with those in other Annexes (1,2, 4, 10, 12 and 15)
Annexes 11 and 14 cannot be audited in isolation. Related provisions in other Annexes must be considered
Expansion also entails the provision of:
Seminar/workshops
Guidance material
COSCAP-NA Briefing on USOAP 2

16. ICAO Auditing practices to-date
Briefing to ICAO TCB Seminar
13 January 2004
ICAO Auditing practices to-date
Annex-by-Annex approach until now.
Adding new Annexes would inflate the resource requirements of the Programme
Lengthy and expensive proposition
Momentum lost when area of concentration changes
Entails replacing staff – and losing expertise at every audit cycle. Cannot keep receiving and monitoring progress from States
Detrimental to the continuity and credibility of the Programme
COSCAP-NA Briefing on USOAP 2

17. Briefing to ICAO TCB Seminar
13 January 2004
Systems approach
A Systems Approach for the conduct of audits is being proposed for the ICAO Safety Oversight Audit Programme
It moves away from the current approach of Annex-by-Annex auditing
It will focus both on Oversight capability of States and the safety oversight critical elements
It will be designed to cover all safety-related Annexes and will provide a more objective and cost-effective approach to auditing
COSCAP-NA Briefing on USOAP 2

18. Systems approach to conducting audits
Briefing to ICAO TCB Seminar
13 January 2004
Systems approach to conducting audits
Implementation of Annex provisions and status of differences would be determined BEFORE the audit mission, through a series of tools and relevant reports
A review of the State’s legislation, regulations, documentation and organization would also be done BEFORE the mission
The final assessment of the organization and the remaining critical elements of a safety oversight system would be made on-site
COSCAP-NA Briefing on USOAP 2

19. Systems approach to conducting audits
Briefing to ICAO TCB Seminar
13 January 2004
Systems approach to conducting audits
On site audit would validate the organization, procedures and processes established to help the State fulfill its safety oversight obligations
Audits will be tailored to the level of activity and complexity of aviation activities in each State
COSCAP-NA Briefing on USOAP 2

20. Systems approach to conducting audits
Briefing to ICAO TCB Seminar
13 January 2004
Systems approach to conducting audits
All Contracting States to be audited at least once in a six-year period
Number, duration, frequency of visits and team size tailored to each State
Core audit staff at Headquarters complemented with Regional Office and State-seconded staff will conduct the audits
Systems approach offers cost advantages over Annex-by-Annex approach as the cost per Annex audited will be significantly reduced
COSCAP-NA Briefing on USOAP 2

21. Briefing to ICAO TCB Seminar
13 January 2004
Conclusions
USOAP’s activities related to Annexes 1, 6 and 8 cannot be abandoned. States have a vested interest and ICAO’s credibility would be at stake
Audits in the expansion areas of Annexes 11, 13 and 14 cannot be conducted in isolation
Support to States with seminar/workshops and guidance material must be continued and increased
COSCAP-NA Briefing on USOAP 2

22. Briefing to ICAO TCB Seminar
13 January 2004
Conclusions
Audits based on the systems approach can be launched in 2005 with a six-year cycle schedule
Council has accepted this concept and the Assembly will be invited to endorse it
The Preparatory work for launching the Systems Approach can be finalized in 2004
If the systems approach is endorsed there will be no need to start audits relating to Annexes 11, 13 and 14, in 2004
COSCAP-NA Briefing on USOAP 2

23. Briefing to ICAO TCB Seminar THANK YOU
13 January 2004
ICAO Universal Safety Oversight
Audit Programme
THANK YOU