TERMS OF REFERNCE FOR THE IMPACT EVALUATION OF COSCAPs Dr. Ansgar Eussner, Chief, Evaluation and Internal Audit Office, ICAO

BACKGROUND AND OVERVIEW EAOs work programme 2011 includes an evaluation of regional projects implemented by TCB. COSCAPs are the most homogenous sub-group, present in all regions, and after about a decade of operations sufficiently advanced to justify a mid-term evaluation. Since 1998, 10 COSCAPs emerged in Asia, Africa, Europe and Latin America, with more than 70 participating States. The aim of COSCAPs is the enhancement of the safety and the efficiency of international air transport through the establishment of self-sustaining sub-regional cooperative entities.

Objective of the evaluation a) a)The objective of the evaluation is to establish whether and to what extent COSCAPs made a difference in participating countries in terms of increasing aviation safety. The evaluation will: a) a)analyze the contribution of COSCAPs to improving the safety oversight systems of States and ultimately by diminishing the actual number of accidents. b) b)take into account that these results do not depend exclusively on COSCAPs but that other activities on national and regional level also have an influence and that the directly causality will sometimes be difficult to prove. c) c)be completed by the end of 2011 and to be presented to the 195th session of the Council in February/March 2012.

COSCAPs Ways of Operation COSCAPS assist States in the harmonizing air laws, establishing Regional Safety Teams - RAST, preparing for USOAP audits, conducting safety oversight inspections and training of personnel. The approach is to establish an office in the region with international expert(s) recruited by ICAO, assisted by inspectors recruited regionally. COSCAPs are guided by a Steering Committee (SC). COSCAPs are funded mainly by participating States. Usually, co-funding is provided in cash or in kind from donor organizations, CAAs, aircraft manufacturing industries, airlines, etc.

a) a)Have COSCAPs been established as planned with the funding, staff and activities scheduled as per project documents? b) b)Have project activities contributed as planned to a harmonization of legislation among member countries, increased the availability of trained inspectors in the regions, and strengthened the national and regional capacities of safety oversight systems? c) c)Has ICAO through the support for COSCAPs contributed to improving the safety oversight systems of States and their mutual assistance? d) d)Have COSCAPs resulted in a measurable and attributable reduction of accidents and fatalities by country and region? Evaluation issues

f) f)Have the measures used been efficient and cost effective? g) g)Has the cooperation between different departments/offices in ICAO (ANB, TCB, Regional Offices) and programmes (in particular between COSCAPs and USOAP) functioned effectively? h) h)What experiences were made in mobilizing donor funding and managing their contributions, including planning, reporting and auditing requirements? i) i)What are the lessons learnt so far from the experiences made in the various regions? j) j)What recommendations can be made to improve the functioning, impact and sustainability of COSCAPs? Evaluation issues (cont.)