1. Results-oriented Monitoring
Comissão Europeia
Results-oriented Monitoring
Seminário sobre Resultados da Ajuda Externa e Metodologias de Avaliação da CE, em Lisboa
9/3/2007
Alexandra Chambel
EuropeAid/ Unidade de Avaliação

2. How is it done? Independent experts Ongoing projects
Rapid assessment using a strictly structured consistent methodology against internationally agreed criteria:
Relevance
Efficiency
Effectiveness
(Potential) Impact
(Potential) Sustainability
12/11/2018

3. Outputs
A two page report is uploaded in a database allowing for aggregations, simple statistics and comparisons.
It provides, short explanations and recommendations on quality improvement + important complementary information to internal monitoring
A response from the field on quality and usefulness of the report is also uploaded, projects synopsis , etc
12/11/2018

4. The assessment gives a picture of the situation helping :
Aims
The assessment gives a picture of the situation helping :
to improve the individual projects quality performance
to have an overview over the performance of the project portfolio; how is it progressing towards results
12/11/2018

5. 4247 projects in 172 Countries have been recorded.
Some Numbers
In 2006
148 countries visited
1402 projects monitored by ROM with a total value of approximately €9.9 billion were visited.
Since inception, in 2000
6721 monitoring reports
4247 projects in 172 Countries have been recorded.
12/11/2018

6. Grading
The basis for the assessment is the programming document of the project => Is the project, with all due flexibility pursuing its planned course and will it attain its objectives? Is it on track?
4 Grades:
4 points (A): Much better than planned , very good
3 points (B): slightly better than planned , good
2 points (C): slightly below plan, minor problems
1 point (D): serious problems
Methodological note:
The baseline parameter for the scoring scale, 2.5 points, has been defined as “implementation according to plan”.
Thus, a project with a score of 2.5 is fully on track and expected to deliver the benefits and impact planned.
12/11/2018

7. Performance Timeline 2002 – 2006
12/11/2018

8. Improved Projects (all regions)
Rom helps also to improve the quality of project performance
See here the ratings for all those projects which were monitored, both, in 2004 and in projects were monitored in year 2005 and 2004, or 33,3 % of monitored population in 2005 (1244 projects)
project performance improved clearly for the projects that were monitored on all criteria.
Some projects were monitored by the same person, some projects were monitored by another contractor altogether (change of contractors between 2004 and 2005 for Tacis and ACP). Analysis made showes that there is no correlation between the the scores given and the fact that the monitor is the same person or not.
* Average ratings for main criteria for the re-monitored projects in 2006, which had been monitored in 2004 and 2005
12/11/2018

9. Table: Results per Sector
Social infrastructure and services (education, health, water, government and civil society etc.) - performing well and in line with the global picture.
Another important sector, economic infrastructure and services with transport, energy and banking, is performing even better with 2.81.
12/11/2018

10. Scope:
The aim is to cover all projects above 1 million EC contribution and a 10% sample of smaller ones
For the time being only ongoing projects are monitored
But
Ex-post monitoring is being tested to get information on impact and sustainability as realized
Sector Policy Support Programme methodology under preparation – Pool Finding + Sector Budget Support
12/11/2018

11. Challenge: best possible access to credible and useful information
Project Evaluations
Challenge: best possible access to credible and useful information
Strategy: global Evaluations + Project performance information (ROM) + in depth project evaluations only where needed
The challenge is to provide all stakeholders with the best possible access to credible and useful information.
The joint Evaluation unit has developed adapted evaluation methodology on the level of sectors, themes, country, regions and for programming.
Concerning projects, an instrument (ROM) has been set up to provide real-time data on performance.
In view of the two instruments above, project evaluation has to be rationalised and focussed on those studies which are needed and provide real value added.
The project evaluation reports must be made more accessible and a database should be created. .
12/11/2018

12. Making project evaluations more useful
Number of project evaluation limited to relevant ones (value - added to the EC)
Focus on quality for devolved environment (delegations)
Common methodology and harmonized terms of reference –> comparability
Compile reports and draw lessons across the board
Project evaluations are important for those who order them, we want :
Not to overload the colleagues with unnecessary obligations
Give them a hand with clear and simple methodology to achieve quality
Use the evaluation reports more largely (beyond project stakeholders), make them better known and draw common lessons.
12/11/2018