Horst Breier Study commissioned by the DAC Network on Development Evaluation June 2005 JOINT EVALUATIONS: RECENT EXPERIENCES, LESSONS LEARNED, OPTIONS FOR THE FUTURE

Horst Breier Background 2000: Publication of Effective Practices in Conducting a Joint Multi-Donor Evaluation in the DAC Evaluation and Effectiveness Series 2004: DAC Network on Development Evaluation commissions a new study on joint evaluations:  To review past experience,  To distil lessons learned, and  To explore options for the future

Horst Breier Background The report is based on  An extensive review of literature and written material, including official documents, manuals, evaluation reports, etc.  More than 100 interviews with evaluators and aid officials, representatives of the research and NGO community, consultants, etc.  The Nairobi workshop with representatives of partner country governments, NGOs and consultants

Horst Breier Background The report consists of three parts:  First, an analysis of key aspects of joint evaluation work carried out between 1990 and today  Second, a review of recent experience with joint evaluations, and a presentation of lessons learned and recommendations  Third, a chapter on options for the future and issues for discussion, primarily addressed to the DAC Evaluation Network

Horst Breier Background Annex 1 contains an annotated table of about 50 joint evaluations that were identified during the research phase. This overview provides a fascinating picture of the variety of joint evaluations that DAC members and others have implemented since 1990 However, this summary table is unlikely to be fully complete. Members are invited to provide any additional information on joint evaluations of which they are aware

Horst Breier Findings The report contains a large number, and a wide range, of findings. This presentation highlights a selection of the most important ones:  Joint evaluations are a dynamic area of development co-operation, with an increasing number of joint evaluations undertaken during the last 4-5 years  The DAC Evaluation Network has been instrumental in leading and forming the debate on this important aspect of evaluation work

Horst Breier Findings  However, DAC members are displaying different levels of involvement in joint evaluation work. Denmark, the Netherlands and Norway are clearly in the lead, followed by a large middle group and then others who are demonstrating less commitment to joint working  There is no evidence of a clear pattern that would explain why particular joint evaluations are undertaken and others are not

Horst Breier Findings The wide variety of configurations of actors, activities, focus, etc, in different joint evaluation work means it is difficult to develop a typology of joint evaluations that remains simple and workable. However, the following categorization, based on the mode of how actors work together, is proposed:

Horst Breier Findings TYPE OF EVALUATION MODE OF WORKING TOGETHER Open to all stakeholders. All evaluation partners participate and contribute equally Open to those who meet certain requirements, such as membership in a particular group of countries, participation in a SWAp, etc. A wide range of mostly complex arrangements for joint work with a general inclination to keep the number of partners down, usually to two or three Classic multi-partner Qualified multi-partner Hybrid multi-partner

Horst Breier Findings  There is a trend over the last 4-5 years towards fuller partner country participation. However, partner countries are too often invited to participate at a relatively advanced stage of the process; when key decisions have already been taken  A clear message came out of the Nairobi workshop that partner countries want to be involved in joint evaluation work more prominently, upstream and with stronger ownership

Horst Breier Findings Chapter 1 explores the wide range of advantages of joint working, along with the challenges and costs Chapter 2 explores more detailed experiences, challenges and possible solutions in joint evaluation work. A wide range of lessons learned and practical recommendations for the future are put forward.

Horst Breier Findings Strengths of joint working include:  A high degree of flexibility in forming and organising evaluation partnerships  Knowledge sharing and mutual learning and capacity building  High quality of products, usually due to the pooling of resources and approaches  Enhanced credibility, legitimacy and impact

Horst Breier Findings  Reduction of transaction costs for partner countries  Reduction of number of conflicting messages emanating from a multitude of donor evaluations  Shared financing of large evaluations - which individual donors might not be able to afford  Promotion of donor harmonisation and alignment  Response to the evaluation challenges emanating from the new modes of harmonised aid delivery; such as GBS, SWAps, basket financing, etc.

Horst Breier Findings Potential weaknesses and challenges include:  Reliance on individual initiative as the prime mover of joint evaluation work  Uncertain and uneven commitment of partners to the outcomes of the joint evaluation  Too little time allocated at the beginning of the process to clarify and agree the objectives, concepts, methodologies and ground rules  Complex supervisory and management structures, with too little delegation of authority, and too much micro- management and control, resulting in bureaucratisation and delayed processes and delivery

Horst Breier Findings  Selection, guidance and supervision of consultants  High direct and indirect costs for the lead donor(s)  Long gestation periods for evaluation results to become available, entailing the risk of untimely delivery of products and outcomes  Risk of conflict between the official and unofficial agendas of the partners involved in a joint evaluation

Horst Breier Findings  Insufficient upstream participation of partner countries - resulting in a lack of ownership and a risk of sidelining the evaluation results within those countries  Relatively weak reflection, in joint evaluation activities so far, of the new aid modalities and of the DAC agenda on Aid Effectiveness

Horst Breier Conclusions The picture that has emerged from this study is one of strong and invaluable leadership of the DAC Evaluation Network in: - Putting joint evaluations on the agenda - Rooting them firmly in the donor community - Providing practical inputs and guidance Without the contributions of DAC Evaluation Network members and observers who have been willing to take up the challenges of joint evaluations, to invest resources, creativity and leadership, and to stay with this idea right from the first experiments in the 1990s, we would be a long way from where we are today.

Horst Breier Conclusions However, there are a number of new and emerging challenges that need to be addressed - by donors individually, by partner countries, and by the DAC Evaluation Network collectively. These challenges are listed in Chapter 3 which focuses on options for the future. Many of the issues raised need to be addressed urgently.

Horst Breier Conclusions These options and challenges for the future are focused around three basic considerations: 1. Improving the efficiency and effectiveness of joint evaluations 2. Working towards enhanced developing country participation and ownership 3. Streamlining the role of the DAC Evaluation Network and focusing on strategic issues

Horst Breier Next Steps Network Members are invited to give feedback and comments on the report, by a deadline of 21 June Feedback and comments should be sent directly to with copy to of the Secretariat.