1 A review of Evaluation and Impact Assessment by DAC Heads of Information in 2006 Ida Mc Donnell Annual Meeting of DAC HoI Canberra May 2006

2

3 Structure Challenges and Next Steps I II III Objectives of the Review Survey Results

4 Challenges and Next Steps I II III Objectives of the Review Survey Results

5Objectives Overview of current practice in evaluating communications, awareness raising, development education Identify common lessons and challenges

6 Challenges and Next Steps I II III Objectives of the Review Survey Results

7 Why evaluate? Source: Questionnaire administered to DAC Heads of Information October 2005

8 Common criteria for evaluating an activity (Govt & NGO) in 11 DAC Members New/pilot activity Cost/size/ importance Depends on activity Ongoing, L-T activity Belgium*** Canada** Denmark* (NGOs only)* Finland** (NGOs)* Germany * (> than euro 7K for NGOs) * Luxembourg* Netherlands* (>euro 20K)* Norway* Sweden* Switzerland** UK* Source: Questionnaire administered to DAC Heads of Information October 2005

9 Tools & Indicators Source: Questionnaire administered to DAC Heads of Information October 2005

10 Challenges and Next Steps I II III Objectives of the Review Survey Results

11 Top challenges Govt Evaluation of Communication How to measure impact Attribution Resource issues - cost and time Indicators, incl. process, short-term, long-term and comparable between different activities. Culture of evaluation Status of communications; not treated seriously/seen as part of development cooperation so less status …

12 Top challenges Evaluation by NGO of Communication Similar to Govt. (cost, time, impact, culture, comparability…) With a few specificities to NGOs: lack of interest in evaluating long-term reach, especially when funded for short-term activities diversity of NGO activities and targets difficulty in having standard indicators no established methodologies few trainers no expertise in evaluation trust to report negative conclusions from evaluation

13 HOI Network can help Sixteen responses 12 think HOI can exchange best/good practice, experiences, and information on evaluation and impact assessment. Nine think HOI can identify relevant methodologies, common indicators, and best practice guidelines on evaluating effective communication and development education. Other: testing/experimenting with new methods; solving the issue of attribution; organise workshop on the art and theory of evaluation; better coordination; share good external evaluators.

14 Options for Next Steps… Organise workshop in early 2007 with expert evaluators, including members of the DAC Evaluation Network HOI network work on/agree on common guidelines/best practice on evaluation? Investigate the possibility of experimenting with methodologies and create a knowledge-base on which activities are more effective in changing behaviour than others? Other?

15 Thank you!