1. Methods and good practices for influential evaluations Uganda Evaluation Week 2014

2. What do we mean by influential? Direct effects on an intervention – a basic objective, but narrow to the intervention Wider influence Changing perceptions – core feature of the role of evidence Setting an agenda - by reframing the way an issue is debated and creating pressure for change Developing capacity - within organisations to allow them to understand and respond to an issue. Changing institutions – influencing policy, strategy, resource allocation within organisations (both government and non-government sectors) 2

3. What matters 3 Influence Timing Credibility Commun -ication Topic relevance

4. Topic relevance: selecting programmes High stakeholders interest Strong evidence base Innovative, or pilot programme Cross-cutting concerns, for example, anti-corruption, absenteeism or value for money Level of contentiousness and risk Financial value Strategic importance to government objectives or is a particular policy priority Evaluability (whether it is possible realistically to evaluate a programme) 4

5. Timing To deliver findings and recommendations in time for key decisions: A new phase of activities Expansion to different locations Annual budget deliberations Financing decisions 5

6. The political context is critical … Programs are political creatures: Identified, designed, debated, endorsed and funded through political processes Values, interests and policy horizons vary Survival is a potent political force Evidence based policy vs. policy based evidence: Selective evidence, data mining, seeking ‘good results’ etc. Evaluation is politics: Evaluation governance Decisions to evaluate or not evaluate Choice of evaluation methods 6

7. Credible design 1. Understand the intervention logic 2. Develop questions 3. Structure according to evaluation criteria 4. Decide on an evaluation theory or approach 5. Select appropriate methods 6. Use suitable tools for data collection 7

8. Improving the quality of evidence 8 Confidence about the evidence Detailed evaluation and statistical design Calculation of error terms and confidence interval for statistics Details of purposive and randomised sampling Notes on triangulation of enquiries Openness about bias and limitations Availability of data for independent analysis/ verification

9. Governance and independence Independent or in-house team? Whether or not to appoint: Steering group Reference group Policy advisers Technical advisers 9

10. The evaluation Analysis – core source of information Commissioning process Stakeholder consultation Video On-line access Press release Statistical synopsis Full Report Short summaries Topic briefs Meetings & workshops Elements of communication 10 Social media

11. END 11