Plan © Plan Assessing programme effectiveness at the global level in a large and complex organisation Presentation delivered to the conference on Perspectives on Impact Evaluation, Cairo 2 April 2009 By Irko Zuurmond

© Plan Context Founded in 1937 as a child-sponsorship organisation Geographical scope: Programmes in 49 countries in Africa, Americas, Asia Fund-raising through 17 National Organisations Programme scope: child health, education, livelihood, water and sanitation, ….. Budget: approx US$600 million Income is increasingly diverse

© Plan Programme evolution: changing the what and the how What: New emerging programme areas, e.g. child protection, child participation How: From needs-based to right-based programming From direct benefits/service delivery to facilitating development processes in which local actors play a key role Increased emphasis on principles informed by human-rights instruments, e.g. inclusion/non-discrimination, accountability, etc. More strategic programming to increase scope and impact; e.g. advocacy, scaling-up, etc. Changes in the ‘road-map’ to contribute to the realisation of the ultimate goal (fulfilment of children’s rights)

© Plan Challenge: How does a large and complex organisation like Plan know whether globally its programmes are effective? How, at the global level, can Plan optimize the use of information that is generated by the myriad of programme initiatives in multiple countries, to generate an informed assessment of its programme effectiveness?

© Plan Why doesn’t Plan’s PME system answer this question? The focus of Plan’s planning, monitoring and evaluation system has been and continues to be at the level of programme countries. Creation of a wealth of data and information: main focus on output, limited focus on outcome and (contribution to) impact Diverse in quality Diverse in nature Aggregation of 26 corporate indicators proved impossible

© Plan Programme Effectiveness Framework: key features Provides a working definition of outcome and impact Unpacks Plan’s theory of change and intervention logic Introduces a lens/analytical framework for assessing programme effectiveness across the organisation Proposes multiple initiatives using of multiple methodologies and multiple sources of information Identifies the types of information and reports that will be generated Identifies mechanisms to ensure the use of effectiveness information

© Plan Working definition of outcome and impact Changes among rights holders, duty bearers and civil society organisations, and the synergies among these social actors (outcome), that contribute to the creation of a lasting, enabling environment in which children’s rights are fulfilled and in which children realise their full potential (impact)

© Plan Levels and dimensions of change 4 TABLES: Levels/actors of change: Rights holders Duty bearers: at different levels Civil society: at different levels Plan Dimensions of change: Each table identifies the changes/transformations for each type of actor that contribute to a lasting fulfilment of the ROC Cross cutting questions: Plan’s role: to what extent has Plan contributed to the above changes? To what extent does the change affect children (+/-)? Non-discrimination and inclusion: Who benefits from the change/who doesn’t/why? Sustainability: to what extent can/will the change be sustained?

© Plan Multiple initiatives, using multiple methodologies and sources of information Systematic Review of annual and evaluation reports generated by 49 Programme Countries Analysis of sponsorship data (basic information on 1.2 million sponsored children) Thematic evaluations Post-intervention studies …………  Global Effectiveness Report (3-year period)

© Plan Use of effectiveness information Internally: Identify decision makers/mechanisms (formal and informal) Identify key decision making points Externally: Indentify key decision makers/mechanisms at the different levels Influence decision making at the different levels Requires reflection on the role and positioning of Plan ….. Requires a strong evidence base.

© Plan Support strategies Programme Effectiveness Framework is dependant on quality of and access to information generated at the level of programme countries  Strengthening the M&E and Research capacity Improving the quality of evaluation and research Knowledge Management strategy