Plan © Plan Post-Intervention Studies Presentation delivered to the conference on Perspectives on Impact Evaluation, Cairo 1 April 2009 By Junaid Habib and Irko Zuurmond international.org)

© Plan Context Founded in 1937 as a child-sponsorship organisation Geographical scope: Programmes in 49 countries in Africa, Americas, Asia Fund-raising through 17 National Organisations Expanding programme scope: child health, education, livelihood, water and sanitation, ….. child protection, child participation Changing programme approach: from a needs- based to a rights-based approach –increased emphasis on underlying processes of change Long-term presence in the community/district Budget: approx US$600 million

© Plan Increased pressure to assess programme effectiveness  Increased institutional commitment Development and adoption of a Programme Effectiveness Framework  A basket of initiatives to assess programme effectiveness using multiple methodologies and multiple sources of information One of the new, proposed initiatives are post- intervention studies Considered an essential component to strengthen future programme design, in particular sustainability

© Plan Sponsorship: phase-out process Standard protocol in place for phase-out from communities/areas/districts Phase-out process starts months prior to phase out and includes justification for phase out. Reasons include reaching of agreed development targets (e.g. immunisation levels, primary school enrolment, etc) Once phased-out, there is no systematic process of going back to the community/area/district  Evaluation gap

© Plan Proposal for introduction of Post-Intervention Studies Aim: To assess Plan’s contribution to long-term changes and document lessons learnt. Objectives: Each post-intervention study will have its context specific objectives, including the analysis and documentation of the lasting results (positive / negative), lessons learnt and factors of sustainability.

© Plan In practical terms To what extent have programme outcomes been maintained after phase-out? What are the contributing and impeding factors for sustaining (positive) programme outcomes?

© Plan Methodology No fixed methodology However, a quasi-experimental study design is proposed. To include a credible counterfactual analysis: A carefully selected control (or comparison) group: E.g. neighbouring community or a physically distant community but with identical socio-economic indicators and comparable characteristics – essential Pre-and-post intervention comparison of both groups - desirable

© Plan Phase-inPhase-outPost- intervention yrs3-5 yrs Factual (with Plan programme) Counter-factual  ?  ?

© Plan Where are we? Concept paper developed and circulated Overall organisational buy-in into the concept of post-intervention studies Participation of Country Offices is on voluntary basis  volunteers have been identified

© Plan Next steps Assess ‘evaluability’  narrow down eligible countries Selection of two pilot countries among those who have volunteered In the first instance, the emphasis will be on developing a sound process and methodology/ methodologies, rather than having a representative sample of countries. Depending on the feasibility of the two pilots, scale-up the approach across Plan.