Assessing Sustainability in Global and Regional Partnership Programs Elaine Wee Ling Ooi AEA Conference November 13, 2009

Main Messages Program Sustainability Partners and Stakeholders Go beyond assessing program sustainability to assessing sustainability of outcomes Partners and Stakeholders A GRPP is only one player among a cast of several Sustainability of outcomes depend on the individual contributions of other development actors, and national/local stakeholders Replicability, Devolution & Exit These achievements need to be assessed, depending on GRPP maturity/type

Special Features of GRPPs in Relation to Sustainability Programs do not start off with predetermined end-points; they may modify business model over time if needed to improve on performance. Programs are typically grant-financed, with little capacity to generate revenue from their own resources. Outcomes and their sustainability depend on the complementary activities of partners, and the capacity of country-level stakeholders. Many programs have phased implementation plans. Long-term sustainability may require devolution or exit from regional/country-level activities.

Assessing Sustainability – Suggested Approach Theory of sustainability Sustainability of the program Complementary activities of development partners Capacity of country-level stakeholders Scaling up and replicability Devolution and/or exit

Theory of Sustainability Sustainability is the continuation of benefits arising from a program’s activities after the activities have been completed. Essential evaluation question: What is the probability or resilience to risk of continued benefit flows over time? Two sub-questions: What are the requirements – in theory – to sustain key outcomes of the program in the future? To what extent are the program and its partners – in practice – paying attention to these requirements in the present? Answers will depend on Types of activities that the program is undertaking – whether networking, technical assistance, or financing global, regional, or national public goods Maturity of the Program.

Good Practice Example: Sustainability of the Program Roll Back Malaria (RBM) Global Coordinator of Malaria Control; also provides TA Waning interest from core donors affected planned ‘roll out’ activities Examined future of partnership Conducted “partner value mapping exercise”, including partners’ expectations of RBM role and functioning Had founding members “adapted” business plans to further RBM goals? Contrasted RBM structure with other partnerships. Was RBM-WHO host relationship optimal to fulfilling mandate? Financial health; resource mobilization capability of RBM Partnership recharged; formal Governing Body created; enabled RBM to position itself for operational expansion

Good Practice Example: Partners’ Complementary Activities at Country Level UNAIDS Global Coordinator for HIV/AIDS; strategic TA Examined contributions of partners’ complementary programs: Used a Web-based UN Partnership Assessment tool Assessed effects of interdependency, over-reliance, coordination, and synergies on overall UNAIDS achievements Also assessed working relationship with PEPFAR, Gates, civil society, and private sector in terms of policy and TA development Evaluations have done a good job of assessing effectiveness of partners’ activities and increasingly their sustainability

Good Practice Example: Capacity of Country-Level Stakeholders African Management Services Company (AMSCO) Provides expatriate CEOs and management training to help SMEs become profitable and sustainable Evaluation assessed: Financial Sustainability of AMSCO – self supporting, deriving income for services rendered to companies Commercial Viability of SMEs using indicators such as profitability, management stability, business and operational changes; and compliance with environmental, health, and safety standards Sustainability of Benefits: Market for expatriate CEOs and management training and establishment of other management consulting firms. Easier to do such an assessment in older programs with more narrowly defined goals and targets

Good Practice Example: Scaling Up and Replicability Millennium Villages Program (MVP) Piloted an “integrated package” of tried and tested low-cost interventions to bring rural communities out of poverty Evaluation purpose: To assess “proof of concept” and learning: Factors necessary for sustainability within and scalability beyond pilot sites Factors considered: Funding; absorptive capacity; cost-effectiveness; preparedness for phase 1; Alignment with national and local priorities. Addressed “proper sequencing” of interventions for “scaling up” Replicate first measures least in need of reform or adaptation Second order measures: Simplification, adaptation and contextual fit Consider local expectations about targets; Contextual fit important

Good Practice Example: Devolution/Exit African Program on Onchocerciasis (APOC) Establishing sustainable community directed treatment (CDTI) systems for riverblindness in 19 countries Evaluation assessed regional- and local-level sustainability Desirability of continuing partnership after 2010 to coordinate and sustain essential trans-boundary activities (surveillance & monitoring) APOC instruments, structures and procedures for devolution to countries and communities Evidence of institutional, political and budget support by countries, communities and NGOs – “sustainability action plans”; results of “independent performance monitoring” and integration with local health systems Criteria for devolution: predefined, agreed upon by APOC partners, beneficiaries and evaluation team

Major Conclusions Beyond Program Sustainability: Evaluation can more concretely measure impact and benefits in older programs, especially those with well defined targets and goals Partners and Stakeholders: Very mature programs with strong country-level activities more cognizant of complementary contributions by other partners. Contextual fit important – strongly consider local expectations about targets in assessments. Replicability and Devolution: Early planning/agreement necessary: Criteria for scaling up and devolution need predefinition; and agreed upon by partners, beneficiaries and evaluators

Thank You