Assessing the Relevance of Global and Regional Partnership Programs (GRPPs) Chris Gerrard Global Programs Coordinator, IEG November 13, 2009 http://www.globalevaluations.org

Main Messages Need to assess the beneficiary demand for the program as well as donor supply of financial resources Need to assess the program’s value added both vertically and horizontally — in relation to the partners’ own activities as well as to other global initiatives Need to assess relevance of the program’s design in addition to the relevance of its objectives

Special Features of GRPPs in Relation to Relevance Often arise out of international conventions, conferences, and meetings Most have been initiated by donors Usually conduct activities at multiple levels – global, regional, national and local Often compete with other programs and activities in the same sector Positive outcomes also depend on complementary activities other development partners and country-level stakeholders

Assessing Relevance -- Suggested Dimensions Supply-side relevance: The existence of an international consensus that global/regional collective action is required. Demand-side relevance: Consistency with the needs, priorities, and strategies of beneficiary countries. Vertical relevance (or subsidiarity): Not competing with the major partners’ own development assistance activities Horizontal relevance: The absence of alternative sources of supply of the same goods and services. Relevance of the design: The appropriateness of the strategies and priority activities of the program for achieving the objectives of the program.

Assessing Relevance – Suggested Approach First of all, come to some understanding on the (higher order) goals, (immediate) objectives, strategies and major activities of the program, and how these have changed over time: Not always clearly articulated in historical program documents or in the evaluation TOR Often some implicit objectives (which may exist for strategic or other reasons) May need to construct at least a rudimentary results framework in order to assess relevance of design Then use an evidence-based analytical approach to assess the objectives against the five dimensions, relative to the time at which the evaluation is taking place.

Extent to Which 60 Evaluations Assessed Different Dimensions of Relevance

Good Practice Examples: Supply-Side Relevance Global Donor Platform for Rural Development (GDPRD) Program Objectives: To improve donor cooperation/ collaboration and coordinated dialogue with partner countries Evaluation Methodology: Analysis of international context in which the program operates, interviews, and stakeholder analysis Evaluation Findings: Objectives are congruent with international consensus for improved donor coordination (e.g. Paris Declaration) But there is a lack of consensus among stakeholders on the definition of the problem, or role of the Platform in addressing it Recommendation: GDPRD Board should initiate a strategic process to clarify the Platform’s role, including the preparation of strategic and operational business plans to guide its future development

Demand-Side Relevance Public-Private Infrastructure Advisory Facility (PPIAF) Program Objectives: To help facilitate public-private partnerships to improve countries’ infrastructure Evaluation Questions: Is there a sustainable demand for PPIAF services across developing countries? Are beneficiary country priorities reflected in the distribution of PPIAF’s services? Methodology: Portfolio analysis of individual grants in relation to country priorities Findings: PPIAF is demand-driven by design, but filtered through WB task managers Types of services provided generally reflect recipient government priorities Recommendation: PPIAF should give greater priority to those countries with clear national policies and strategies for infrastructure development

Vertical Relevance -- Subsidiarity Global Alliance for Vaccines and Immunization (GAVI) Program Objectives: To increase access to immunization in poor countries Evaluation Question: The extent to which GAVI was competing or substituting for the activities of its major partners (WHO, UNICEF). Findings: GAVI has achieved better immunization outcomes than its partners could have achieved individually by (a) engaging players in the field, (b) raising global immunization funding to an unprecedented level, and (c) applying innovative approaches to address urgent immunization problems GAVI has been less successful in advocating the immunization agenda at the country level Recommendation: GAVI should accelerate its efforts to build stronger country ownership and commitment to the immunization agenda.

Horizontal Relevance – Absence of Alternative Sources of Supply Water and Sanitation Program (WSP) Program Objectives: To improve access of the poor to improved water and sanitation services. Evaluation Methodology: Detailed analysis of the value added of WSP in relation to other global programs in the water sector , namely, WSSCC and GWP Findings: Each of these three programs has a distinct role in the sector which is complementary, but the programs are working largely independently of each other Recommendation: WSP should improve its working relationships with these other programs, and explore avenues of improved cooperation to achieve shared objectives.

Relevance of Design International Center for Tropical Agriculture (CIAT) Program Objectives: Collaborative research to improve agricul-tural productivity & natural resource management in the tropics. Evaluation Methodology: In-depth analysis of CIAT’s research activities. Findings: A large number of its research activities were not clearly linked to its objectives, creating inefficiencies in deploying its resources and reducing its development impact. Recommendation: CIAT should revise its strategic plan around small system-based outcome lines rather than separate activities/products and concentrate more on regional as opposed to global level research.

Overall Conclusions Need to confirm that the international consensus underlying the program extends beyond the donors to include the intended beneficiaries Need to assess the value added – both vertical and horizontal – of those programs that are not providing global or regional public goods Need to pay particular attention to program design. Few programs have a well-articulated theory of change – from inputs and outputs to desired outcomes and impacts – along with measurable performance indicators.

Thank You