1. Applying Contribution Analysis in a Global Thematic Evaluation: Evaluating Denmark’s Global Strategy for Support to Sexual and Reproductive Health and Rights (2014) IPDET Presentation by Ted Freeman, Evaluation Team Leader June 12, 2015

2. The Evaluation Team Ted Freeman: Team Leader (GGI) Susannah Mayhew, PHD: Reproductive Health and Rights Expert (LSHTM) Cristiano Matsinhe, PHD: Mozambique Expert Dr. Alex Nazzar MD: Ghana Expert Sara Van Belle, PhD: Researcher – Realist Evaluation and Global ToC (ITM Antwerp)

3. Overview: Main Points Contribution Analysis (CA) is an effective approach to evaluating the importance of a national (donor) contribution to addressing a global challenge No template or operational guidelines for using CA in evaluation so practitioners must innovate while applying the core principles The Euro Health Group/KIT Group team was able to find solutions to challenges of applying CA to evaluating a global strategy/programme Evaluation community needs to develop a community of practice for CA (guidelines?)

4. Basic Elements of CA in Evaluation Theory-Based Evaluation Requires a Theory of Change (ToC) Combines elements of impact evaluation (a focus on documenting results) and process evaluation (a focus on the mechanisms for contributing to outcomes) Requires the Evaluation Team to “challenge” the ToC in order to validate a credible contribution story for the programme.

5. Challenges of Applying CA in Evaluating Denmark’s Support to SRHR Required developing and testing ToC at Global Level and in two Countries (Ghana and Mozambique) The problem of “granularity” how to select the appropriate level for “nested”: ToCs (and how to make the connection between global and country levels) Where to apply the challenge function (contribution pathways versus linkages) given a complex ToC? What criteria for the challenge function? Displaying and reporting findings on the ToC

6. Responses by the Team 1.Focused on a limited number of important “Contribution Pathways” 2.Developed a common set of criteria for assessing the strength of contribution pathways (the challenge function) and applied them at global and country level 3.Developed a graphical approach to displaying findings on Denmark’s contribution through each pathway 4.Ghana Country Study as An Example

7. How Complex is the ToC?

8. Contribution Pathways (Ghana) 1.Programme Area Pathway One: Support to Health Systems: (MOH/GHS and CHAG) 2.Programme Area Pathway Two: Support to INGOS Active in SRHR in Ghana 3.Programme Area Pathway Three: Support to the National HIV and AIDS Response 4.Programme Area Pathway Four: Support to UNFPA in Ghana

9. All Contribution Pathways (slide 2)

10. Pathway Assessment Criteria 1.Necessity of the supported intervention (within a set of other supports/initiatives) 2.Clarity of the linkages from Danida support to results in SRHR 3.Significance of Danida support 4.Immediacy of expected results 5.Adequacy/Realism of strategies for addressing Risk

11. Sample Pathway Graphic: Ghana Pathway One - Support to Central Health Systems Danida Support SBS to the POW of the MOH/GHS TA in Planning, M&E and accountability to MOH/GHS NHIA Policy Dialogue on SRHR by RDE Earmarked Funding to CHAG Sec. and Member Institutions Long Term TA to CHAG Secretariat Intermediary Outcomes MOH/GHS Invest in SRH Service Improvements MOH Plans and Programmes Prioritize SRHR Improved monitoring of and accountability for services in SRH CHAG Sect. Supports Strategic and operational planning by members CHAG member institutions improve Service quality in SRH TA in PFM for NHIA/NHIS NHIS Strengthened as a source of funding and includes SRH Services Intermediary Outcomes Continued DP Support to MOH/GHS POW Outcomes Some Increased demand for improved services including FP and SBA Outcomes Some Increased access to services in SRH, Including FP and SBS Outcomes Improvements in Outcome Indicators in SRH including CPR and SBA Policy Changes Improve Delivery of FP and other SRH Services

12. Ghana Country Graphic 1. Health Systems Support: Central 2. Support to INGOs Active in SRHR 3. Support to HIV and AIDS Programming 4. Core Support to UNFPA Programmes in Ghana 1. Gender Equality and Women’s Empowerment 2. Young people’s access to information and services in SHR. 3. Improving Sexual and Reproductive Health (MDG 5) 4. Linking HIV and AIDS Response and SRHR

13. Lessons Contribution Pathways approach provided a solution to problem of granularity but meant reports were lengthy and not easy to summarize Findings were very context specific and required a major investment in understanding/documenting changing contexts Criteria were relatively easy to apply and document at the Pathway level – not all were equally useful Graphic approach helped communicate the content and the strength of pathway linkages Findings were easy to defend and the level of evidence was seen as very strong by readers/clients

14. Questions?