2. Process Evaluation of GoU Family Planning Programmes and Policies Increasing Demand for FP through Male Involvement and Promotion among Young People Presentation to the Uganda Evaluation Association Denis Muhangi, PhD Socio-Economic Data Centre Ltd. July 28, 2016

3. Introduction Study conducted as part of the 3ie Uganda Policy Window studies in four sectors coordinated by OPM and funded by DFID Other studies covered UPE, youth livelihood program, and decentralized governance and service delivery.

4. Background FP services is one of the most cost-effective interventions to prevent maternal, infant & child deaths One quarter to 1/3 of all maternal deaths could be prevented only thru. reduction in the No. of unintended pregnancies GoU and partners have been investing resources in FP programmes Limited comprehensive impact evaluation of FP programmes & hence limited evidence on achievement of objectives

5. Background -2 Remaining challenges – Unmet need for FP remains at 34% – CPR is lowest amongst married young people aged 15–24 years at only 11% – High rates of teenage pregnancy: 25% of teenage girls in Uganda are either pregnant or have given birth – High school dropout rate among girls attributed to pregnancy

6. The current GoU FP Programme The current GOU FP Programme is provided under the HSDP 2014/15 – 2019/20, & the Costed Implementation Plan (CIP-FP) for the same period. Of the 5 Strategic Priorities of CIP, two are relevant to the present study. – For Young people: “Increase age-appropriate information, access, & use of FP amongst young people, ages 10–24 years”. – For Male involvement: “Promote & nurture change in social & individual behaviour to address myths, misconceptions, side effects & improve acceptance and continued use of FP to prevent unintended pregnancies”. The CIP identifies demand creation as one of the key strategies to promote FP uptake, and recognises male involvement as crucial to the success of this strategy.

7. Key Interventions & Outcomes Empower male partners with knowledge about reproductive, maternal and newborn care services – Expected outcome: Men support the use of contraception for themselves and their partners. Establish / functionalize adolescent friendly corners at all levels of care Train health workers on youth-friendly services Increase FP service delivery hours to include outside school hours – Expected outcome: Young people 10–24 years of age are knowledgeable about FP & are accessing FP services

8. Process evaluation and Impact evaluation Process evaluation Conducted when a programme is already on-going to assess, among others: – Whether the programme objectives are relevant to the priorities of the target group – Whether the inputs and outputs of the programme are adequate for achieving the intended objectives; and – The extent to which the programme’s objectives are valid in that context Impact evaluation An assessment of how an intervention affects outcomes Establishes whether the intervention had a welfare effect and whether this effect can be attributed to the concerned intervention Often requires a counterfactual of what those outcomes would have been in the absence of the intervention

9. What was the planned course of the study? Carry out a process evaluation of the GOU FP programmes focusing on young people (15- 24) and adult males (26-54) and propose a rigorous and policy-relevant impact evaluation design based on the findings – Design and conduct a process evaluation – Use the findings to design and conduct an impact evaluation

10. Study process Bid proposal Consultative meetings – Meetings with OPM – Consultative meetings with user ministry (MOH) – Interactions with other teams Literature review – FP literature – Process and impact evaluation literature – counterfactuals – Theory of change literature Decide focus of the study; formulate title Draft inception report – Formulate study design, sample size etc – Draft theory of change Theory of change workshop Follow up consultations with MoH and partners Finalize Inception Report and tools Study implementation – Data collection – Data processing, analysis and report writing Validation/dissemination meetings

11. Theory of Change for FP demand among young people and adult men Refer to sheet in Ms Word

12. Conceptual Framework (Saunders et al., 2005 framework for process evaluation of health promotion programs) Fidelity: The extent to which the intervention was delivered as planned. Fidelity represents the quality and integrity of the intervention as conceived by the developers. Dose delivered: The amount and quality of the intervention delivered. Dose delivered is a function of efforts of the intervention providers. Dose received: The extent to which target groups actively engage with and value the intervention. Dose received is a characteristic of the target audience. It assesses the extent of engagement, exposure and satisfaction of target groups with the intervention. Reach: The proportion of the target audience that comes into contact with or participates in the intervention. Reach is a characteristic of the target audience. Recruitment: The procedures used to approach and attract target groups, and their relevance. Context: Other aspects of the larger social, political, and economic environment that may influence intervention implementation and uptake.

13. Elements of a Process Evaluation Research Questions A: Fidelity A1: To what extent does the family planning programme design suit the local needs of male adults and young people, and the context? A2: To what extent was the FP programme targeting adult males and young people rolled out or delivered as planned and what is the scope of the existing interventions? B: Dose delivered B1: How many FP interventions (services, commodities, sessions, etc) have been delivered? C: Dose received/ Uptake C1: What is the level of acceptance of the programme amongst key stakeholder groups, and what are the perceptions of the beneficiaries regarding the programme? C2: To what extent do beneficiaries value and get satisfaction from the programme interventions? C3: What are the current levels of knowledge, attitudes and practices among adult males and young people [men 15-54] in relation to FP in Uganda? C4: What is the likelihood of the programme to achieve its outputs and objectives? D: Reach D1: To what extent is the programme reaching the intended beneficiaries? Who is being reached and who is being left out? E: Recruitment E1: What procedures are used to generate demand and attract target groups, and what is their relevance? E2: What are the enabling and inhibiting factors to the provision of FP services to young people and adult men? F: ContextF1: What other aspects of the larger social, political, and economic environment influencing intervention uptake? Moving from the conceptual framework to research questions

14. Final Research Questions 1.To what extent does the FP programme design suit the local needs of male adults & young people, and the context? 2.To what extent was the FP programme targeting adult males & young people delivered as planned and what is the scope of existing FP interventions? 3.What is the level of acceptance of FP programmes targeting young people & males amongst key stakeholder groups, and what are the perceptions of the stakeholders regarding the programmes? 4.To what extent do beneficiaries value and get satisfaction from the programme interventions?

15. Research Questions cont’d 5.What are the levels of knowledge, attitudes and practices among adult males & young people in relation to FP? 6.To what extent is the programme reaching the intended beneficiaries? 7.What are the enabling and inhibiting factors to provision of FP services to young people and men? 8.What procedures are used to generate demand and what is their relevance?

16. Methods Study Design – A cross-sectional mixed method design was used – Qualitative and quantitative approaches – Quantitative component Household survey among male adults (24-54): N=1,071 Survey among young people (15-24): N=1,428 – Qualitative methods used in a sample of communities with selected individuals & groups. Study population – Young people aged 15-24 (male and female) – Adult men aged 25-54 – Secondary study population: WRA, VHTs, HWs, DHT, local leaders

17. RegionSampled Districts Northern Oyam Kitgum West Nile Arua Moyo Eastern Katakwi Sironko Central Mubende Ssembabule Western Kanungu Isingiro KaramojaNakapiripirit Sampled Districts by Region

18. Data Collection Methods Structured interviews with adult men (25-54) & young people [male and female] (15-24) FGDs – held separately with: – men aged 26-54; women aged 15-49, – young men aged 15-24; young women aged 15-24, – VHTs In-depth interviews with key informants: – District health staff – Health workers in a sample of HFs – Political and religious leaders at various levels – Staff from partner agencies involved in FP work, Review of documents

19. Processing and analysis of data Field editing, coding of open-ended questions, data entry using Epidata, analysis using SPSS, data cleaning, analysis at descriptive and bi- variate and levels Transcripts of qualitative notes from KIIs and FGDs, thematic and content analysis, Triangulation of quantitative and qualitative data

20. Challenges & limitations FP is a broad area, and issues are interconnected Drawing the boundary between process and impact – and the temptation to evaluate impact Accessing people and information from MoH and partners Inadequate technical support

21. Thank you Comments?