11 Validating AFP’s Theory of Change Michelle J Hindin, MHS PhD AFP Partners Meeting, Kenya Scientist,

22 AFP’s Theory of Change  At each stage: – Use AFP-SMART to identify barrier and quick win(s) – Use cascade analysis to document quick win – Use decision tree to track progress towards impact (stage 4) – Provide additional documentation that barrier is addressed – Consider feasibility of going to next stage Filename Stage 1: FP Regulation Change or budget allocated Stage 2: Resources allocated to increase FP services Stage 3: Change in provision of FP services Stage 4: Change in FP uptake or service quality

33 Stage 1: FP policy change or FP resources allocated  What additional documentation could be provided?  Is it feasible to go to Stage 2? If not, why? Filename Stage 1: FP Regulation Change or budget allocated Stage 2: Resources allocated to increase FP services Stage 3: Change in provision of FP services Stage 4: Change in FP uptake or service quality  Kenya Quick Win: Siaya, Kakamega, Tharaka Nithi and Kitui counties allocate a combined total of $246,279 to FP Fiscal Year 2015/2016  Documentation: Country budget documents, annual reports

44 Stage 2: Resources allocated for improved FP services  DRC Quick Win: After DRC’s commitment to FP2020, the government purchases S$300,000 worth of contraceptives  Documentation Programme National de Santé de la Reproduction (PNSR)  What additional documentation could be provided?  Is it feasible to go to Stage 3? If not, why? Filename Stage 1: FP Regulation Change or budget allocated Stage 2: Resources allocated to increase FP services Stage 3: Change in provision of FP services Stage 4: Change in FP uptake or service quality

55 Stage 3: Change in provision of FP services  Zanzibar Quick Win: After refresher trainings for forecasting FP commodities for key staff in 10 districts, reduced stock outs and increased availability and access to FP commodities  Documentation Tanzania health management information systems reports.  What additional documentation could be provided?  Is it feasible to go to Stage 4? If not, why? Filename Stage 1: FP Regulation Change or budget allocated Stage 2: Resources allocated to increase FP services Stage 3: Change in provision of FP services Stage 4: Change in FP uptake or service quality

66 Stage 4: Increase of FP use  Indonesia: In Karanganyar district Indonesia, after releasing funds, increase in LAPM services, enhanced data collection, LAPM users increased (32.4%-34.2%)  Documentation: FP village team recording and reporting form, District BKKBN service statistics  What additional documentation could be provided?  Can this be scaled up? Why or why not? Filename Stage 1: FP Regulation Change or budget allocated Stage 2: Resources allocated to increase FP services Stage 3: Change in provision of FP services Stage 4: Change in FP uptake or service quality

77 How can we measure impact in Stage 4?

88 Limitations of data sources: Household Surveys  Infrequent (DHS, MICS every 5 years) though PMA2020 is more frequent. Other national surveys  Disaggregation (often cannot get reliable estimates at the subnational level)  Measures of appropriate use are limited, especially for many short-term methods Are they really taking OCs every day? Using condoms for every sex act? Getting new injections at the appropriate time?

99 Limitations of measuring mCPR with Facility Data  Limited electronic record keeping in many facilities  No guarantee that short-term contraceptive commodities are used. Using condoms for every sex act? Taking OCs every day  Commodity stock fluctuations  Geography – Women make seek sources outside their communities – Short-term methods can be obtained from other sources that are not tracked with the public facilities Pharmacies Other private sector Community health workers

10 Don’t give up--  Triangulate Data! Filename

11 Filename Thank You

12 What can we do to get better estimates?  Household survey: – Link source data with household survey data – Present data with confidence intervals  Facility data: – Move to electronic log books  Combined household and facility – Aggregate month-to-month or quarterly data in facilities to annual or semi-annual data – Aggregate PMA2020 to annual or every two years  Triangulate – Use more data sources Commodity tracking data (e.g. JSI) Community health worker data