ICFP panel: Quality Metrics for Family Planning – the “Secret Sauce”? Benchmarking FP quality: A standardized measure of facility-level family planning readiness Ben Bellows ICFP panel: Quality Metrics for Family Planning – the “Secret Sauce”? 26 January 2016

Background Metrics Working Group (MWG) helps social franchises to report performance beyond utilization numbers Health impact (DALYs) Cost-effectiveness Additionality (formerly “health market expansion” and “new users”) Sustainability (TBD) Equity (DHS wealth quintiles) Quality http://sf4health.org/measuring-performance

Differing priorities, differing attributes Program Managers: Easy to collect (fast, inexpensive) Easy to interpret by managers and field staff Agency Headquarters: Standardized Comparable across countries Easy to explain Other Stakeholders: Comparable to global or national standards Rigorous Objective: To create an index of quality that differentiates high and low performing facilities while remaining meaningful in the national context of FP quality.

Assessing quality of care: Donabedian framework Structure Process Outcome Facility characteristics - Services available - General infrastructure - Availability of supplies, drugs, equipment, guidelines - Basic and comprehensive care Staff characteristics - Skill mix - Training Clinical care - Knowledge and skills - Examinations performed - Services provided - Explanations given Interpersonal relations - Waiting time Privacy, & confidentiality, Staff friendliness, Communication Dignity Health status - Functional status - Symptom relief User evaluation - Satisfaction - Awareness Quality of family planning services assessed using Donabedian’s multi dimensional framework: structure; process and outcome

35 structure/ readiness questions

Methods Test internal consistency of 35 questions (Cronbach’s alpha) Use principal components analysis (PCA) to generate distribution of scores from national SPA facilities Use PCA factor weights from national SPA facilities to generate distribution of scores from local Bungoma facilities Compare scores between local Bungoma and national SPA facilities

Kenya SPA 2010 facility scoring on FP readiness (n=576) SPA quintile Nat’l referral hospital Prov. referral hospital Dist hosp. Sub-dist. Hosp. Other hosp. Health center Clinic Disp. Maternity Stand alone VCT Total Q1 1 24 18 31 40 6 122 Q2 8 11 5 14 12 55 4 110 Q3 13 20 15 21 114 Q4 19 25 Q5 35 10 116 2 7 71 63 85 91 76 133 44 576 0-24% 25-49% 50-74% 75-100%

Bungoma public facilities scoring on FP readiness (n=43) SPA quintile Nat’l referral hospital Prov. referral hospital Dist hosp. Sub-dist. Hosp. Other hosp. Health center Clinic Disp. Maternity Stand alone VCT Total Q1   1 Q2 Q3 5 17 22 Q4 11 Q5 2 30 43 0-24% 25-49% 50-74% 75-100%

Limitations Lack of “must have” quality indicators. This strategy is orthogonal to normative questions of content. E.g. Are affirmative answers to all 35 questions required to be considered a functional “FP-ready” facility? The index is limited by what’s available in SPA survey Further reflection warranted on time difference between national SPA survey (2010) and Bungoma facility assessment (2014)

Going forward Repeat this approach with other facility surveys & SPA datasets Development of quality process index

Thank you Comments, feedback to Ben Bellows bbellows@popcouncil.org