1. PEPFAR OVC Survey Toolkit Janet Shriberg, EdD Senior Evaluation Advisor Office of HIV/AIDS USAID **please note that slides were created with work done by MEASURE Evaluation

2. PEPFAR Mission & Goal (OVC)  Mission: To mitigate the social, emotional and economic impacts of HIV/AIDS on children and to reduce their risk and vulnerability while increasing their resilience  A goal: To care for 5 million orphans and vulnerable children  10% funding earmark for OVC

3. The problem  High investment in OVC programs BUT impact is unclear & questions regarding “what works” in improving household well-being  Part of the challenge: lack of standardized measures and tools for child and household outcomes (well-being) 3

4. A proposed solution Standardized questionnaires for use in a survey of children ages 0-17 years and their adult caregivers

5. Who are these tools for?  Local and international research institutions and other implementing organizations with evaluation agenda  USAID Forward – supporting local researchers

6. The purpose  Standardize population-level child and caregiver well- being data beyond what is available from routine surveys  Produce actionable data to inform programs and enable mid-course corrections  Enable comparative assessments of child and caregiver well-being and household economic status across a diverse set of interventions and regions

7. Focus on PEPFAR OVC programs  Indicators need to reflect & be amenable to change by PEPFAR program intervention  HH interventions led by home visitors  Community interventions  Low direct funding per target, focus on linkages  Often inadequate services in vicinity

8. How did we start…  Two step, participatory process:  Build consensus around core impact indicators for PEPFAR-funded OVC programs  Develop OVC program evaluation (survey) tools

9. Distilling the core indicators  Our starting point: 6+1 domains of PEPFAR OVC programming  Step 1: Extensive literature search  Step 2: Gaps (HES, PSS) filled through targeted research  Result: >600 child/HH wellbeing questions/indicators  Step 3: Analysis against 8 criteria  Result: shorter list of questions for discussion 9

10. Finalizing the core indicators  External working group: s olicited review from 49 stakeholders  Finalized core set of 12 child and 3 household measures 10

11. Developing the tools  Tools drafted with strong stakeholder input  Draft tools piloted in Zambia and Nigeria  Cognitive interviews to test key concepts (e.g. social support)  Household pre-test of full tools, procedures

12. Structure and content 1.Caregiver questionnaire (including questions on household) 2.Child questionnaire (ages 0-9 years), administered to caregiver 3.Child questionnaire (ages 10-17), administered to child with parental consent & child assent

13. SectionsCore questionsOptional modules Section 1: Household schedule  Household schedule* (10)  Changes in household composition (4) Section 2: Background Information on Caregiver and Household  Demographic information* (7)  Work* (3)  Access to money (3)  Shelter (1)  Household Economic Status (forthcoming)  Progress out of Poverty Index or similar (country specific) Section 3: Food Security  Household food security (6)  Dietary Diversity (1) Section 4: Caregiver Well- being and Attitudes  General health (2)  Caregiver support (4)  Parental self-efficacy (1)  Perceptions and experience of child discipline, violent discipline (forthcoming)  Gender roles and decisionmaking power* (9) Section 5: HIV/AIDS Testing, Knowledge, Attitudes  Basic HIV/AIDS knowledge* (7)  HIV testing* (3)  Attitudes to condom educ (1)  HIV/AIDS attitudes* (4) Section 6: Access to HIV Prevention, Care & Support  Household access to services (1) *DHS, bold=core indicator Caregiver questionnaire

14. SectionsCore questionsOptional modules Section 1: Child Health and Welfare  Confirm demographics (5)  General health & disability (4)  Birth certificate (2)  Vaccinations (11)  Fever (<5 years)* (1)  Diarrhea (<5 years)* (1)  Experience of neglect (2)  Slept under mosquito net* (1)  HIV testing experience* (2)  Fever: extended* (4)  Diarrhea: extended* (3)  Health for children living with HIV/AIDS (forthcoming) Section 2: Education and Work  School attendance*, progression/repeats, drop-outs, missed school days (5+ years) (9)  Work for wages (2)  Early childhood stimulation (2) Section 3: Food Consumption  Food consumption (2+ years) (8) Dietary diversity (1) Section 4: Access to HIV Prevention, Care & Support  Child access to services (1) Section 5: Anthropometric Measures (of Children)  Weight*, Height*, MUAC Child questionnaire (ages 0-9) *DHS, bold=core indicator

15. Child questionnaire (ages 10-17) SectionsCore questionsOptional modules Section 1: Background Information on Child  Confirm demographics* (5)  Identity of caregiver (1) Section 2: Diary  Daily log (6) Section 3: Education  School attendance*, progression/repeats, drop- outs (9) Section 4: Chores & Work  Chores (3)  Work (7) Section 5: Food & Alcohol Consumption  Food consumption (8)  Alcohol consumption (3)  Dietary diversity (1) Section 6: Health, Support & Protection  Birth certificate (2)  General health & disability (3)  General support (4)  Health for children living with HIV/AIDS (forthcoming)  Perceptions/experience of violence (forthcoming) Section 7: HIV Testing, Knowledge, and Attitudes  Basic HIV/AIDS knowledge* (7)  HIV testing * (3)  Child development knowledge (6)  HIV/AIDS attitudes and beliefs (4) Section 8: Sexual ExperienceSexual behavior (13-17 yrs) (5) Section 9: Access to HIV Prevention, Care & Support  Child access to services (1) Section 10: Anthropometric Measures: Weight and Height  Weight, Height, MUAC

16. Using the Tools  There is no single data collection tool that can meet all OVC program M&E requirements. YesNo Situation analysisx TargetingX Case managementX Program monitoringX Evaluationx

17. When are these the right tools? Tools are useful if your question is: 1.Is my program having, or did my program have an impact on the children and households it reached? 2.What are the characteristics of children and their caregivers in my country, state/province or district/area, in terms of education, health, protection, and psychosocial support? 3.Where do the children most in need of program support live? 4.Approximately how many children need services or support? 5.What are the needs of my program’s registered beneficiaries, in terms of education, health, protection, and psychosocial support?

18. These are not the right tools for you if… You want to know:  Which children in selected communities to target  How a particular child/household is faring  Which households, children or caregivers are worst off  What services to provide or refer for a particular child / household  The number of children/households that are receiving program support, and the types of support received  Whether staff are carrying out their responsibilities  Whether interventions are being implemented as planned

19. What’s in the toolkit?  Tools & Manual  Template protocol with consent/assent forms  Methodological guidance  Data analysis guide  Data collector training manual and materials  French translations

20. Global Monitoring, Evaluation and Reporting (“ the MER”)  PEPFAR shift towards outcomes and impact  All technical areas participating (CARE, PREVENTION, TREATMENT)  Specific Changes for OVC Programs  2 indicators at Level 1 (required, reported to HQ)  Outcomes Package at Level 2 (required, reported at country level; indicators elevated from survey toolkit)  2 Education indicators: attendance and progression  Systems Strengthening and Social Protection at Level 3 (recommended)

21. Where can I find out more? Go to MEASURE Evaluation website: http://www.cpc.unc.edu/measure/ our-work/ovc Email: Janet Shriberg jshriberg@usaid.gov

22. Special Thanks to…  Dr. Jenifer Chapman, MEASURE Evaluation* (*Technical Lead for Toolkit, slideset creator)  Dr. Lisa Parker, MEASURE Evalution  OVC TWG Working Group, HQ and Missions  Stakeholders globally working on OVC programming  IATT Education for Guidance on education questions