1. MER Essential Survey Indicators Jenifer Chapman, PhD & Lisa Parker, PhD February 2, 2015

2. Learning outcomes  You will become familiar with the indicators and think about what they mean for your programs  You will know how to get the data  You will better understand how to choose programs for outcomes monitoring from your country HKID portfolio

3. MER Essential Survey Indicators  Move to outcomes, linked to program goals  Mandatory  Every two years

4. 9 Essential Indicators  Representing holistic measures of child and family wellbeing  Linked to broader HIV response goals  Linked to broader child protection response goals  Vetted by broad stakeholder community

5. Indicator criteria  Amenable to change due to PEPFAR OVC programs in a 2 year period  Easy to measure by trained data collectors  Relevant over time and place  Indicators including questions that could be validated were prioritized  Many pilot tested

6. HIV status Percent of children whose primary caregiver knows the child’s HIV status Rationale: If HIV status is unknown, child will not access care & treatment (proxy for testing) Source: PEPFAR OVC TWG

7. Nutrition Percent of children <5 years of age who are undernourished Rationale: linked to infant morality and long- term child health and development Source: World Health Organization

8. Health Percent of children too sick to participate in daily activities Rationale: PEPFAR supports critical linkages to health services to improve functional well- being Source: MEASURE Evaluation

9. Legal protection Percent of children who have a birth certificate Rationale: Required to access essential services Source: DHS

10. Education: School attendance Percent of children regularly attending school Rationale: Important for child development; children in school are less likely to acquire HIV Source: UNESCO

11. Education: Progression in school Percent of children who progressed in school during the last year Rationale: More education is linked to better HIV awareness, higher contraceptive use, improved child well-being (among others) Source: MEASURE Evaluation

12. Early childhood development Percent of children <5 years of age who recently engaged in stimulating activities with any household member over 15 years of age Rationale: Early childhood stimulation is linked to long term child health and development Source: MICS

13. Perception of violence Percent of caregivers who agree that harsh physical punishment is an appropriate means of discipline or control in the home or school Rationale: Perception of violence is linked to use of violence; children experiencing violence show greater HIV risk behaviors Source: MEASURE Evaluation

14. Household economic resilience Percent of households able to access money to pay for unexpected household expenses Rationale: Resilience to economic shocks is linked to poverty, which impacts child and family well-being Source: MEASURE Evaluation

15. Disaggregation  By sex  By age group (where relevant):  0-4 years  5-9 years  10-14 years  15-17 years

16. Let’s discuss (30 mins) What questions do these indicators raise for you with respect to program implementation?  Hint: how does this affect targeting beneficiaries?

17. Feedback What did you discuss?

18. Getting the data  Figuring out which programs  Figuring out which approach: 1.Outcomes monitoring 2.Evaluation  Figuring out who will collect the data

19. Which programs?  Appropriate proportion of budget  Agency representation  Appropriate program scope and timeline  Strategic effort  Paying heed to other data collection efforts *Countries with total HKID funding of <1M USD/year are exempt from requirement

20. So we might have data from multiple programs in one country?  Yes. They cannot be aggregated.

21. What approach? Outcomes monitoring vs. Evaluation

22. Considerations  The information you need  Why you need it  When you need it Note that we have developed data collection tools for both approaches

23. EvaluationOutcomes monitoring Outcomes may be attributed to program* Attribution cannot be established Data valid at population level Data may be valid at local level* Larger number of indicatorsVery limited number of indicators 3-5 years usuallyEvery 2 years Complex samplingSimpler sampling Complex analysisSimpler analysis* Higher costLower cost

24. Methods for outcomes monitoring Cluster sample surveys vs. Lot quality assurance sampling (LQAS)

25. MethodAdvantagesDisadvantages Cluster sampling Sample large enough for sub-group analyses Simpler sampling design No weighting* Statistician needed for sample size calculation More expensive (larger) Lot quality assurance sampling Provides information valid at supervision area (SA) level May be cheaper depending on number of SAs Sampling frame needed for each SA Sample size will need to be increased for some indicators Values need to be weighted

26. Data collection tool

27. Which partner?  Surveys must be undertaken by appropriate institution, that is NOT providing services under program  Proven institutional capes:  Survey design and sampling (all methods)  Ethical and safe data collection  Data management and analysis

28. When? APR FY15 And again in two years.

29. Implications for your work (30 min)  What programs might require outcome monitoring in your country?  What are your next steps?

30. Feedback

31. Where can I find out more? Go to our website: http://www.cpc.unc.edu/measure/ our-work/ovc Email: Jenifer Chapman: jchapman@futuresgroup.com or Lisa Parker: lparker@futuresgroup.com jchapman@futuresgroup.com lparker@futuresgroup.com

32. The research presented here has been supported by the President’s Emergency Plan for AIDS Relief (PEPFAR) through the United States Agency for International Development (USAID) under the terms of MEASURE Evaluation cooperative agreement AID-OAA-L-14-00004. Views expressed are not necessarily those of PEPFAR, USAID or the United States government. MEASURE Evaluation is implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures Group, ICF International, John Snow, Inc., Management Sciences for Health, and Tulane University.