Effective HIV & SRH Responses among Key Populations in Myanmar Module 4: Monitoring and Evaluation

Monitoring and Evaluation M&E and Using Data for Decision Making Session 1 Monitoring and Evaluation M&E and Using Data for Decision Making

Overview Monitoring & Evaluation – Why? The Core Package M&E Overview Monitoring & Evaluation of Integrated Services - How? The Data Feedback Cycle Unique Identifier Codes Defining true targets Data Presentation Using Data for Decision Making

Monitoring & Evaluation Power of numbers best practice Resource mobilisation Stories behind statistics

The Core Package for HIV Prevention Amongst Key Populations in Myanmar “Monitoring and Evaluation provides implementers, programme managers, decision makes and donors with information to assess the extent to which programmes are being implemented and objectives and impact achieved. This information can demonstrate whether resources are being effectively employed, and inform the planning and design of future responses and resource allocation”

The Core Package for HIV Prevention Amongst Key Populations in Myanmar Monitoring & Evaluation is recognised as a fifth element in the Core Package All M&E Activities and Indicators are guided by Myanmar’s National M&E Plan which describes core indicators used to assess the performance and results of the national response Recommends use of UIC system Notes that national guidelines outlining the frequency that key populations should receive services still need to be developed Set of indicators at end of each element chapter.

The Core Package for HIV Prevention Amongst Key Populations in Myanmar

Monitoring & Evaluation Programme M&E systems are driven by: Local data demands Funding agency requirements National M&E systems: The Three Ones All work under different sets of indicators: PEPFAR, USAID, DFAT Australian Aid Global Fund, UNAIDS, UNGASS National Indicators “Rarely do you need to design new indicators. Resist the temptation. If you think it will be a better indicator than any other existing indicator, ask why no one else has used it before.” David Pencheon

M&E of Integrated Services Monitoring & Evaluation of Integrated Services Avoid duplication of numbers Establish early M&E framework among services Align with one national system & reporting Align with funding reporting requirements Validate data through established mechanisms Complete data feedback cycle with integrated service information distribution to KAPs

The Data Feedback Cycle Data Collection Data Management Data Validation Data Analysis Data Translation Data Feedback

The Data Feedback Cycle Global Guidance Group Check In: The Data Feedback Cycle Which steps of the Data Feedback Cycle are the strongest in organisations in Myanmar? Which steps of the Data Feedback Cycle are the strongest for the national response in Myanmar? Which steps of the Data Feedback Cycle require attention to improve HIV and SRH data quality and flow?

Unique Identifier Codes Provide an anonymous and reliable system for tracking key populations through prevention, treatment and care services A unique code for each individual is created, based on a combination of answers to 4-8 questions that are relevant to the specific context and epidemic of the country M&E systems support monitoring of individuals, not absolute numbers, within and across outreach programs and health services.

Unique Identifier Codes Promote monitoring of key populations and accurate tracking against targets by: Creating a confidential client recognition system that minimises barriers to HIV services for key populations Avoiding duplication in counting of clients attending services Identifying new individuals engaging with services Enabling analysis of treatment cascades Assessing mobility of key populations through outreach and health services Assisting reorienting of services to meet needs and attendance patterns of key populations

Unique Identifier Codes Developing a UIC system Establish a set of questions that elicit a unique code from individual (<2% repeat) Mixture of letters and numbers is ideal Easy to recall, consistent data that is acceptable to key populations and supports required disaggregation Example questions: First two letters of family name? First two letters of mother’s maiden name? First two letters of birthplace? Right or left handed? Gender (Male 1, Female 2, TG 3) Last two digits of birth year?

Unique Identifier Codes Developing a UIC system requires national dialogue and collaboration between health and programmatic local, sub-national and national organisations.

Unique Identifier Codes National Dialogue → National Implementation Define the objective, the key populations and the UIC Define method of data collection & develop tools Pilot the UIC in defined outreach sites Train pilot staff Establish confidentiality and pilot study dialogue Collect pilot data on feasibility & acceptability of UICs Revise UIC protocol based on pilot study findings Rollout UIC to outreach programs with training & tools Pilot the UIC in defined health sites Create connections between outreach and health data National system tracking coverage of key populations Reorient health and programme services to meet needs and mobility of key populations – minimum package of services Ongoing evaluation, follow up & support for scale-up.

Unique Identifier Codes Challenges with UIC systems Initiating National Dialogue & Funding Collaboration between national, sub-national and local health and programmatic partners Change management: a new way of thinking Setting realistic reach based targets Transliteration in countries with multiple language scripts Establishment of revised M&E systems and databases. Establishing trust with key populations and assurance of confidentiality & anonymity

Unique Identifier Codes Successes with UIC systems Increased engagement of key populations with health and outreach services Improved service focus on key populations Population size estimates Coverage data for individuals and populations Minimum package of services targets: Individuals receiving defined outreach and health services Referrals and reporting collaboration Definition of coverage required to reduce risk behaviours Project partnerships and advocacy (regional response).

Unique Identifier Codes COUNTING PEOPLE, NOT NUMBERS

Unique Identifier Codes Group Check In: Experience of Unique Identifier Codes? Which programmes are using UICs? What are the questions that form the UIC? Challenges and opportunities of implementing UICs? How has the programme addressed confidentiality? How is the UIC data collected and managed? How is the UIC data used? Have UICs improved experience of services by key populations?

Treatment Cascade Where does data for each level generate from? Key Populations Living with HIV KPs with HIV that access testing KPs with HIV that know and understand their result KPs with HIV that are linked to treatment, care and support services KPs with HIV start ART KPs with HIV maintain viral suppression Where does data for each level generate from?

Defining True Targets Use of UICs demands revision of targets Lower targets justified by greater accuracy (no duplication) Aim for SMART targets in line with national guidelines In absence of national guidelines, speak with other local programs, funders and review international guidelines Grade targets and celebrate milestones Improve systems to improve data

Conclusion Monitoring & Evaluation of Integrated Services presents challenges and opportunities for measuring engagement of key populations across prevention, treatment and care Indicator sets should align with national and funding systems Population Size Estimates provide a denominator in measurement of programme success A Comprehensive Package of Services defined interventions to be monitored Reduced targets are not negative reflections of practice but positive steps towards accurate measurements Good M&E is innovative and supports the complete data feedback cycle

Monitoring & Evaluation THANK YOU

References Baldwin, S., Boisen, N., & Power, R. (2008). Managing information: Using systematic data collection to estimate process and impact indicators related to harm reduction services in Myanmar. The International Journal of Drug Policy, 19S, S74-S79. doi:10.1016/j.drugpo.2007.12.005. DDRP. (2007). DDRP Best Practice Collection: Unique Identifier Code. Fielden, S. (2013). Protocol: Creating the UIC Code & Evaluation Plan. Tools for Establishing UIC in HIV Outreach for Key Populations in Morocco. Gray, R. & Hoffman, L. (2007). Tracking coverage on the silk road: Time to turn theory into practice. The International Journal of Drug Policy, 19S, S15-S24. doi:10.1016/j.drugpo.2007.12.011. Thien Nga, N., Jacka, D., Van Hai, N., Kieu Trinh, N., Boisen, N., & Neukom, J. (2012). Innovative data tools: a suite for managing peer outreach to key affected populations in Viet Nam. WPSAR, Vol.3, No.3, 29-34. doi:10.5365/wpsar.2012.3.2.003. UNAIDS. An introduction to indicators.