Adding value to project implementation through a learning/CLA approach October 16, 2018

The evolution of M&E: the early days Our projects were doing a good job of collecting and reporting data But not much emphasis on data use, applied research, learning and feedback loops to take learning to action We started focusing on learning and on “telling the story” of projects We started integrating our M&E approaches with applied research and USAID’s CLA guidance

What does the MERLA approach look like?

What is MERLA? MERLA is the intentional application of results-focused monitoring, evaluation and research… …to inform continuous learning and adaptation… …for improving program effectiveness and policy decision making.

MERLA in relationship to M&E MERLA does not replace monitoring and evaluation (M&E), but rather adds more value and augments M&E MERLA helps projects move away from M&E as a data reporting requirement and toward a dynamic, collaborative, and adaptive process to increase program efficiency and impact MERLA integrates research and CLA (collaborating, learning, and adapting) into M&E, thereby making M&E stronger

What does MERLA do for a project? MERLA improves program implementation by ensuring that: Data collected through M&E systems are timely, of high quality, analyzed and used continuously M&E data are complemented and strengthened by filling gaps in evidence through operations research (OR) Key results and evidence from M&E and research are synthesized as learning Learning is continuously incorporated into program re-design for improving implementation, documenting and communicating lessons, and informing policy decisions

Examples of MERLA products developed for proposals and at project start up Performance Monitoring Plan (PMP) with indicators, results framework, logframes Project dhis2 platforms Learning Agenda and Communications Plan Operations research questions and protocols Approaches and tools for ensuring CLA

dhis2 platforms

Learning and adapting approaches and tools Internal and external pause and reflect sessions Establishment of learning platforms and communities of practice Data to action guides Virtual cross-country roundtable discussions

Learning agenda key questions What results is a given project achieving? Why and how are we achieving – or not achieving - results? At what cost are we achieving these results? How can we use learning to make rapid programmatic course correction and influence policy discussions? And for effective communication?

Example 1: how learning helps improve program implementation RTI’s StopPalu+ malaria prevention project in Guinea aims to motivate pregnant women to seek mRDT Project implemented SMS program to reach out to pregnant women As part of MERLA, we set up intervention and comparison groups Set up feedback loops to ensure continuous program learning and adapting

Learning and adapting is key Image source: Counseling, testing and enrollment of women in StopPalu program. Patrick Adams, RTI International Already seeing findings that are helping us tweak the SMS program as it progresses Learning: Women with no education and those registering between 1-4 weeks into their first trimester less likely to seek 2nd antenatal care visit Adapting: SMS program is targeting messages for women early on in pregnancy and those likely to become pregnant In terms of study design, this will be a new arm: important to track results and costs for each aspect of program Lastly, let’s remember why and who it is we are learning and adapting for: not for the sake of learning in itself, but for the benefit of mothers and children

Example 1: how learning helps improve program implementation SMS intervention group had 8% more follow-up antenatal care visits than the comparison group Cost per woman enrolled dropped from $20.50 in first month to $2.95 by end of first year SMS group saw 23 fewer neonatal deaths over a one-year period than the comparison group It cost the program $650 per neonatal death averted

Example 1: how learning helps improve program implementation Feedback loops and reflection sessions enabled course correction on continuous basis Women with no or little education were less likely to seek follow-up antenatal care Based on this learning, we developed targeted and visual messages for these women Saw increased uptake of antenatal care following these refinements Government is using these learnings to provide guidance for other SMS programs

Example 2: how learning helps improve program implementation RTI’s LuzonHealth project in Philippines aims to prevent adolescent pregnancies in Luzon province Project was tasked with implementing a government-approved teen pregnancy prevention program in schools Concerns that the program wasn’t really promoting pregnancy prevention

Example 2: how learning helps improve program implementation We put together a MERLA plan & protocol Comprehensive desk review of curriculum Pre and post test in intervention and comparison schools Qualitative study exploring perceptions and benefits of curriculum, barriers to implementation, etc.

Example 2: how learning helps improve program implementation Teen pregnancy prevention module had no info on teen pregnancy prevention: message was abstinence But HIV prevention and responsible parenthood modules covered condom use and pills Intervention schools saw significant increase in teen pregnancy prevention knowledge and behavior intentions Identified concrete ideas for curriculum improvements Dept of Education partnering with Dept of Health for curriculum mainstreaming

Learning from learning CLA/MERLA not rocket science. We’ve all been doing it in various ways for decades But important to have concrete approaches and tools to ensure deliberate learning and adapting CLA is so much easier on paper! The soft skills (patience, diplomacy, time for engagement) related to CLA are as important as the hard skills/results The rewards are worth it: strong local buy in and ownership, more likelihood of learning to action

Moving to cross-sectoral learning CLA is helping us break M&E/MERLA siloes within RTI Established cross-RTI MERLA Community of Practice in 2017 Engaging closely with USAID Learning Lab to share our CLA experiences Focusing on communicating our CLA stories: in-country and cross-country learning platforms, learning events, conferences, publications