Institute for International Programs An international evaluation consortium Institute for International Programs An international evaluation consortium.

Slides:



Advertisements
Similar presentations
February 2006 WHO's Contribution to Scaling Up towards Universal Access to HIV/AIDS Prevention, Care and Treatment Department of HIV/AIDS.
Advertisements

Introduction to the Results Framework. What is a Results Framework? Graphic and narrative representation of a strategy for achieving a specific objective.
UNICEFs Social Policy Agenda and the Global Study on Child Poverty and Disparities Regional Workshop, Panama 1-2 July 2008 Gaspar Fajth Chief, Social Policy.
Introduction to Monitoring and Evaluation
Intensified action on seven behaviours by all development partners Session objectives 1.To review status of intensified action: progress, issues and challenges.
February Dakar, Senegal
Donald T. Simeon Caribbean Health Research Council
Lucila Beato UNMIL/HRPS
Sabelo Mbokazi Senior Policy Officer HIV/AIDS, TB, Malaria & OID
Almost 14 years ago all countries endorsed a set of 8 Millennium Development Goals (or MDGs). 3 of those 8 Goals focus on health – that being child mortality,
Comprehensive M&E Systems
Preparing National Water Resources Management and Efficiency Plans Taking stock of progress since Johannesburg-are we meeting the 2005 target? Swedish.
What is H(M)IS?. Purpose of HIS “is to produce relevant information that health system stakeholders can use for making transparent and evidence-based.
The SEEAW in the context of Integrated Water Resource Management Roberto Lenton Chair, Technical Committee Global Water Partnership.
1 Collective Efficiencies Development Finance Architecture Workshop Prerna Banati - July
Regional Integration in the Context of HIV and AIDS, TB and Malaria: The case of SADC Presentation at the XIX International AIDS Conference, Washington.
Every Newborn Global Action Plan - Group 4 Vision and Goal Vision: Option 2 preferred – a more positive framework: A world in which preventable maternal.
Common recommendations and next steps for improving local delivery of climate finance Bangkok, October 31, 2012.
Assessment of laws and policies for promoting rights of children Dr Bernadette Daelmans Department of Maternal, Newborn, Child and Adolescent Health World.
Elizabeth Mason Department of Child and Adolescent Health and Development New Strategic Directions Tracking progress in child survival Countdown to 2015.
The SEEAW in the context of Integrated Water Resource Management and the MDGs Roberto Lenton Chair, Technical Committee Global Water Partnership.
Performance Monitoring and Financial Reports Performance Monitoring and Financial Reports UNAIDS and Unified Budget and Workplan (UBW)
Multiple Indicator Cluster Surveys (MICS) Contribution of MICS4 to Monitoring of National and International Commitments Sarah Ahmad Mirza 7 th December.
T HE M ULTISECTORAL A PPROACH, I NVESTMENT T HINKING AND N ATIONAL AIDS R ESPONSE C OORDINATION M ESFIN G ETAHUN & B ENJAMIN O FOSU -K ORANTENG N OVEMBER.
Evaluating the system-wide effects of HIV scale-up: methodological gaps, challenges and recommendations David Hotchkiss Health Systems 20/20/Tulane University.
1 OPHS FOUNDATIONAL STANDARD BOH Section Meeting February 11, 2011.
Health Indicators Mortality indicators Morbidity indicators
Evidence and Information for Policy Health Metrics Network Strengthening Country-Level and Global Tracking of Health Outcomes.
1 RBM Background Development aid is often provided on a point to point basis with no consistency with countries priorities. Development efforts are often.
Unit 10. Monitoring and evaluation
Toolkit for Mainstreaming HIV and AIDS in the Education Sector Guidelines for Development Cooperation Agencies.
SECTOR POLICY SUPPORT PROGRAMMES A new methodology for delivery of EC development assistance. 1.
Framework for evaluation of scaling up of programmes Consultative workshop on Health Impact Accounts Brussels 6 February 2012.
The State of the World’s Children 2008 Themes Child Survival: Where we stand Lessons learned from evolving health- care systems and practices Community.
Framework for Measurement of Universal Health Coverage Ties Boerma, WHO Beijing, 3 November 2012 Based on meeting at Rockefeller Center, Bellagio,
Aid for Development Effectiveness -Managing for Development Results- Terence D. Jones UNDP Third International Roundtable Managing for Development Results.
Evaluation of sector programmes and budget support operations in the context of EU development cooperation 1 st M&E Network Forum 07 to 08 November 2011.
DRAFT V1 National Vaccine Supply Chain Innovations: Country Commitment to Ownership, Sustainability & Impact GAVI Partners’ Forum WHO – UNICEF – GAVI -
M&E Basics Miguel Aragon Lopez, MD, MPH. UNAIDS M&E Senior Adviser 12 th May 2009.
Monitoring UA 2010 in health sector 1 |1 | Monitoring progress towards Universal Access 2010 in the health sector Kevin M De Cock Ties Boerma.
April_2010 Partnering initiatives at country level Proposed partnering process to build a national stop tuberculosis (TB) partnership.
Regional Strategy on Human Resources for Health (WHO Western Pacific Region) Presentation by Dr Ezekiel Nukuro Regional Adviser, Human Resources.
21/4/2008 Evaluation of control measures 1. 21/4/2008 Evaluation of control measures 2 Family and Community Medicine Department.
HUMAN DEVELOPMENT PRODUCTIVITY AND EMPLOYMENT. OUTLINE Introduction 1. Summary of issues 2.What is working 3.Looking ahead: Focus on outcomes 4.What makes.
What do donor’s think? Opportunities and challenges for stigma reduction programs and research R. Cameron Wolf, PhD Senior HIV/AIDS Advisor for Key Populations.
Steve Hodgins Imperial Royale Hotel, Kampala 26 June 2013.
DEVELOPMENT COOPERATION FRAMEWORK Presentation by Ministry of Finance 10 December 2013.
A short introduction to the Strengthened Approach to supporting PFM reforms.
Consultant Advance Research Team. Outline UNDERSTANDING M&E DATA NEEDS PEOPLE, PARTNERSHIP AND PLANNING 1.Organizational structures with HIV M&E functions.
Vito Cistulli - FAO -1 Damascus, 2 July 2008 FAO Assistance to Member Countries and the Changing Aid Environment.
Financing for Reproductive, Mother, Newborn, Child, and Adolescent Health for UHC ACCELERATING PROGRESS ON EARLY ESSENTIAL NEWBORN CARE September,
Changing the way the New Zealand Aid Programme monitors and evaluates its Aid Ingrid van Aalst Principal Evaluation Manager Development Strategy & Effectiveness.
Human resources for maternal, newborn and child health: opportunities and constraints in the Countdown priority countries Neeru Gupta Health Workforce.
Selecting performance indicators and setting targets Joseph Naimoli The World Bank October 2008 Presented by Kate Gilroy Institute for International Programs,
 2007 Johns Hopkins Bloomberg School of Public Health Section B Logic Models: The Pathway Model.
Measurement of Universal Health Coverage Based on meeting at Rockefeller Center, Bellagio, September 2012.
Global Partnership for Enhanced Social Accountability (GPESA) December 19, 2011 World Bank.
Comprehensive M&E Systems: Identifying Resources to Support M&E Plans for National TB Programs Lisa V. Adams, MD E&E Regional Workshop Kiev, Ukraine May.
Implementing operational research for HIV treatment scale-up in resource-limited settings TB/HIV Research Priorities in Resource-Limited Settings Expert.
NFM: Modular Template Measurement Framework: Modules, Interventions and Indicators LFA M&E Training February
GFF Third Investors Group Meeting Geneva, Switzerland June 23-24, 2016 Financing for RMNCAH: complementary financing Global Fund’s engagement with the.
Outline The Global Fund Strategy emphasizes the Key Populations
UGANDA: Budgeting, resource tracking and domestic advocacy practices.
M&E Basics Miguel Aragon Lopez, MD, MPH
Review of integrated PSM resources and tools and introduction to group work Upjeet Chandan ICCM FTT 17th February 2016.
Lecture 9: PHC As a Strategy For HP Dr J. Sitali
Joint session with IHP+ introduction
April 2011.
SRH & HIV Linkages Agenda
monitoring & evaluation THD Unit, Stop TB department WHO Geneva
Presentation transcript:

Institute for International Programs An international evaluation consortium Institute for International Programs An international evaluation consortium A common evaluation framework for the scale-up to achieve the health MDGs International Evaluation Consortium Institute for International Programs, Johns Hopkins Bloomberg School of Public Health May 2008 Including the results framework for maternal, newborn and child health

Presentation Outline 1.Background, rationale, methods and partners 2.The strategic framework 3.The results framework for maternal and child health and nutrition 4.Evaluation challenges

BACKGROUND, RATIONALE, METHODS AND PARTNERS Section 1:

Rationale for a common framework  To support the comparison of results and costs within and across countries that may use different approaches to implementation  To promote systematic attention to the methodological challenges of evaluating large- scale approaches  To inform pay-for-performance approaches Victora CG, Black RE, Bryce J. Learning from new initiatives in maternal and child health. Lancet 2007; 370:

What is included in an evaluation framework? 1.A strategic framework including general principles 2. A results framework that includes:  A conceptual model specifying how activities will lead to outcomes and impact  A set of compatible designs for evaluation of country-level initiatives, to allow comparisons across places and time  A set of common indicators and other measures

Framework development process Global business plan Global campaign IHP+ Catalytic initiative BMGF/PMNCH Rapid Scale Up Doris Duke MCH results framework Overall strategic framework Common evaluation framework Initiatives Evaluators & products

THE COMMON STRATEGIC FRAMEWORK Section 2:

Framework Terminology TermDefinition Evaluationrigorous, science-based collection of information about program activities, characteristics, outcomes and impact that determines the merit or worth of a specific program or intervention Monitoringroutine tracking and reporting of priority information about a program and its intended outputs and outcomes…to assess whether resources are spent according to plan and whether the program is resulting in the expected outputs Monitoring performance project or programme monitoring which aims to provide feedback for improved performance/ implementation Performance- based funding provision of financial incentives based on measured progress Data quality audits a method to assess recording and reporting systems in the context of performance-based funding Adapted from Boerma T, Bos E, Walford V et al. International Health Partnership+. A common framework for monitoring performance and evaluation of the scale up for better health. Draft 4, February 2008.

Principles Collective action: Primary focus on the contribution of the collective efforts to scale-up the health sector response in countries Alignment with country processes: build upon national processes that countries have established to M&E progress in the implementation of national plans Balance between country participation and independence: driven by country needs but conducted in a manner which maintains independence of evaluation Harmonised approaches: common protocols and standardized outcome indicators and measurement tools, with appropriate country adaptations Capacity building and health information system strengthening: systematic involvement of country institutions Adequate funding: between 5% and 10% of the overall scale-up funds set aside for monitoring performance, evaluation, operational research and strengthening health information systems Adapted from Boerma T, Bos E, Walford V et al. International Health Partnership+. A common framework for monitoring performance and evaluation of the scale up for better health. Draft 4, February 2008.

Inputs Outputs Outcomes Impact Funding Domestic sources International sources Plan Coherent, prioritised and funded Harmonization Aligned international efforts with national plan Well coordinated and harmonized support National plan implementation Systems strengthening Priority interventions scale-up Capacity building Programmes Institutions People Accountability Performance monitoring Results focus and evaluation Use for better practices Health system strengthened Governance, HR, medical products, information Increased service utilization and intervention coverage Reduced inequity (e.g. gender, socio- economic position) Responsiveness No drop-off non- health sector interventions (e.g. water & sanitation) Improved survival Child mortality Maternal mortality Adult mortality due to infectious diseases Improved nutrition Children Pregnant women Reduced morbidity HIV, TB, malaria, repr. health Improved equity Social and financial risk protection Reduced impoverishment due to health expenditures Improved services Access, safety, quality, efficiency Process Aid process monitoring Resource tracking Strengthen country health information systems Evaluation: process, health systems strengthening, impact M & E action Health system monitoringCoverage monitoringImpact monitoring Implementation Monitoring Strategic Framework Adapted from Boerma T, Bos E, Walford V et al. International Health Partnership+. A common framework for monitoring performance and evaluation of the scale up for better health. Draft 4, February 2008.

THE RESULTS FRAMEWORK FOR MATERNAL, NEWBORN AND CHILD SURVIVAL Section 3:

Three components of a results framework 1. A conceptual model specifying how activities will lead to outcomes and impact 2. A set of compatible designs for evaluation of country-level initiatives, to allow comparisons across places and time 3. A list of common indicators and other measures

What is a conceptual model in the context of a results framework? A description of the activities and pathways that will lead from program inputs to impact on health and nutrition Inputs Funding Planning & policies Harmonization & efficiency Inputs Funding Planning & policies Harmonization & efficiency Outputs Health services delivery Quality Behavioural Interventions & knowledge Outputs Health services delivery Quality Behavioural Interventions & knowledge Outcomes Service utilization and intervention coverage Behavioural change Reduced inequity Outcomes Service utilization and intervention coverage Behavioural change Reduced inequity Impact Mortality Morbidity Nutrition Impact Mortality Morbidity Nutrition Process Training & Capacity building Procurement and supply Guidelines IEC Community mobilization Process Training & Capacity building Procurement and supply Guidelines IEC Community mobilization A generic conceptual model based on the strategic framework

Why is a conceptual model essential in a results framework?  To clarify expectations of program planners/ developers  To define the evaluation questions and select indicators  To support the design and estimate sample sizes  To guide analysis and attribution of results  To compare and interpret results across sites  To track changes in assumptions as they evolve in response to evaluation findings  To stay honest about what was expected The generic conceptual model must be adapted to reflect the plans and expectations of each country/program.

Common elements of the conceptual model for the global scale-up  MORBIDITY AND MORTALITY High and equitable coverage with proven interventions Country-specific implementation & health system strengthening Inputs Outputs Outcomes Impact Process

Designs linked to national monitoring The evaluation framework must be implemented in ways that build institutional capacity for evaluation in countries. National program monitoring Impact evaluation Policies/plans/ resources Implementation (provision & quality) OutcomesImpact

Evaluation challenges  Complete tracking of program and contextual factors  Full consideration of equity  Evaluating “scaling-up” and “catalytic actions”  Measuring “community engagement”  Assessing “health system strength”  Capturing changes in mortality within short time frames – new approaches needed

Summary The common evaluation framework for the global scale-up to the health MDGs  There is a common evaluation framework; all countries and partners should use it as a starting point for planning their evaluations  Main purpose of evaluating global scale up is assessment of program effectiveness  Requires comprehensive prospective evaluation designs  Complementary evaluation goals of independence and capacity building  Key constructs need further definition (e.g., “scaling-up”; “catalytic”, “health system strength”)