M&E of policy influence some guidance thoughts…. Elements of a monitoring frame.. Policy objective and theory of change Indicators Data collection tools.

Slides:

Advertisements

Similar presentations

Knowledge without boundaries. Outcome evaluation of PLIP projects: what, why and how? Ugne Lipeikaite PLIP Impact Manager

Advertisements

Operationalizing Political Analysis and Communication Strategy Caby Verzosa September 11, 2008.

Scaling-Up Early Childhood Intervention Literacy Learning Practices Maurice McInerney, Ph.D. American Institutes for Research Presentation prepared for.

Sidney B. Westley East-West Center Let’s Talk about Reaching Wider Audiences with our Research Results.

OECD/INFE High-level Principles for the evaluation of financial education programmes Adele Atkinson, PhD OECD With the support of the Russian/World Bank/OECD.

Supporting continuous improvement in the replication process Getting to Grips with Replication Seminar 3: Monitoring, evaluation & continuous improvement.

Measuring policy influence: like measuring thin air? John Young:

Monitoring and Evaluation of Influence John Young.

Measurement. T EAM STEPPS 05.2 Mod Page 2 Measurement Objectives  Describe the importance of measurement  Describe the Kirkpatrick model of training.

Corry Bregendahl Leopold Center for Sustainable Agriculture Ames, Iowa.

Márcia Paterno Joppert Brazilian Monitoring and Evaluation Network October 2013 Multipaper session: The State of Evaluation Practice in the Early 21st.

Dissemination pathways Science and policy

Launch Meeting 26 October 2005 Loughborough University.

HEInnovate A self-assessment tool for higher education institutions (HEIs) wishing to explore their entrepreneurial and innovative potential.

Evaluation. Practical Evaluation Michael Quinn Patton.

+ Monitoring, Learning & Evaluation Questions or problems during the webinar?

Sustaining Local Public Health and Built Environment Programs Fit Nation NYC November 3, 2011 Annaliese Calhoun.

TeamSTEPPS TM National Implementation Measurement The following slides are not part of the TeamSTEPPS Instructor Guide. Due to federal 508 compliance requirements.

HOW TO WRITE A GOOD TERMS OF REFERENCE FOR FOR EVALUATION Programme Management Interest Group 19 October 2010 Pinky Mashigo.

THE MYSTERY OF GETTING RESEARCH INTO USE… THE ONGOING MYSTERY OF GETTING RESEARCH INTO USE.

Establishing Research and Evaluation Network on Child Issues in Indonesia Nuning Akhmadi Indonesia Research and Evaluation Network Manila, 7 – 8 November.

Reporting and Using Evaluation Results Presented on 6/18/15.

Lessons from RAPID’s work on research-policy links John Young.

Dr Abdul Ghaffar Executive Director Influencing policy-making Results of a multi-country study (London, 13 December, 2011)

Development and Implementation of a National Multisectoral Output Monitoring System (SHAPMoS) for HIV Responses in Swaziland: Challenges and lessons learned.

Impact assessment framework

Unit 8: Uses and Dissemination of HIV Sentinel Surveillance Data #3-8-1.

Program Evaluation and Logic Models

1 Introduction to Evaluating the Minnesota Demonstration Program Paint Product Stewardship Initiative September 19, 2007 Seattle, WA Matt Keene, Evaluation.

Beyond impact factors: making your research count through better translation Clinical and Public Health Seminar April 2014 Associate Professor Harriet.

Fundamentals of Evaluation for Public Health Programs ROBERT FOLEY, M.ED. NIHB TRIBAL PUBLIC HEALTH SUMMIT MARCH 31,

OECD/INFE Tools for evaluating financial education programmes Adele Atkinson, PhD Policy Analyst OECD With the support of the Russian/World Bank/OECD Trust.

Objectives 1. Build capacity for Regional Coordinators to better support the NOs. 2. Prepare the Malaysian Programme field staff on local level advocacy.

Communications Noora-Lisa Aberman RENEWAL 3 Regional Workshop 12 th March 2007.

Sub-Regional Workshop for GEF Focal Points in West and Central Africa Accra, Ghana, 9-11 July 2009 Tracking National Portfolios and Assessing Results.

Tracking national portfolios and assessing results Sub-regional Workshop for GEF Focal Points Western and Central Africa Dakar, May 2007.

3ie Grantees Communication for Policy Influence Clinic Day 2.

RAPID Outcome Mapping Approach Simon Hearn, ODI 16 April 2010 Bern, Switzerland.

The NEKIA Knowledge Utilization Initiative Board of Directors Meeting Monday April 11, 2005 Montreal.

Stakeholder analysis for project design Ingvild Oia, Programme Specialist,UNDP Photo by: Konomiho/flickr.

BUILDING THE FOUNDATIONS FOR A STATE M&E SYSTEM Information for M&E in the State of Yucatán 1.

Designing Local Curriculum Module 5. Objective To assist district leadership facilitate the development of local curricula.

Learning Lessons from Experience: good practice case studies Inclusive Education Title: Inclusive education planning in school and local authority levels.

EDPQS in 10 minutes: Overview of European Drug Prevention Quality Standards (EDPQS) With financial support from the Drug Prevention and Information Programme.

 Read through problems  Identify problems you think your team has the capacity and interest to solve  Prioritize the problems and indicate the.

Bridging the gap between research and policy making? Ajoy Datta Foundation for the Future, Dead Sea, Jordan Civil Society Partnerships Programme

Better knowledge for better practice SUMMARY SLIDE n SCIE’s Role n 5 Key messages about SCIE n How SCIE can help.

BCO Impact Assessment Component 3 Scoping Study David Souter.

Roma Inclusion pilot project: Information and awareness raising DG REGIO conference “Monitoring and Evaluation of Roma projects and policies” 30 November.

Hosted by: Funded by: Research into Reality Overcoming the challenges of knowledge exchange Foundation for the Future Research Conference 26 th – 28 th.

Alain Thomas Overview workshop Background to the Principles Definitions The National Principles for Public Engagement What.

Knowledge Management of Gender Statistics in MDG- based Poverty Reduction Strategies IA and EGM on Gender and Statistics December 2006, New York.

PARIS21/CARICOM Workshop on NSDS, Trinidad and Tobago, July 2009 How can PARIS21 help? Presentation by PARIS21 Secretariat.

Consultant Advance Research Team. Outline UNDERSTANDING M&E DATA NEEDS PEOPLE, PARTNERSHIP AND PLANNING 1.Organizational structures with HIV M&E functions.

1 The project is financed from the European Union funds within the framework of Erasmus+, Key Action 2: Cooperation for innovation and the exchange of.

Documentation, Data, and Monitoring Section 9 Page 38 1.

Influencing Policy through Research: Introduction to Principles and Tools Arnaldo Pellini:

M & E System for MI’s Training Program & Guidelines for MI’s Completion Report Presented by Monitoring and Evaluation Officer Mekong Institute January.

Connect2Complete Theory of Change Development for Colleges and State Offices November 10, 2011 OMG Center for Collaborative Learning.

Public Policy Process and Public Administration

Session 2: Developing a Comprehensive M&E Work Plan.

Florita Montgomery West Virginia University Extension Allison Nichols West Virginia University Extension Teresa McCoy University of Maryland Extension.

Chapter 1 Toward Accountability. The Faces of Accountability Professional Accountability.Service Delivery Accountability.Coverage Accountability.Cultural.

Evaluation Capacity Building at Country Level: GEF Focal Points 1 Osvaldo Néstor Feinstein AEA 2011 Conference GEF Evaluation Office Panel.

CAPACITY DEVELOPMENT for the CLEAN DEVELOPMENT MECHANISM for CAMBODIA (CD4CDM-CAM) National Workshop on Capacity Development for the Clean Development.

European Agency for Development in Special Needs Education Project updates Marcella Turner-Cmuchal.

Supporting community action on AIDS in developing countries ‘Evaluating HIV and AIDS-related advocacy’ Skills Building Session International.

Introduction to advocacy

Supporting Learning and Organisational Effectiveness

Who Are Champions, What Are Their Impacts and How Do We Know

Presentation transcript:

M&E of policy influence some guidance thoughts…

Elements of a monitoring frame.. Policy objective and theory of change Indicators Data collection tools

Starting with policy objective and TOC.. To identify key moments Change in resettlement practices Access to Secretary, Ministry of Highways Public pressure to change Workshops with RDA officers Media dissemination Workshops with ADB, JICA ? ?

Thinking about indicators… Output level Meetings, presentations…. Outcome level Uptake: citations, media clippings…. Changes in Knowledge, Attitudes, Behaviors: changes in interest, participation in events… Policy Impact level Changes in policy / practice: changes in policy documents, legislation, replication….

Choosing data collection tools.. Sources of data The IE team The users - policy makers / practitioners Academics, other researchers Media ….. Possible tools o AIIM o After Action Reviews o User Surveys o Bellwether Method o Outcome Mapping o Most significant change o …… Possible tools o AIIM o After Action Reviews o User Surveys o Bellwether Method o Outcome Mapping o Most significant change o ……

After action reviews Useful for evaluating quality of outputs Guiding Questions What was supposed to happen? What actually happened? Why were there differences? What worked? What didn’t? Why? What would you do differently next time? Turn conscious learning into tacit knowledge..

User surveys Useful for monitoring uptake - to get perceptions and reactions of a range of users Can be in the form of Large scale questionnaires Small focus group discussions Useful for monitoring uptake of websites, policy briefs

Bellwether Method Bellwethers Defined ‘Influentials’ in the public and private sectors whose positions require that they are politically informed and that they track a broad range of policy issues ► Policymakers ► Administrators ► Media ► Advocates ► Funders ► Business ► Associations ► Policymakers ► Administrators ► Media ► Advocates ► Funders ► Business ► Associations Useful for monitoring if policy messages have ‘broken through’

Group Exercise Question: What indicators can you suggest to monitor whether the IE’s policy objective(s) are being met? To the plenary, please present the following Proposed indicators together with rationale, for the selected project any “lessons” from the exercise that the other projects in the group can take forward into their own projects

How to monitor policy influence: in summary Many activities, many contexts, much still to be learnt on how to monitor But some lessons are available: Select few key Indicators Use fit-for-purpose, easy to use data collection tools Where to begin? Policy Objective and Theory of Change If you want to read more… A guide to monitoring and evaluating policy influence by Harry Jones Making a difference: M&E of policy research by Ingie Hovland If you want to read more… A guide to monitoring and evaluating policy influence by Harry Jones Making a difference: M&E of policy research by Ingie Hovland