Introduction to Evaluation

Slides:

Advertisements

Similar presentations

- Results-based Management - Monitoring & Evaluation An overview.

Advertisements

Linking regions and central governments: Indicators for performance-based regional development policy 6 th EUROPEAN CONFERENCE ON EVALUATION OF COHESION.

Theories of Change and Logic Modeling: Matching Means and Ends

Integrating quantitative and qualitative approaches: an approach applied to the study of Intensive Care Presented by Ruth Kowalczyk.

Outputs, outcomes and impacts Using Theory of Change to plan, develop and evaluate your initiative.

The complex evaluation framework. 2 Simple projects, complicated programs and complex development interventions Complicated programs Simple projects blue.

Intelligence Step 5 - Capacity Analysis Capacity Analysis Without capacity, the most innovative and brilliant interventions will not be implemented, wont.

Evaluating health informatics projects Reasons for and problems of evaluation Objective model Subjective model.

Donald T. Simeon Caribbean Health Research Council

HANDLING DISCIPLINE & GRIEVANCES EMPLOYEE RIGHTS.

Program Evaluation and Measurement Janet Myers. Objectives for today… To define and explain concepts and terms used in program evaluation. To understand.

T EN QUESTIONS TO CONSIDER IN DESIGNING AN IMPACT EVALUATION.

Evaluation is a professional and ethical responsibility and is a core part of PHN professional practice Commitment to evaluation helps build the PHN intelligence.

PPA 502 – Program Evaluation

Person-centred Care & Patient Activation Richard Owen NHS England Dr Natalie Armstrong University of Leicester.

1 Qualitative Evaluation Terms Coding/categorization The process of condensing qualitative data through the identification of common themes. Data Matrix.

ISTEP: Technology Field Research in Developing Communities Instructor: M. Bernardine Dias CAs: Sarah Belousov and Ermine Teves Spring 2009.

Evidence based research in education Cathy Gunn University of Auckland.

The BC Clinical Care Management Initiative as a Case Study in Large Scale Change CARES International Conference on Realist Approaches, October 29,

EVALUATION APPROACHES Heather Aquilina 24 March 2015.

Implementation and process evaluation: developing our approach Ann Lendrum University of Manchester Neil Humphrey University of Manchester Gemma Moss Institute.

Assessment/Evaluation 1) Determine Desired Outcomes 2) Determine Activities to achieve outcome 3) Determine Indicators to measure whether you are making.

Impact Evaluation in Education Introduction to Monitoring and Evaluation Andrew Jenkins 23/03/14.

Impact evaluations: lessons from AFD’s experience Phnom Penh SKY evaluation meeting 4-5 October 2011.

 Welcome, introductions  Conceptualizing the evaluation problem  stakeholder interests  Divergent and convergent processes  Developing evaluation.

Begin at the Beginning introduction to evaluation Begin at the Beginning introduction to evaluation.

restricted external Evaluating the vinspired 24/24 programme Ewan King, director OPM 30 September

Workshop A. Development of complex interventions Rob Anderson, PCMD Nicky Britten, PCMD.

Improving skills and care standards in the support workforce: a realist synthesis of workforce development interventions Jo Rycroft-Malone, Christopher.

CHAPTER 7 Apply Phase. What is Application? The application of the theory to the problem, phenomenon or issue in the world of practice.

Theory of Change Articulating your project’s design theory.

Developing a Monitoring & Evaluation Plan MEASURE Evaluation.

Dr Ana School of Sociology & Social Policy University of Leeds- United Kingdom European Evaluation.

Monitoring and Evaluation Presentation by Kanu Negi Office of Development Effectiveness (ODE), DFAT and David Goodwins Coffey International Development.

Otago.ac.nz/uowsummerschool 10. Evaluating Health in All Policies: South Australia’s Experience Carmel Williams, Manager Strategic Partnerships Public.

Module 8 Guidelines for evaluating the SDGs through an equity focused and gender responsive lens: Overview Technical Assistance on Evaluating SDGs: Leave.

Assessing Impact: Approaches and Designs

Project monitoring and evaluation

MATERI #6 Proses Perancangan Intervensi

Making Causal Claims in Non-Experimental Settings

Seven day working: evaluating the impact of extending occupational therapy services for older adults in the acute setting.

Gathering a credible evidence base

Monitoring and Evaluation Frameworks

Workshop to develop theories of change

Technical Assistance on Evaluating SDGs: Leave No One Behind

Food and Agriculture Organization of the United Nations

Introduction to Comprehensive Evaluation

Corporate-level Evaluation of IFAD’s Decentralization Experience

Assess Plan Do Review Resource 1: Types of Evaluation – which type of approach should I use? There are a number of different types of evaluation, the most.

How do we know that two heads are better

Conducting Efficacy Trials

Measuring Outcomes of GEO and GEOSS: A Proposed Framework for Performance Measurement and Evaluation Ed Washburn, US EPA.

Program Evaluation Essentials-- Part 2

Development of the Evaluation of an Interprofessional Hotspotting Curriculum Charles Baron BA, Lauren Collins MD Background Evaluation Development Super-utilizers.

Logic Models and Theory of Change Models: Defining and Telling Apart

Resource 1. Evaluation Planning Template

THEORY OF CHANGE APPROACH

A3 – Improving State Level Supports and Stakeholder Engagement through Effective Evaluation Kim Gulbrandson, Justyn Poulos – Wisconsin RtI Center Key.

Evaluating agricultural value chain programs: How we mix our methods

How Effective are You? A study on how supervisory organizations in health care are concerned with the effectiveness of supervision Wouter Sparreboom.

CATHCA National Conference 2018

What is a Logic Model? A depiction of a program showing what the program will do and what it is to accomplish. A series of “if-then” relationships that.

Regulated Health Professions Network Evaluation Framework

Change Control Diagram

To RCT, or not RCT: that is the question

Changing the Game The Logic Model

Evaluation network MS - DG REGIO 14th April 2011, Kai Stryczynski

OGB Partner Advocacy Workshop 18th & 19th March 2010

What is a Logic Model? A depiction of a program showing what the program will do and what it is to accomplish. A series of “if-then” relationships that.

Root Cause Analysis Identifying critical campaign challenges and diagnosing bottlenecks.

Presentation transcript:

Introduction to Evaluation Simon Bishop

Why evaluate? Previous evaluations? Theory based evaluation Methods

Why evaluation? 3 good reasons To evidence progress towards desired outcomes To provide feedback to guide implementation To test and develop existing ideas/knowledge/theories (Kubish et al 1998) + Many other ‘less good’ ones!

Evaluations of Integrated Care Torbay Evaluation Kings Fund (2011) Longitudinal action research case study Qualitative and quantitative measures of process and some broad outcomes Overview of long term change Davey et al (2005) Comparative study of co-located and non co-located teams Standardized interviews Identify contextual factors that impact on team development and patient/client outcomes DoH Integrated Care Pilots - Rand / Ernst and Young - (2012) Comparison of 16 diverse care pilots Qualitative (‘living documents’ + Interviews) and quantitative (HES) Process improvements, but changes for patients mixed. Outcomes difficult to unpick Lessons from Sedgefield- Hudson (2002, 2007) Process of front line team integration Mixed qualitative methods Unpicks factors contributing to positive outcome for the teams,

Theory Based Evaluation Theory on relationship between programme, outputs and outcomes Different schools Theory driven evaluation Theory of change Realist evaluation Acknowledges complexity Take into account context Making theory explicit improves shared understanding Improves transferability of findings (as looks at relationship between conditions and outcomes)

‘Assumptions’ and ‘Mechanisms’ Bring to light the underlying assumptions about why it is believed that elements are linked (for example how actions linked to outcomes) Complex initiatives likely to involve a large number of assumptions Unpick intermediate steps in the (assumed/hypothetical) causal chain Impact of context on assumptions needs to be considered Help to identify links between causes and assumed effects

Theories of Change Identifying long-term goals Mapping and connecting outcomes Identifying Assumptions Developing Indicators Identifying Interventions

Why Integrate? What are the underlying assumptions/ mechanisms behind integrated approaches to care? What contextual factors are these assumptions dependent upon?

Methods of Data Collection Quantitative data Outcome Hospital Episode Statistics Health/wellbeing indicators Surveys Process Meetings Teamwork Qualitative Staff interviews Ethnographic methods Documentary evidence Reports / structured questions A combination of different methods usually required to understand the process and outcomes of change …. or no change