Program evaluation Sebastian Uijtdehaage 310.794.9009.

Slides:

Advertisements

Similar presentations

Program Evaluation: What is it?

Advertisements

STRATEGIES FOR COURSE REDESIGN EVALUATION Laura M. Stapleton Human Development and Quantitative Methodology University of Maryland, College Park

Introduction to Impact Assessment

Introduction to Monitoring and Evaluation

Using Logic Models in Program Planning and Grant Proposals The Covenant Foundation.

Overview M&E Capacity Strengthening Workshop, Maputo 19 and 20 September 2011.

Slide 1 Change Management Change is a process …. Where are you today Where you want to be Process of change Time.

MERC Ten Steps to Designing an Evaluation for Your Educational Program Linda Perkowski, Ph.D. University of Minnesota Medical School.

Program Evaluation and Measurement Janet Myers. Objectives for today… To define and explain concepts and terms used in program evaluation. To understand.

Getting on the same page… Creating a common language to use today.

Program Evaluation In A Nutshell 1 Jonathan Brown, M.A.

Introduction to Assessment Jennifer Lowman Coordinator, Student Persistence Research Credit for the content of this presentation should go to Marilee Bresciani.

Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA.

How to Develop the Right Research Questions for Program Evaluation

+ STARS Evaluation Assistant Webinar 1 September 19, 2014 Evaluation Projects.

Step 2. Describe Program Logic Models & Outcome Measurement

Impact Evaluation: Initiatives, Activities, & Coalitions Stephen Horan, PhD Community Health Solutions, Inc. September 12, 2004.

This project is funded by the EUAnd implemented by a consortium led by MWH Logical Framework and Indicators.

IntendedActual IntendedActual IMPLEMENTATION EVALUATION Training for Teachers in Artful Learning & Character Ed. Training for Teaching Artists Programming.

Evaluation – the why’s, the what’s and the how’s 2014 Dr Basma Ellahi (PI) Cat Crum (PhD Student)

Program Evaluation & Research In Service-Learning Service-Learning Mini-Institute Lynn E. Pelco, Ph.D. Division of Community Engagement.

Too expensive Too complicated Too time consuming.

Rethinking Homelessness Their Future Depends on it!

Semester 2: Lecture 9 Analyzing Qualitative Data: Evaluation Research Prepared by: Dr. Lloyd Waller ©

Contribution Analysis Made Do-Able Rod Laird Organisation 18 September 2007 Euston, London Mike Hendricks, Ph.D.

Logic Models and Theory of Change Models: Defining and Telling Apart

Julie R. Morales Butler Institute for Families University of Denver.

Nancy L. Weaver, PhD, MPH Department of Community Health School of Public Health Saint Louis University 16 July 2010 LOGIC MODEL FUNDAMENTALS.

Integrating Knowledge Translation and Exchange into a grant Maureen Dobbins, RN, PhD SON, January 14, 2013.

Copyright © 2014 by The University of Kansas Refining the Program Intervention Based on Research.

Dr. Shulagna Sarkar Assistant Professor , Jt. Coordinator – PGDM – HRM

EVALUATION OF HRD PROGRAMS Jayendra Rimal. The Purpose of HRD Evaluation HRD Evaluation – the systematic collection of descriptive and judgmental information.

The Major Steps of a Public Health Evaluation 1. Engage Stakeholders 2. Describe the program 3. Focus on the evaluation design 4. Gather credible evidence.

Katherine Perdomo Alanna Pugliese CATEGORIES OF EVALUATION.

Using Logic Models in Program Planning and Grant Proposals The Covenant Foundation.

 2007 Johns Hopkins Bloomberg School of Public Health Introduction to Program Evaluation Frances Stillman, EdD Institute for Global Tobacco Control Johns.

Does Your Academic Program Make a Difference? Knowing What Your Students Learned Rosslyn M. Smith (moderator) Katherine A. Austin Valerie Osland Paton.

Texas Homeless Education Office The University of Texas at Austin Charles A. Dana Center 2901 N IH 35,

Logic Models: Laying the Groundwork for a Comprehensive Evaluation Office of Special Education Programs Courtney Brown, Ph.D. Center for Evaluation & Education.

Program Evaluation Overview. Definitions of Program Evaluation systematic collection of information abut the activities, characteristics, and outcome.

Support the spread of “good practice” in generating, managing, analysing and communicating spatial information Evaluating and Reflecting on the Map-making.

Planning Evaluation Setting the Course Source: Thompson & McClintock (1998) Demonstrating your program’s worth. Atlanta: CDC/National Center for Injury.

 2007 Johns Hopkins Bloomberg School of Public Health Section B Logic Models: The Pathway Model.

Basic Program Logic RESOURCES/INPUTS The ingredients you need to implement your program! YOUR PROGRAM What you do to achieve your departmental goals! RESULTS/IMPACT.

Assessment Small Learning Communities. The goal of all Small Learning Communities is to improve teaching, learning, and student outcomes A rigorous, coherent.

Logic modeling. “Would you tell me, please, which way I ought to go from here?” “That depends a good deal on where you want to get to.” said the Cat.

Kaplan University Writing CenterSaturday, March 05, IX520 Needs Assessment and Evaluation in Education and Training Unit 7 Seminar September 13,

Are we there yet? Evaluating your graduation SiMR.

Introduction to Monitoring and Evaluation. Learning Objectives By the end of the session, participants will be able to: Define program components Define.

.  Evaluators are not only faced with methodological challenges but also ethical challenges on a daily basis.

Introduction to Group Work. Learning Objectives The goal of the group project is to provide workshop participants with an opportunity to further develop.

Assessment/Evaluation Make evaluation a central part of planning – not an afterthought 1) Determine Needs 2) Determine Desired Outcomes 3) Determine Activities.

Planning Planning is considered the most important element of the administrative process. The higher the level of administration, the more the involvement.

ADRCs Do What? Using Logic Models to Document and Improve ADRC Outcomes Glenn M. Landers.

Logic Models How to Integrate Data Collection into your Everyday Work.

Development/Succession Planning Program

Designing Effective Evaluation Strategies for Outreach Programs

Mathematics Assessment (Math 321 & Math 334)

Using Logic Models in Program Planning and Grant Proposals

Introduction to Program Evaluation

Short term Medium term Long term

Strategic Prevention Framework - Evaluation

Determine Grant Impacts through Ripple Effects Mapping

Logic Models and Theory of Change Models: Defining and Telling Apart

FRAMEWORK OF EVALUATION QUESTIONS

Logic modeling.

Performance and Quality Improvement

Using Logic Models in Project Proposals

Framing Grants for policy Research

Presentation transcript:

Program evaluation Sebastian Uijtdehaage

Why do you evaluate?

Five reasons of evaluating 1.FORMATIVE informs program development 2.SUMMATIVE demonstrate program’s success and impact 3.UNDERSTANDING how and why did program work? 4.DISSEMINATION sharing best practices 5.RECOGNITION scholarship, academic credit, leadership Stufflebeam 2003

What type of evaluation should I conduct?

Outcome & Impact Did it work?

Dan Stufflebeam: CIPP Model of Program Evaluation Needs Assessment What should we do? Program Planning How shall we do it? Implementation Are we doing as planned? Outcome & Impact Does it work? INPUT CONTEXT PROCESS PRODUCT

What type of evaluation should I conduct? Formative or Summative?

formative & summative evaluation Formative Evaluation ContextWhat needs to be done? InputHow should it be done? ProcessIs it being done? ProductIs it working? Stufflebeam 2003

formative & summative evaluation Formative EvaluationSummative Evaluation ContextWhat needs to be done? Were important needs addressed? InputHow should it be done? Was program plan defensible? ProcessIs it being done?Was the plan well executed? ProductIs it working?Did the effort make a difference? Stufflebeam 2003

What type of evaluation should I conduct? Quantitative or Qualitative?

Quantitative or qualitative measures? Quantitative measures HOW MUCH…? How much did the participant change? How much did the community change? Qualitative measures WHY and HOW? Why did the program work? How did participants change as result?

What makes good evaluation?

Evaluation must be planned ahead Evaluation should be part of program planning Planning and resource allocation makes evaluation more successful – Example: long-term tracking of participants

Confidence being admitted in a medical school

What else makes good evaluation? Scientific rigor Evaluation produces useful information that helps make decisions – Assessment should be done with a particular purpose in mind Evaluation is tailored to your program objectives Evaluation is done with conceptual framework in mind

LOGIC MODEL ResourcesActivitiesOutputOutcomesImpact The Planned Work The Intended Results CONDITIONS and DRIVERS

LOGIC MODEL CONDITIONS and DRIVERS Current situation that precipitates the need for the program

LOGIC MODEL ResourcesActivitiesOutputOutcomesImpact CONDITIONS and DRIVERS Resources and inputs needed to operate the program: personnel, facilities, materials, equipment

LOGIC MODEL ResourcesActivitiesOutputOutcomesImpact CONDITIONS and DRIVERS Everything the program does if it had access to the resources

LOGIC MODEL ResourcesActivitiesOutputOutcomesImpact CONDITIONS and DRIVERS The “deliverables”—the events, products and services Number of participants, classes, brochures, advisors, etc The “deliverables”—the events, products and services Number of participants, classes, brochures, advisors, etc

LOGIC MODEL ResourcesActivitiesOutputOutcomesImpact CONDITIONS and DRIVERS If the program delivers, the participants will benefit in certain ways: Knowledge, skills, behavior, attitudes Immediate, mid-term and long-term outcomes If the program delivers, the participants will benefit in certain ways: Knowledge, skills, behavior, attitudes Immediate, mid-term and long-term outcomes

LOGIC MODEL ResourcesActivitiesOutputOutcomesImpact CONDITIONS and DRIVERS If these benefits to participants are achieved, then certain changes in organizations and communities might be expected to occur

LOGIC MODEL ResourcesActivitiesOutputOutcomesImpact The Planned Work The Intended Results CONDITIONS and DRIVERS

LOGIC MODEL Links processes and outcomes: – Funders love this! Great for program planning – Work backwards Evaluation is built in! Ain’t perfect

Your turn…