Program Evaluation: What is it?

Slides:



Advertisements
Similar presentations
Program Evaluation Alternative Approaches and Practical Guidelines
Advertisements

Focusing an Evaluation Ben Silliman, Youth Development Specialist NC 4-H Youth Development.
Professional Learning Communities Connecting the Initiatives
Introduction to Monitoring and Evaluation
S EPTEMBER SESSION 1: R ESEARCH Q UESTIONS & S ELECTING M EASURES The first steps toward getting your EA research project underway.
Overview M&E Capacity Strengthening Workshop, Maputo 19 and 20 September 2011.
Catherine Kost Heather McGreevy E VALUATING Y OUR V OLUNTEER P ROGRAM.
What You Will Learn From These Sessions
Designing an Effective Evaluation Strategy
Program Evaluation and Measurement Janet Myers. Objectives for today… To define and explain concepts and terms used in program evaluation. To understand.
Laura Pejsa Goff Pejsa & Associates MESI 2014
Chapter 2 Flashcards.
Impact and outcome evaluation involve measuring the effects of an intervention, investigating the direction and degree of change Impact evaluation assesses.
Volunteer Recognition Honoring and recognizing individuals for their unique contribution to educational program efforts Honoring and recognizing individuals.
Title I Needs Assessment and Program Evaluation
Evaluation. Practical Evaluation Michael Quinn Patton.
Measuring Learning Outcomes Evaluation
Choosing Your Primary Research Method What do you need to find out that your literature did not provide?
Assessing Financial Education: A Practitioner’s Guide December 2010.
1 Classroom-Based Research: How to Be a Researcher in Your Classroom Basic Skills Initiative Teaching and Learning Workshop October 2009 Darla M. Cooper.
How to Develop the Right Research Questions for Program Evaluation
Chapter 4 Principles of Quantitative Research. Answering Questions  Quantitative Research attempts to answer questions by ascribing importance (significance)
REC 375—Leadership and Management of Parks and Recreation Services Jim Herstine, Ph.D., CPRP Assistant Professor Parks and Recreation Management UNC Wilmington.
Product Evaluation the outcome phase. Do the magic bullets work? How do you know when an innovative educational program has “worked”? How do you know.
McGraw-Hill © 2006 The McGraw-Hill Companies, Inc. All rights reserved. The Nature of Research Chapter One.
Qualitative Research.
Evaluation Basics Principles of Evaluation Keeping in mind the basic principles for program and evaluation success, leaders of youth programs can begin.
Assessment 101 Center for Analytics, Research and Data (CARD) United Church of Christ.
Too expensive Too complicated Too time consuming.
Monitoring and Evaluation in MCH Programs and Projects MCH in Developing Countries Feb 10, 2011.
Fundamentals of Evaluation for Public Health Programs ROBERT FOLEY, M.ED. NIHB TRIBAL PUBLIC HEALTH SUMMIT MARCH 31,
Elizabeth Godfrey 1.  Periodic assessment of results Appropriateness, effectiveness, efficiency, impact, sustainability  Identifies intended and unintended.
Semester 2: Lecture 9 Analyzing Qualitative Data: Evaluation Research Prepared by: Dr. Lloyd Waller ©
Overview of Evaluation Designs. Learning objectives By the end of this presentation, you will be able to: Explain evaluation design Describe the differences.
1 PROJECT EVALUATION IT’S ALL ABOUT STUDENTS. 2 In partnership, we help America’s students stay in school and graduate by: Reducing gaps in college access.
Introduction to research methods 10/26/2004 Xiangming Mu.
Quantitative and Qualitative Approaches
Developing an Assessment System B. Joyce, PhD 2006.
Community Planning Training 5- Community Planning Training 5-1.
Program Evaluation.
“A Truthful Evaluation Of Yourself Gives Feedback For Growth and Success” Brenda Johnson Padgett Brenda Johnson Padgett.
QUANTITATIVE RESEARCH Presented by SANIA IQBAL M.Ed Course Instructor SIR RASOOL BUKSH RAISANI.
Monitoring and Evaluation in MCH Programs and Projects MCH in Developing Countries Feb 24, 2009.
 2007 Johns Hopkins Bloomberg School of Public Health Introduction to Program Evaluation Frances Stillman, EdD Institute for Global Tobacco Control Johns.
Evaluation Designs Adrienne DiTommaso, MPA, CNCS Office of Research and Evaluation.
Adrienne DiTommaso, MPA, CNCS Office of Research and Evaluation
Developing SEA Change’s Evaluation Plan
1 The project is financed from the European Union funds within the framework of Erasmus+, Key Action 2: Cooperation for innovation and the exchange of.
Program Evaluation Principles and Applications PAS 2010.
Evaluation: from Objectives to Outcomes Janet Myers, PhD MPH AIDS Education and Training Centers National Evaluation Center
Monitoring and Evaluation in MCH Programs and Projects MCH in Developing Countries Feb 9, 2012.
Introduction to Research for Physical Therapy Students.
Data Driven Planning and Decision-making for Continuous School Improvement: Developing a Program Evaluation Plan Leadership for Innovative Omani Schools.
Basic Concepts of Outcome-Informed Practice (OIP).
Session 13: Monitoring and Evaluation PubH325 Global Social Marketing Donna Sherard, MPH November 30, 2009.
Are we there yet? Evaluating your graduation SiMR.
4 th Biennial SAMEA Conference Meaningful Evaluation: Improving Use and Results Evaluation of Teacher-Directed ICT Initiatives Presenter Mokete Mokone.
Research And Evaluation Differences Between Research and Evaluation  Research and evaluation are closely related but differ in four ways: –The purpose.
Assessment/Evaluation Make evaluation a central part of planning – not an afterthought 1) Determine Needs 2) Determine Desired Outcomes 3) Determine Activities.
Evaluation Nicola Bowtell. Some myths about evaluation 2Presentation title - edit in Header and Footer always involves extensive questionnaires means.
PROGRAM EVALUATION A TOOL FOR LEARNING AND CHANGE.
Part 1: Program Evaluation: What is it?
Designing Effective Evaluation Strategies for Outreach Programs
Chapter 2 Sociological Research Methods
Program Evaluation Essentials-- Part 2
What is a Logic Model? A depiction of a program showing what the program will do and what it is to accomplish. A series of “if-then” relationships that.
Group Experimental Design
What is a Logic Model? A depiction of a program showing what the program will do and what it is to accomplish. A series of “if-then” relationships that.
Data for PRS Monitoring: Institutional and Technical Challenges
Presentation transcript:

Program Evaluation: What is it? Brad Rose, PhD. Program Development & Funding Data Collection & Outcome Measurement Feedback & Continuous Improvement Impact & Assessment Reporting

Definition Program Evaluation Is an applied research process Systematically collects, analyzes and interprets data Addresses activities, characteristics, and outcomes of programs

Goal of Program Evaluation To assist stakeholders in making data-informed judgments about a specific program: Impact Effectiveness Value

How Does Evaluation Differ from Research? Research is an investigation that seeks to find out what is. Evaluation is an investigation into how, why, and to what extent objectives or goals are achieved. Evaluation is research that compares what is with what should be. It makes a judgment against criteria, expectations, standards.

What is a Program? Structured, intentional, intervention to improve the well-being of people, groups, organizations, or communities General effort that mobilizes staff and resources toward some defined and funded goals Programs vary in size, scope, duration, clarity, and specificity of goals

Basic Purposes/Kinds of Evaluations Formative Evaluation Program improvement and strengthening. Summative Evaluation (Outcome Evaluation) Ultimate value, merit and impact. Should the program continue? Be refunded, etc.?

10 Evaluation Questions General questions to consider when conducting a program evaluation: Why is the evaluation being conducted? What is the "it" that is being evaluated?   What are the intended outcomes of the program? What is the program meant to achieve? What are the key questions to be answered?   Who is the evaluation for?

10 Evaluation Questions (cont.) Which information will stakeholders find useful? How will evaluation findings be used?   What will be the “product” of the evaluation? What are the potential sources of information data? What is the optimal design for the evaluation? Which methods will yield the most valid, accurate, and persuasive evaluation findings and conclusions?

Typical Methods and Tools Interviews (Individual and Focus Group) Surveys Observations Review of existing data Collection and statistical analysis of quantitative data

Evaluation Design Quantitative Non-experimental design: Pre- and post-test, “a single group interrupted time series” (Observation Treatment Observation) Experimental design: Compare outcomes among “treatment” and control groups Random Observation Treatment Observation Observation No Treatment Observation

Evaluation Design Qualitative Interviews/focus groups with participants (staff and community members) Observations of program activities Document analysis Case studies And more…

Evaluation Design Form follows function Design follows questions The key to evaluation design The evaluation design should be determined by the kind of questions you want to answer.

Review: The Fundamental Questions What will be changed or different as a result of the operation of the program? Attitudes Knowledge Behavior Competencies/Skills What will success “look like”? How will we show that changes occurred? (i.e., which measures/indicators?)

“If you can‘t say it clearly, you don’t understand it yourself” -- John Searle “Things should be made as simple as possible, but not any simpler.” -- Albert Einstein “82.7% of all statistics are made up on the spot.” --Steven Wright

Questions? On-line Resources: www.bradroseconsulting.com  617-412-4709