Program Evaluation The use of scientific methods to judge and improve the planning, monitoring, effectiveness, and efficiency of health, nutrition, and.

Slides:



Advertisements
Similar presentations
Pre-Experimental Designs
Advertisements

Evaluation Procedures
8.2 PRE-EXPERIMENTAL DESIGNS (NONDESIGNS) One-Shot Case Study X O One-Group Pretest-Posttest Design O 1 X O 2 Intact-Group Comparison X O 1 ─ ─ ─ O 2.
Defining Characteristics
Types of Experimental Designs Non-experimental –Post-test only –Pre-post test Experimental –Pre & post-test with control group –Pre & multiple post-test.
CHAPTER OVERVIEW The Nonequivalent Control Group Design The Static Group Comparison Single-Subject Design Evaluating Single-Subject Designs.
Research Design and Validity Threats
Wed Oct 29, 2003 Exams after class Mean(SD): 71.6(11.6)% Range: 44-95% Appointment Research day 1 week.
Experimental Design Week 9 Lecture 1.
Experimentation. What experiments are not: Two commonsense uses of “experiment”: – any study – to “try something out” – …both of these understandings.
Experimental Design.
Who are the participants? Creating a Quality Sample 47:269: Research Methods I Dr. Leonard March 22, 2010.
Quasi & Non-Experimental Designs
SINGLE CASE, QUASI- EXPERIMENT AND DEVELOPMENT RESEARCH.
Experimental and Quasi- experimental Designs Having fun with X’s and O’s.
TRAINING EVALUATION. STEPS TO EFFECTIVE TRAINING 1.Assess Needs 2.Design Training 3.Conduct/Deliver Training 4.Ensure Transfer –Support –Consequences.
Types of Group Designs ____________ group design. The experiment compares groups that receive or _______________ the IV (control group) e.g., behavior.
Evaluation Research COMT 502. topics Evaluation research –Process of determining whether intervention has its intended result.
RESEARCH METHODS Lecture 34. EXPERIMENTAL RESEARCH.
9 Quantitative Research Designs.
McGraw-Hill © 2006 The McGraw-Hill Companies, Inc. All rights reserved. Experimental Research Chapter Thirteen.
Experimental Research
Problem identification Research questions Constructs & Variables Research design.
Experimental Design The Gold Standard?.
I want to test a wound treatment or educational program but I have no funding or resources, How do I do it? Implementing & evaluating wound research conducted.
INTRODUCTION TO OPERATIONS RESEARCH Niranjan Saggurti, PhD Senior Researcher Scientific Development Workshop Operations Research on GBV/HIV AIDS 2014,
I want to test a wound treatment or educational program in my clinical setting with patient groups that are convenient or that already exist, How do I.
Research Design for Quantitative Studies
Learning Objectives 1 Copyright © 2002 South-Western/Thomson Learning Primary Data Collection: Experimentation CHAPTER eight.
Day 6: Non-Experimental & Experimental Design
Methodology Describe Context & setting Design Participants Sampling Power Analysi s Interventions Outcome (study variables) Data Collection Procedures.
Learning Objectives Copyright © 2004 John Wiley & Sons,Inc Primary Data Collection: Experimentation CHAPTER Seven.
Copyright ©2008 by Pearson Education, Inc. Pearson Prentice Hall Upper Saddle River, NJ Foundations of Nursing Research, 5e By Rose Marie Nieswiadomy.
Chapter 9 Causal Research Designs and Test Markets
Evaluating HRD Programs
Chapter Four Experimental & Quasi-experimental Designs.
Types of Research and Designs This week and next week… Covering –Research classifications –Variables –Steps in Experimental Research –Validity –Research.
Organizational Design, Diagnosis, and Development Session 15 The Evaluation of Change.
1 Experimental Research Cause + Effect Manipulation Control.
Training Evaluation- Design Issues Basic Designs –Post-test only –Pre-rest/Post-test Complex Designs –Post-test with control group –Pre-test/Post-test.
After giving this lecture the student should be able to do the following: After giving this lecture the student should be able to do the following: List.
Chapter 10 Common Experimental Research Designs
Chapter 10 Experimental Research Gay, Mills, and Airasian 10th Edition
Nonexperimental and Quasi- Experimental Designs Distinction is the degree of control over internal validity.
Quasi Experimental and single case experimental designs
Diagram of a quasi-experimental design with two groups
1 Module 3 Designs. 2 Family Health Project: Exercise Review Discuss the Family Health Case and these questions. Consider how gender issues influence.
Experimental & Quasi-Experimental Designs Dr. Guerette.
Quasi-Experimental Designs Slides Prepared by Alison L. O’Malley Passer Chapter 11.
Chapter 11.  The general plan for carrying out a study where the independent variable is changed  Determines the internal validity  Should provide.
SOCW 671: #6 Research Designs Review for 1 st Quiz.
Chapter 8 Experiments Topics Appropriate to Experiments The Classical Experiment Selecting Subjects Variations on Experimental Designs An Illustration.
The Experiment Chapter 7. Doing Experiments In Everyday Life Experiments in psychology use the same logic that guides experiments in biology or engineering.
Types of Experimental Designs (Educational research) True Experimental Quasi-Experimental.
IREL 561: Research Methods Fall 2013 Week 10 Based largely on Neuman’s Basics of Social Research, Chapter 8 Prepared by Craig Webster, Ph.D.
Experiments Uniquely suited to identify cause-effect relationships To study effect of one variable (treatment) on another (outcome/dependent variable)
Evaluation Research Dr. Guerette. Introduction Evaluation Research – Evaluation Research – The purpose is to evaluate the impact of policies The purpose.
Chapter Nine Primary Data Collection: Experimentation and
Outcomes Evaluation A good evaluation is …. –Useful to its audience –practical to implement –conducted ethically –technically accurate.
School of Public Administration & Policy Dr. Kaifeng Yang 研究设计 : 实验研究的基本问题.
Chapter 11 Experimental Designs PowerPoint presentation developed by: Sarah E. Bledsoe & E. Roberto Orellana.
Research designs Research designs Quantitative Research Designs.
Educational Research Experimental Research Chapter 9 (8 th Edition) Chapter 13 (7 th Edition) Gay and Airasian.
Chapter 9 Scrutinizing Quantitative Research Design.
Types of Designs.
Designing an Experiment
بسم الله الرحمن الرحیم.
The Nonexperimental and Quasi-Experimental Strategies
Types of Designs: R: Random Assignment of subjects to groups
Reminder for next week CUELT Conference.
Presentation transcript:

Program Evaluation The use of scientific methods to judge and improve the planning, monitoring, effectiveness, and efficiency of health, nutrition, and other human service programs

Why Evaluate a Program? See table 10-1, page 309, Boyles and Morris

Types of Program Evaluation Process evaluation Impact or outcome evaluation Fiscal or efficiency evaluation

Process Evaluation Evaluate process objectives Provides information for why program may or may not have reached its outcome objectives If program is delivered from a variety of sites, provides information on why some sites may have been more successful than others

Six Steps for Program Evaluation 1. Determine objectives of program Evaluate for: a. Appropriateness of objectives b. Effectiveness in meeting objectives c. Efficiency of program d. Side effects of program

Steps in Program Evaluation 2. Determine characteristics to be measured Measurements should be: Valid Reliable Precise

Steps in Program Evaluation 3. Measure characteristics

Steps in Program Evaluation 4. Make comparisons May use: Control groups Similar groups Standards Pre vs post measurements

Steps in Program Evaluation 5. Draw conclusions 6. Make recommendations

Common Biases Introduced During Evaluations Selection Testing History Maturation Halo effect

Evaluation Design 1. Experimental design 2. Quasi-experimental design 3. Non-experimental design

Steps for experimental design 1. Experimental and control groups randomly assigned 2. Each group measured 3. Intervention or program provided 4. Groups measured again--if experimental group improved more than control, program was successful

Examples of Designs of True Experiments Pre-test post-test control group design R O X O R O O

Examples of Designs of True Experiments After only control group R XO R O

Examples of Experimental Design Solomon 4 group R O X O R O O R X O R O

Quasi-experimental design Steps similar to experimental, but rigid control not met. Random selection may not be done Subjects may be volunteers Nonequivalent control groups may be used

Nonexperimental design Random selection not used No control group or nonequivalent control group used

Examples of Non-experimental design After only or one-shot case study X O Nonequivalent control group study X O O Pre-test-Post-test design O X O

Fiscal or Efficiency Evaluations Cost-benefit analysis Cost-effectiveness analysis

Cost benefit analysis Decision making framework used in allocating resources among competing uses. Both costs and benefits are expressed in dollars

Costs Direct Costs --Cash expenditures Indirect Costs –All other costs such as –Spillover effects –Costs to client –Costs to organization not covered by program Opportunity costs Intangible costs--grief, suffering pain

Benefits All costs that would be avoided if the program were in effect Direct benefits--values of resources which the program saves Negative benefits Indirect benefits--other costs averted Intangible benefits--happiness, bonding from breastfeeding

Discount rate Based on deferred benefits

Cost-Benefit Analysis of Attending School Direct costs Indirect costs Intangible costs Direct benefits Indirect benefits Intangible benefits Discount rates

Cost effectiveness analysis Determines the most efficient way of meeting a predetermined set of objectives Costs measured in dollars Effectiveness measured by outcomes, e.g.. lives saved, increase in birth weight, etc

Communicating Evaluation Results See pages