Other Quasi-Experimental Designs. Design Variations Show specific design features that can be used to address specific threats or constraints in the context.

Slides:



Advertisements
Similar presentations
EVAL 6970: Experimental and Quasi-Experimental Designs
Advertisements

8.2 PRE-EXPERIMENTAL DESIGNS (NONDESIGNS) One-Shot Case Study X O One-Group Pretest-Posttest Design O 1 X O 2 Intact-Group Comparison X O 1 ─ ─ ─ O 2.
Increasing your confidence that you really found what you think you found. Reliability and Validity.
Defining Characteristics
Types of Experimental Designs Non-experimental –Post-test only –Pre-post test Experimental –Pre & post-test with control group –Pre & multiple post-test.
Hybrid Experimental Designs. Hybrid Designs l Variations on randomized designs l Help to address specific threats l Incorporates different design features.
Randomized Experimental Design
CHAPTER OVERVIEW The Nonequivalent Control Group Design The Static Group Comparison Single-Subject Design Evaluating Single-Subject Designs.
Experimental Research Designs
Quasi-Experimental Design Ken Koedinger Based on Trochim Chpt 10.
Quasi-Experimental Design
Correlation AND EXPERIMENTAL DESIGN
Research Design: The Experimental Model and Its Variations
Research Design and Validity Threats
Wed Oct 29, 2003 Exams after class Mean(SD): 71.6(11.6)% Range: 44-95% Appointment Research day 1 week.
Statistical Analysis of the Regression Point Displacement Design (RPD)
Experimental Design.
Who are the participants? Creating a Quality Sample 47:269: Research Methods I Dr. Leonard March 22, 2010.
Experimental and Quasi- experimental Designs Having fun with X’s and O’s.
Types of Group Designs ____________ group design. The experiment compares groups that receive or _______________ the IV (control group) e.g., behavior.
Chapter 11 Quasi-Experimental Designs ♣ ♣ Introduction   Nonequivalent Comparison Group Design   Time-Series Design   Regression Discontinuity Design.
Quasi-Experimental Designs Whenever it is not possible to establish cause-and-effect relations because there is not complete control over the variables.
Chapter 9 Experimental Research Gay, Mills, and Airasian
McGraw-Hill © 2006 The McGraw-Hill Companies, Inc. All rights reserved. Experimental Research Chapter Thirteen.
Experimental Research
Utilizing PBS “Cyberchase” Math Computer Games with Struggling Math Students Michelle Brennan Education 7202T Seminar in Applied Theory and Research II.
EVALUATING YOUR RESEARCH DESIGN EDRS 5305 EDUCATIONAL RESEARCH & STATISTICS.
Experimental Research Take some action and observe its effects Take some action and observe its effects Extension of natural science to social science.
Experimental Design The Gold Standard?.
Quasi-Experiments. The Basic Nonequivalent Groups Design (NEGD) l Key Feature: Nonequivalent assignment NOXONOONOXONOO.
2.4. Design in quantitative research Karl Popper’s notion of falsification and science – If a theory is testable and incompatible with possible empirical.
Experimental and Quasi-Experimental Designs
Program Evaluation Using qualitative & qualitative methods.
I want to test a wound treatment or educational program in my clinical setting with patient groups that are convenient or that already exist, How do I.
Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning,
Quasi-Experimental Designs Manipulated Treatment Variable but Groups Not Equated.
Quasi Experimental Methods I Nethra Palaniswamy Development Strategy and Governance International Food Policy Research Institute.
Paul Bakker – Social Impact Squared
Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved.
URBDP 591 A Lecture 8: Experimental and Quasi-Experimental Design Objectives Basic Design Elements Experimental Designs Comparing Experimental Design Example.
URBDP 491 A Lecture 7: Research Approaches Objectives How to compare alternative approaches Experimental vs. non-experimental approaches Cross-sectional.
How can giving ELL students access to learning games on a computer help them learn in the classroom? By: Lisa Cruz.
Applying impact evaluation tools A hypothetical fertilizer project.
Chapter 10 Experimental Research Gay, Mills, and Airasian 10th Edition
How can giving ELL students access to learning games on a computer help them learn in the classroom? By: Lisa Cruz.
Nonexperimental and Quasi- Experimental Designs Distinction is the degree of control over internal validity.
Experimental & Quasi-Experimental Designs Dr. Guerette.
Research Design. Time of Data Collection Longitudinal Longitudinal –Panel study –Trend study –Cohort study Cross-sectional Cross-sectional.
Chapter 11.  The general plan for carrying out a study where the independent variable is changed  Determines the internal validity  Should provide.
 The basic components of experiments are: 1) taking action 2) observing the consequence of that action  Experimental model is most closely linked to.
Types of Experimental Designs (Educational research) True Experimental Quasi-Experimental.
Experiments Uniquely suited to identify cause-effect relationships To study effect of one variable (treatment) on another (outcome/dependent variable)
Evaluation Research Dr. Guerette. Introduction Evaluation Research – Evaluation Research – The purpose is to evaluate the impact of policies The purpose.
Journalism 614: Experimental Methods Experimental Research  Take some action and observe its effects –Extension of natural science to social science.
©2005, Pearson Education/Prentice Hall CHAPTER 6 Nonexperimental Strategies.
EMR 6550: Experimental and Quasi- Experimental Designs Dr. Chris L. S. Coryn Kristin A. Hobson Fall 2013.
Table of Contents  Research Design  Threats to Internal Validity  Threats to External Validity  Proposed Data  Pre-Test Post-Test Data Results 
William M. Trochim James P. Donnelly Kanika Arora 8 Introduction to Design.
Research Designs for Explanation Experimental, Quasi-experimental, Non-experimental, Observational.
Experimental and Quasi-Experimental Research
Experimental Research
Chapter 8 Experimental Design The nature of an experimental design
12 Inferential Analysis.
Research Methods: Concepts and Connections First Edition
Quasi-Experimental Design
ED 571 Introduction to quantitative research
The Nonexperimental and Quasi-Experimental Strategies
12 Inferential Analysis.
Non-Experimental designs: Correlational & Quasi-experimental designs
Reminder for next week CUELT Conference.
Presentation transcript:

Other Quasi-Experimental Designs

Design Variations Show specific design features that can be used to address specific threats or constraints in the context

Proxy Pretest Design l Pretest based on recollection or archived data l Useful when you weren’t able to get a pretest but wanted to address gain NO1XO2NO1O2NO1XO2NO1O2

Separate Pre-Post Samples l Groups with the same subscript come from the same context. l Here, N 1 might be people who were in the program at Agency 1 last year, with those in N 2 at Agency 2 last year. l This is like having a proxy pretest on a different group. N1ON1XON2ON2ON1ON1XON2ON2O

Separate Pre-Post Samples l Take random samples at two times of people at two nonequivalent agencies. l Useful when you routinely measure with surveys. l You can assume that the pre and post samples are equivalent, but the two agencies may not be. R1OR1XOR2OR2OR1OR1XOR2OR2O N N

Double-Pretest Design l Strong in internal validity l Helps address selection-maturation l How does this affect selection-testing? NOOXONOOONOOXONOOO

Switching Replications l Strong design for both internal and external validity l Strong against social threats to internal validity l Strong ethically NOXOONOOXONOXOONOOXO

Nonequivalent Dependent Variables Design (NEDV) l The variables have to be similar enough that they would be affected the same way by all threats. l The program has to target one variable and not the other. NO1XO1NO2O2NO1XO1NO2O2

NEDV Example l Only works if we can assume that geometry scores show what would have happened to algebra if untreated. l The variable is the control. l Note that there is no control group here.

NEDV Pattern Matching l Have many outcome variables. l Have theory that tells how affected (from most to least) each variable will be by the program. l Match observed gains with predicted ones. l If match, what does it mean?

NEDV Pattern Matching l A “ladder” graph. l What are the threats? r =.997

NEDV Pattern Matching l Single group design, but could be used with multiple groups (could even be coupled with experimental design). l Can measure left and right on different scales (e.g., right could be t-values). l How do we get the expectations?

Regression Point Displacement (RPD) l Intervene in a single site l Have many nonequivalent control sites l Good design for community-based evaluation N (n=1) OXO NOO

RPD Example l Comprehensive community-based AIDS education l Intervene in one community (e.g., county) l Have 29 other communities (e.g., counties) in state as controls l measure is annual HIV positive rate by county

RPD Example X Y

X Y Regression line

RPD Example X Y Regression line Treated community point

RPD Example X Y Regression pine Treated community point Posttest displacement