Chapter 12: Quasi-Experimental Designs

Slides:



Advertisements
Similar presentations
Non-Experimental designs: Developmental designs & Small-N designs
Advertisements

Defining Characteristics
Non-experimental Designs
GROUP-LEVEL DESIGNS Chapter 9.
Group Discussion Describe the fundamental flaw that prevents a nonequivalent group design from being a true experiment? (That is, why can’t these designs.
Experimental Research Designs
©2005, Pearson Education/Prentice Hall CHAPTER 5 Experimental Strategies.
Correlation AND EXPERIMENTAL DESIGN
Issues in Research Reliability: The consistency and stability of a measure or score. Validity: the extent to which a measure actually measures what it.
Quasi-Experimental Designs
Who are the participants? Creating a Quality Sample 47:269: Research Methods I Dr. Leonard March 22, 2010.
Quasi & Non-Experimental Designs
SINGLE CASE, QUASI- EXPERIMENT AND DEVELOPMENT RESEARCH.
Non-Experimental designs: Developmental designs & Small-N designs
Non-Experimental designs: Developmental designs & Small-N designs
Chapter 13: Descriptive and Exploratory Research
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Chapter 10 Using Specialized Research Designs.
Questions What is the best way to avoid order effects while doing within subjects design? We talked about people becoming more depressed during a treatment.
PSYC512: Research Methods PSYC512: Research Methods Lecture 15 Brian P. Dyre University of Idaho.
Non-Experimental designs
Group Discussion Describe the similarities and differences between experiments , non-experiments , and quasi-experiments. Actions for Describe the similarities.
PSYC512: Research Methods PSYC512: Research Methods Lecture 14 Brian P. Dyre University of Idaho.
Chapter 9 Experimental Research Gay, Mills, and Airasian
McGraw-Hill © 2006 The McGraw-Hill Companies, Inc. All rights reserved. Experimental Research Chapter Thirteen.
Experimental Research
Basic Principles of Research Design
METHODS IN BEHAVIORAL RESEARCH NINTH EDITION PAUL C. COZBY Copyright © 2007 The McGraw-Hill Companies, Inc.
Chapter 8 Experimental Research
Experimental Design The Gold Standard?.
I want to test a wound treatment or educational program but I have no funding or resources, How do I do it? Implementing & evaluating wound research conducted.
Research Methods in Psychology
Quantitative Research Designs
Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning,
Chapter 11 Experimental Designs
Chapter 8 Experimental Design: Dependent Groups and Mixed Groups Designs.
SINGLE - CASE, QUASI-EXPERIMENT, AND DEVELOPMENT RESEARCH © 2012 The McGraw-Hill Companies, Inc.
Chapter 3 Research Methods Used to Study Child Behavior Disorders.
Chapter 2 Research in Abnormal Psychology. Slide 2 Research in Abnormal Psychology  Clinical researchers face certain challenges that make their investigations.
Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved.
URBDP 591 A Lecture 8: Experimental and Quasi-Experimental Design Objectives Basic Design Elements Experimental Designs Comparing Experimental Design Example.
1 Experimental Research Cause + Effect Manipulation Control.
Single-Subject Experimental Research
Introduction section of article
Chapter 6 Research Validity. Research Validity: Truthfulness of inferences made from a research study.
Chapter 10 Experimental Research Gay, Mills, and Airasian 10th Edition
Chapter 10 Finding Relationships Among Variables: Non-Experimental Research.
Copyright © 2011 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin Using Specialized Research Designs.
Quasi Experimental and single case experimental designs
Experimental Research Methods in Language Learning Chapter 4 Experimental Research Designs.
Quasi-Experimental Designs Slides Prepared by Alison L. O’Malley Passer Chapter 11.
Chapter 11.  The general plan for carrying out a study where the independent variable is changed  Determines the internal validity  Should provide.
Slides to accompany Weathington, Cunningham & Pittenger (2010), Chapter 15: Single-Participant Experiments, Longitudinal Studies, and Quasi-Experimental.
©2005, Pearson Education/Prentice Hall CHAPTER 6 Nonexperimental Strategies.
CJ490: Research Methods in Criminal Justice UNIT #4 SEMINAR Professor Jeffrey Hauck.
1. /32  A quasi-experimental design is one that looks like an experimental design but lacks the key ingredient -- random assignment. 2.
© 2011 Pearson Education, Inc. All rights reserved. This multimedia product and its contents are protected under copyright law. The following are prohibited.
Educational Research Experimental Research Chapter 9 (8 th Edition) Chapter 13 (7 th Edition) Gay and Airasian.
Experimental Design Ragu, Nickola, Marina, & Shannon.
CHAPTER 13: THE NUTS AND BOLTS OF QUASI- EXPERIMENTS.
8 Experimental Research Design.
Chapter 11: Quasi-Experimental and Single Case Experimental Designs
Experimental Research Designs
Chapter 8 Experimental Design The nature of an experimental design
Ron Sterr Kim Sims Heather Cruz aka “The Carpool”
Introduction to Design
Threats to Internal Validity
The Nonexperimental and Quasi-Experimental Strategies
Chapter 11 EDPR 7521 Dr. Kakali Bhattacharya
Non-Experimental designs: Correlational & Quasi-experimental designs
Types of Designs: R: Random Assignment of subjects to groups
Presentation transcript:

Chapter 12: Quasi-Experimental Designs When researchers can not manipulate the independent variable, rather it is a grouping variable (gender, age, disability) and equivalence between the groups can not ensured Researchers can not randomly assign participants to groups thus lack control over extraneous variables Quasi-independent variable: is not a true independent variable but usually occurs naturally or can not be manipulated. Researchers still look for effect of the quasi-independent variable.

Types of Quasi-Experimental Designs Quasi-experimental designs usually have lower internal validity than true experiments. Types of Quasi-Experimental Designs Pretest-Postest designs Test participants before an after the quasi-independent variable One group: measure participants before and after the quasi-independent variable. Only have one group of participants (those that experienced the quasi-independent variable) Test reading before children at school X start reading program and then test their reading after they finish the reading program. O1 X O2

Threats to internal validity Maturation: students may have matured over the reading program. They may be better at reading just because of time and not due to the program. History Effects: something other than the independent variable may have occurred between the pretest and posttest. Pretest sensitization: taking the pretest may change the participants reaction to the posttest. Regression to the mean: Tendency for extreme scores on pretest to regress (move towards) the mean on a subsequent test (posttest).

If participants are selected because they have extreme scores on the pretest (e.g. select a school with very poor reading ability) there may be other factors due to measurement error that resulted in such low scores at the pretest (tired, bad day etc.) that may have slightly deflated their scores. Measurement error causes extreme scores to be biased in the extreme direction (away form the mean). So when you test them a second time it is unlikely that you will have those same factors that may have deflated their scores and their scores will increase a bit and make it look like the program is have an effect.

Nonequivalent Control Group Design: We cannot randomly assign participants to control and study group, so we select a control group that is similar to the group that gets the quasi-independent variable. Posttest-only: measure both groups after one group has received the treatment. Measure reading in School A and School B after School A has participated in the reading program. Quasi-experimental group X O Nonequivalent control group -- O Selection bias: we do not know whether the two groups were similar before the intervention

Pretest-Posttest design: Test both groups before one group gets the intervention, then test both groups again after one group gets the intervention (quasi-independent variable) Quasi-experimental group O1 X O2 Nonequivalent control group O1 -- O2 Allows researchers to see if the two groups scored similarly on the dependent variable before the introduction of the treatment. To determine if the quasi-independent variable had an effect you want scores to change between pretest and posttest ONLY for the Quasi-experimental group and NOT for the Nonequivalent control group.

Simple interrupted time series design Time Series Designs Measure the dependent variable many times before and after the quasi-independent variable is introduced. Simple interrupted time series design Researchers make a series of observations of the dependent variable before and after the treatment is introduced. O1 O2 O3 O4 X O5 O6 O7 O8 Evidence for a treatment effect occurs when there are abrupt changes in the time-series data at the time the treatment was implemented.

Percentage of cavities after the introduction of fluoride into toothpaste in 1970

Percentage of cavities after the introduction of fluoride into toothpaste in 1970

Percentage of cavities after the introduction of fluoride into toothpaste in 1970

This design helps to distinguish changes due to maturation from the quasi-independent variable Contemporary History: Observed effect could still be due to another event that occurred at the same time as the quasi-independent variable Perhaps the electric toothbrush was introduced in 1970, or there was a major TV add campaign that promoted brushing teeth.

Interrupted time series with a reversal Researchers measures the dependent variable before and after the treatment is introduced and then again after the treatment is removed O1 O2 O3 O4 X O5 O6 O7 O8 -X O9 O10 O11 We can see what happens to the dependent variable after the quasi independent variable is introduced and then again after it is removed. If the quasi-independent variable was really having an effect we would expect performance to change back to normal after it is removed

Percentage of cavities after the introduction of fluoride into toothpaste in 1970

Limitations: Researchers may not have the ability to remove the quasi-independent variable remove fluoride from toothpaste, remove a seatbelt law Some effects of the quasi-independent variable may remain even after it is removed If you did a time series study before and after introduction of reading program, and then removed program, reading may not decrease, the children may not regress because they did learn to read. Removal of the quasi-independent variable may produce effects that are not due to the quasi-independent variable

Control Group Interrupted time series Measure more than one group on several occasions, but only one group receives the quasi-independent variable. O1 O2 O3 O4 X O5 O6 O7 O8 O1 O2 O3 O4 -- O5 O6 O7 O8 Helps to rule out history effects, and we can be more certain the a change was due to X rather than an outside influence. Could still have a local history effect.

Time serves as the quasi-independent variable Longitudinal Designs Time serves as the quasi-independent variable Commonly used in developmental research Allows researchers to eliminate generational effects (when effects differ depending on the era in which people grew up). In longitudinal research you are studying people of the same generation over time. O1 O2 O3 O4 O5 O6 O7 O8 Allows researchers to examine how individuals change with age (not just group differences).

Limitations: Difficult to obtain samples who are willing to be repeatedly tested over time. Difficult to keep track of participants over time (attrition may occur). Takes a lot of time and money.

Program Evaluation Used to assess effectiveness of interventions (or programs) and provide feedback to the administrators Assess needs, process, outcome, and efficiency of social services. Considered applied research.

Evaluating Quasi-Experimental Designs The presumed cause usually precedes the presumed effect. These designs do allow researchers to determine if the two variables covary together. BUT they can not eliminate effects of extraneous variables and ensure randomization.

Chapter 13: Single-Case Research Examine individual participants rather than a group of participants. Idiographic approach: describe, analyze, and compare individual behavior Nomothetic approach: Describe, analyze, and compare behavior across individuals and make broad generalizations to a group. Many research areas started in single case-research (Ebbinhgaus’s memory research on himself, Skinner and Pavlov research)

Three main criticisms of group designs Error variance: error variance in group data does not always reflect variability in behavior (rather is due to the design). Group designs examine inter-participant variance which is across or between participants (individual differences) Rather single-case researchers emphasize intra-participant variance which is variability in an individual’s behavior. Generalizability: single case researchers suggest that group designs have limited generalizability.

Group designs usually reflect an average of the participants behavior which may not represent the response of any particular participant. Average number of children adults have 2.1 Average anxiety score is 10 (but most people may be at either high or low end 3-4 and 17-18). Reliability: group designs may test an effect once, but do not always replicate it to see if the effect holds up and is reliable. Single-case researchers often test an effect in the same participant a few times (intraparticipant replication) or determine whether the same effect is found in a few other participants (interparticipant replication)

Single-Case Experimental Designs ABA design: observe participants in absence of independent variable, baseline (A), then introduce independent variable, experimental period (B), then remove independent variable and observe behavior (A). Sometimes called a reversal design Difficult to determine if some other event occurring at the same time as the independent variable resulted in the effect The independent variable may produce permanent changes in a participants behavior, so it may not go back to baseline.

Present varying nonzero levels of the independent variable Multiple-I Designs Present varying nonzero levels of the independent variable ABC design: baseline (A), introduce IV (B), then remove this IV and introduce another level of the IV (C). ABACA: have baseline condition between each level of the independent variable Multiple Baseline Designs: two or more behaviors are examined simultaneously. After the baseline data, the researcher examines the effect of the independent variable on both behaviors. Usually test to see if independent variable affects the hypothesized behavior.

Data Analysis Graphic analysis: display all of a participants data points (before and after independent variable) on a graph. Visually examine the graph to determine whether it looks like the independent variable produces an effect. Uses of single-case designs: Conditioning research (reinforcement and punishment effect) Behavior modification techniques (phobias) Demonstrational purposes.

Critiques of single-case research Not necessarily generalizable because the participant is not usually chosen at random Difficult to study interactions among variables Ethical issues in ABA designs when the researcher may remove a very helpful treatment

Case Study Research An intensive description and analysis of a single individual, or sometimes a group Usually gather a narrative description of information about the individual(s). Common when describing rare phenomena (rare brain injuries or disorders, prodigies, assassins) Psychobiography: use psychological theories to understand lived of famous people (study Nixon) Used to make anecdotes and to illustrate general principles.

Limitations of Case Studies Very difficult to control extraneous variables. Usually unable to asses and rule out alternative explanations. Observer Biases