Chapter 11 Experimental Designs Power Point Slides by Ronald J. Shope in collaboration with John W. Creswell

Educational Research 2e: Creswell Key Ideas Definition and uses for experimental designs Procedures used in experimental research Internal and external validity threats in experimental research Types of experimental designs Steps in conducting experimental research Evaluating experimental research Educational Research 2e: Creswell

What is experimental research? In an experiment, you test an idea (or practice or procedure) to determine whether it influences an outcome or dependent variable. Determine activities with which to “experiment” Assign individuals to experience it (and have some individuals experience something different) Determine whether those who experienced the activities (or practice or procedure) performed better on some outcome than those who did not experience it. Educational Research 2e: Creswell

When do you use experimental procedures? To establish cause and effect Between independent and dependent variables Control for all variables that might influence the outcome Used when testing theories Used when comparing two or more groups Educational Research 2e: Creswell

How did experiments develop? Psychological experiments (late 19th early 20th century) Procedures for comparing groups (McCall 1925) Statistics for comparing groups (e.g. Chi-Square) Identification of types of experimental designs (Campbell and Stanley 1963) Types of basic designs and threats (Cook and Campbell 1979) Complicated experiments with many variables (Since 1980) Educational Research 2e: Creswell

Key Characteristics of Experimental Designs Random assignment Control over extraneous variables Manipulation of the treatment conditions Outcome Measures Group Comparisons Threats to validity Educational Research 2e: Creswell

Educational Research 2e: Creswell Random assignment Randomly assign individuals at random conditions or to groups Random assignment equates groups and distributes variability between or among groups and conditions Different from Random Selection which is selection of participants at random Educational Research 2e: Creswell

Control over extraneous variables Extraneous Variables: influences in participant selection, procedures, statistics, or the design likely to affect the outcome and provide an alternative explanation results than what was expected. Random assignment helps to control for extraneous variables Done before the experiment begins Educational Research 2e: Creswell

Control over extraneous variables Other control procedures pretest/posttest covariates matching participants selecting homogenous samples using blocking variables Educational Research 2e: Creswell

Pre-Test and Post-Tests Intervention Time 1 Time 2 Pre-Test Post-Test Educational Research 2e: Creswell

Controlling for Covariates Independent Variable Dependent Variable No Covariates Covariate Introduced Independent Variable: Type of Instruction Dependent Variable: Rates of Smoking Covariate: Parents Who Smoke Variance Removed Variance Educational Research 2e: Creswell

Matching Process Based on Gender John Jim James Josh Jackson Jane Johanna Julie Jean Jeb Experimental Group Control Group Educational Research 2e: Creswell

Manipulation of the treatment conditions Identify a treatment variable Identify the conditions or levels of the treatment variable Manipulate the treatment conditions Educational Research 2e: Creswell

The Experimental Manipulation of a Treatment Group Independent Variables 1. Age (can’t manipulate) 2. Gender (can’t manipulate) 3. Types of Instruction (can manipulate) a. Lecture (control) b. Lecture + Hazard Instruction (Comparison) c. Lecture + Hazard Instruction + slides of damaged lungs (experiment) Dependent Variable Frequency of Smoking Educational Research 2e: Creswell

Educational Research 2e: Creswell Outcomes Criterion or effect variable Outcome variable Measured on a continuous scale Educational Research 2e: Creswell

Group comparisons in an experiment Phase 1: Relationship Picture Error Correction Treatment Spelling Accuracy Phase 2: Timeline Picture Test 1 Test 2 Test 3 6 Weeks 6 Weeks 6 Weeks Class A: Regular Spelling Practice (Control) Class B: Reduced word list (Comparison) Phase 3: Statistical Comparisons Class A Class B Class C F-value Test 1 Test 2 Test 3 10.3 (3.6) 10.7 (3.3) 11.1 10.8 (4.3) 10.6 (3.8) 9.9 (3.9) 13.9 (4.2) 13.1 0.27 4.90* 3.31* Class C: Error Correction (Experimental) + * p<.05 Educational Research 2e: Creswell

Threats to internal validity History Maturation Regression Selection Mortality Interactions with selection Diffusion of treatments Compensatory equalization Compensation rivalry Resentful demoralization Testing Instrumentation Educational Research 2e: Creswell

Threats to external validity Interaction of selection and treatment Interaction of setting and treatment Interaction of history and treatment Educational Research 2e: Creswell

Types of experiments: Between groups True Experiments Pre- and posttest Posttest Only Quasi Experiments Pre Posttest Factorial Designs Educational Research 2e: Creswell

Means and Main Effects of Eight groups in Factorial Design Depression Mean rate of smoking Low Health lecture Type of Instruction Standard lecture Main Effects of Type of Instruction Main Effects of Medium High Educational Research 2e: Creswell

Graphs Showing Main and Interaction Effects Standard lecture High Low Medium No Interaction Effects (Parallel) Health lecture Extent of Smoking High Low Medium (b) Interaction Effects (Crossed) Standard lecture Extent of Smoking Health lecture (c) Interaction Effects (Not Parallel) High Low Medium Standard lecture Extent of Smoking Health lecture Educational Research 2e: Creswell

Types of Experiments: Within-group or individual Time series experiments interrupted uninterrupted Repeated measures experiments Single-subject experiments A/B design Multiple baseline design Alternating treatments Educational Research 2e: Creswell

Steps in Conducting Experimental Research Decide if an experimental design fits research problem Form cause/effect hypotheses Select experimental unit and identify study participants Select an experimental treatment and introduce it Choose a type of experimental design Conduct the experiment Organize and analyze the data Develop an experimental research report Educational Research 2e: Creswell

Criteria for Evaluating Experimental Research Does the experiment have a powerful intervention? Does it employ few treatment groups (e.g. only two)? Will participant gain from the intervention(s)? Is there a systematic way the researcher derived the number of participants (cell size)? Was there an adequate number of participants used in the study? Were valid, reliable, and sensitive measures or observations used? Did the study control for extraneous factors? Did the researcher control for threats to internal validity? Educational Research 2e: Creswell