Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition Chapter 11 Experimental Designs

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition 11.2 By the end of this chapter, you should be able to: Define the purpose of an experiment and the uses for experimental designs Identify procedures used in experimental research Describe internal and external validity threats in experimental research Identify types of experimental designs List the steps in conducting experimental research Evaluate experimental research

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition 11.3 Experimental Research In an experiment, you test an idea (or practice or procedure) to determine whether it influences an outcome or dependent variable. Experimental design process –Determine if an experiment is the appropriate research approach for your study –Determine activities with which to “experiment” –Assign individuals to experience the experiment (and have some individuals experience something different) –Determine whether those who experienced the activities (or practice or procedure) performed better on some outcome than those who did not experience the same thing

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition 11.4 When to Use Experimental Procedures Used to establish cause and effect –Between independent and dependent variables –Control for all variables that might influence the outcome Used when testing theories Used when comparing two or more groups

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition 11.5 The Development of Experiments Psychological experiments (late 19th, early 20th century) Procedures for comparing groups (McCall, 1925) Statistics for comparing groups (e.g., chi-square) Identification of types of experimental designs (Campbell and Stanley,1963) Types of basic designs and threats (Cook and Campbell, 1979) Complicated experiments with many variables (since 1980)

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition 11.6 Characteristics of Experimental Designs Random assignment Control over extraneous variables Manipulation of the treatment conditions Outcome measures Group comparisons Guard against threats to validity

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition 11.7 Random Assignment Randomly assign individuals at random to conditions or to groups Random assignment equates groups and distributes variability between or among groups and conditions Different from random selection which is selection of participants at random

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition 11.8 Control Over Extraneous Variables Extraneous variables: Influences in participant selection, procedures, statistics, or the design likely to affect the outcome and provide an alternative explanation of results than what was expected Random assignment of participants before beginning the experiment Other control procedures: –Pre- and posttests –Matching participants –Covariates –Homogenous samples –Blocking variables

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition 11.9 Pretests and Posttests Time 1 Time 2 Intervention Pretest Posttest

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition Controlling for Covariates Dependent Variable Independent Variable No Covariates Covariate Introduced Covariate: Parents Who Smoke Variance Removed Variance Dependent Variable: Rates of Smoking Independent Variable: Type of Instruction

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition Matching Process Based on Gender Experimental Group Control Group John Jim James Josh Jackson Jane Johanna Julie Jean Jeb

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition Manipulation of the Treatment Conditions Identify a treatment variable Identify the conditions or levels of the treatment variable Manipulate the treatment conditions

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition The Experimental Manipulation of a Treatment Group Independent variables 1. Age (cannot manipulate) 2. Gender (cannot manipulate) 3. Types of instruction (can manipulate) a. Lecture (control) b. Lecture + hazard instruction (comparison) c. Lecture + hazard instruction + slides of damaged lungs (experiment) Dependent variable Frequency of smoking

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition Outcomes Criterion or effect variable Outcome variable Measured on a continuous scale

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition Group Comparisons in an Experiment + * p <.05 Phase 1: Relationship Picture Error Correction Treatment Spelling Accuracy Phase 2: Timeline Picture Test 1 Test 2 Test 3 6 Weeks 6 Weeks 6 Weeks Class A: Regular Spelling Practice (control) Class B: Reduced Word List (comparison) Phase 3: Statistical Comparisons Class A Class B Class C F value Test 1 Test 2 Test (3.6) 10.7 (3.3) 11.1 (3.3) 10.8 (4.3) 10.6 (3.8) 10.3 (3.6) 9.9 (3.9) 13.9 (4.2) 13.1 (3.8) * 3.31* Error Correction Treatment 6 Weeks 6 Weeks 6 Weeks Class C: Error Correction (experimental )

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition Threats to Internal Validity History Maturation Regression Selection Mortality Interactions with selection Diffusion of treatments Compensatory equalization Compensation rivalry Resentful demoralization Testing Instrumentation Internal validity: The observed changes that took place are a result of your intervention or your program and are not the result of other causes. Threats

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition Threats to External Validity Interaction of selection and treatment Interaction of setting and treatment Interaction of history and treatment External validity: The degree to which the findings are generalizable to a population Threats

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition Types of Experiments: Between Groups True experiments –Pre- and posttest –Posttest only Quasi-experiments –Pretest –Posttest –Factorial designs

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition Means and Main Effects of Eight Groups in Factorial Design Depression Mean rate of smoking Low Mean rate of smoking Mean rate of smoking Mean rate of smoking Mean rate of smoking Mean rate of smoking Health lecture Type of instruction Standard lecture Main effects of type of instruction Main effects of depression MediumHigh

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition Graphs Showing Main and Interaction Effects High Low MediumHigh Low MediumHigh Low MediumHigh (a) No Interaction Effects (Parallel) Extent of Smoking (b) Interaction Effects (Crossed) (c) Interaction Effects (Not Parallel) Standard lecture Health lecture Standard lecture Health lecture Standard lecture Health lecture

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition Types of Experiments: Within-Group Time series experiments –Interrupted –Uninterrupted Repeated measures experiments Single-subject experiments –A/B design –Multiple baseline design –Alternating treatments

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition Steps in Conducting Experimental Research 1. Decide if an experimental design fits research problem 2. Form cause/effect hypotheses 3. Select experimental unit and identify study participants 4. Select an experimental treatment and introduce it

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition Steps in Conducting Experimental Research (cont’d) 5. Choose a type of experimental design 6. Conduct the experiment 7. Organize and analyze the data 8. Develop an experimental research report

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition Criteria for Evaluating Experimental Research Does the experiment have a powerful intervention? Does it employ few treatment groups (e.g., only two)? Will participant(s) gain from the intervention(s)? Is there a systematic way the researcher derived the number of participants (cell size)?

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, third edition Criteria for Evaluating Experimental Research (cont’d) Was an adequate number of participants used in the study? Were valid, reliable, and sensitive measures or observations used? Did the study control for extraneous factors? Did the researcher control for threats to internal validity?