Experimental Design

True Experimental Designs n Random assignment n Two comparison groups n Controls threats to internal validity n Strongest evidence for cause-effect

Quasi Experimental Designs n Lacks randomization or n Lacks control group n Accommodates for natural setting

Group Assignment n Completely randomized (between subjects) n Randomized block design n Repeated measures design (within subjects)

Selecting a Design n How many IV are being tested? n How many levels does each IV have, and are these levels experimental or control conditions? n How many groups of subjects are being tested?

Selecting a Design n How will subjects be selected, and how will they be assigned to groups? n How often will observations of responses be made? n What is the temporal sequence of interventions and measurements?

True Experimental Designs n Single factor designs for independent groups n One way design

Design 1A n Pretest-Posttest control group design n R O 1 X O 2 n O 1 O 2 n Standard in Clinical research for cause - effect n Randomized controlled trial (RCT)

Design 1B n Two group pretest-posttest design n O 1 X 1 O 2 n R O 1 X 2 O 2 n New vs. old n Difference between 2 treatments

Design 1C n Multigroup pretest-posttest control group design n O 1 X 1 O 2 n R O 1 X 2 O 2 n O 1 O 2 n Threat to internal validity is attrition n Threat to external validity is the interaction of treatment and testing

Analysis of Design 1 n Utilize change scores n Interval-ratio data 2 groups unpaired t-test n 3 groups ANOVA n Ordinal data 2 groups Mann-Whitney U n 3 groups Kruskal-Wallis analysis of variance

Design 2 n Posttest-only design n R X 1 O 2 n O 2 n True experimental design n Internal validity is strong n External validity is strong n Analysis is identical to Design 1

Multifactor Designs for Independent Groups n Design 3 Factorial n Design 4 Randomized Block n Design 5 Nested

Design 3 Factorial Design n 2 or more Independent variables n Has factor and levels n 2 x 2 n 3 x 3 n 4 x 4

Two Way Factorial Design n Main effect n Interaction effect n Analysis is two way or three way analysis of variance

Design 4: Randomized Block Design n Concern over an extraneous variable n Attributed variable is crossed with an active IV n Analysis is 2 way analysis of variance or multiple regression analysis

Design 5: Nested Design n Some attribute variables cannot be crossed

Repeated Measures Design n Design 6 One Way Repeated Measures Design n Design 7 Crossover Design n Design 8 Latin Square Design

Design 6: One Way Repeated Measures Design n All subjects exposed to all levels of one treatment variable n X 1 O 1 n X 2 O 2 n X 3 O 3 n Internal threat is carry over n Analysis one-way ANOVA

Design 7: Cross Over Design n Minimizes bias of test sequence n Counterbalances treatment conditions n Washout period n R O 1 X 1 O 2... O 3 X 2 O 4 n O 1 X 2 O 2... O 3 X 1 O 4 n Analysis: Paired T-test or 2 way ANOVA n Ordinal data Wilcoxon signed ranks test

Design 8: Latin Square Design n Sequencing of treatment order is concern n Randomizes treatment order n 3 x 3 or 4 x 4

Multifactor Designs for Repeated Measures n Design 9 Two-way design with two repeated measures n Design 10 Mixed design

Design 9: Two-way Design with Two Repeated Measures n Analysis: Two way analysis of variance with two repeated measures to measure main and interaction effects

Design 10: Mixed Design n Analysis: 2 way anova with one repeated measure for main effect two way design with one repeated factor for interaction effects

Quasi Experimental Designs n Do not use random assignments n Potential bias of sample n Strong control over design

Quasi Experimental Designs n Design 11 One-group pretest-posttest design n Design 12 Time series design n Design 13 Nonequivalent pretest-posttest control group design n Design 14 Nonequivalent posttest-only control group design

Design 11: One-group Pretest- posttest Design n O 1 X O 2 n Threat to internal validity-no comparison group n Threat to external validity-potential interactions with selection n Analysis: Paired t-test for ratio/internal n Wilcoxon signed ranks test for ordinal

Multigroup Design n Design 12 Time series design n O 1 O 2 O 3 O 4 X O 5 O 6 O 7 O 8 n Effect of tx on physiological or psychological variables over time n IV is time n Threat to internal validity is history n Variations 2 or more groups, tx withdrawal

Design 13: Nonequivalent Pretest-posttest Control Design n O 1 X O 2 O 1 O 2 n Threat to internal validity is interaction of selection with history and maturation n Analysis: Unpairted T-test, anova-interval or ratio n Mann-Whitney U-test-Ordinal

Design 14: Nonequivalent Posttest- Only Control Group Design n X O O n Threat to internal validity is selection bias and attrition n Analysis: Regression