© 2001 Dr. Laura Snodgrass, Ph.D.1 Basic Experimental Design Common Problems Assigning Participants to Groups Single variable experiments –bivalent –multivalent –baseline Multivariate –factorial –converging series

© 2001 Dr. Laura Snodgrass, Ph.D.2 Common Problems Confounds Lack of control group(s) Nonequivalent control groups Why control groups –history –maturation –testing –instrument decay –statistical regression

© 2001 Dr. Laura Snodgrass, Ph.D.3 Assigning Participants to Groups Independent or Random Groups Design –between groups Repeated Measures –within groups

© 2001 Dr. Laura Snodgrass, Ph.D.4 Between Groups Advantages –generalizable –collect more data at a given level –shorter time for each participant Disadvantages –may not be random –unequal N –potential confounds –requires more participants

© 2001 Dr. Laura Snodgrass, Ph.D.5 Between Groups Matching to equate groups and decrease error variance How –correlated variables –pairs –yoked controls –performance criterion

© 2001 Dr. Laura Snodgrass, Ph.D.6 Matching Advantages –equates groups –increase power of experiment –decrease number of participants needed Disadvantages –extra work –extra testing –lose individual differences - less generalizable

© 2001 Dr. Laura Snodgrass, Ph.D.7 Repeated Measures Advantages –fewer participants needed –impt for special groups –statistically more powerful Disadvantages –not naïve after first trials –order effects practice and fatigue non-symmetric or differential transfer

© 2001 Dr. Laura Snodgrass, Ph.D.8 Counterbalancing Vary order of treatment to distribute or measure order effects Complete counterbalancing –within participants ABBA –between AB for some, BA for others Latin Squares –each cond at each ordinal position –precedes and follows each other once

© 2001 Dr. Laura Snodgrass, Ph.D.9 Counterbalancing Randomized blocks Time interval between trials –mortality

© 2001 Dr. Laura Snodgrass, Ph.D.10 Single Variable Experiments Bivalent –one independent variable with two levels Multivalent (functional) –one independent variable with three or more levels Baseline

© 2001 Dr. Laura Snodgrass, Ph.D.11 Bivalent Two levels of the independent variable –experimental and control groups –two different levels of the variable Post-test only vs. pre-test/post-test Advantages –easy to interpret and analyze –decide if IV is worth studying

© 2001 Dr. Laura Snodgrass, Ph.D.12 Bivalent Disadvantages –limited theoretical value –conclusions may be based on arbitrary choice of levels –negative findings are not conclusive –does not describe shape of relationship therefore you may over generalize for non-linear relationships interpolation and extrapolation plateau or asymptote

© 2001 Dr. Laura Snodgrass, Ph.D.13 Multivalent (functional) Gives more info about the shape of the relationship Advantages –better estimate true relationship –individual choice of levels becomes less critical Disadvantages –more: time, effort, cost, subjects –more complex statistics and interpretation

© 2001 Dr. Laura Snodgrass, Ph.D.14 Baseline Only works with certain types of variables –will not work with variables that cause permanent change Procedure: –establish baseline or steady-state response level –introduce IV until stable transition –allow subject to return to baseline

© 2001 Dr. Laura Snodgrass, Ph.D.15 Baseline Advantages –rules out most confounds –easy to interpret (often no statistics) –flexible and replicable –investigate behavior of an individual Disadvantages –does not show small changes –may not generalize

© 2001 Dr. Laura Snodgrass, Ph.D.16 Multivariate Experiments Factorial Designs –two or more independent variables, each with two or more levels –variables can be all between, all within, or mixed in many combinations Converging series –series of small experiments in which a variable manipulated in an earlier experiment becomes a control variable in a later experiment

© 2001 Dr. Laura Snodgrass, Ph.D.17 Factorial Design matrix –produces a family of functions –study main effects and interactions Advantages –study interactions –increases precision and generalizability –decrease statistical error and increase power –theoretical value

© 2001 Dr. Laura Snodgrass, Ph.D.18 Factorial Disadvantages –increases time, money and number of subjects increases dramatically as number of cells increases –assumptions of ANOVA may not be met –N-way interactions are very difficult to interpret

© 2001 Dr. Laura Snodgrass, Ph.D.19 Converging series for applied problems Optimal designs –e.g. car, medical treatment, office Find an optimal level of a variable and turn it into a control variable –lose higher order interactions

© 2001 Dr. Laura Snodgrass, Ph.D.20 Converging Operations Converge on a single hypothesis –start with several possible hypotheses or explanations –each experiment eliminates one or more until only one remains (hopefully) For example: –perceptual defense against vulgar words –isolation tank

© 2001 Dr. Laura Snodgrass, Ph.D.21 Converging Series Advantages –flexible, many choice points –efficient, leave out factors that have no effect –built in replications Disadvantages –interactions are lost –almost always between subjects –analyze and interpret prior before next experiment so can take a long time