1 Experimental Designs HOW DO HOW DO WE FIND WE FIND THE ANSWERS ? THE ANSWERS ?

2 Characteristics of Experimental Designs  Manipulation of one or more factors  (Independent Variables)  Measurement of the effects of manipulation  (Dependent Variables)  Validity  Are we in control?  Reliability  Can the results be replicated?  Sensitivity  Are we measuring what we want to measure?

3 Validity (Internal)  amount of control over experimental conditions  allows conclusion that the IV causes an effect on the DV  allows exclusion of other variables causing an effect on the DV

4 Internal Validity ä Challenges to Internal Validity ä Using intact groups ä (such as classes of students) ä Not balancing extraneous variables ä (individual differences) ä * hypnosis volunteers early or late in term ä Subject Loss ä mechanical subject loss (equipment failure) ä selective subject loss (related to paradigm?)

5 Validity (External) ä Can findings be generalized ä to other species ä to other individuals ä to other settings or situations ä to other conditions

6 Validity (External) ä For some lab experiments we do not establish external validity ä Is external validity needed? ä Mook (1983) argues that external validity is irrelevant if we are testing a specific hypothesis in a laboratory setting ä Lab experiments typically try to test a specific hypothesis instead of imitating a typical situation

7 Validity (External) ä External validity needed when results are to generalized to a population ä External validity requires a representative sample ä Partial replication (repeating some but not all of the experimental conditions) can provide evidence for external validity

8 Sensitivity ä Is our measure appropriate for the effect we are looking for? ä Are we measuring enough of the effect? ä Are we measuring too much of the effect (even if we get an effect, is it meaningful?)

9 Issues of Control ä Methods of Control ä Manipulation ä Holding conditions constant ä Balancing

10 Control ä Manipulation ä Systematic varying of an Independent Variable

11 Control ä Hold Conditions Constant ä Make the IV the only variable the differentiates between the groups ä Example: Use only males to hold the gender effect constant

12 Control ä Balancing ä Technique used to control for individual differences of participants ä Used in independent groups designs ä Insures that all groups are equivalent in areas such as age, motivation, sex, intelligence, etc.

13 Independent Groups Design ä Each group represents a different condition ä Conditions are defined by the level of the IV ä Groups are formed by participants being assigned to conditions ä Nature of group formation makes balancing a major consideration of control

14 Random Groups Design ä Groups are formed prior to introducing the IV ä Subjects are sampled in such a way that the selection of one subject in no way influences the selection of another subject ä All subjects have an equal chance of being in any given group ä May be accomplished by random selection or random assignment

15 Random Selection ä Requires a well defined population ä Requires randomization processes for selection of subjects ä Subjects are randomly selected for each group

16 Random Assignment ä Used when random selection is not possible ä Most samples are accidental and not from well defined populations (Intro Psyc students) ä Random assignment is then used to randomize subjects into different groups instead of random selection

17 Random Assignment ä Block Randomization ä Most often used for random assignment ä Have number of blocks = number of subjects in each condition ä Randomize conditions in each block ä Assign subjects to each condition in each block until all blocks are filled

18 Matched Groups Design ä Used when comparable groups is required ä Instead of random assignment ä the researcher makes the groups equivalent by matching the subjects in each group ä Most useful if a good matching task is used

19 Matched Group Design ä Example ä 1) pretest for dependent variable (BP) ä 2) match subjects by BP level and group by the number of conditions ä 3) randomly assign to conditions ä 4) compare BP of subjects by condition at posttest

20 Natural Groups Design ä Subjects are selected based on levels of IV ä Used when impossible to manipulate IV ä age, gender, personality traits, etc. ä Used when not ethical to manipulate IV ä married, divorced, widowed, etc.

21 Repeated Measures Design ä One group of subjects ä Subjects receive all levels of the IV ä Eliminates problem of Individual Differences ä Reduces the number of subjects required ä Counterbalancing necessary for control

22 Counterbalancing ä Counterbalancing necessary to control practice effect. ä ABBA design optimal (IVs A & B) ä ABBA (complete counterbalance)

23 Counterbalancing ä Problems with ABBA design ä Each subject has to complete all presentations of IV ä ABBA ä As IV levels increase design becomes unmanageable ä IVs A, B, & C ä ABCACBBACBCACABCBA (Complete Counterbalance)

24 Counterbalancing ä Alternate to complete ABBA design ä Use Incomplete design ä ½ of group receives conditions AB ä ½ of group receives conditions BA ä IVs ABC (complete design) ä ABCACBBACBCACABCBA (Complete Counterbalance – each subject receives 18 conditions) ä ABC ACB BAC BCA CAB CBA ä (Incomplete Counterbalance – each subject receives 3 conditions)

25 Design Problems and Solutions ä Independent Groups Design ä Individual Differences ä (Differences between subjects in each group) ä Use Repeated Measures Design to eliminate individual differences (using same subjects) ä Repeated Measures Design ä Differential Transfer ä (Carryover effects) ä Use Independent Groups Design to eliminate differential transfer

26 Complex (Factorial) Designs ä Main Effects ä Effects of the Main IVs ä Two possible Main Effects in your experiment ä Difference in RT between Caffeine and No Caffeine ä (IV # 1 or “A”) ä Difference in RT of PH and NPH ä (IV # 2 or “B”)

27 Complex (Factorial) Designs ä Interaction Effects ä How one IV (A) may impact another IV (B) ä Will Caffeine influence RT in one hand but not the other hand? ä Four possible Interaction Effects in your experiment ä ä This is a 2 X 2 Factorial Design ä 2 IVs (Caffeine & Handedness) ä Each has 2 levels (Caffeine or No Caffeine & PH or NPH) CaffeineNo Caffeine PHRT (Caff&PH)RT (NoCaff&PH) NPHRT (Caff&NPH)RT (NoCaff&NPH)

28 Analysis of Factorial Designs ä Analyze with a Factorial ANOVA (F test) ä F test analyses reflects ä Systematic variance due to manipulation ä Error variance due to confounds ä Including Individual differences of subjects ä F = variation between groups ä variation within groups ä F = error variation + systematic variation ä error variation

29 Analysis of Factorial Designs ä F test analyses reflects ä F = variation between groups ä variation within groups ä F = error variation + systematic variation ä error variation ä F test may indicate significant differences in Main Effects and Interaction Effects ä Requires a Post Hoc text to determine differences

30 Analysis of Factorial Designs ä Post Hoc test for Main Effects ä One-Way or Repeated Measures ANOVAS if needed ä Post Hoc test for Interaction Effects ä Graph data ä Parallel lines indicate no interaction ä Converging or Intersecting lines indicate interactions

31 Experimental Designs EXPERIMENTAL DESIGNS THAT ARE CORRECTLY EXECUTED EXPERIMENTAL DESIGNS THAT ARE CORRECTLY EXECUTED RESULT IN SUCESSFUL OUTCOMES RESULT IN SUCESSFUL OUTCOMES

32 Experimental Designs ä QUESTIONS?