Research Design & Analysis II: Class 10 Announcements colloquium More complicated Experimental designs Designs with more than one IV Factorial Designs 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Winter Colloquium Series Dr. Jennifer Hendrick QEII Health Science Center, Halifax Psychological aspects of HIV February 10, 2000, 3:30 p.m. BAC 236 Supported by the 1999-2000 Bank of Montreal Visiting Lectureship fund 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Sensory Modality Effects on Memory 20 subjects, 10 randomly assigned to two groups Auditory Group 10 min travel film no emotional shock written instructions, 1 o’clock group testing, etc. IV: Auditory Price Information Measure memory - D.V. Visual group 10 min travel film no emotional shock written instructions, 1 o’clock group testing, etc. IV: Visual Price Information Measure memory - D.V. Statistically Compare differences in measured memory This bivalent experiment answers the question of whether auditory or visual information is remembered better 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Psyc2023 Class #10 (c) Peter McLeod Hypothetical Data 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Emotional Shock Effects on Memory 20 subjects, 10 randomly assigned to each group Shock Group 10 min travel film Auditory Price Information written instructions, 1 o’clock group testing, etc. IV: Emotional Shock Measure memory - D.V. No Shock Group 10 min travel film Auditory Price Information written instructions, 1 o’clock group testing, etc. IV: No Emotional Shock Measure memory - D.V. Statistically compare differences in measured memory This bivalent experiment answers the question of whether emotional shock disrupts memory or not. 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Psyc2023 Class #10 (c) Peter McLeod Hypothetical Data 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Advantages of the Factorial Design Might wonder if the effect of emotional shock on memory is equivalent for information presented aurally or visually. Could do another bivalent experiment With one factorial design answer all three questions 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Advantages of the Factorial Design Answers three questions: Is auditory or visual information remembered better? Does emotional shock disrupt memory? Does the effect of emotional shock depend upon whether the information was received via auditory or visual channel? (Or does the effects of modality of presentation depend upon the presence of emotional shock?) This third question addresses the generalizability of the main effects. 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Advantages of the Factorial Design With one factorial experiment, you can answer all three questions effectively and efficiently. In psychology, we often have informal hypotheses in part because there are more than one causal factor determining behaviour (thinking …) When a combination of independent variables acting together simultaneously determines the outcome (DV), a factorial design is uniquely suited. More complex (and accurate) causal explanations can be tested with these designs. 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Designing a Factorial Experiment Three steps: 1) Identify each hypothesized causal factor (Independent variable) of interest. 2) Decide how many levels of each factor you want (simplest case being two levels of each). 3) Determine all possible combinations (create a matrix of treatment combinations). 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Designing a Factorial Experiment 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Designing a Factorial Experiment 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Designing a Factorial Experiment 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Designing a Factorial Experiment 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Designing a Factorial Experiment 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Designing a Factorial Experiment 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Designing a Factorial Experiment 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Compare the means from the four treatment combinations 2x2 Factorial Experiment 40 subjects, 10 randomly assigned to each of 4 groups Group 1 10 min film Control variables emotional shock auditory information Measure memory Group 2 10 min film Control variables emotional shock visual information Measure memory Group 3 10 min film Control variables no emotional shock auditory information Measure memory Group 4 10 min film Control variables no emotional shock visual information Measure memory Compare the means from the four treatment combinations 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Psyc2023 Class #10 (c) Peter McLeod Factorial Designs May seem like IVs are confounded, but this is still an analytic experiment, the “gold standard” of science. 1) Random selection of subjects from defined population. 2) Random assignment of subjects to treatment groups. 3) Concurrent control of extraneous variables and contrast (statistical comparison) of the measured DV. 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Psyc2023 Class #10 (c) Peter McLeod Factorial Designs All conditions are still held constant except that in the factorial design it is the combination of treatments that are manipulated. Because all possible combinations of levels of the the IVs are represented, we can statistically separate their effects. This is done with ANOVA. 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Analysis of Variance: ANOVA ANOVA is a way of analyzing the total variability among dependent variable scores and dividing or partitioning this total variability among the factors assumed to cause it; Called sources of variance 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Sources of Variance 1) subject variables 2) experimental error (measurement or recording error) 3) combination of the IV treatments (levels) 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Psyc2023 Class #10 (c) Peter McLeod 2-Way ANOVA For factorial designs, ANOVA calculates an F-ratio for each main effect and interaction. For main effects, F is the ratio of between-group variability to within-group variability. Within-group variability is due to: differences among subjects and/or experimental error. Between group variability can also be due to: differences among subjects, experimental error, and/or variability caused by the combined treatment effects of the IVs. 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Evaluating the Significance of F Print outs - give probabilities of getting an F-value as big, or bigger, than the observed by chance for each main effect and interaction possible in the factorial design. For a 2-way ANOVA (2-factor design), probabilities are given for both main effects and the interaction. 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Terminology of Factorial Designs Term factor is used interchangeably with independent variable A number is given for each IV (factor) The number used refers to the number of levels that factor has Design? 2 x 2 Factorial That there are two digits indicates there are two IVs That both digits are “2”s indicates each IV has two levels. 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Terminology of Factorial Designs In addition, may specify how the variables were manipulated, within subjects, between subjects or both (called a mixed design). 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Types of Inferential Statistics There are many, each intended, and appropriate, for use under specific conditions that depend upon: Scale of measurement: Shape of the sample distributions Experimental Design #of I.V.s #levels of each I.V. How manipulated (within, between, mixed) #D.V.s 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Factorial Data: From Tables 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Factorial Data: Main Effects From Tables 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Factorial Data: Main Effects From Tables 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Factorial Data: Main Effects From Tables 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Factorial Data: Main Effects From Tables Comparing these values determines the main effect of IV #1 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Factorial Data: Main Effects From Tables Comparing these values determines the main effect of IV #2 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Factorial Data: From Tables Comparing values to determine the main effects of IVs 1 and 2 requires statistical analyses to determine if the observed differences are greater than would be expected by chance for a given, a priori researcher determined, alpha error rate. This is done using ANOVA. 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Factorial Data: 2-way Interactions From Tables Comparing the differences in these differences determines the interactive effect of IVs 1&2 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Factorial Data: 2-way Interactions From Tables Or, comparing the differences in these differences evaluates the interaction between IVs 1&2 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Factorial Data: 2-way Interactions From Tables You can also compare these values to determine the interaction 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Factorial Data: 2-way Interactions From Tables Or, you can also compare these values to determine the interaction 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Factorial Data: 2-way Interactions From Tables Comparing any of the differences in same coloured values determines the interactive effect of IVs 1&2 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Factorial Data: From Tables Comparing values to determine the interaction between IVs 1 and 2 also requires statistical analyses to determine if the observed differences in differences are greater than would be expected by chance for a given, a priori researcher determined, alpha error rate. This is also provided by ANOVA. 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Some (many) Examples for practice

Interpreting Factorial Data: From Tables Assess the interaction: 2-2=0 compared with 4-4=0 Or: 2-4=-2, compared with 2-4=-2 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Factorial Data: Figures 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Main Effects: Figures Visually average across one IV to determine main effect of the other IV 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Main Effects: Figures Visually average across one IV to determine main effect of the other IV. Compare these values. 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Main Effects: Figures Visually average across one IV to determine main effect of the other IV 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Main Effects: Figures Visually average across one IV to determine main effect of the other IV. Compare these values. 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Interactions: Figures Visually find the differences within levels of one IV to determine the interaction 4-2=2 4-2=2 If the differences are the same, there is no interaction 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Interpreting Interactions: Figures Visually find the differences within levels of one IV to determine the interaction 4-4=0 2-2=0 If the differences are the same, there is no interaction 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Complete Counterbalancing Split subjects into groups and give the different groups the treatments in all possible orders Assuming the effects of order will be balanced equally across treatment conditions (levels of the IV), order effects will not confound your results. Can test this assumption, by having the order in which treatments are received an independent variable (becomes a mixed factorial design, with order manipulated between- and treatment within-subjects) 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Stress Induced Analgesia: Revisited Hypothetical data: Fully Counterbalanced Design? 2 x 2 Mixed Factorial the two digits indicates there are two IVs That both digits are “2”s indicates each IV has two levels. 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Stress Induced Analgesia: Revisited Hypothetical data: Fully counterbalanced Three Questions: Main effect for Stress? Main effect for Order? Interaction? Stress - Yes Order - No Interaction - No 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Stress Induced Analgesia: Revisited 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Stress Induced Analgesia: Revisited Three Questions: Main effect for Stress? Main effect for Order? Interaction? Stress - No Order - Yes Interaction - No 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Stress Induced Analgesia: Revisited 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod

Stress Induced Analgesia Three Questions: Main effect for Stress? Main effect for Order? Interaction? Stress - No Order - No Interaction - Yes 9/16/2018 Psyc2023 Class #10 (c) Peter McLeod