Reasoning in Psychology Using Statistics 2017

Announcements Quizzes 1 & 2 are posted Quiz 2 due Fri (Feb 3) Don’t forget Exam 1 is coming up (Feb 8) In class part – multiple choice, closed book In labs part – open book/notes Today Nuts and bolts of experimental & observational designs Announcements

Designing your research Generally the process involves a number of steps: identification of your research questions identifying your variables specify your hypotheses (how are the variables related to one another) selecting a research design collecting your data, analyzing your data drawing conclusions from your data about your hypotheses Designing your research

Designing your research Generally the process involves a number of steps: identification of your research questions identifying your variables specify your hypotheses (how are the variables related to one another) selecting a research design collecting your data, analyzing your data drawing conclusions from your data about your hypotheses Today’s focus Designing your research

Design Decision Tree Context Statistical tests Research design Measurement scale What is known about the distributions Statistical tests Which tests to conduct How to set up your datafile Design Decision Tree

Selecting your research method Issue: What is the best way to study for a test? Cram the night before - or - Spread your studying out over several nights How could we investigate this question? What research methods should we use? Selecting your research method

Selecting your research method Issue: What is the best way to study for a test? Observational Experimental involve examining variables as they already are involve manipulation of variables How could we investigate this question? What research methods should we use? Selecting your research method

Selecting your research method Issue: What is the best way to study for a test? Randomly select individuals Watch their study habits See how they do on a test Observational Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test Experimental Selecting your research method

Observational designs Issue: What is the best way to study for a test? Randomly select individuals Watch their study habits See how they do on a test Here our analysis looks for a relationship between the explanatory (study method) and response (test performance) variables Observational The scale of measurement used for the variables is important Observational designs

Experiments Experimental Why random assignment? To try to minimize bias, potential systematic differences between the people in the different groups Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test Experimental Experiments

Experiments Experimental Why 2 groups? Randomly select individuals All experiments make some sort of comparison. Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test Experimental Experiments

Experiments Why 2 groups? Experiment All experiments make some sort of comparison. This is an observational design - called “one-shot case study design” Experiment ?? If only one group: There is natural variability in your DV. No comparison group, so hard to see if: Randomly select individuals Tell 10 students to distribute their studying See how they do on a test change in the IV change in the DV Should not make causal claims Special case: when our “comparison” group is the population Experiments

Experiments Why 2 groups? Experiment All experiments make some sort of comparison. The systematic reduction and control of variability of the IV(s) allows us to examine whether they are sources of (causes) variability in the DV Experiment If two groups: Still some natural variability in your DV. But have constrained one source of the variability (crammed or distributed studying) Can see if there is an effect of the manipulation Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test change in the IV change in the DV We will discuss these designs with two groups later in the semester Experiments

Experiments Issue: What is the best way to study for a test? Variables Independent Dependent Extraneous Control Random Confound Experiment Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test Experiments

Experiments Issue: What is the best way to study for a test? Variables Independent Dependent Extraneous Control Random Confound Experiment – These are the variables that are manipulated by the experimenter Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test A number of ways to manipulate your IV Event/Stimulus manipulations – manipulate characteristics of the stimuli, context, etc. Instructional manipulations – different groups are given different instructions Subject manipulations – there are (pre-existing mostly) differences between the subjects in the different conditions (typically results in quasi-experimental designs) Will typically use a Categorical Scale of Measurement Nominal or Ordinal – names corresponding to different levels of the IV Experiments

Experiments Issue: What is the best way to study for a test? Variables Independent Dependent Extraneous Control Random Confound Experiment 1 IV (factor): study type 2 levels: Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test Crammed study Study all in one night Distributed study Study over five nights Experiments

Experiments Issue: What is the best way to study for a test? Variables Independent Dependent Extraneous Control Random Confound Experiment Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test These are the variables that are measured by the experimenter, they are “dependent” on the independent variables For most of our designs these will be measured on Continuous Scales of Measurement (Ratio or Interval) Test performance Experiments

Experiments Issue: What is the best way to study for a test? Variables Independent Dependent Extraneous Control Random Confound Experiment Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test – Holding things constant Controls for excessive random variability Crammed study Study for 5 hours Distributed study Study for 1 hour each night Total study time = 5 hours Experiments

Experiments Issue: What is the best way to study for a test? Variables Independent Dependent Extraneous Control Random Confound Experiment Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test - May freely vary, to spread variability equally across all experimental conditions Experiments

Experiments Issue: What is the best way to study for a test? Variables Independent Dependent Extraneous Control Random Confound Experiment Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test – Other variables, that haven’t been accounted for (manipulated, measured, randomized, controlled) that co-vary with the IV(s) and can impact changes in the dependent variable(s) Experiments

Experiments Issue: What is the best way to study for a test? Variables Crammed study Distributed study Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test Variables Independent Dependent Extraneous Control Random Confound Experiment Underlying third variable that varies with the IV Studied vocabulary and practice problems Studied only vocabulary Is this difference due to study method? Or is it due to what was studied? Test performance 85% 73% Experiments

Selecting your research method Which research method is best? - it depends Observational Experimental - May see patterns of complex behaviors - Good first step - May learn about something unexpected - Precise control possible - Precise measurement possible - Can make causal claims Advantages - Shouldn’t make causal claims - Directionality of the relationship isn’t known - Threats to internal validity due to lack of control - Sometimes the results are not reproducible - Artificial situations may restrict generalization to “real world” - Complex behaviors may be difficult to measure Disadvantages Selecting your research method

Your design largely determines your statistical analysis Design Decision Tree

Design Decision Tree SPSS Your design largely determines your statistical analysis Which test do I use? Students often find this one of the hardest parts of the course – the key is understanding the context Design Decision Tree SPSS

Wrap up Today’s lab Questions? Practice developing and evaluating experiments Questions? Statistics Learning Centre Choosing which statistical test to use (~10 mins) Erich Goldstein Choosing a Statistical Test (~13 mins) RStatsInstitute Independent vs. Paired t Tests Setup in SPSS (~3 mins) Dr. Todd Grande Comparison of Independent-Samples T Test and Paired-Samples T Test using SPSS (~20 mins) Wrap up