1. Reasoning in Psychology Using Statistics
2019

2. Announcements Don’t forget Exam 1 is coming up (Feb 6) Today
In class part – multiple choice, closed book
In labs part – open book/notes
Today
Nuts and bolts of experimental & observational designs
Announcements

3. Designing your research
Generally the process involves a number of steps:
identification of your research questions
identifying your variables
specify your hypotheses (how are the variables related to one another)
selecting a research design
collecting your data, analyzing your data
drawing conclusions from your data about your hypotheses
Designing your research

4. Designing your research
Generally the process involves a number of steps:
identification of your research questions
identifying your variables
specify your hypotheses (how are the variables related to one another)
selecting a research design
collecting your data, analyzing your data
drawing conclusions from your data about your hypotheses
Today’s focus
Designing your research

5. Design Decision Tree Context Statistical tests Research design
Measurement scale
What is known about the distributions
Statistical tests
Which tests to conduct
How to set up your datafile
Design Decision Tree

6. Selecting your research method
Issue: What is the best way to study for a test?
Cram the night before
- or -
Spread your studying out over several nights
How could we investigate this question?
What research methods should we use?
Selecting your research method

7. Selecting your research method
Issue: What is the best way to study for a test?
Observational
Experimental
involve examining variables as they already are
involve manipulation of variables
How could we investigate this question?
What research methods should we use?
Selecting your research method

8. Selecting your research method
Issue: What is the best way to study for a test?
Randomly select individuals
Watch & classify their study habits
See how they do on a test
Observational
Randomly select individuals
Randomly assign to groups
Crammed study group
Distributed study group
See how they do on a test
Experimental
Selecting your research method

9. Observational designs
Issue: What is the best way to study for a test?
Randomly select individuals
Watch & classify their study habits
See how they do on a test
Here our analysis looks for a relationship between the explanatory (study method) and response (test performance) variables
Observational
The scale of measurement used for the variables is important
Observational designs

10. Experiments Experimental Why random assignment?
To try to minimize bias, potential systematic differences between the people in the different groups
Randomly select individuals
Randomly assign to groups
Crammed study group
Distributed study group
See how they do on a test
Experimental
Part 1 of the movie
“Every Statistical test starts with an appropriate selection of subjects”
Experiments

11. Experiments Experimental Why 2 groups? Randomly select individuals
All experiments make some sort of comparison.
Randomly select individuals
Randomly assign to groups
Crammed study group
Distributed study group
See how they do on a test
Experimental
Experiments

12. Experiments Why 2 groups? Experiment
All experiments make some sort of comparison.
This is an observational design - called “one-shot case study design”
Experiment ??
If only one group:
There is natural variability in your DV.
With no comparison group, so hard to see if:
Randomly select
individuals
Tell 10 students to distribute their studying
See how they do on a test
change in the IV
Causes
change in the DV
Should not make causal claims
Special case: when our “comparison” group is the population
Experiments

13. Experiments Why 2 groups? Experiment
All experiments make some sort of comparison.
The systematic reduction and control of variability of the IV(s) allows us to examine whether they are sources of (causes) variability in the DV
Experiment
If two groups:
Still some natural variability in your DV.
But have constrained one source of the variability (crammed or distributed studying)
Can see if there is an effect of the manipulation
Randomly select
individuals
Randomly assign to groups
Crammed study group
Distributed study group
See how they do on a test
change in the IV
Causes
change in the DV
We will discuss these designs with two groups later in the semester
Experiments

14. Experiments Issue: What is the best way to study for a test? Variables
Independent
Dependent
Extraneous
Control
Random
Confound
Experiment
Randomly select
individuals
Randomly assign to groups
Crammed study group
Distributed study group
See how they do on a test
Experiments

15. Experiments Issue: What is the best way to study for a test? Variables
Independent
Dependent
Extraneous
Control
Random
Confound
Experiment
– These are the variables that are manipulated by the experimenter
Randomly select
individuals
Randomly assign to groups
Crammed study group
Distributed study group
See how they do on a test
A number of ways to manipulate your IV
Event/Stimulus manipulations – manipulate characteristics of the stimuli, context, etc.
Instructional manipulations – different groups are given different instructions
Subject manipulations – there are (pre-existing mostly) differences between the subjects in the different conditions (typically results in quasi-experimental designs)
Will typically use a Categorical Scale
Nominal or Ordinal – names corresponding to different levels of the IV
Experiments

16. Experiments Issue: What is the best way to study for a test? Variables
Independent
Dependent
Extraneous
Control
Random
Confound
Experiment
1 IV (factor): study type
2 levels:
Randomly select
individuals
Randomly assign to groups
Crammed study group
Distributed study group
See how they do on a test
Crammed study
Study all in one night
Distributed study
Study over five nights
Experiments

17. Experiments Issue: What is the best way to study for a test? Variables
Independent
Dependent
Extraneous
Control
Random
Confound
Experiment
Randomly select
individuals
Randomly assign to groups
Crammed study group
Distributed study group
See how they do on a test
These are the variables that are measured by the experimenter, they are “dependent” on the independent variables
For most of our designs these will be measured on a Continuous Scale of Measurement (Ratio or Interval)
Test performance
Experiments

18. Experiments Issue: What is the best way to study for a test? Variables
Independent
Dependent
Extraneous
Control
Random
Confound
Experiment
Randomly select
individuals
Randomly assign to groups
Crammed study group
Distributed study group
See how they do on a test
– Holding things constant
Controls for excessive random variability
Crammed study
Study for 5 hours
Distributed study
Study for 1 hour each night
Total study time = 5 hours
Control Variable: total time studying
Experiments

19. Experiments Issue: What is the best way to study for a test? Variables
Independent
Dependent
Extraneous
Control
Random
Confound
Experiment
Randomly select
individuals
Randomly assign to groups
Crammed study group
Distributed study group
See how they do on a test
- May freely vary, to spread variability equally across all experimental conditions
Random variables:
How much sleep they get
What they eat for lunch
Experiments

20. Experiments Issue: What is the best way to study for a test? Variables
Independent
Dependent
Extraneous
Control
Random
Confound
Experiment
Randomly select
individuals
Randomly assign to groups
Crammed study group
Distributed study group
See how they do on a test
– Other variables, that haven’t been accounted for (manipulated, measured, randomized, controlled) that co-vary with the IV(s) and can impact changes in the dependent variable(s)
Experiments

21. Experiments Issue: What is the best way to study for a test? Variables
Underlying third variable that varies with the IV
Possible Confound variable:
Order of studied material, always start with vocab
Variables
Independent
Dependent
Extraneous
Control
Random
Confound
Crammed study
Distributed study
Randomly select
individuals
Randomly assign to groups
Crammed study group
Distributed study group
See how they do on a test
Experiment
Studied vocabulary and practice problems
Studied only vocabulary
Is this difference due to study method?
Or is it due to what was studied?
Test performance
85%
73%
Experiments

22. Selecting your research method
Which research method is best?
- it depends
Observational
Experimental
- May see patterns of complex behaviors
- Good first step
- May learn about something unexpected
- Precise control possible
- Precise measurement possible
- Can make causal claims
Advantages
- Shouldn’t make causal claims
- Directionality of the relationship isn’t known
- Threats to internal validity due to lack of control
- Sometimes the results are not reproducible
- Artificial situations may restrict generalization to “real world”
- Complex behaviors may be difficult to measure
Disadvantages
Selecting your research method

23. Your design largely determines your statistical analysis
Design Decision Tree

24. Design Decision Tree SPSS
Your design largely determines your statistical analysis
Which test do I use?
Students often find this one of the hardest parts of the course – the key is understanding the context
Design Decision Tree SPSS

25. Wrap up Today’s lab Questions?
Practice developing and evaluating experiments
Questions?
Statistics Learning Centre Choosing which statistical test to use (~10 mins)
Erich Goldstein Choosing a Statistical Test (~13 mins)
RStatsInstitute Independent vs. Paired t Tests Setup in SPSS (~4 mins)
Dr. Todd Grande Comparison of Independent-Samples T Test and Paired-Samples T Test using SPSS (~20 mins)
Wrap up