1. Using the scientific method Observational Methods & Research Designs
Brief How-to on reviewing the literature too
Psych 231: Research Methods in Psychology

2. Announcements ReggieNet Quizzes Next week’s labs: Group Projects –
Quiz 2 is Due Fri. at midnight.
Next week’s labs:
Download and read the Assefi & Garry (2003) article before labs (a pdf is available in the ReggieNet Resources folder)
Group Projects –
An experiment, at least 1 independent variable and 1 dependent variable
Limited resources: per group of run time, one room (but we can put folks in the hall to wait their turn: Thursday sections could work differently because the two sections meet at same time), ~20 participants (but thinking about ways to try to increase this)
Announcements

3. Reviewing the literature
Why do a review of the literature?
What is the literature?
How do you search the literature?
Reviewing the literature

4. Why review the literature?
What are the underlying motivations for doing a review of the literature?
Getting ideas.
What has been done, what hasn’t been done?
Understanding the relevant theories.
What variables are important?
Avoid past mistakes.
Source Types: Primary and Secondary
Why review the literature?

5. Primary Sources - essentially reading the original report
Journal articles
Edited books (sometimes)
Professional meetings
Electronic publishing (fairly new, pluses and minuses)
Recommendation: stick to peer review journals, there are some that just require authors to pay-to-publish
Faculty members & other personal communications
What is the literature?
Primary vs. Secondary Sources (~3 mins)

6. Secondary Sources - reading a report of the report
Literature Reviews
Psychological Bulletin, Annual Review of Psychology
Text books
Citations in books and articles
Advantages:
Good starting place
Often reviews a lot of relevant literature
Relatively brief descriptions
Disadvantages:
Somebody else’s description
May be incorrect
May be biased
Not enough detail
What is the literature?
Primary vs. Secondary Sources (~3 mins)

7. Reviewing the literature
How do you search the literature?
PsycInfo is a powerful tool
At Milner Library web page
Some tutorials: video (~4 mins) | video (~5 mins) | video (~55 mins) | more videos here
Google Scholar is an increasingly powerful tool
Use the reference sections at the end of articles: An “old school method” that is still a really good one
Reviewing the literature

8. Developing your Research Idea
Claim: People perform best with a good night of sleep.
Sleep walking story (2)
Sleep and high school (2)
Science of sleep
To begin to answer the claim we’ve got to FOCUS the idea
Break the general idea down into smaller more specific ideas
Develop theories/hypotheses as to how & why
EVALUATE the idea (e.g., the ROT test)
TEST the idea: using research methods to test parts of the theory (hypotheses)
Developing your Research Idea

9. Developing your Research Idea
Claim: People perform best with a good night of sleep.
Focusing the idea
What do we mean by “perform best”?
Academic performance?
Physical performance?
What do we mean by “good night sleep”?
8 hrs?, Uninterrupted?, 2 hours of REM?
What is the underlying theory? What hypotheses do we test?
e.g., Consolidation of memories happen during REM sleep, so getting more REM sleep should lead to better recall
Operational definitions
Developing your Research Idea

10. Developing your Research Idea
Claim: People perform best with a good night of sleep.
Evaluating the idea (ROT)
Can we replicate the research, do we get similar results?
Answer may depend on how you choose to make your observations (your research methods)
How do we observe performance? How do we observe good sleep?
Recall tests, recognition tests, “brain waves,” ,,,
Are our predictions testable?
Developing your Research Idea

11. Developing your Research Idea
Claim: People perform best with a good night of sleep.
How might we go about trying to test this claim?
What are the things (variables) of interest?
What is the hypothesized relationship between these variables?
How should we test it?
How do we observe the behavior?
What research design should we use? What are our goals?
Developing your Research Idea

12. Conducting Research Observational approaches: Data collection
How do we observe the behaviors of interest?
Types of research designs
What kinds of research questions are you investigating?
E.g., Cause and effect? Descriptive?
Descriptive: Describe variables as they exist
Observational
Survey
Case studies
Correlational - measures two (or more) variables in order to describe the relationship between them
Experimental: Systematic manipulation and observation of non-naturally occurring events
Conducting Research

13. Observational Methods
Observational approaches: Data collection
How do we observe the behaviors of interest?
Naturalistic observation
Participant observation
Survey & interviews
Archival data
Systematic (contrived/analog) observation
Experiments
Observation without manipulation
Direct Observation
Descriptive: Describe variables as they exist
Observational
Survey
Case studies
Correlational - measures two (or more) variables in order to describe the relationship between them
Experimental: Systematic manipulation and observation of non-naturally occurring events
Observational Methods

14. Observational Methods
Naturalistic Observation: Observation and description of behaviors within a natural setting
High external validity
Good for behaviors that don’t occur (as well) in more controlled settings
Often a first step in the research project
Can be difficult to do well
Hard not to influence things (reactivity effect)
Takes a long time
Need multiple observers to agree
Jane Goodall
helps to identify what some of the important variables are
then the next step is to move into more controlled settings for further tests
Dian Fossey
Observational Methods

15. Observational Methods
Participant Observation: The researcher engages in the same behaviors as those being observed
May allow observation of behaviors not normally accessible to outside observation
Internal perspective from direct participation
But could lead to loss of objectivity
Potential for contamination by observer
Observational Methods

16. Observational Methods
Survey methods: Questionnaires and interviews that ask people to provide information about themselves
Widely used methodology
Best way to collect some kinds of information:
Descriptive, behavioral, and preferential
e.g., demographic information, recreational behavior, and attitudes
Large amounts of data can be collected quickly with relatively little cost (effort, time, etc.)
But they’re often not as “cheap” as you may think
Done correctly, can be a very difficult method
Widely used methodology
those annoying things in the mail and by phone
US Census
Can collect a lot of data
Lots of participants in a short amount of time
Can collect subjective information
Done correctly, can be a very difficult method
constructing good questions, rating scales, etc.
Doesn’t provide clear cause-effect patterns
Observational Methods

17. Observational Methods
Archival data: Rather than making direct observations, researcher examines existing public or private records
If the appropriate existing records can be found, no need for data collection
Data set may be more extensive than what you could collect yourself
However, you are limited to the data that exists, may be no way to collect follow-up data
Data may be of observations that you cannot (ethically) collect or manipulate
E.g., murder rates, who marries whom, etc.
Word of caution: be aware of how and where the data were collected
Observational Methods

18. Observation without manipulation
Advantages
Complex patterns of behavior in particular settings
Useful when little is known about the subject of study
May learn about something that never would have thought of looking at experimentally
Disadvantages
Causality is a problem
Threats to internal validity because of lack of control
Every confound is a threat
Lots of alternative explanations
Directionality of the relationship isn’t known
Sometimes the results are not reproducible
Observational Methods

19. Observational Methods
Systematic (Contrived/Analog Observation): The observer sets up the situation that is observed
Observations of one or more specific variables made in a precisely defined setting
Much less global than naturalistic observations
Often takes less time
However, since it is not a natural setting, the behavior may be changed
Observational Methods

20. Types of research designs
Case studies
Intensive study of a small set of individuals and their behaviors
Correlational
Looking for a co-occurrence relationship between two (or more) variables
Quasi-experimental
Experimental designs with one or more non-random variables
Experimental
Investigating the cause-and-effect relationship between two (or more) variables through the manipulation of variables
Descriptive: Describe variables as they exist
Observational
Survey
Case studies
Correlational - measures two (or more) variables in order to describe the relationship between them
Experimental: Systematic manipulation and observation of non-naturally occurring events
Types of research designs

21. Intensive study of a single person, a very traditional method
Intensive study of a single person, a very traditional method. Typically:
Descriptive (and non-experimental).
Interesting (and often rare) case. Fits well with clinical work.
Phineas Gage (Sci. Am. Show)
Sept 13, 1848 Explosion propelled a railroad tamping rod through his brain
Changed personality
This view has some disadvantages
There may be poor generalizabilty
There are typically a number of possible confounds and alternative explanations
Intensive study of a single person, a very traditional method
Get a very detailed description
Fits well with clinical work
Typically an interesting (and often rare) case
The man who mistook his wife for a hat
NA (fencing accident, amnesia)
HM (surgery for epilepsy, amnesia)
PZ (Korsakoff’s syndrome, amnesia, wrote autobiography a couple of years before the memory loss)
Case Histories
See: Oliver Sacks’ books for some other great examples
interview

22. Correlational Methods
Measure two (or more) variables for each individual and see if the variables co-occur (suggesting that they are related)
Used for:
Predictions
Establishing Reliability and Validity
Evaluating theories
Limitation: Shouldn’t make casual claims X Y ? or or
Correlational Methods

23. X Y Causal claims X Y We’d like to say: To be able to do this: or
There must be co-variation between the two variables
The causal variable must come first
Directionality problem
Happy people sleep well
Or is it that sleeping well when you are happy?
Need to eliminate plausible alternative explanations
Third variable problem X Y
causes X Y or
Do Storks bring babies?
Neyman (1952) reported a strong positive correlation between number of babies and stork sightings
Causal claims

24. Causal claims r = 0.63 Do Storks bring babies? Source: Kronmal (1993)
Neyman (1952) reported a strong positive correlation between number of babies and stork sightings
Birthrate: To adjust for population size both the number of storks and number of babies was divided by the number of women (per 10,000 women) to form the birth rate and stork rate.
Causal claims

25. Theory 1: Storks deliver babies
Is killing storks and effective method of controlling birth rates?
Theory 1: Storks deliver babies

26. Theory 2: Underlying third variable

27. The experimental method
Manipulating and controlling variables in laboratory experiments
Must have a comparison
At least two groups (often more) that get compared
One groups serves as a control for the other group
Variables
Independent variable - the variable that is manipulated
Allows for the testing of causal hypotheses
Dependent variable - the variable that is measured
Control variables - held constant for all participants in the experiment
Experimental control will be discussed in more detail in a later lecture. It includes keeping the value of a variable constant for all conditions as well as allowing the variable to vary randomly (and hopefully equally across the experimental conditions)
The experimental method

28. The experimental method
Manipulating and controlling variables in laboratory experiments
Advantages
Precise control possible
Precise measurement possible
Theory testing possible
Can make causal claims
Disadvantages
Artificial situations may restrict generalization to “real world”
Complex behaviors may be difficult to measure
The ability to make causal claims comes from the use of control:
Holding variables constant
Selecting the levels of the independent variable
Random assignment to conditions
The experimental method