1. Lecture Outline Introduction to experimental research
Introduction to correlational research
Three types of research validity
Threats to research validity

2. Experimental Research Overview and Major Features
An independent variable (IV) is manipulated
A dependent variable(s) (DV) is measured
Many basic experiments consist of two levels of the independent variable
experimental group
control group
Control over extraneous variables
holding constant
randomizing effects
A causal relationship between the independent and dependent variables can be established

3. Basic Goal of Experimental Design
Manipulate possible values of the IV
Observe behavioral changes as indexed by values of the DV

4. Basic Characteristics of Experimental Designs
As part of randomizing effects of extraneous variables, subjects are randomly assigned to research groups
A given participant must be equally likely to be assigned to any of the given groups
Random number table
Flip a coin
In experimental designs, a causal relationship between the independent and dependent variables can be established

5. Independent Variable Differing values of the IV are called “levels”
May vary quantitatively
10 mg day
20 mg day
e.g. here are 2 levels of variable dosage
May vary qualitatively
Zoloft
Imipramine
Prozac
E.g. here are 3 levels of the variable medication type

6. Operational Definitions
An operational definition is a clearly defined set of procedures for obtaining a measure of the construct of interest.
The key to an acceptable operational definition is that the procedure is specified precisely enough to allow replication by others.
Examples:
quality of memory -- accuracy of recall in a certain task
depression -- Beck Depression Inventory (survey) score
arousal -- galvanic skin response (conductivity of the surface of the skin)

7. Operational Definition
Operational Definitions: Examples
He is a good car
salesperson
He:
1. Sells many cars
2. Points out + features
3. Is nice
4. Sells only good cars
Verbal Statement
Operational Definition

8. Operational Definition
Operational Definitions: Examples
Stress
1. Environment/Number of stressors
a. Number negative Life events
b. Number of Examinations
2. Self-report
Measurable Nervous mood
3. Behavior
a. number of fidgets/minute
b. Psychophysiological
responses elevated
Verbal Statement
Operational Definition

9. Identify Key Experimental Features
Social loafing refers to people’s tendency to slack on group tasks. In one study on social loafing (Latane, Williams and Harkins) college participants cheered alone, and in groups of 2, 4, or 6 ppl. They were told to cheer as loud as possible at specific times so researchers could determine how much noise people made in social settings. The amount of noise made by each participant dropped as group size increased.
1) What is the IV?
2) What is the DV?
3) Why is random assignment important here?

10. Correlational Research: Major Features
No independent variables are manipulated
Two or more dependent variables are measured and a relationship established
Correlational research does not show causality

11. When To Use Correlational Research
Gathering data in early stages of research
Manipulating an independent variable is:
Impossible
Unethical
Relating > 2 naturally occurring variables

12. Correlational Research
2 Variables
Positive and Linear
Negative and Linear

13. Example of Non-Linear Relationship Yerkes-Dodson Law
Better
Performance
Worse
Low
Arousal
High

14. Correlational Research: Major Points
Don’t confuse statistics with research design
Correlation coefficients ( a statistic) can be used in correlational or experimental research designs (although they are more commonly used in correlational designs)
Correlational research simply denotes:
an ASSOCIATION or
a RELATIONSHIP between variables
Correlational research is “silent” about causality

15. Correlation Does Not Imply Causation
Causality – must demonstrate that variance in one variable can only be due to influence of the other variable
Directionality of Effect Problem
Third Variable Problem

16. Directionality of Effect ProblemX Y X Y X Y

17. Directionality of Effect ProblemX Y
Class
Attendance
Higher
Grades X Y
Class
Attendance
Higher
Grades

18. Directionality of Effect ProblemX Y
Aggressive Behavior
Viewing Violent TV X Y
Aggressive Behavior
Viewing Violent TV
Aggressive children may prefer violent programs or
Violent programs may promote aggressive behavior

19. Third Variable ProblemX Y Z

20. that influences both X and Y
Class Exercise
Identify the
third variable
that influences both X and Y

21. Third Variable Problem+
Class
Attendance
GPA

22. Third Variable Problem+
Number of
Churches
Crime
Rate

23. Third Variable Problem+
Ice Cream
Consumed
Number of
Drownings

24. Measurement Considerations
Variables should be measured accurately and consistently.
Accuracy
Validity
Bathroom scale gives correct weight
Consistency
Reliability
Bathroom scale gives same weight consistently

25. Evaluating Research: 3 Validities
Validity
Are we measuring/manipulating what we say we are?
Three validities
Construct validity
Internal validity
External validity

26. Construct Validity Schacter and Singer (1962)
The degree to which the construct can be inferred from the operational definition of that construct
Never one perfect operationalization
Schacter and Singer (1962)
Two-factor theory of emotion:
Emotions require arousal and label
Factor 1: AROUSAL
Participants received
epinephrine shots and either
informed them of side-effects or not
Factor 2: LABEL
Participants around "euphoric"
or "angry“ confederate

27. How Valid are These Manipulations?
Euphoria
Confederate shoots rubber bands, plays with hula hoops and crumples up paper and practices hook shots into wastebasket
Anger
Confederate complains about the injection, adds negative comments about the questionnaire they are filing out and ends up ripping up the questionnaire.

28. Simplified Results Euphoria Condition Anger Informed of
epinephrine effects
Uninformed of

29. Internal and External Validity
Internal validity
The extent to which the observed effect is caused only by the experimental treatment condition
The ability to draw conclusions about a causal relationship from our data
Experiments usually high in this
External validity
The extent to which the results can be applied to and across different persons, settings and times
AKA generalizability
Experiments often low in this

30. Generalizing From the Lab Setting
Mundane realism
The extent to which an experiment is similar to real-life situations
Psychological realism
The extent to which an experiment triggers relevant psychological and group processes
Key point
Lab experiments not automatically artificial
When an experiment lacks mundane realism it may be very realistic in terms of psychological realism

31. Assess Internal & External Validity
Does lighting affect productivity?
Study 1:
Give people a production task in a laboratory. The lab room is either bright or dim. Measure and compare their productivity (how quickly and accurately they perform the task).
Study 2:
Give employees in a well-lit office and those in a dim office a production task. Measure and compare their productivity (how quickly and accurately they perform the task).

32. The Tradeoff Real life situations Lab settings High external validity
Hard to rule out other explanations
Low internal validity
Lab settings
High internal validity
Often artificial and findings can’t be generalized
Low external validity

33. Overview: Threats to Validity
1) People are Different
2) People Change
3) Process of Studying People Changes Them
4) Variables with Treatment Change People

34. 1) People are Different: Individual Differences
Can’t determine if outcome is due to IV or to individual differences
Pitfall of one group experiments
Pseudo-experiments
No control group
Threat to internal validity

35. 1) People are Different: Selection Bias/ Nonresponse Bias
Choosing participants from a nonrepresentative sample
Threat to external validity
President Alf Landon?
Nonresponse bias
When a substantial proportion of those invited to participate refuse to do so

36. 2) People Change History
Events occurring between first and second measurements but not part of the manipulation.
Subjects
Smoking
Measure
Training
Program
DV: Pretest
DV: Posttest
Independent
Variable
Pitfall of one group
pretest-posttest design
Threat to internal validity

37. 2) People Change Maturation
People change over time
Grow older wiser, stronger , healthier, more tired, more bored…
Third
graders
Math Exam
Training
Program
Pitfall of one group pretest-posttest design
Threat to internal validity

38. 3) People Change Regression Toward The Mean
AKA statistical regression
Extreme scores (high or low) change over time (regress to the mean)
High scores become lower
Low scores become higher
Hex of the Sports Illustrated cover
Pitfall of one group
pretest-posttest design

39. Does Studying Something Change it?
"The more precisely the POSITION is determined, the less precisely the MOMENTUM is known"
The Hawthorne effect
An increase in worker productivity produced by the obtrusive observation of that process

40. 3) Studying People Changes People Testing Effects
Simply taking the pretest changes the participant!
May cue people in on what is being studied
Make them more adept at a skill
Students
Science
Exam
Program
Pitfall of one group pretest-posttest design
Threat to internal validity

41. 3) Studying People Changes People Mortality (Attrition)
No, this isn’t that people in your study are dying
Participants drop out of the study before completing it
Couples
Relationship
Status
Marriage
Counseling
Homogeneous attrition
Attrition rates equal across experimental conditions
Threat to external validity
Heterogeneous attrition
Attrition rates different across experimental conditions
Threat to internal validity

42. 3) Studying People Changes People Participant Reaction Bias
Research participants realize they are being studied and behave in a way they normally would not
Good subject role
Participant reactance
Evaluation apprehension
Threats to internal validity

43. Applicant Hiring Experiment
INSTRUCTIONS
This study assesses job applicants capacity to handle stress
You will be interviewing an unemployed person
Your task is to read questions to the job applicant and then give negative responses
After the first eight questions you should read the 'Stress Remarks'
SCENARIO
The applicant appears well-qualified and experienced
The answers to the first eight questions are intelligent, lucid and to the point.
Stress Remark:
"This job is much too difficult for you according to the test".

44. Jane
Jane remembered the mix of male and female participants in the waiting area and noted the fact that the researcher was male. She decided that the experimenter had predicted that female subjects will be less likely to make the stress remarks and she so simply ignored them and said "well done you are doing well, aren't you!" in order to help the researcher's project.

45. Good-Subject Role
Participants consciously/unconsciously try to behave consistent with experimenters hypothesis
Demand characteristics
Feature of an experiment that may inform participants of the purpose of the study

46. Su
Su remembered the mix of male and female participants in the waiting area and noted the fact that the researcher was male. She also decided that the experimenter has predicted that female subjects will be less likely to make the stress remarks and made a point of not only making the stress remarks, but saying it in an aggressive manner - in order to disabuse the researcher of female stereotypes

47. Participant Reactance
Participants try to disconfirm perceived hypotheses

48. John
John looked at the highly professional instruction cards and the expensive video recording equipment and decided this was a test of management skills. So, he attempted to stay calm and read out the questions in a controlled, professional manner in order to show the experimenter that he was made of stern stuff and competent at handling difficult interpersonal situations. However, he was so nervous about doing the task well that he mixed up the order of questions and read them out too quickly.

49. Evaluation Apprehension
Participants try to behave in a way that will portray them most favorably.

50. Controlling for Demand Characteristics
Keep participants in the dark
Cover stories
Schacter and Singer’s emotion study
Unobtrusive observations
One-way mirror
Hot sauce
Radio stations
Ensure anonymity

51. 4) Treatment Changes People Experimenter Bias
AKA expectancy effects
Experimenter expectations biasing results
Two sources of experimenter bias
Unintentionally treat participants differently
Experimenters record the behaviors differently
Threat to internal validity
The story of Clever Hans

52. Expectancy Effects Rosenthal and Jacobson (1966) Students IQ tested
Told teachers some students were going to be “intellectual bloomers”
These students actually chosen at random
At end of school year their IQ tested again
Results
“Ready to bloomers” saw marked improvements
Self-fulfilling prophecy
Teacher expectations are communicated to students who in turn respond to those cues by adjusting their behavior.
The result: the original expectation becomes true.

53. Solution to Expectancy Effects
1) Practice makes perfect (or nearly so)
2) Run all participants in one fell swoop
3) Go high tech!
4) Ignorance can be bliss
Unaware of hypotheses
Unaware of condition
Single blind
Double blind

54. 4) Treatment Changes People Confounds
Some additional variable varies systematically with the independent variable
Really, all threats we have discussed the problem was that something else was accompanying the independent variable.