1. Chapter 8 Experimental Design
RCS 6740
Supplemental

2. Refresher of the Experimental Method
Experimental Method: A method of determining whether variables are related in which the researcher manipulates the INDEPENDENT variable and controls all other variables (DEPENDENT) either by randomization or by direct experimental control (Cozby, 2004).

3. Refresher of the Experimental Method
Random Sampling vs. Random Assignment
Random Sampling: Using a random table of numbers, a researcher selects people from a sample to become participants in a study
Random Assignment: Participants (already chosen using random sampling) are assigned to groups using a random table of numbers

4. Refresher of the Experimental Method Cont.
So, a true experimental study needs:
Randomization and
Control
Independent Variable: The variable that is manipulated to observe its effect on the dependent variable.
Dependent Variable: The variable that is the subject’s response to, and dependent on, the level of the manipulated independent variable

5. Refresher of the Experimental Method Cont.
Example: To measure the effects of crowding (Independent variable) on cognitive performance (Dependent variable), participants are randomly assigned to one of two groups:
Group 1 takes a test in a crowded room
Group 2 takes a test alone in a room
Is this a good example of the experimental method?

6. Refresher of the Experimental Method Cont.
Yes, if you accounted for and minimized other differences between the groups!
Confounding Variable: A variable that is not controlled in an experiment and that can effect the dependent variable.
So, from the study on crowding and cognitive performance, what are possible confounding variables?

7. Refresher of the Experimental Method Cont.
Possible Confounding Variables in the Crowding Experiment
Groups took tests in different rooms, one with a window, one without
One group took an algebra test, the other took a statistics test
One group took the test at 7:30 AM, the other at 7:30 PM
One group was proctored by Dr. Spitznagel, the other by Snoop Dog

8. Refresher of the Experimental Method Cont.
Good experimental designs involve eliminating confounding variables that may result in alternative explanations
When results of an experiment can confidently be attributed to the effect of the independent variable on the dependent variable, a study is said to have good Internal Validity.
Therefore, researchers need to design experiments that account for an minimize possible alternative explanations

9. Types of Experimental Design
Basic Experiments
Posttest only Design
Pretest-Posttest Design
Independent Groups Design
Repeated Measures Design
Counterbalancing
Matched Pairs Design
Developmental Research Designs
Cross-Sectional Design
Longitudinal Method
Sequential Method

10. Coding for Lecture O = observation or measure
To simplify the designs, the following coding system will be used throughout the remainder of this lecture
O = observation or measure
X = experimental intervention or manipulation
R = random assignment to groups

11. Basic Experiments The most basic experimental design has two variables
Independent Variable
Dependent Variable
The independent variable has two Levels
Experimental Group (Usually receives treatment)
Control Group (Usually does not receive treatment)
A study can also have two different amounts of an independent variable
10 mg of Prozac for one group and 20 mg of Prozac for another group
Example: A Randomized and Controlled study looking at the effects of exercise (Independent) on body fat (Dependent)
Group 1 exercises 3 times a week for 6 weeks
Group 2 does not exercise at all for three weeks
Researchers will compare the body fat of those who exercise to those who do not. The question is, when should body fat be measured?

12. Basic Experiments: Posttest Only
Posttest Only Design
A researcher using a Posttest only design must:
Obtain two equivalent groups*
Introduce the independent variable
Measure the effect of the independent variable on the dependent variable
* To eliminate selection differences among participants, a researcher needs to either randomly assign participants to conditions or have the same participants participate in both conditions

13. Basic Experiments: Posttest Only
(R) X O
(R) O
(R) X (Exercise/IV) O (Body Fat Measure/DV)
(R) O (Body Fat Measure/DV)
Upon completion of all interventions, the researcher will measure the effects of the independent variable on the dependent variable to look for statistically significant differences

14. Basic Experiments: Posttest Only
Posttest only design using our exercise example:
Participants are randomly assigned to one of two groups. Group 1 exercises 3 times a week for 6 weeks and Group 2 does not exercise for 6 weeks. After 6 weeks, both groups will undergo a test to gauge their body fat. It was found that people in the exercise group had 5% less body fat than the people who did not exercise. Results of this study are statistically significant.

15. Basic Experiments: Posttest Only
Advantages of the Posttest Only Design
Less expensive and time consuming than Pretest-Posttest
Does not sensitize participants to what you are studying
Pretests are not usually given in the real world so this design provides the researcher with the most confidence for findings generalization
In this design the possibility of the interaction of the pretest effects and the independent variable are not present, therefore that source as a potential threat to external validity is eliminated

16. Basic Experiments: Posttest Only
Disadvantages of Posttest Only Design
Does not enable a researcher to account for mortality
Mortality: People dropping out in studies
Smoking Study example
Does not establish a baseline
Does not give a researcher before and after data to analyze statistically

17. Basic Experiments: Pretest-Posttest Design
With a Pretest-Posttest design, a researcher must:
Obtain two equivalent groups*
Introduce the independent variable
Measure the effect of the independent variable on the dependent variable
* A larger sample size increases the likelihood that the that the groups will differ. A good rule of thumb is 20 to 30 people per condition

18. Basic Experiments: Pretest-Posttest Design
(R) O X O
(R) O O
Participants are randomly assigned to one of two groups. Group 1 exercises 3 times a week for 6 weeks and Group 2 does not exercise for 6 weeks. Before the study begins, a measure of body fat is taken from participants in each group. After 6 weeks, both groups will undergo the same body fat test. It was found that the people in the exercise group reduced their body fat (pretest to posttest) by a mean of 10%. The people in the non-exercise group reduced their body fat by a mean of 1%. Results of this study are statistically significant.

19. Basic Experiments: Pretest-Posttest Design
Advantages of the Pretest-Posttest Design
Allows a researcher to account for mortality
Establishes a baseline
Enables a researcher to do before and after analyses of data
Allows the researcher to establish whether or not groups were random/equal to begin with
Allows researcher the opportunity to select participants for a study
Matching (discussed in detail later in lecture)
Smoking Example: I smoke regularly but answers only 1 a day on pretest

20. Basic Experiments: Pretest-Posttest Design
Disadvantages of the Pretest-Posttest Design
More costly and time consuming than Posttest only design
May sensitize participants to the study
A researcher can disguise the pretest by:
Having a different experimenter give it in a different setting
Embed the pretest with a few irrelevant measures
Does not allow for confidence in generalizing results as pretests are not given in the real world

21. Combination of Pretest only and Pretest-Posttest Designs
The Solomon Four Group Design combines pretest-posttest and posttest only designs
1/2 participants receive pretest and posttest
1/2 participants receive only posttest
Randomly assigned
To examine and control for effects of pre-test
If there is no impact of the pretest, posttest scores for both groups (pre and pre/post) should be equivalent

22. Solomon Four Group Design: No Pretest Effect (Posttest Scores)

23. Solomon Four Group Design: Pretest Effect (Posttest Scores)

24. Assigning Participants to Experimental Conditions
Independent Groups Design
Participants are randomly assigned to various conditions so that each participates in only one group
Repeated Measures Design
Participants participate in all conditions and are eventually assigned to all levels of the Independent Variable

25. Independent Groups Design
Participants are randomly assigned to each condition
Example: Using our exercise and body fat study, participants are randomly assigned to one of two groups
Group 1 exercises 3 times a week for 6 weeks
Group 2 does not exercise for 6 weeks

26. Repeated Measures Design
Participants participate in all groups/conditions and are measured repeatedly on the dependent variable after each condition
Example: To study the efficacy of Speech and Language Pathology treatments on Aphasia, each participant is assigned to 3 groups: Melodic Intonation, Errorless Traditional, and Errorless Mapping. After completion of each treatment, the participants will take the Jackson Speech Assessment Inventory.
Remember, each participant participates in each group

27. Repeated Measures Design
Advantages of the Repeated Measures Design
Fewer Participants are needed
Saves time and money
Repeated Measure Designs are extremely sensitive to statistical differences (significant) that are not as evident with the Independent Groups Design
Allows individual differences and random error to be seen

28. Repeated Measures Design
Score of Jackson Measure after ET
Score of Jackson Measure after EM
Difference
Participant 1 68 64 +4
Participant 2 81 78 +3
Participant 3 92 85 +7

29. Repeated Measures Design
Disadvantages of the Repeated Measures Design
Different conditions must be presented in a particular sequence
Order Effect: The order of treatments may effect the dependent variable
Example: In our speech study, lets say that everyone received Melodic Intonation first. Most of the participants improved on the Jackson Speech Assessment Inventory after the other two treatments (ELT & ELM) due to a technique learned from Melodic Intonation. Can you attribute this gain to the effects of ELT or ELM?

30. Repeated Measures Design
Types of Order Effects
Practice Effect
Improvement in performance as a result of repeated practice
Fatigue Effect
Drop in performance due to being tired, bored, or distracted
Contrast Effect
Responses to the second condition of an experiment are altered because both conditions are contrasted to one another
Crime Study Example
Seeing a Mild Crime First and then Seeing a Severe Crime Second
Seeing a Severe Crime First and then seeing a Mild Crime Second
Basically, seeing one first will influence how you perceive the other

31. Repeated Measures Design
There are two way to account for and minimize order effects
Employ Counter Balancing Techniques
Ensure that adequate time elapses between conditions

32. Counterbalancing
Counterbalancing entails including all possible orders of treatment/conditions in an experiment
It helps researchers identify any order effects
Example: Lets use our speech study and focus on ELM and ELT…

33. Counterbalancing Participants 1 through 10 receive ELT first
Participants 1 through 10 receive ELM second
Participants 11 through 20 receive ELM first
Participants 11 through 20 receive ELT second

34. Counterbalancing
Counterbalancing can be extended to experiments with three or more groups
What if the number of possible orders becomes too big?
One study has 3,628,800 orders!
A Latin Square controls for order effects without having all possible orders

35. Latin Square
A Latin Square is a limited set of orders constructed to ensure that:
Each condition appears at each ordinal position
Each condition precedes and follows each condition one time
For experiments with an extreme number of orders, a researcher may choose to randomly select a few orders to construct a Latin Square

36. Latin Square (3 Groups, 3 Treatments)

37. Randomized Blocks
Some Repeated Measures Design experiments with multiple treatment orders are repeated over and over again to look for effects
Example: Assigning participants a “lucky number” and seeing what mood it puts them in.
Each repetition of the basic experiment (i.e. assigning of the first lucky number) is called a Block of Trials
To control for order effects, the assigning of the numbers should be done in a random manner

38. Time Interval Between Treatments
In addition to counterbalancing, researchers need to carefully control how much time elapses between treatments/interventions
Examples:
Giving a sufficient rest period after a month of therapy
Allowing enough time to let a drug wear off
Also, ensuring that too much time does not elapse as participants may become upset and drop out

39. When to Choose an Independent Groups Design or a Repeated Measures Design
Basically, time, money, and number of participants will affect your choice
Also, it will depend on how your study generalizes to the “real world”
Example: If you want to look at how characteristics of a defendant affect a juror, you would probably use an Independent Groups Design (jurors focus on only one defendant)
If you wanted to see the effects of job applicants on employers, you would probably use the Repeated Measures Design (employers interview multiple applicants)

40. Matched Pairs Design
The Matched Pairs design is a way of assigning participants to groups by using, and focusing on, certain participant characteristics (Matching Variable)
The goal is to achieve the same equivalency of groups as the Repeated Measures Design without having the same participants in multiple conditions

41. Steps of the Matched Pairs Design
Obtain a measure of the matching variable from each participant
Example: IQ scores
Rank the participants from highest to lowest on the matching variable
Example: (136, 126, 118, 118, 103, 101)
Match pairs that are approximately equal on the matching variable
Example: (136 and 126, 118 and 118, 103 and 101)
Finally, randomly assign members of each pair to one of two treatment groups
Example: 136 goes to ELT and 126 goes to ELM, 118 goes to ELM and 118 goes to ELT, 103 goes to ELT and 101 goes to ELM

42. Analysis of Covariance
Analysis of Covariance is another way to statistically control the correlation between a subject variable (e.g. IQ scores) and the dependent variable.
Basically, information on the subject variable is collected after the fact and analyzed statistically to determine if it is causing any effect on the dependent variable

43. Developmental Research Designs
Cross-Sectional Method
Longitudinal Method
Sequential Method

44. Cross-Sectional Method
In the cross-section method individuals at different ages are studied at one point in time on the dependent variable.
Health care use (DV) and age (IV)
Example: Health care use of a 10, 15, 20, 30, 45, and 60 year old person are examined at the same time
This method saves time, but uses different participants to study the effects of the independent variable (aging).
Beware of the Cohort Effect where change may be based on comparisons between cohorts (same age group of people) rather than the developmental change of age
Did your Grandparents have a Playstation?

45. Longitudinal Method
The longitudinal method studies the same individuals over time and take measurement on the dependent variable at specified times.
Example: Health care use of a group of 10 year olds is gauged. The same group will have their health care use gauged at 15, 20, 25, and 30 years of age
Good design but people drop out, move, and pass away

46. Sequential Method
The sequential method is a mixed method beginning with the cross-sectional method and then using the longitudinal method for at least one more set of measurements on the dependent variable
Example: Health care use of 55 and 65 year olds are gauged. Then, 5 years later, both groups’ use of health care will be gauged again

47. Questions or Comments ?