1. Experimental Research Designs

2. Experimental Design Advantages Disadvantages
Best establishes cause-and-effect relationships
Disadvantages
Artificiality of experiments
Feasibility
Unethical
Difficult to establish cause-and-effect.
Correlational research often done first to establish relationships that may be examined for cause-and-effect.
Cause-and-effect are not established by statistics but rather by logical thinking and sound research design. You must establish that no other plausible explanation exists for the changes in the DV except the manipulation done to the IV.

3. Causality Temporal precedence Covariation between IV and DV
Eliminate alternative explanations

4. Types of Experimental Designs
Simple True Experimental
Complex True Experimental
Quasi-Experimental

5. Types of Experimental Designs
Simple True Experimental
Complex True Experimental
Quasi-Experimental

6. Simple True Experimental
Characteristics
Types
Variations

7. Characteristics of True Designs
Manipulation (treatment)
Randomization
Control group
Characteristics of simple true designs
One IV with 2 levels (T, C)
One DV

8. Types Randomized posttest control group design
Randomized pretest-posttest control group design
Random groups, controls for past history, maturation, testing, and sources of invalidity based on nonequivalent groups (statistical regression, selection biases, selection-maturation interaction
Investigator must control present history, instrumentation, experimental mortality

9. Randomized posttest control group design
R T Post
R C Post

10. Randomized pretest-posttest control group design
R Pre T Post
R Pre C Post
Makes it possible to ascertain that groups were equivalent at the beginning of the study.
Not necessary if
Randomization was used
Large sample size

11. Advantages & Disadvantages
Advantages of pretest design
Equivalency of groups
Can measure extent of change
Determine inclusion
Assess reasons for and effects of mortality
Disadvantages of pretest design
Time-consuming
Sensitization to pre-test

12. Solomon four-group design
R Pre T Post
R Pre C Post
R T Post
R C Post

13. Variations Independent groups (between groups)
Repeated measures (within groups)

14. Repeated Measures Design
Advantages:
Fewer subjects needed (less costly)
Sensitive to finding statistical differences
Disadvantages:
Order effect (practice, fatigue, carry-over)
Advantages:
Fewer subjects needed (less costly)
Sensitive to finding statistical differences because of control over participant differences
Disadvantages:
Order effect (practice, fatigue, carry-over)

15. Dealing with Order Effects
Counterbalancing n!
Latin squares
Counterbalancing - all possible orders of presentation are included in experiment
Latin squares – a limited set of orders constructed to ensure that
Each condition appears at each ordinal position
Each condition precedes and follows each condition one time

16. Latin Squares 1 2 3 4 Row 1 Row 2 Row 3 Row 4 A (60) B (0) D (120) C
(180)
Row 2
Row 3
Row 4

17. Dealing with Order Effects
Counterbalancing n!
Latin squares
Randomized blocks
Time interval between treatments
Time interval – may counteract the effects of treatment, but also increases time demands on subjects

18. Consider external validity when deciding which design to use.
Variations
Independent groups (between) vs. repeated measures (within) designs
Consider external validity when deciding which design to use.

19. Types of Experimental Designs
Simple True Experimental
Complex True Experimental
Quasi-Experimental
Pre-Experimental

20. Characteristics of True Designs
Manipulation (treatment)
Randomization
Control group
Characteristics of simple true designs
One IV with 2 levels (T, C)
One DV

21. Complex True Experimental
Randomized matched control group design
Increased levels of IV
Factorial design
Multiple DVs

22. Complex True Experimental
Randomized matched control group design
Increased levels of IV
Factorial design
Multiple DVs

23. Randomized matched control group design
M R T Post
M R C Post
Obtain measure of matching variable from each subject
Rank from highest to lowest based on score
Form matched pairs
Randomly assign members of pairs to conditions
Used in small samples
 cost in time & money

24. Complex True Experimental
Randomized matched control group design
Increased levels of IV
Factorial design
Multiple DVs

25. Increased Levels of IV
Provides more complete information about the relationship between the IV & DV
Detects curvilinear relationships
Examines effects of multiple treatments

26. DV
Performance level (% complete) IV
Amount of reward promised ($)

27. Increased Levels of IV DV IV Performance level (% complete)
Amount of reward promised ($)

28. Complex True Experimental
Randomized matched control group design
Increased levels of IV
Factorial design
Multiple DVs

29. Factorial Design 2X2 >1 IV (factor)
Simultaneously determine effects of 2 or more factors on the DV (real world)
Between Factor vs. Within Factor
ID’d by # of factors and levels of factors
Use Figures 10.1 and 10.2 to demonstrate graphically at end of slide.
2X2

30. Do differing exercise regimens (hi, med, lo intensity) have the same effect on men as they do on women?
3 X 2 (Exercise Regimen X Gender)
2 factors
Exercise Regimen – 3 levels
Gender – 2 levels
Between factors
DV?
Experimental IVs or Participant IVs?

32. Do strength gains occur at the same rate in men as they do in women over a 6 mo. training period? Measurements are taken at 0, 2, 4, 6 mo.
2 X 4 (Gender X Time)
? factors
Time – 4 levels
Gender – 2 levels
Between or within factors?
DV?
Experimental IVs or Participant IVs?

34. Cell means, Margin means Main Effects, Interactions
Grand mean

36. Parallel lines indicate no interaction.
Is there
a main
effect?

37. Is there
a main
effect?

38. Non-parallel lines indicate an interaction.
Is there
a main
effect?

39. Is there
a main
effect?

40. Interpretation Always interpret the interaction first (graphical)
If no significant interaction, interpret main effects

41. Consider external validity when deciding which design to use.
Advantages of factorial designs:
Greater protection against Type I error
More efficient
Can examine the interaction
Disadvantages:
 subject # for between factor designs
Advantages of factorial designs:
Greater protection against Type I error
More efficient (1 analysis vs. multiple one-ways)
Can examine the interaction (not possible with one-way ANOVAs)
Disadvantages:
Only for fixed models (levels of IV chosen by researcher) and when subjects are assigned randomly
Consider external validity when deciding which design to use.

42. IV A: Exposure to Violence – violent vs
IV A: Exposure to Violence – violent vs. nonviolent video IV B: Gender – male vs. female DV: # ads recalled (0-8) B1 B2 9 B 1 2 5 3 9 7 A 5 1 1 2 A

43. IV A: Exposure to Violence – violent vs
IV A: Exposure to Violence – violent vs. nonviolent video IV B: Gender – male vs. female DV: # ads recalled (0-8) B1 B2 9
A: Yes
B: No
AxB: Yes 5 1 1 2 A

44. Complex True Experimental
Randomized matched control group design
Increased levels of IV
Factorial design
Multiple DVs

45. Do strength gains occur at the same rate in men as they do in women over a 6 mo. training period? Measurements are taken at 0, 2, 4, 6 mo.

46. Types of Experimental Designs
Simple True Experimental
Complex True Experimental
Quasi-Experimental

47. Characteristics of True Designs
Manipulation (treatment)
Randomization
Control group
Less control
More real-world
Program evaluation

48. Randomized posttest control group design
R T Post
R C Post

49. Randomized pretest-posttest control group design
R Pre T Post
R Pre C Post
Makes it possible to ascertain that groups were equivalent at the beginning of the study.
Not necessary if
Randomization was used
Large sample size

50. Quasi-experimental Designs
One group posttest-only design
One group pretest-posttest design
Non-equivalent control group design
Non-equivalent control group pretest-posttest design
Time series
Single subject designs (Case study)
Developmental designs

51. Quasi-experimental Designs
One group posttest-only design
One group pretest-posttest design
Non-equivalent control group design
Non-equivalent control group pretest-posttest design
Time series
Single subject designs (Case study)
Developmental designs

52. Randomized posttest control group design
R T Post
R C Post

53. One group posttest-only design (One shot study)
T Post
No control of IV threats
Use?

54. Quasi-experimental Designs
One shot study
One group pretest-posttest design
Non-equivalent control group design
Non-equivalent control group pretest-posttest design
Time series
Single subject designs (Case study)
Developmental designs

55. Randomized pretest-posttest control group design
R Pre T Post
R Pre C Post
Makes it possible to ascertain that groups were equivalent at the beginning of the study.
Not necessary if
Randomization was used
Large sample size

56. One group pretest-posttest design
Pre T Post
History
Maturation
Testing
Instrument decay
Regression
Use control group

57. Quasi-experimental Designs
One shot study
One group pretest-posttest design
Non-equivalent control group design
Non-equivalent control group pretest-posttest design
Time series
Single subject designs (Case study)
Developmental designs

58. Randomized posttest control group design
R T Post
R C Post

59. Non-equivalent control group design (Static group comparison design)
T Post
C Post
Selection bias

60. Quasi-experimental Designs
One shot study
One group pretest-posttest design
Non-equivalent control group design
Non-equivalent control group pretest-posttest design
Time series
Single subject designs (Case study)
Developmental designs

61. Randomized pretest-posttest control group design
R Pre T Post
R Pre C Post
Makes it possible to ascertain that groups were equivalent at the beginning of the study.
Not necessary if
Randomization was used
Large sample size

62. Non-equivalent control group pretest-posttest design
Pre T Post
Pre C Post
Can check selection bias

63. Quasi-experimental Designs
One shot study
One group pretest-posttest design
Non-equivalent control group design
Non-equivalent control group pretest-posttest design
Time series
Single subject designs (Case study)
Developmental designs

64. Pre Pre Pre Pre T Post Post Post Post
Time series
Pre Pre Pre Pre T Post Post Post Post

65. Quasi-experimental Designs
One shot study
One group pretest-posttest design
Non-equivalent control group design
Non-equivalent control group pretest-posttest design
Time series
Single subject designs (Case study)
Developmental designs

66. Quasi-experimental Designs
One shot study
One group pretest-posttest design
Non-equivalent control group design
Non-equivalent control group pretest-posttest design
Time series
Single subject designs (Case study)
Developmental designs

67. Developmental Research Designs
Longitudinal
Powerful (within subject)
Time consuming
Attrition
Testing effect
Cross Sectional
Less time consuming
Cohorts problem
Developmental research – studies the ways that individuals change as a function of age; age is the independent variable
Longitudinal (similar to repeated measures)
Powerful (within subject) but several problems
Time consuming
Attrition due to move, death, school rezoning may change sample characteristics (e.g., more obese subjects die, leaving non-obese subjects in sample – knowledge about obesity is not changing but rather sample is changing)
Subjects become familiar with test items (learning effect or items may cause change in behavior)
Cross-sectional (similar to independent groups)
Less time consuming, but problems
Cohorts – a group of people born at about the same time, exposed to same events in society, and influenced by same demographic trends such as divorce rates and family size. Are all age-groups really from same population? Are environmental circumstances that affect jumping performance the same today for 6 yr olds as they were when the 10 yr olds were 6?

68. Choosing a Research Design
Best addresses the problem
Ethics
Cost in time and money
Validity (internal & external)