1. Section 1.3
Experimental Design
Larson/Farber 4th ed.

2. Section 1.3 Objectives
Describe observational studies, experiments, surveys and simulations
Identify control groups, placebo effects, completely randomized experiments
Discuss potential pitfalls that might your data unreliable
Larson/Farber 4th ed.

3. Experiment vs. Observational Study: What’s the Difference?
Chapter 1
Experiment vs. Observational Study: What’s the Difference?
Interpret.
How would you conduct the observational study? How would you conduct the experiment? Compare.
Larson/Farber 4th ed.
Larson/Farber 4th ed

4. Procedures: Observational Study vs. Experiment
Sample two types of people from the population: those who work vs. those who do not.
Find the average energy level.
Compare results.
Larson/Farber 4th ed.

5. Procedures: Observational Study vs. Experiment
Sample a group of people from the population.
Randomly assign them to two groups: work out vs. no workout.
Find average energy level in each group.
Compare results.
Which approach do you think would yield more meaningful results? Discuss and defend your argument.
Larson/Farber 4th ed.

6. How Do We Compare Energy Levels of Different People?
Chapter 1
How Do We Compare Energy Levels of Different People?
Compare.
What are characteristics of each approach? Compare and contrast.
Larson/Farber 4th ed.
Larson/Farber 4th ed

7. Observational Study: What are the Limitations?
Chapter 1
Observational Study: What are the Limitations?
Differences between energy levels cannot be attributed solely to working out
People in better shape might be more likely to work out
People in better shape might have better energy levels
Compare.
Larson/Farber 4th ed.
Larson/Farber 4th ed

8. Experiment: What are the Advantages?
Chapter 1
Experiment: What are the Advantages?
Variables that might contribute to the outcome are equally represented due to random assignment
Random assignment tends to eliminate bias
Differences in average energy levels can be assumed to be the result of working out
We can make a causal statement
Draw conclusions.
Larson/Farber 4th ed.
Larson/Farber 4th ed

9. Confounding Variables
Chapter 1
Confounding Variables
Interpret.
What do you think a confounding variable is?
Larson/Farber 4th ed.
Larson/Farber 4th ed

10. Confounding Variables
Chapter 1
Confounding Variables
Examine the data below. Can you identify the confounding variables?
We control the explanatory variable. We measure the response variable.
Larson/Farber 4th ed.
Larson/Farber 4th ed

11. Confounding Variables
Extraneous variables that can affect the response variable
Confounding occurs when two or more variables are associated in such a way that their effects on a response variable cannot be distinguished from each other.
Larson/Farber 4th ed.

12. Lurking Variables Confounding variables that are not measured
Fuel Economy Study: The following variables would be lurking variables if they were not measured:
temperature , terrain, level of traffic congestion
Example: Blood Pressure Experiment
Larson/Farber 4th ed.

13. Example: Experimental Design
Step
Example
Identify the claim to be studied.
Which fertilizer produces the best crop yields?
Determine the factor(s) that affect the response variable.
Brand of fertilizer. Amount of rain.
Determine the number of experimental units.
We will test 45 different plots of land.
Determine the level(s) of each factor.
We will apply three different fertilizer bands: A,B and C
Conduct the experiment.
Apply fertilizers. Measure response variable (i.e. mean crop yield)
Test the claim
Focus of later work. Requires inferential statistics.

14. Example: Experimental Design

15. What is a Completely Randomized Design?
Chapter 1
What is a Completely Randomized Design?
In a completely randomized design, all the treatments are assigned to all the experimental units completely by chance.
Some experiments may include a control group that receives an inactive treatment or an existing baseline treatment.
Group 1
Group 2
Treatment 1
Treatment 2
Compare Results
Random Assignment
Experimental Units
Interpret.
Larson/Farber 4th ed.
Larson/Farber 4th ed

16. Randomized Experiment
Chapter 1
Randomized Experiment
Designed so that the value of the explanatory variable for each unit is determined randomly before the response variable is measured
Treatment 1
Treatment 2
Group 1
Group 2
Compare Results
Random Assignment
Experimental Units
Larson/Farber 4th ed.
Larson/Farber 4th ed

17. Randomized Experiment
Chapter 1
Randomized Experiment
Will minimize the effect of confounding variables through the process of randomization
Helps to establish a causal relationship between two variables when an association is established
Treatment 1
Treatment 2
Group 1
Group 2
Compare Results
Random Assignment
Experimental Units
Larson/Farber 4th ed.
Larson/Farber 4th ed

18. Key Elements of Experimental Design: Control
Control Group
Used to account for the influence of other known or unknown variables that might be the cause of an underlying response in an experimental group
Control groups frequently receive a dummy treatment
Larson/Farber 4th ed.

19. Chapter 1
What is Blocking?
A block is a group of experimental units that are known before the experiment to be similar in some way that is expected to affect the response to the treatments.
In a randomized block design, the random assignment of experimental units to treatments is carried out separately within each block.
Analyze.
Larson/Farber 4th ed.
Larson/Farber 4th ed

20. Example: Randomized Block Design
Chapter 1
Example: Randomized Block Design
Test Effects of Weight Loss Drink
Interpret.
Larson/Farber 4th ed.
Larson/Farber 4th ed

21. Matched Pairs Experiment – Matched Subjects
Chapter 1
Matched Pairs Experiment – Matched Subjects
Experimental units are paired up
Each individual of the pair is assigned to a different treatment
Matched subjects are represented by matching color patterns.
Larson/Farber 4th ed.
Larson/Farber 4th ed

22. Matched Pairs Experiment – Matched Subjects
Chapter 1
Matched Pairs Experiment – Matched Subjects
Matched subjects are represented by matching color patterns.
Larson/Farber 4th ed.
Larson/Farber 4th ed

23. Matched Pairs Experiment – Matched Subjects
Chapter 1
Matched Pairs Experiment – Matched Subjects
Pair up similar students (i.e. same GPA)
All students take pretest
All students take SAT test
Compare test scores
Matched subjects are represented by matching color patterns.
Treatment A: SAT Prep course
Control: No SAT Prep course
Larson/Farber 4th ed.
Larson/Farber 4th ed

24. Summary: Matched Pairs Experiment- Matched Subjects
Chapter 1
Summary: Matched Pairs Experiment- Matched Subjects
Type
Description
Paired Subjects are identical twins
Each pair receives the same treatment. In each pair, we compare results to judge the effectiveness of the treatment.
Matched pairs are identical twins.
Larson/Farber 4th ed.
Larson/Farber 4th ed

25. Matched Pair Design – Single Subject
Description: Evaluate new medication for dry eyes
Drops are applied to each eye. Each subject receives two treatments in randomized order. Patient responses are compared.
Larson/Farber 4th ed.

26. Summary: Matched Pairs Experiment Single Subject
Chapter 1
Summary: Matched Pairs Experiment Single Subject
Interpret.
Larson/Farber 4th ed.
Larson/Farber 4th ed

27. What is the Placebo Effect?
A subject reacts favorably to a placebo when in fact he or she has been given no medical treatment at all
Larson/Farber 4th ed.

28. What is a Double-Blind Study?
Chapter 1
What is a Double-Blind Study?
Larson/Farber 4th ed.
Larson/Farber 4th ed

29. What is Wrong With This Experimental Design?
Chapter 1
What is Wrong With This Experimental Design?
We cannot determine whether the results are due to the method, or the teacher.
Each class is an experimental unit, since the treatment is assigned to the class
Each treatment is applied to just one experimental unit
Larson/Farber 4th ed.
Larson/Farber 4th ed

30. Improved Experimental Design: Why?
Chapter 1
Improved Experimental Design: Why?
Each class is an experimental unit
Each treatment is applied to several experimental units
This design utilizes replication
Analyze.
Larson/Farber 4th ed.
Larson/Farber 4th ed

31. Replication: Multiple Experimental Units
Repeated application of treatment under investigation
Each treatment is applied to a number of experimental units
Larson/Farber 4th ed.

32. Replication: Multiple Populations
Chapter 1
Replication: Multiple Populations
Example: The previous experiment is repeated over and over utilizing all the third grade classes in different elementary schools
Using this design, we can attribute differences to the method. The randomization will eliminate the teacher as a factor.
Larson/Farber 4th ed.
Larson/Farber 4th ed

33. Summary: Principles of Experimental Design
Control:
We control sources of variation other than the factors we are testing by making conditions as similar as possible for all treatment groups.
Randomize:
Randomization allows us to equalize the effects of unknown or uncontrollable sources of variation.
Larson/Farber 4th ed.

34. Summary: Principles of Experimental Design
Replicate:
Repeat the experiment, applying the treatments to a number of experimental units
Block: [Optional]
Sometimes, attributes of the experimental units that we are not studying and that we can’t control may nevertheless affect the outcomes of an experiment.
If we group similar individuals together and then randomize within each of these blocks, we can remove much of the variability due to the difference among the blocks.
Larson/Farber 4th ed.

35. Section 1.3
Surveys
Larson/Farber 4th ed.

36. What is the Purpose of a Survey?
Larson/Farber 4th ed.

37. Data Collection Survey
An investigation of one or more characteristics of a population.
Commonly done by interview, mail, or telephone.
Example: A survey is conducted on a sample of female physicians to determine whether the primary reason for their career choice is financial stability.
Larson/Farber 4th ed.

38. Surveys: What can Go Wrong?
Larson/Farber 4th ed.

39. Identify the Bias
You want to know how all sophomores feel about the upcoming basketball game. You only ask sophomore basketball players what they think.
This is called selection bias.
Selection bias is the tendency for samples to differ from the corresponding population as a result of systematic exclusion of some part of the entire population
Larson/Farber 4th ed.

40. Identify the Bias
You ask the following question: “You don’t want the school day to start any earlier than 6 AM, do you?
This is called response bias.
Response bias is the tendency for samples to differ from the corresponding population as a result of the influence of the interviewer
Larson/Farber 4th ed.

41. Identify the Bias
You are conducting a phone survey. The person whom you call just hangs up the phone on you.
This is called non-response bias
Non-response bias is the tendency for samples to differ from the corresponding population because data are not obtained from all individuals selected for inclusion in the sample
Larson/Farber 4th ed.

42. What is a Simulation?
Larson/Farber 4th ed.

43. Simulation
Uses a mathematical or physical model to reproduce the conditions of a situation or process.
Computers are often used
Examples:
Automobile manufacturers use simulations with dummies to study the effects of crashes on humans.
Airlines use simulators to train pilots on different models of aircraft
Larson/Farber 4th ed.

44. Example: Methods of Data Collection
Consider the following statistical studies. Which method of data collection would you use to collect data for each study?
A study of the effect of changing flight patterns on the number of airplane accidents.
Solution:
Simulation (It is impractical to create this situation)
Larson/Farber 4th ed.

45. Example: Methods of Data Collection
A study of the effect of eating oatmeal on lowering blood pressure.
Solution:
Experiment (Measure the effect of a treatment – eating oatmeal)
Larson/Farber 4th ed.

46. Example: Methods of Data Collection
A study of how fourth grade students solve a puzzle.
Solution:
Observational study (observe and measure certain characteristics of part of a population)
Larson/Farber 4th ed.

47. Example: Methods of Data Collection
A study of U.S. residents’ approval rating of the U.S. president.
Solution:
Survey (Ask “Do you approve of the way the president is handling his/her job?”)
Larson/Farber 4th ed.

48. Group Exercises
Group Exercise 1: Researchers want to investigate whether taking aspirin regularly reduces the risk of heart attack. At the end of the study, researchers count the number of men in each group who have had heart attacks. Group Exercise 2: The Smell & Taste Treatment and Research Foundation conducted a study to investigate whether smell can affect learning. Analyze the experiment.
Larson/Farber 4th ed.

49. Group Exercises
Group Exercise 3: A study is done to determine the average tuition that San Jose State undergraduate students pay per semester. Each student in the following samples is asked how much tuition he or she paid for the Fall semester. What is the type of sampling in each case?
Larson/Farber 4th ed.

50. Section 1.3 Summary Examined variety of sampling techniques
Explored different experimental designs
Contrasted different data collection techniques
Described simulations, observational studies and experiments
Described control groups and placebo effects
Discussed potential pitfalls that might data unreliable
Larson/Farber 4th ed.