1 Chapter 5: Producing Data 5.1Designing Samples 5.2Designing Experiments 5.3Simulating Experiments

2 Types of Studies Observational Study –We observe individuals and measure variables of interest, but no attempt is made to affect the outcomes. Examples: Interviews, correlation studies, surveys Experiment –We deliberately impose some treatment on individuals in an attempt to affect the outcomes. An experiment is the “gold standard” for researchers, because we can make conclusions about cause and effect. Example: New treatment for cocaine addiction. Example: The effects of machine speed on woven fabric quality.

3 Diagram of a Two-Factor Experiment

4 Populations and Samples A population is the entire group of individuals for which we want information. –Note that this varies for each study. –“Individuals” can mean people, rats, yarn bobbins, … A sample is a part of the population that we actually examine. –We do this for time and cost reasons, mainly.

5 Taking Samples

6 Poor Sampling Methods and Bias Voluntary response sample Convenience sampling Bias –The design of a study is biased if it systematically favors certain outcomes.

7 Problems 5.2, p , p , p , p. 274

8 Simple Random Sampling We have chosen a simple random sample of size n out of a population if each individual had an equal chance of being selected. –What are some ways to choose a SRS? Note: The above statement is true, but in itself is not a complete definition of SRS. –Reading from BVD text. –Statement in your text, p. 275.

9 Choosing a SRS Put all names in a hat and draw them out to get the appropriate sample. Use software or a calculator. –Use RANDint function (under MATH—PRB) Use a table of random digits. –p. 275 –Example 5.4, p. 276 Exercise 5.9, p. 279

10 Exercises Problems: –5.13, p. 283 –5.17, p. 284

11 Stratified Random Sample A stratified random sample is one obtained by separating the population elements into non- overlapping groups (called strata), and then selecting a simple random sample from each stratum. Reasons for choosing a stratified random sample: –Possible reduction in the variation of the estimators. –Estimates are often needed for the subgroups of the population. –Focuses on important subpopulations but ignores irrelevant ones.

12 Example A real-world example of using stratified sampling would be for a US political survey. If we wanted the respondents to reflect the diversity of the population of the United States, the researcher would specifically seek to include participants of various minority groups such as race or religion, based on their proportionality to the total population.

13 Disadvantages of Stratified Sampling Can be difficult to select relevant stratification variables Not useful when there are no homogeneous subgroups Can be expensive Requires accurate information about the population

14 Weighting the Strata Proportionate allocation uses a sampling fraction in each of the strata that is proportional to that of the total population. If the population consist of 60% in the male stratum and 40% in the female stratum, then the relative size of the two samples (one males, one females) should reflect this proportion.

15 Cluster Sampling Cluster sampling is generally employed because of cost effectiveness or because no adequate sampling frame is available. Once the clusters are identified, clusters are chosen at random and each element within the cluster is measured.

16 Cluster Sampling Example Suppose we wish to estimate the average income per household in a large city. We might consider using stratified random sampling, but we must be able to determine the strata and also have the sampling frame (that is, the elements). However, we could divide the city into regions such as blocks (the clusters), then select a simple random sample of blocks from the population. This task is easily accomplished by using a frame that lists all city blocks. Then the income of every household within each sampled block could be measured.

17 Cautions About Sample Surveys We need an accurate and complete list of the population. –Else, undercoverage can result. Non-response bias Wording of questions! –Ex. 5.7, p. 282 Sample size must be adequate to make inferences about the population. –Chapter 10

18 Designing a Survey:Things to Consider Purpose statement Instructions to respondent –Confidentiality, etc. Method of administering survey Sample size: –All? Some? How do we choose? Type of questions Number of questions Specific wording of questions

19 Exercises, p

20 Homework Reading in section 5.2, pp Exercises, p. 284: –5.15, 5.18

21 Section 5.2 Experimental Design

22 Example Experimental Design Explanatory variables in an experimental design are often called factors. –Mozart effect factor(s)? A specific experimental condition applied to the units is called a treatment. –What treatments did we have here? Many experiments study the joint effects of several factors. In these experiments, each treatment is formed by combining a specific value or level of each of the factors. Many times, we measure several response variables on the experimental units (called subjects when dealing with people).

23 Diagram of a Two-Factor Experiment

24 Exercises 5.32, p , p. 293

25 Benefits of Using Experimental Design Experiments can give good evidence for causation. Experiments allow us to study the specific factors we are interested in, while controlling the effects of the lurking variables. They also allow us to study the combined effects of several factors, as we saw above. Experimental control! –First basic principle of statistical design of experiments.

26 What is the “placebo effect”? l.htmlhttp:// l.html

27 Placebo Effect Example 5.11, p. 292 –Placebo –Control group –Comparative experiment

28 Problems Exercises: –5.34, p. 293 –5.36, p. 294 –5.39, p. 298 –5.42, p. 299

29 Homework Reading in section 5.2, pp

30 Randomization Systematic differences among the groups of experimental units in a comparative experiment cause bias. –Textile example Randomization is the statistician’s remedy against bias. –Example 5.12, p. 295

31 Questions about Example 5.12 Are the two groups of rats guaranteed to be identical? Do you think it is wise to include many experimental units? –We hope that we can have the effects of chance to average out with multiple experimental units. –Replication!

32 Exercise 5.38, p. 298

33 Experimental Control Definition of experimental control: –Taking account of extraneous variables in the experimental design, most simply by the use of equivalent groups for comparison. How can we, as researchers, achieve experimental control?

34 Principles of Experimental Design Experimental control Randomization Replication

35 More Items in Section 5.2 Double-blind experiment Matched pairs designs Block designs

36 Treating Each Subject Identically When we do a randomized comparative experiment we must take special care to see that each subject (or each experimental unit) is treated identically. –Is there researcher influence? Did other things not in our control change during the course of the experiment (e.g., ambient conditions, machines, people, etc.)? Double-blind experiment –Gold standard in medical research –5.43, p. 303

37 Matched Pairs Designs A matched pairs design helps in experimental control. See Example 14 from handout: –Moore, David S. (1991). Statistics: Concepts and controversies (3 rd ed.). W.H. Freeman: New York.

38 Block Designs Example 15, p. 108 in: –Moore, David S. (1991). Statistics: Concepts and controversies (3 rd ed.). W.H. Freeman: New York. Blocks are a way of holding fixed an extraneous variable that would otherwise cause large variations in the experimental results.

39 Where did we get “blocking?” Research in agronomy –Researchers created blocks in the field.

40 Block Design Example * An agronomist wishes to compare the yield of five corn varieties. The field in which the experiment will be carried out increases in fertility from north to south. The agronomist divides the field into 30 plots of equal size, and wants to employ a randomized complete block design. Identify the experimental units, the treatments, and the blocks. Describe the arrangement of the randomized complete block design, and perform the appropriate randomization. * Moore, David S. (1991). Statistics: Concepts and controversies (3rd ed.). W.H. Freeman: New York.

41 Problems From the handout (Moore): –p. 112: 2.36, 2.37, 2.38

42 Lack of Realism Examples 5.14 and 5.15, p. 300 Subjects being monitored/experimented on may or may not be like the population.

43 Statistical Significance Even if we have controlled properly, randomized, and replicated, we can still get differences among our experimental units on our response variables that are caused by chance. We use the laws of probability, which give mathematical descriptions of chance behavior, to learn if the treatment effects are larger than we would expect to see if only chance were operating. –If they are larger than to be expected by chance, we call them statistically significant.

44 Homework Problems: –5.44, p. 303 –5.46, p. 304 –5.49, p. 306 –5.53, p. 307

Simulating Experiments We can use technology to simulate what might happen if an experiment were actually carried out. –TI-83/84/89 random number generator –We can also use dice or coins or cards, though these methods are less efficient. Simulation is an effective tool for finding likelihoods of complex results once we have a trustworthy model.

46 Monte Carlo Methods

47 Example 5.21, p. 310 Simulation steps: –1. Describe the experiment –2. State the assumptions E.g., likelihood of an event occurring; independence of events. Independence—result of one trial does not affect the result of another (e.g., coin toss) –3. Assign digits to represent outcomes. Change: 0-4 heads; 5-9 tails –4. Simulate many repetitions. Each group of 2 simulate 5 reps, where a rep is 10 flips. –5. Estimated probability?

48 Problems 5.59, p , p , p. 317

49 Homework Simulation problems, pp : –5.62, 5.65 Chapter 5 Test on Wednesday

50 Review Problems 5.74, 5.77, 5.79, 5.87 –pp