1. Chapter 2 Collecting Data Sensibly Note: Correct usage of the vocabulary in this chapter is VERY important!

2. Consider the following headlines which occurred on September 25, 2009. “Spanking lowers a child’s IQ” (Los Angeles Times) “Do you spank” Studies indicate it could lower your kids’ IQ.” (SciGuy, Houston Chronicle) “Spanking can lower IQ” (NBC4i, Columbus, Ohio) “Smacking hits kids’ IQ” (newscientist.com) In this study, two groups of children were followed for 4 years; 806 children ages 2 to 4 and 704 children ages 5 to 9. IQ was measured at the beginning of the study and again four years later. Researchers found that the average IQ of children, ages 2 to 4, who were not spanked was 5 points higher than those who were spanked and 2.8 points higher for children, ages 5 to 9. These headlines imply that spanking is the CAUSE of the observed difference in IQ. Is this conclusion reasonable?

3. Observation versus Experimentation Look at the following two examples: A social scientist studying a rural community wants to determine whether gender and attitudes toward abortion are related. Using a telephone survey, 100 residents are contacted at random and their gender and attitude toward abortion are recorded. A professor might wonder what would happen to final test scores if the required lab time for a chemistry course is increased from 3-hours to 6-hours. For 100 chemistry students, half were randomly assigned to the 3-hour lab and half to the 6-hour lab. The rest of the course remained the same for the two groups. The difference in their final test scores will be examined. How do these two examples differ? Think about: How the groups were determined? Were any variables controlled? What did the researcher do? Which is the experiment and which is the observational study?

4. Definitions: Observational study – a study in which the researcher observes characteristics of a sample selected from one or more populations. Experiment - a study in which the researcher observes how a response variable behaves when one or more explanatory variables (factors) are manipulated. A well-designed experiment can result in data that provides evidence for a cause- effect relationship.

5. Let’s return to the study on spanking and IQ In this study, two groups of children were followed for 4 years; 806 children ages 2 to 4 and 704 children ages 5 to 9. IQ was measured at the beginning of the study and again four years later. Researchers found that the average IQ of children, ages 2 to 4, who were not spanked was 5 points higher than those who were spanked and 2.8 points higher for children, ages 5 to 9. Does spanking “CAUSE” a decrease in IQ? Why or why not? Are there other variables connected to the response (decreased IQ) and the groups of children? These are called confounding variables.

6. Definition: Confounding variable – a variable that is related to both group membership and the response variable of interest in a research study Because observational studies may contain confounding variables, their results can NOT be used to show cause- effect relationships.

7. Observational studies CAN be generalized to the population if the sample is randomly selected from the population of interest, but CANNOT show cause-effect relationships. Well-designed experiments CAN show cause-effect relationships, but CANNOT be generalized to the population if the groups are volunteers or are not randomly assigned.

8. Sampling Section 2.2

9. Do frog fairy tale... How good is a census? Do frog fairy tale... The answer is 83!

10. Census versus Sample Why might we prefer to take select a sample rather than perform a census? 1. Measurements that require destroying the item Measuring how long batteries last Safety ratings of cars 2.Difficult to find entire population Length of fish in a lake 3. Limited resources Time and money Obtaining information about the entire population is called a census. Most common reason to use a sample

11. Sampling design refers to the method used to choose the sample from the population

12. Jelly Blubber Activity Select 10 Jelly blubbers that you think are representative of the population of blubbers in regards to length. Find the mean length of your sample

13. Methods of selecting random samples Simple Random Sample (SRS) A sample of size n is selected from the population in a way that ensures that every different possible sample of the desired size has the same chance of being selected. It has to be possible for all 100 students in the sample to be seniors – or any other combination of students! A simple random sample does NOT guarentee that the sample is representative of the population. Suppose a local school has 2000 students. We want to survey 100 students about the current cell phone policy. A sample of students can be selected by putting each students’ name on individual (but identical) slips of paper and placing them in a large container. After mixing well, randomly select 100 names from the container, one at a time. This is an example of a simple random sample.

14. Simple Random Sample (SRS) continued A sample of size n is selected from the population in a way that ensures that every different possible sample of the desired size has the same chance of being selected. Sampling frame – list of all the objects or individuals in the population. Methods of selecting random samples Another way to select a simple random sample is to create a list of all the students in the school (called a sampling frame). Number each student with a unique number from 1 to 2000. Use a random digit table or random number generator (a calculator or computer software) to select the 100 students for the sample.

15. How to use a Random digit table The following is part of the random digit table found in the back of your textbook: Row 609387679956256584264 741010220475119479751 864736345123118004820 980287938404208912332 Since our students are numbered 1-2000, we will select 4-digit numbers from the table. If the number is not within 1-2000, we will ignore it. We would continue in this fashion until we had selected 100 numbers. It would be faster to use a random number generator.

16. Simple Random Sample (SRS ) continued A sample of size n is selected from the population in a way that ensures that every different possible sample of the desired size has the same chance of being selected. Although sampling with and without replacement are different, they can be treated as the same when the sample size n is relatively small compared to the population size (no more than 10% of the population). Methods of selecting random samples Most often sampling is done without replacement. That is once an individual or object is selected, they are not replaced and cannot be selected again. Sampling with replacement allows an object or individual to be selected more than once for a sample.

17. Jelly Blubber Activity Find ten random numbers between 1 & 100. Locate the corresponding Jelly Blubbers and record their lengths. Find the mean length of your sample

18. SRS Advantages – Unbiased – Easy Disadvantages – Large variance – May not be representative – Must have sampling frame (list of population)

19. Methods of selecting random samples Stratified Random Sample Population is divided into non-overlapping subgroups called strata Simple random samples are selected from each stratum Sometimes easier to implement and is more cost effective than simple random sampling Sometimes allows more accurate inferences about a population than simple random sampling Strata are groups that are similar (homogeneous) based upon some characteristic of the group members. Instead of a simple random sample to answer our survey about the cell phone policy at school, suppose we were take four simple random samples of size 25 from each grade level, freshman, sophomore, junior, and senior. This would be an example of a stratified random sample.

20. Jelly Blubber Activity Suppose we divide the Jelly Blubbers into 5 strata based upon their size. Choose 2 Jelly Blubbers at random from each stratum by choosing a random number. Locate the Blubbers & record their lengths. Find the mean length of your sample

21. Stratified Advantages – More precise unbiased estimator than SRS – Less variability – Cost reduced if strata already exists Disadvantages – Difficult to do if you must divide stratum – Formulas for SD & confidence intervals are more complicated – Need sampling frame

22. Methods of selecting random samples Systematic Sampling A value k is specified (for example k = 50 or k = 200). One of the first k individuals is selected at random. Then every k th individual in the sequence is included in the sample. This method works reasonably well as long as there are no repeating patterns in the population list. Suppose we randomly select a number between 1 and 20. Using a alphabetical list of students at our school, select the student whose name is at that number in the list. Then choose every 20 th student from there. This is an example of a systematic random sample.

23. Jelly Blubber Activity A system is used to select the sample. Use the original page of Blubbers and select five Blubbers. Choose a random number between 1 & 10. This is the first Blubber. Then add 10 to each previous number to find the other nine Blubbers. Record their lengths. Find the mean length of your sample

24. Systematic Random Sample Advantages –Unbiased –Don’t need sampling frame –Ensure that the sample is spread across population –More efficient, cheaper, etc. Disadvantages –Large variance –Can be confounded by trend or cycle –Formulas are complicated

25. Methods of selecting random samples Cluster Sampling Population is divided into non-overlapping subgroups called clusters Randomly select clusters and then all the individuals in the clusters are included in the sample Cluster sampling is often easier to perform and more cost effective. Clusters are often based upon location. It is best if the clusters are heterogeneous subgroups from the population. Let’s look at another way to select a sample of students to answer our survey on the current cell phone policy at our school. One way to do this would be to randomly select 5 classrooms during 2 nd period. Survey all the students in those rooms! This is an example of a cluster sample.

26. Jelly Blubber Activity The Jelly Blubbers exist in clusters. Suppose the groups shown on the Cluster Page are actually spaced further apart. Choose 2 random numbers between 1 & 20 and find that cluster. Record the lengths of the all the Blubbers in the clusters. Find the mean length of your sample

27. For the Jelly Blubber colony:  = 19.41

28. Cluster Samples Advantages – Unbiased – Cost is reduced – Sampling frame may not be available (not needed) Disadvantages – Clusters may not be representative of population – Formulas are complicated

29. Identify the sampling design 1)The Educational Testing Service (ETS) needed a sample of colleges. ETS first divided all colleges into groups of similar types (small public, small private, medium public, medium private, large public, and large private). Then they randomly selected 3 colleges from each group. Stratified random sample

30. Identify the sampling design 2) A county commissioner wants to survey people in her district to determine their opinions on a particular law up for adoption. She decides to randomly select blocks in her district and then survey all who live on those blocks. Cluster sampling

31. Identify the sampling design 3) A local restaurant manager wants to survey customers about the service they receive. Each night the manager randomly chooses a number between 1 & 10. He then gives a survey to that customer, and to every 10 th customer after them, to fill it out before they leave. Systematic sampling

32. Consider the following example: In 1936, Franklin Delano Roosevelt had been President for one term. The magazine, The Literary Digest, predicted that Alf Landon would beat FDR in that year's election by 57 to 43 percent. The Digest mailed over 10 million questionnaires to names drawn from lists of automobile and telephone owners, and over 2.3 million people responded - a huge sample. At the same time, a young man named George Gallup sampled only 50,000 people and predicted that Roosevelt would win. Gallup's prediction was ridiculed as naive. After all, the Digest had predicted the winner in every election since 1916, and had based its predictions on the largest response to any poll in history. But Roosevelt won with 62% of the vote. The size of the Digest's error is staggering. This is a classic example of how bias affects the results of a sample! Bias is the tendency for samples to differ from the corresponding population in some systematic way.

33. Sources of bias Selection bias Occurs when the way the sample is selected systematically excludes some part of the population of interest –called undercoverage May also occur if only volunteers or self- selected individuals are used in a study Suppose you take a sample by randomly selecting names from the phone book – some groups will not have the opportunity of being selected! People with unlisted phone numbers – usually high-income families People without phone numbers – usually low- income families People with ONLY cell phones – usually young adults

34. Sources of bias Convenience sampling Using an easily available or convenient group to form a sample. –The group may not be representative of the population of interest –Results should not be generalized to the population Can also occur when samples rely entirely on volunteers to be part of the sample – called voluntary response Suppose we decide to survey only the students in our statistics class – why might that cause bias in a survey? An example would be the surveys in magazines that ask readers to mail in the survey. Other examples are call-in shows, American Idol, etc. Remember, the respondent selects themselves to participate in the survey!

35. Sources of bias Response bias Measurement or Response bias Occurs when the method of observation tends to produce values that systematically differ from the true value in some way –Improperly calibrated scale is used to weigh items –Tendency of people not to be completely honest when asked about illegal behavior or unpopular beliefs –Appearance or behavior of the person asking the questions –Questions on a survey are worded in a way that tends to influence the response Suppose we wanted to survey high school students on drug abuse and we used a uniformed police officer to interview each student in our sample – would we get honest answers? People are asked if they can trust men in mustaches – the interviewer is a man with a mustache. A Gallup survey sponsored by the American Paper Institute (Wall Street Journal, May 17, 1994) included the following question: “It is estimated that disposable diapers accounts for less than 2% of the trash in today’s landfills. In contrast, beverage containers, third-class mail and yard waste are estimated to account for about 21% of trash in landfills. Given this, in your opinion, would it be fair to tax or ban disposable diapers?”

36. Sources of bias Nonresponse occurs when responses are not obtained from all individuals selected for inclusion in the sample To minimize nonresonse bias, it is critical that a serious effort be made to follow up with individuals who did not respond to the initial request for information People are chosen by the researchers, BUT refuse to participate. NOT NOT self-selected! This is often confused with voluntary response! The phone rings – you answer. “Hello,” the person says, “do you have time for a survey about radio stations?” You hang up! How might this follow-up be done?

37. Identify a potential source of bias. 1) Before the presidential election of 1936, FDR against Republican ALF Landon, the magazine Literary Digest predicting Landon winning the election in a 3-to-2 victory. A survey of 2.3 million people. George Gallup surveyed only 50,000 people and predicted that Roosevelt would win. The Digest’s survey came from magazine subscribers, car owners, telephone directories, etc. Undercoverage – since the Digest’s survey comes from car owners, etc., the people selected were mostly from high-income families and thus mostly Republican! (other answers are possible)

38. Identify a potential source of bias. 2) Suppose that you want to estimate the total amount of money spent by students on textbooks each semester at a local college. You collect register receipts for students as they leave the bookstore during lunch one day. Convenience sampling – easy way to collect data or Undercoverage – students who buy books from on-line bookstores are excluded.

39. Undercoverage – leaves out homes that are not for sale or homes that are listed with different realtors. (other answers are possible) Identify a potential source of bias. 3) To find the average value of a home in Plano, one averages the price of homes that are listed for sale with a realtor.

40. Comparative Experiments Sections 2.3 & 2.4

41. Suppose we are interested in determining the effect of room temperature on the performance on a first-semester calculus exam. So we decide to perform an experiment. What variable will we “measure”? the performance on a calculus exam What variable will “explain” the results on the calculus exam? the room temperature This is called the response variable. Response variable – a variable that is not controlled by the experimenter and that is measured as part of the experiment This is called the explanatory variable. Explanatory variables – those variables that have values that are controlled by the experimenter (also called factors)

42. We decide to use two temperature settings, 65° and 75°. How many treatments would our experiment have? the 2 treatments are the 2 temperature settings Room temperature experiment continued... Experimental condition – any particular combination of the explanatory variables (also called treatments)

43. Suppose we have 10 sections of first-semester calculus that have agree to participate in our study. On who or what will we impose the treatments? the 10 sections of calculus How would we determine which sections would be in rooms with the temperature set at 65° and which sections in rooms set at 75°? we need to randomly assign them to the treatments Room temperature experiment continued... These are our subjects or experimental units. Experimental units – the smallest unit to which a treatment is applied. Random assignment of subjects to treatments or treatments to trials ensures that the experiment does not systematically favor one treatment over another.

44. Sections assigned Treatment 1 (65°) Treatment 2 (75°) 3 9 85 7 Room temperature experiment continued... To randomly assign the 10 sections of first- semester calculus to the 2 treatment groups, we would first number the classes 1-10. Place the numbers 1-10 on identical slips of paper and put them in a hat. Mix well. Randomly select 5 numbers from the hat. Those will be the sections that have the room temperature set at 65°. The remaining sections will have the room temperature set at 75°. 97583 124610

45. Sections assigned Treatment 1 (65°)97583 Treatment 2 (75°)124610 Room temperature experiment continued... Notice that there are five sections assigned to each treatment. This is called replication. Replication ensures that we have multiple observations for each treatment. Why is replication an important trait of a well-designed experiment?

46. Remember – the explanatory variable is the room temperature setting, 65° and 75°. The response variable is the grade on the calculus exam. Are there other variables that could affect the response? Room temperature experiment continued... T i m e o f d a y ? Instructor? T e x t b o o k ? A b i l i t y l e v e l o f s t u d e n t s ? These other variables are called extraneous variables. An extraneous variable is a variable that is NOT one of the explanatory variables (factors) but it is thought to affect the response. In an experiment, these extraneous variables need to be “controlled”. Direct control is holding the extraneous variables constant so that their effects are not confounded with those of the experimental conditions (treatments). Can the experimenter control these extraneous variables? If so, how? What about the variables that the experimenter can’t directly control? What can be done to avoid confounding results? Remember - two variables are confounding if their effects on the response cannot be distinguished from each other.

47. Suppose that there were five instructors who taught the first-semester calculus. We do not have direct control of this variable; however, we could have each instructor teach 2 sections. Then we could randomly assign which one of the 2 sections would have a temperature setting of 65° and the other would have a temperature setting of 75°. Room temperature experiment continued... This is an example of blocking. Blocking is process by which an extraneous variable’s effects are filtered out. Similar groups, called blocks, are created. All treatments are tried in each block.

48. What about extraneous variables that we cannot control directly or that we cannot block for or that we don’t even think about? Random assignment should evenly spread all extraneous variables, that are not controlled directly or that are not blocked, into all treatment groups. We expect these variables to affect all the experimental groups in the same way; therefore, their effects are not confounding. Room temperature experiment continued...

49. Would the students in each section of calculus know to which treatment group, 65° or 75°, they were assigned? If the students knew about the experiment, they would probably know which treatment group they were in. So this experiment is probably NOT blinded. Room temperature experiment continued... An experiment in which the subjects do not know which treatment they were in is called a single-blind experiment. A double-blind experiment is one in which neither the subjects nor the individuals who measure the response knows which treatment is received.

50. In the room temperature experiment, we only have 2 treatment groups, 65° and 75°. We do NOT have a control group. Control group is an experimental group that does NOT receive any treatment. The use of a control group allows the experimenter to assess how the response variable behaves when the treatment is not used. This provides a baseline against which the treatment groups can be compared to determine whether the treatment had an effect.

51. Consider Anna, a waitress. She decides to perform an experiment to determine if writing “Thank you” on the receipt increases her tip percentage. She plans on having two groups. On one group she will write “Thank you” on the receipt and on the other group she will not write “Thank you” on the receipt. Which of these is the control group?

52. Suppose we want to test an herbal supplement to determine if it aided in weight loss. Why would it not be beneficial have two groups in the experiment; one that takes the supplement and a control group that takes nothing? What could be done to remedy this problem? Give one group the supplement and give the other group a pill that is the same size, color, taste, smell, etc. as the supplement, but contains no active ingredient. This is called a placebo. A placebo is something that is identical to the treatment group but contains no active ingredient.

53. Random assignment removes the potential for confounding variables. Blocking uses extraneous variables to create groups (blocks) that are similar. All treatments are then tried in each block. Let’s recap some ideas- Direct control holds extraneous variables constant so their effects are not confounded with the treatments.

54. Example 1: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with twobdifferent types of fertilizers. Sixteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared. Experimental units? Factors? Response variable? How many treatments? Plots of land Type of fertilizer Yield of crop 2

55. Suppose that the sixteen plots of land are not similar. Some are close to a river with trees along the bank, while others are farther away from the river. See the diagram below. A A A A AA AA B B B B B B B B Suppose the researcher decides to fertilizer the 8 plots closest to the river with fertilizer A and the remaining plots with B. He has a greater yield with fertilizer A. Does this mean that A is the better fertilizer? NO! The amount of water or soil type is confounded with the type of fertilizer!

56. Experimental Designs 1.Completely randomized design – experimental units are assigned at random to treatments or treatments are assigned at random to trials Experimental Units Measure response for A Treatment B Treatment A Measure response for B Random Assignment Compare treatments Let’s look at two examples of completely randomized experiments.

57. Fertilizer experiment continued: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with three different types of fertilizers. Fifteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared. Why is the same type of seed used on all 15 plots? What are other potential extraneous variables? Does this experiment have a placebo? Explain It is part of the controls in the experiment. Type of soil, amount of water, etc. NO – a placebo is not needed in this experiment

58. Suppose that the sixteen plots of land are not similar. Some are close to a river with trees along the bank, while others are farther away from the river. See the diagram below. B B A A AB BA B B A B B A A A Suppose the researcher decides to randomly assign fertilizer A to 8 plots and the remaining plots with B. He has a greater yield with fertilizer A. Does this mean that A is the better fertilizer? Yes! The effects of water and soil type are evenly spread into both treatments. Thus there is no confounding.

59. Example 2: A consumer group wants to test cake pans to see which works the best (bakes evenly). It will test aluminum, glass, and plastic pans in both gas and electric ovens. There are 30 boxes of cake mix to use for this experiment. Experiment units? Factors? Response variable? Name the treatments? Two factors - type of pan (aluminum, glass, and plastic) and type of oven (electric and gas) How evenly the cake bakes Aluminum pan in electric oven, aluminum pan in gas oven, glass pan in electric oven, glass pan in gas oven, plastic pan in electric oven, and plastic pan in gas oven Cake mixes

60. Cake experiment continued: A consumer group wants to test cake pans to see which works the best (bakes evenly). It will test aluminum, glass, and plastic pans in both gas and electric ovens. There are 30 boxes of cake mix to use for this experiment. Describe how to randomly assign the cake mixes to the treatments so that there is an even number in each treatment. Number the boxes of cake mix from 1 to 30. Write the numbers 1 to 30 on identical slips of paper and place into a hat. Mix well. Randomly select 6 numbers from the hat and assign those boxes to the treatment of aluminum pan in electric oven. Randomly select 6 more numbers and assign those boxes to the treatment aluminum pan in gas oven. Continue this process, randomly assigning 6 boxes to each treatment glass pan in electric oven, glass pan in gas oven, and plastic pan in electric oven. The remaining 6 are assigned to plastic pan in gas oven This is just one way that you can perform this randomization. Could we roll a die for each box? If we roll a “1” assign the box to the first treatment (aluminum pan in electric oven). If we roll a 2, assign the box to the 2 nd treatment, and so on.

61. 2. Randomized block – units are blocked into groups (homogeneous) and then randomly assigned to treatments Experimental Designs Continued... Block 2 Measure response for A Treatment B Treatment A Measure response for B Random Assignment Compare treatments for block 2 Block 1 Measure response for A Treatment B Treatment A Measure response for B Random Assignment Compare treatments for block 1 Experimental Units Create blocks Compare the results from the 2 blocks Units should be blocked on a variable that effects the response!!!

62. Fertilizer experiment revisited: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with two different types of fertilizers. Sixteen plots of land (8 plots are along a river and 8 plots are away from the river) are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared. Can the experimenter directly control the types of soil in the different plots of land? What can be done to account for this variable? No – they must use the plots that are available They could block by type of soil

63. Fertilizer experiment revisited: Describe how to create the blocks of land and then to randomly assign plots to the 2 types of fertilizer. First create 2 blocks of land. Block 1 would be the 8 plots that are by the river. Block 2 would be the 8 plots away from the river. Number the 8 plots in block 1 from 1 to 8. Write the numbers 1 to 8 on identical slips of paper and place into a hat. Mix well. Randomly select 4 numbers from the hat and assign those boxes to fertilizer A. The remaining 4 are assigned to Fertilizer B. Number the 8 plots in block 2 from 1 to 8. Write the numbers 1 to 8 on identical slips of paper and place into a hat. Mix well. Randomly select 4numbers from the hat and assign those boxes to fertilizer A. The remaining 4 are assigned to Fertilizer B.

64. Suppose that the sixteen plots of land are not similar. Some are close to a river with trees along the bank, while others are farther away from the river. See the diagram below. A A B A BB AB B A B B A B A A Suppose the researcher decides to block the 16 plots into two blocks based on their proximity to the river. Yes! Blocking is a way to “control” the effects of extraneous variables. Then the researcher randomly assigns fertilizer A to 4 plots in the red block and to 4 plots in the blue block. The remaining plots receive fertilizer B. He has a greater yield with fertilizer A. Does this mean that A is the better fertilizer?

65. 3. Matched pairs - a special type of block design where the blocks consist of 2 experimental units that are similar with each being randomly assigned to a treatment OR the block consist of individual units that are assigned both treatments in random order Experimental Designs Continued...

66. Suppose that the sixteen plots of land are not similar. Some are close to a river with trees along the bank, while others are farther away from the river. See the diagram below. A B B A BB AA A A B B B A A B Yes! Matched pairs is a special type of blocking. Then the researcher randomly block the plots of land into groups of two so that they are as alike as possible. Randomly select which of the two get fertilizer A. The other will get fertilizer B. Analyze the differences between the pairs. He has a greater yield with fertilizer A. Does this mean that A is the better fertilizer?

67. Example 3: Two new word-processing programs are to be compared by measuring the speed with which a standard task can be completed. One hundred volunteers are will perform the same task on each of the programs in random order and their speeds will be measured. Explain why this is a matched pairs design. How could we determine which program the volunteers use first? Each block consist of an individual who will do both treatments We could flip a coin for each volunteer; heads they do program A first, tails they do program B first.

68. The ONLY way to show a cause-effect relationship is with a well-designed, well-controlled experiment !!!