Unit 4--Lesson 2

Lesson Objectives At the end of the lesson, students can: Identify common issues with sampling and surveys Design an experiment using appropriate experimental design concepts.

Bias ERROR favors certain outcomes might be attributed to the researchers, the respondent, or to the sampling method

Sources of Bias things that can cause bias in your sample cannot do anything with bad data

Sources of Bias Volunteer Response Sampling People chose to respond Usually only people with very strong opinions respond

Sources of Bias Convenience Sampling Ask people who are easy to ask Produces bias results

Sources of Bias Undercoverage some groups of population are left out of the sampling process

Sources of Bias Nonresponse occurs when an individual chosen for the sample can’t be contacted or refuses to cooperate telephone surveys 70% nonresponse

Sources of Bias Response Bias occurs when the behavior of respondent or interviewer causes bias in the sample wrong answers

Sources of Bias Wording of the Question wording can influence the answers that are given connotation of words use of “big” words or technical words

Identify the Sources of Bias Before the presidential election of 1936, FDR against Republican ALF Landon, the magazine Literary Digest predicted Landon winning the election in a 3-to-2 victory. A survey of 10 million people by George Gallup surveyed only 50,000 people and predicted that Roosevelt would win. The Digest’s survey came from magazine subscribers, car owners, telephone directories, etc. Undercoverage

Identify the Sources of Bias Suppose that you want to estimate the total amount of money spent by students on textbooks each semester at Rice. You collect register receipts for students as they leave the bookstore during lunch one day. Convenience sampling

Identify the Sources of Bias To find the average value of a home in Friendswood, one averages the price of homes that are listed for sale with a realtor. Undercoverage

Identify the Sources of Bias Mrs. Hood wants to find out what the Wake County high school students think about the effectiveness of their math teachers. a)Mrs. Hood sends out a survey to 1500 high school students randomly selected from the Wake County Public Schools records. 941 surveys are returned, indicating that 80% of the respondents believe their math teacher is effective. What is the population? What is the sampling frame? What is the sample? What type of sample is this? Is there any bias?

Experimental Design Refresh: Observational study - observe outcomes without imposing any treatment Experiment - actively impose some treatment in order to observe the response

I’ve developed a new rabbit food, Hippity Hop. Rabbit Food Makes fur soft & shiny! Increases energy! 100% of daily vitamins & essential oils!

Can I just make these claims? What must I do to make these claims? Who (what) should I test this on? What do I test? NO Do an experiment Rabbits The type of food

Experimental Design Experimental unit – the single individual (person, animal, plant, etc.) to which the different treatments are assigned Factor – is the explanatory variable Level – a specific value for the factor

Experimental Design Response variable – what you measure Treatment – a specific experimental condition applied to the units

I plan to test my new rabbit food. What are my experimental units? What is my factor? What is the response variable? Rabbits Type of food How well they grow

Hippity Hop I’ll use my pet rabbit, Lucky! Since Lucky’s coat is shinier & he has more energy, then Hippity Hop is a better rabbit food!

Experimental Design Control group – a group that is used to compare the factor against; can be a placebo or the “old” or current item Placebo – a “dummy” treatment that can have no physical effect

Old Food Hippity Hop Now I’ll use Lucky & my friend’s rabbit, Flash. Lucky gets Hippity Hop food & Flash gets the old rabbit food. WOW! Lucky is bigger & shinier so Hippity Hop is better!

Old Food Hippity Hop Number the rabbits from 1 – 10. Place the numbers in a hat The first five numbers pulled from the hat will be the rabbits that get Hippity Hop food. I evaluated the rabbits & found that the rabbits eating Hippity Hop are better than the old food! The remaining rabbits get the old food

Experimental Design blinding - method used so that units do not know which treatment they are getting double blind - neither the units nor the evaluator know which treatment a subject received

Rabbit Food Hippity Hop Rabbit Food makes fur soft and shiny, & increases energy for ALL types of rabbits! Can I make this claim?

Example 1: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with three different types of fertilizers. Fifteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared. Experimental units? Factors? Levels? Response variable? How many treatments? Plots of land Type of fertilizer Fertilizer types A, B, & C Yield of crop 3

Example 2: A consumer group wants to test cake pans to see which works the best (bakes evenly). It will test aluminum, glass, and plastic pans in both gas and electric ovens. Experiment units? Factors? Levels? Response variable? Number of treatments? Two factors - type of pan & type of oven Type of pan has 3 levels (aluminum, glass, & plastic & type of oven has 2 levels (electric & gas) How evenly the cake bakes 6 Cake batter

Example 3: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with three different types of fertilizers. Fifteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared. Why is the same type of seed used on all 15 plots? What are other potential extraneous variables? Does this experiment have a placebo? Explain It is part of the controls in the experiment. Type of soil, amount of water, etc. NO – a placebo is not needed in this experiment

The ONLY way to show cause & effect is with a well-designed, well-controlled experiment! The ONLY way to show cause & effect is with a well-designed, well-controlled experiment!! The ONLY way to show cause & effect is with a well- designed, well-controlled experiment!!!

Jelly Blubber Activity

Look at the worksheet for 5 seconds. Estimate the average length of a blubber.

Judgmental Sample – Select 5 Jelly Blubbers that, in your judgment, are representative of the population on this sheet. Record the lengths of your five Blubbers and find the mean. ____ ____ ____ ____ ____ = _______ x

Simple Random Sample - Find five random numbers between 1 & 100. Locate the corresponding Jelly Blubbers and record their lengths. Find the mean. ____ ____ ____ ____ ____ = _______ x

Stratified Sample - Suppose one shakes the Jelly Blubbers, causing the larger ones to sink and the smaller ones to rise. The result is the sheet of stratified Blubbers, where the sizes are divided into 5 strata. Choose one Blubber at random from each stratum by choosing a random number. Locate the Blubbers & record their lengths. Find the mean. _______ _______ _______ _______ _______ = _______ x

Systematic Sample - A system is used to select the sample. Use the original page of Blubbers and select five Blubbers. Choose a random number between 1 &20. This is the first Blubber. Then add 20 to each previous number to find the other four Blubbers. Record their lengths and find the mean. _______ _______ _______ _______ _______ = _______ x

Cluster Sample - Cluster samples select whole groups or clusters of individuals (usually based on location). Choose a random number (r) between 1 & 20 and multiply by 5. Your sample will be that number (5r) and the four numbers preceding it. (Example: r = 12, 5r = 60, so sample is 60, 59, 58, 57, 56) Record the lengths of the Blubbers and find the mean. _______ _______ _______ _______ _______ = _______ x