5.1 – Designing Samples

Observe individuals and measure variables of interest, but do not attempt to influence the responses Observation: Experiment: Deliberately impose some treatment in order to observe their responses

Entire group of individuals that we want information about Population: Sample: Part of the population that represents the population of interest Sampling: Studying a part in order to gain info about the whole Census: Attempts to contact every individual in the entire population

Sampling Method: Process used to choose the sample from the population Bias: Systematically favors certain outcomes

Voluntary Response Sampling: People who choose themselves by responding to a general appeal. Biased because people with strong opinions, especially negative ones, are more likely to respond.

Convenience Sampling: Choosing individuals who are easiest to reach

Simple Random Samples (SRS): Consists of n individuals from the population chosen in such a way that every set of n individuals has an equal chance to be in the sample selected Each group AND each person in each group has an equal chance!

Random Digit Table: A table with a long string of digits 0-9 where: Each entry in the table is equally likely to be chosen The entries are independent of each other

Choosing an SRS 1. Label. Assign a number to each individual 2. Table. Use Table B to select labels at random 3. Stopping Rule. Know when to stop sampling 4. Identify Sample. Use the labels to identify the subjects

Calculator Tip: Random Integers Math – Prob – RandInt (smallest #, largest #, n)

Example #1 Suppose I wish to choose ten people from three Statistics classes to receive a bonus of 20 extra credit points. Picking the first 10 students that come to mind would be biased and unfair. How do I get a SRS where everyone would have an equal chance to receive the extra credit?

We must use the Random Number Table STEPS: Suppose there are a total of 90 students in Statistics. We must assign each student name a number: 01, 02, 03, 04, 05, …… 55, 56, 57 ……… 88, 89, 90 Randomly choose a row and column to start on in the random number table: Row109, Column 1.

______ _______ ______ _______ _______ Enter the numbers you read (in pairs) horizontally: ______ _______ ______ _______ _______ If you complete a whole row and need to keep going, drop down to the beginning of the next row. If the number is not in your range, ignore it and move to the next two digits. If the number is a repeat of one you already selected, skip it and move to the next two digits.

______ _______ ______ _______ _______ 36 00 91 93 65 Enter the numbers you read (in pairs) horizontally: ______ _______ ______ _______ _______ 36 00 91 93 65 15 41 23 96 38 85 45 34 68 16 If you complete a whole row and need to keep going, drop down to the beginning of the next row. If the number is not in your range, ignore it and move to the next two digits. If the number is a repeat of one you already selected, skip it and move to the next two digits.

Stratified Random Sampling: The population is first separated into groups with similar characteristics, called STRATA and then a SRS is done within each stratum. The samples are combined to make the full sample. There is little variability inside the strata, but is variability between the strata. Men Choose 50 Women Choose 50

Cluster Sampling: Divides the population into groups or clusters. There is variability in the cluster, little difference between clusters. All the individuals in the randomly chosen clusters are selected. AP Stats - SCHS AP Stats - WHHS AP Stats - MMHS AP Stats - VHS

Systematic Random Sampling: Choose an n to start with and then survey every nth person after that.

Undercoverage: When some groups in the population are left out of the process of choosing the sample Nonresponse: When an individual chosen for the sample can’t be contacted or does not cooperate.

Response Bias: When the interviewer influences the response. (i.e. Female surveying male) Wording of Questions: Leading questions or the wording of the question that elicit a specific answer

Questions to Ask before you Believe a Poll Who carried out the survey? Even a political party should hire a professional sample survey firm whose reputation demands that they follow good survey practices.

What was the population? That is, whose opinions were being sought? How was the sample selected? Look for mention of randomly selecting How large was the sample? What percent of the population was surveyed?

What was the response rate? What percent of the original subjects actually provided information? How were the subjects contacted? By telephone? Mail? Face-to-Face? When was the survey conducted? Was it just after some event that might have influenced opinion?

What were the exact questions asked? Did the questions elicit a specific response?

Example #1 Identify any potential bias you can detect from the following: a. A business magazine mailed a questionnaire to the human resources directors of all the Fortune 500 companies, and received responses from 23% of them. Those responding reported that they did not find that such surveys intruded significantly on their workday. Nonresponse

Example #1 Identify any potential bias you can detect from the following: b. A question posted on the Lycos Web site on June 18th, 2000, asked visitors to the site to say whether they thought that marijuana should be legally available for medicinal purposes. Volunteer

Subject bias – just left bar! Example #1 Identify any potential bias you can detect from the following: c. Researchers waited outside a bar they had randomly selected from a list of such establishments. They stopped every 10th person who came out of the bar and asked whether he or she thought drinking and driving was a serious problem. Subject bias – just left bar!

Undercoverage, must have phone Example #1 Identify any potential bias you can detect from the following: d. Consumers Union called 400 random people and asked whether they have used alternative medical treatments, and if so, whether they had benefited from them. For almost all of the treatments, approximately 20% of those responding reported cures or substantial improvement in their condition. Undercoverage, must have phone

Wording of question Example #1 Identify any potential bias you can detect from the following: e. Researchers asked 300 random people, “Given that 18-year olds are old enough to vote and to serve in the military, is it fair to set the drinking age at 21?” Wording of question

Interviewer bias, males not comfortable to answer Example #1 Identify any potential bias you can detect from the following: f. Female researchers asked 200 random men if they had ever used Viagra. 1% of men said yes. Interviewer bias, males not comfortable to answer

Example #1 Identify any potential bias you can detect from the following: g. Concerned about reports of discolored scales on fish caught downstream from a newly sited chemical plant, scientists set up a field station in a shoreline public park. For one week they asked fishermen there to bring any fish they caught to the field station for a brief inspection. At the end of the week, the scientists said that 18% of the 234 fish that were submitted for inspection displayed the discolorization. Volunteer

TERMONOLOGY MATCH-UP 1. Population f. All the units or subjects you wish to study.

TERMONOLOGY MATCH-UP 2. Sample d. Subset of the entire population

TERMONOLOGY MATCH-UP 3. Convenience Sample a. To study the most common purchases at grocery stores, I choose my sample from the Ralph’s grocery store two blocks from my house.

TERMONOLOGY MATCH-UP 4. Voluntary Response Sample j. Extra TV show asks people to call in and vote for the hottest new male actor.

TERMONOLOGY MATCH-UP 5. SRS e. A sample in which each subject has an equally likely chance of being selected

TERMONOLOGY MATCH-UP 6. Undercoverage h. A study is conducted to determine the amount of fat Americans eat but Alaska and Hawaii aren’t contacted.

TERMONOLOGY MATCH-UP 7. Response Bias i. The interviewer (who is smoking) asks the interviewee: “Do you think smoking should be banned from public places?”

TERMONOLOGY MATCH-UP 8. Non-Response b. I call a particular household that has been chosen to be a part of my sample and no one ever answers.

TERMONOLOGY MATCH-UP 9. Systematic Random Sample g. A manufacture of chemicals wants to select 4 containers from each lot of 16 containers of a reagent to test for purity. He randomly selects the second container and every 4th container after that.

TERMONOLOGY MATCH-UP 10. Stratified Random Sample c. A university has 2000 male and 500 female faculty members. The equal opportunity employment agency officer wants to poll their opinions so he does a random sample of 200 males and 200 females to make up his sample.

5.2 - Designing Experiments

Experimental Units: Individuals the experiment is being done on Subjects: When the units are human beings Treatment: Experimental condition applied to the units

Factors: Explanatory variables Level: Amount of each factor Control: Effort made to minimize variability

Placebo: Something to make the subject believe they are receiving the treatment. Placebo Effect: Respond to the experiment because they believe they are receiving a treatment.

Control Group: Group of people who receive no treatment or the placebo Statistically Significant: An observed effect so large that it would rarely occur by chance

Principles of Experimental Design Control the effects of lurking variables on the response, most simply by comparing two or more treatments Replicate each treatment on many units to reduce chance variation in the results Randomize – use chance to assign experimental units to treatments.

Subjects: 20 volunteers Treatments: 3 drinks Response Variables: Example #1 For each of the following studies name the experimental units or subjects, the factors, the levels, the treatments, and the response variables. To test the effects of alcohol on driving performance, 20 volunteers were each asked to take a driving test under two conditions: sober and after three drinks. The order under which they took the driving test were randomized. An evaluator watched them drive on a test course and rated their accuracy on a scale from 1 to 10, without knowing which condition they were under each time. Subjects: 20 volunteers Treatments: 3 drinks Response Variables: Factors: 2 – # of drinks and driving test Driving ability

40 volunteers suffering from insomnia b. Is diet or exercise effective in combating insomnia? Some believe that cutting out desserts can help alleviate the problem while others recommend exercise. Forty volunteers suffering from insomnia agreed to participate in a month-long test. Half were randomly assigned to a special no-desserts diet; the others continued desserts as usual. Half of the people in each of these groups were randomly assigned to an exercise program, while the others did not exercise. Those who ate no desserts and engaged in exercise showed the most improvement. Subjects: 40 volunteers suffering from insomnia Factors: Dessert and Exercise D – E D – no E N – E N – no E Treatments: Response Variables: Insomnia improvement

Subjects: Migraine sufferers Factors: Drug and Ice water D – I c. Some people claim they can get relief from migraine headache pain by drinking a large glass of ice water. Researchers plan to enlist several people who suffer from migraines in a test. When a participant experiences a migraine headache, he or she will take a pill that may be a standard pain reliever or a placebo. Half of each group will also drink ice water. Participants will then report the level of pain relief they experience. Subjects: Migraine sufferers Factors: Drug and Ice water D – I D – no I P – I P – no I Treatments: Response Variables: Level of pain relief

Experimental designs Completely Randomized Design: Divides subjects into groups randomly

Group 1 T1 Random Allocation Group 2 T2 Subjects compare Group 2 T3

Block design: Dividing subjects into similar grouping to control for possible lurking variables

T1 T2 compare Block 1 Random Assignment T3 Subjects Block 2 T1 Random Assignment T2 compare T3

Matched Pairs design: Compares just 2 treatments. Subjects are matched into pairs of like people. Twins are popular to use. You may also be your own pair!!!

T1 compare Pair 1 Random Assignment T2 T1 Subjects Pair 2 Random Assignment compare T2 T1 Pair 3 Random Assignment compare T2

Example #2 Do carrot plants grown with a new fertilizer produce bigger and better tasting carrots to plants raised without the fertilizer under similar conditions? Suppose we obtain 24 carrot plants of the same variety and want to test three different treatments: no fertilizer, some with 16 grams of fertilizer, and some with 32 grams of fertilizer. Create a Completely Randomized Diagram for this experiment. Group 1 No fert. Compare size and taste of carrots 24 carrots Random Allocation Group 2 16g fert. Group 3 32g fert.

Example #3: Does St. John’s Wart supplement improve mood? Suppose researchers decide to test this theory by selecting 6 women and pairing them according to attitude, age, diet and exercise habits. One woman of the pair would receive a St. John’s Wart supplement and the other would receive an inert pill looking identical to the herbal supplement. Create a Matched-Pairs Diagram for this experiment.

St. J Compare mood Pair 1 Random Assignment Plac. St. J 6 women Pair 2 Random Assignment Compare mood Plac. St. J Pair 3 Random Assignment Compare mood Plac.

Example #4 Is diet effective in combating insomnia? Some people believe that cutting out desserts can help alleviate the problem, however, there may be a difference depending on gender. 20 men and 20 women suffering from insomnia were selected. Half of the men and half of the women were assigned to a no-dessert diet; the others continued desserts as usual. Create a Block Diagram for this experiment.

10 women with Dessert compare sleep patterns 20 women Random Assignment 10 women with No Dessert Subjects 10 men with Dessert 20 men Random Assignment compare sleep patterns 10 men with No Dessert

Cautions about Experiments Both the researcher and subject doesn’t know if they are receiving treatment Double-Blind: Lack of Realism: Subject is aware it is an experiment and skews results