1. Producing Data: Sampling Inference: Probabilities and Distributions Week 8 March 1-2, 2011 (REVISED March 8 22:30)

2. To start…. To start today I just want to briefly go over the material we did last time on Chapter 8 and then spend the bulk of tonight on Material from Chapter 10 and 11

3. Producing Data DON’T!!!!!!!MY BEST ADVICE REGARDING THE GENERATION OF DATA SUITABLE FOR STATISTICAL ANALYSIS IS DON’T!!!!!!! HOWEVER, having noted that, you do need to be aware of how people go about collecting the data you are using and the limits that are imposed on your work by these methods

4. The first half of today’s class will look at sampling Some Key Terms –Population (the entire group of individual cases about which we want information) –Sample (a part of the population from which we actually collect information and from which we will draw conclusions about the entire population) –Sampling Design (the method we use to choose a sample from the population) –Inference (the process of generalizing our results from our sample to the wider population)

5. The Ideal Sample A sound sampling design is essential if we are going to produce results that are generalizable from our sample to the entire population In the ideal sample design: 1.Each individual case has an equal probability of being drawn as part of the sample 2.Each characteristic of relevance to the study is present in equal probability in our sample and the population-at-large. ****NOTE: by the end of today’s lesson we will see that this second characteristic is not essential as long as: 1.The first point is realized 2.and the sample we draw is large enough However, for now we will assume it is a good idea to aim for

6. Garbage In… Garbage Out… The truth of the matter is –It is very hard to absolutely achieve both of the ideal characteristics of a sample For some populations it might be impossible such as relatively small groups (think about policy-makers and advisors for example) –It is getting harder all the time to do so –The best we can probably ever hope to do is “approximate” the two characteristics. The closer we approximate the two characteristics The better will be the accuracy of our inferences

7. Sample types Convenience Samples Voluntary Response Samples Purposive Samples –Snowball Samples Simple Random Samples –Not only need to randomly choose the individual cases, but also need to figure out the correct size for the sample so that every possible sample has an equal chance to be chosen

8. Stratified Random Sample –Sometimes you want to go beyond a Simple Random Sample so as to ensure certain characteristics are not left to even the chance of error. This is common in public opinion polling where several Simple Random Samples will be drawn for populations identified along geographic lines.

9. The next slide portrays a multistage stratified random sample

10. BCPrairieON QU Atlantic Randomly Select A set of Postal Sorting Areas for each Region and Randomly Select People in Those PSAs National Stratified Random Sample Greater Vancouver Greater Victoria Other Van. Island The South Interior The North

11. Stratification need not be done on geographic lines (though it often is) Any characteristic can be used as the basis of stratification. What matters is that it is something of importance to your research

12. Caution As the book notes, a sample’s quality is built on more than just the sampling technique We need to have reliable information about the population before we can even begin to select a decent sample Non-responses Response bias And many other factors can also degrade a sample’s quality and the quality of the data we can get out of it.

13. A funny little thing called probability As we noted earlier, when we take a sample and conduct a study we want to generalize (or infer) the results of our study to the wider population our sample is drawn from. If 40 percent of our sample say they will vote Conservative we would like to estimate that this is the situation among the general population However, we know there is a chance that if we had drawn two separate samples and done two simultaneous studies, we might have gotten different results for each sample

14. If the variation of results between samples is too great, then we cannot generalize our results to the wider population. Therefore we need to know about probability to estimate the chance that our results will vary from the actual situation in the population at large. The statistician might tell us there is a 95% chance that our survey result is accurate plus or minus 3% (meaning there is a 95% chance that the real support for the Conservatives among the general population is between 37% and 43%)

15. In the Long-Run The law of probability is based on a regularly documented observation that while chance can produce erratic results over the short-term or when small numbers are looked at, it generates regular and predictable outcomes over the long-term and as numbers increase

16. Random –Outcomes are uncertain but there is nonetheless a regular distribution of outcomes in a large number of repetitions (this is not a pattern). Probability –The probability of any outcome of a random phenomenon is the proportion of times the outcome would occur in a very long series of repetitions.

17. Examples of Commonly Known Probable Long-term Results Coin Tossing –Fifty/Fifty Stockmarket returns –Reversion to the mean The fall of cards in a game (if you are able to count them properly and quickly enough)

18. Randomness Perfect Randomness is very rare and the ability to select numbers totally at random (so that each and every other number had just as good a chance of being selected and no pattern can ever be predicted) is valuable

19. Probability Models Sample Space “S” of a random phenomenon is the set of all possible outcomes An Event is an outcome or a set of outcomes of a random phenomenon (the roll of the dice, the flip of the coin, etc.) A Probability Model is a mathematical description of a random phenomenon consisting of –A sample space S –A way of assigning probability to events

20. As in figure 10.2 in the book: There are 36 possible combinations if you roll two standard dice. If we wanted to define a sample space S for (5) it would be comprised of the four possible ways to roll 5 (i.e. the four “events” that result in 5 A ={ roll 1 & 4, roll 2 & 3, roll 3 & 2, roll 4 &1}

21. Some Formal Probability Rules A probability is a number between 0 and 1 –An event with a probability of 0 ought never to occur, An event with a probability of 1 out to always occur All possible outcomes together must equal 1 If two events have no outcomes in common, the probability that one or the other occurs is the sum of their individual probabilities. Eg. If one event occurs in 40% of cases, and the other in 25% and the two cannot occur together then the probability of one or the other occurring is 65% The probability that an event does not occur is 1 minus the probability that it does occur

22. Discrete vs. Continuous Models Discrete Probability Models –Assume that the sample space is finite –To assign probabilities list the probabilities of all the individual outcomes (must be between 1 and 0 and add up to 1).The probability of an event is the sum of the outcomes making up the event. –Think of our dice example: What is the probability of rolling a five? roll 1 & 4 = 1/36 roll 2 & 3 = 1/36 roll 3 & 2 = 1/36 roll 4 &1 = 1/36 Total Probability = 4/36 = 1/9 = 0.111

23. Continuous Probability Models –Assign probabilities as areas under a density curve (such as the normal curve) –The area under the curve and above any range of values is the probability of an outcome in that range –This is what we did in chapter 3!

24. Sampling Distributions Some Key Words –Parameter: a number that describes the population. We often can only speculate on this as we only have data for a sample. –Statistic is a number that can be computed from the sample data without making use of any unknown parameters. We often use statistics to estimate parameters.

25. The Law of Large Numbers: –Draw observations at random from any population with finite mean µ. –As the number of observations drawn increases, the mean of the observed values gets closer and closer to the mean µ of the population.

26. Two types of distribution of variables (be careful) The Population Distribution of a variable is the distribution of values of the variable in the population The Sampling Distribution of a statistic is the distribution of values taken by the statistic in all possible samples of the same size from the same population (in other words, how a statistic varies in many samples drawn from the same population)

27. For those who like math Suppose that is the mean of a SRS of size “n” drawn from a large population with mean and standard deviation Then the sampling distribution of has mean and standard deviation

28. Bottom Line and Caution If we can compute the average among a sample we have a decent guess as to what the average is in the population The average of a sample is generally a better guess of the average of the population than any one case in the population. In other words, based on a survey of incomes, I can tell you with a reasonable level of certainty what the average income of Canadians is. I cannot tell you just from that what the income of any specific Canadian is.

29. The Central Limit Theorem When the number of samples is large enough, the sample distribution of the mean will look like a normal distribution even if the population from which the sample is drawn does not exhibit a normal distribution for the variable as long as the population has a finite standard deviation This means we can use the known properties of the normal curve to explore our data and study it, even if we do not know for certain what the parameter values are for variables in the general population.