Confidence Intervals feeling comfortable with error Richard Lambert, Ph.D.

General Overview Our goal in conducting a study is often to estimate a particular parameter of interest with as little error as possible. Our goal in conducting a study is often to estimate a particular parameter of interest with as little error as possible. The % that report they will vote for a given candidate is a good example. The % that report they will vote for a given candidate is a good example.

General Overview The % in your sample reporting they will vote for a given candidate if the election were held today is your point estimate of the corresponding parameter. The % in your sample reporting they will vote for a given candidate if the election were held today is your point estimate of the corresponding parameter. It is your estimate of the population value given the sample results. It is your estimate of the population value given the sample results. However, you know it not likely to be exactly correct because of the influence of sampling error. However, you know it not likely to be exactly correct because of the influence of sampling error.

General Overview Therefore, the most honest way to express your results is to report a confidence interval around your best guess of the population parameter. Therefore, the most honest way to express your results is to report a confidence interval around your best guess of the population parameter. The confidence interval expresses a range of plausible values that we are confident contains the true population value and gives a picture of the possible influence of sampling error on the results. The confidence interval expresses a range of plausible values that we are confident contains the true population value and gives a picture of the possible influence of sampling error on the results.

Political Poll Examples Follow this link: Follow this link: Hillary takes big lead Hillary takes big lead Hillary takes big lead Hillary takes big lead This poll showed that 50% of democrats favored Hillary. If this poll was conducted with +/- 4% error, that means that the 95% confidence interval for these results would be 46%-54%. This poll showed that 50% of democrats favored Hillary. If this poll was conducted with +/- 4% error, that means that the 95% confidence interval for these results would be 46%-54%.

Example Suppose you want to conduct a survey of all the teachers in your school (N=150). Suppose you want to conduct a survey of all the teachers in your school (N=150). Your survey includes just one question: “Are you in favor of switching to block scheduling?” Your survey includes just one question: “Are you in favor of switching to block scheduling?” You hope every teacher in the school will complete the simple online survey and return it by the deadline. You hope every teacher in the school will complete the simple online survey and return it by the deadline.

Example Suppose you have responses from 30 teachers and you are interested in finding out what these results tell you. Assume that there is no systematic non-response bias, meaning the 30 really are representative of the 150. Suppose you have responses from 30 teachers and you are interested in finding out what these results tell you. Assume that there is no systematic non-response bias, meaning the 30 really are representative of the %, or 14 teachers, said “Yes” (in favor of block scheduling) and 53.3%, or16 teachers, said “No” (not in favor of block scheduling). 46.7%, or 14 teachers, said “Yes” (in favor of block scheduling) and 53.3%, or16 teachers, said “No” (not in favor of block scheduling). You know that your best guess of the percentage of teachers in favor of block scheduling, 46.7%, probably suffers from some influence of sampling error given the small sample size (n=30). You know that your best guess of the percentage of teachers in favor of block scheduling, 46.7%, probably suffers from some influence of sampling error given the small sample size (n=30).

Example Using the procedure your text, take a few minutes and calculate the 95% Confidence Interval for this situation. Using the procedure your text, take a few minutes and calculate the 95% Confidence Interval for this situation. You should arrive at the following results: You should arrive at the following results: Lower Limit=28.8%, Upper Limit=64.5%. Lower Limit=28.8%, Upper Limit=64.5%. Therefore, the most honest way to report this findings would be to say that 46.7% of our sample was in favor of block scheduling. We are 95% confident that between 28.8% and 64.5% of the teachers in our school are in favor of block scheduling. Therefore, the most honest way to report this findings would be to say that 46.7% of our sample was in favor of block scheduling. We are 95% confident that between 28.8% and 64.5% of the teachers in our school are in favor of block scheduling.

Example Now let’s look at some simulation results that will help us understand what this confidence interval means. Now let’s look at some simulation results that will help us understand what this confidence interval means. Suppose the true percentage of teachers in your school that favor block scheduling is 50%. Suppose the true percentage of teachers in your school that favor block scheduling is 50%. Let’s look at what would happen if we took repeated samples of size 30 from this population. Let’s look at what would happen if we took repeated samples of size 30 from this population.

Simulation Results  This picture represents 50 separate samples of size 30. For each sample a 95% confidence interval has been constructed around the estimate of the percentage of the population that would say yes to the survey question. This picture assumes that the real population percentage in favor is 50%.

Simulation Results  Notice how the pink point, the sample result, changes from sample to sample due to sampling error. Notice how the upper limit and lower limit to the confidence intervals also change from sample to sample. What does not change is the true population value. If we were to conduct this simulation over an infinite number of trials, 95% of the intervals we would construct would capture the true value of 50%.

Simulation Results  Notice in this example that most of the intervals, even though they are not the same, do include the real population value of 50%. How many do not? How many would you expect to capture or not capture 50%?

Simulation Results There were two intervals that did not capture 50% and 48 that did. There were two intervals that did not capture 50% and 48 that did. A 95% confidence interval means that we expect 95 out of every 100 intervals that were constructed using this method to capture (or include) the true population parameter, in this case 50%. A 95% confidence interval means that we expect 95 out of every 100 intervals that were constructed using this method to capture (or include) the true population parameter, in this case 50%. We know that sampling error will move the estimate around from sample to sample. Sometimes we will over-estimate 50% and sometimes we will under- estimate 50%. We know that sampling error will move the estimate around from sample to sample. Sometimes we will over-estimate 50% and sometimes we will under- estimate 50%.

Simulation Results However, we can be 95% confident that any one interval actually contains the correct value. However, we can be 95% confident that any one interval actually contains the correct value. By expressing our results with the confidence interval, we are reporting the possible influence of sampling error on our results. By expressing our results with the confidence interval, we are reporting the possible influence of sampling error on our results.