Evaluate Statistically Based Reports ( AS 3.12) Dru Rose (Westlake Girls High School) Workshop AJ Margin of Error :Clarifying the rules of thumb

The purpose of this workshop To clarify the rules of thumb for estimating MoE and their relationship to theory. To demonstrate the power of technology for developing the concept of margin of error (making the topic accessible to a wider diversity of students than a theoretical approach relying on the central limit theorem and the normal distribution). To share two activities I have developed for clarifying the rules of thumb with students. Dru Rose

Where do the rules of thumb come from? 1.Single poll % Media reports use a 95% level of confidence. Usual theoretical formula for standard error of a single proportion Dru Rose

2. Comparison within one group MoE for the difference ≈ 2 x MoE Where do the rules of thumb come from? 2 x MoE Dru Rose

Where do the rules of thumb come from? 3. Comparison between 2 independent groups MoE for the difference ≈ 1.5 x Average MoE Dru Rose

Developing the rules of thumb with students: Use of technology (Central Limit Theorem and normal approximation to binomial distribution no longer in NZEA Level 3 ) Dru Rose

n=100

We can use the coverage spreadsheet developed by Chris Wild and Dave Smith to show that the rule of thumb gives about 95% coverage for realistic sample sizes of around say n= 600

Developing the rules of thumb with students : 2. Comparison within one group 2 x MoE We cannot use iNZight this time. The rule hinges on the premise that there are two main options with others having very small support. When this is the case, the following argument is valid: Suppose one option has 55% support and the second has 45% support, with a poll MoE of 5%. The first option could have as low as 50% support and the second as high as 50% support. We need a poll% difference between them of more than 2 x MoE (i.e.>10%) to conclude that the first option has more support.

Developing the rules of thumb with students: 2. Comparison within one group 2 x MoE Use of technology: We can use the coverage spreadsheet to demonstrate that the 2 x MoE rule generally gives about 95% coverage provided there are two main options with close to 50% support and other options having very little support. when support for other options is substantial, e.g. current Green Party support, the 2 x MoE rule for the main players over-estimates the MoE for the difference)

Developing the rules of thumb with students: 3. Comparison between 2 independent groups 1.5 x Average MoE We can use the KareKare cards and bootstrap VIT module in iNZight, with the sample within groups option. (Students can watch the differences arrow flip direction and note how often negative differences are produced.)

We can use the coverage spreadsheet to demonstrate that the “ 1.5 x Average MoE” rule gives about 95% coverage in most real polling situations: Developing the rules of thumb with students: 3. Comparison between 2 independent groups

How can we help students sort out which rule to apply when testing claims in the media?

Testing claims in the media 54.7% 59.2% 54.7%-1.2 p. pts 8.3p.pts 17.8 p pts (63.2 – 54.9 )

3. In the former Auckland City, a higher proportion of residents disagreed than agreed with the new policy. 4. Can it be claimed that support is consistently higher outside the old central Auckland area? p. pts 2.7p.pts 19.1 p pts Claim NOT supported -2.4 p. pts 19.6p.pts 41.6 p pts

Research New Zealand Survey 28/02/2012: “ Are Our Buildings Safe to Occupy? ”