2016 US Presidential Election Why Are Polls So Wrong? Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 2016 US Presidential Election Why Are Polls So Wrong? Milo Schield Augsburg College Minneapolis Critical Thinking Club Copy of these slides at: www.StatLit.org/pdf/2016-Schield-CTC1-Slides.pdf 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 1 1

Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 2 2016 Aug 12 Trump 12% Nate Silver 538 . 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 2 2

Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 3 2015 Oct 22 Trump 14% Nate Silver 538 . 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 3 3

2016 Nov 5 Trump 35% Nate Silver 538 . 4 Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 4 2016 Nov 5 Trump 35% Nate Silver 538 . 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 4 4

Final Tally . 5 Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 5 Final Tally . 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 5 5

How Far Off? . 6 Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 6 How Far Off? . 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 6 6

Blame the Polls . 7 Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 7 Blame the Polls . 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 7 7

Why the anti-Trump bias? Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 8 Why the anti-Trump bias? Some Plausible Explanations: Pollsters are liberals or have a liberal bias. Ignored statistical margin of error (plus/minus 3 points) Ignored correlation between state margins. Big “Undecided” up to election day. [Allocated 50/50] Ignored time. Predicted using static analysis. Actual election-prediction error Selection bias by everyone. 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 8 8

1. Liberal Bias . 9 Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 9 1. Liberal Bias . 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 9 9

2a Not Statistically Significant Popular Vote: 11/05 Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 10 2a Not Statistically Significant Popular Vote: 11/05 . 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 10 10

2b Not Statistically Significant Electoral Votes: 11/05 Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 11 2b Not Statistically Significant Electoral Votes: 11/05 . 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 11 11

3. State Margins: Correlated Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 12 3. State Margins: Correlated OH-MI: 0.90 OH-WI: 0.86 MI-WI: 0.86 IA-WI: 0.84 IA-OH: 0.84 IA-MI: 0.83 MN-OH: 0.81 MN-WI: 0.80 MN-MI: 0.79 PA-MN: 0.72 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 12 12

4. Undecided: Twice as big Schield ICOTS Analyzing Numbers in the News 15 May 2008 2014 13 4. Undecided: Twice as big . Polls normally split undecided 50-50 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 13 13

4. Undecided… Already Decided for Trump Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 14 4. Undecided… Already Decided for Trump Some "undecided" voters had already decided in favor of Trump, but didn't want to admit it. Some polls showed Trump getting 0% of the black vote in Pennsylvania and Ohio. In exit polls, Trump got about 8% of the black vote. . 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 14 14

Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 15 4. Last-Week Deciders Voters who said they were “undecided” until the election (last-week deciders) typically voted for Trump. And they did so – by big margins! . 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 15 15

5. Late-Breaking “Change” Ignored time Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 16 5. Late-Breaking “Change” Ignored time Should have “predicted” next week for each state. 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 16 16

6. Average of all Polls Is Not Very Accurate Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 17 6. Average of all Polls Is Not Very Accurate . 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 17 17

6. Average of all National Polls Is Not Very Accurate Schield ICOTS Analyzing Numbers in the News 15 May 2008 2014 18 6. Average of all National Polls Is Not Very Accurate . 95% Margin of Error: 3.6 Points 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 18 18

6. State Error is Typically More than National Error Schield ICOTS Analyzing Numbers in the News 15 May 2008 2014 19 6. State Error is Typically More than National Error . 95% Margin of Error: 4.8 Points 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 19 19

6. Election Polls are more ‘Art” than ‘Science” Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 20 6. Election Polls are more ‘Art” than ‘Science” We gave some good pollsters the same data. They gave very different results!!! 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 20 20

7. Selection Bias. National Polls were OK Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 21 7. Selection Bias. National Polls were OK Nate Silver (11/08) predicted a 3.6 point margin for Hillary: Clinton: 48.5% Johnson 5.0% Trump: 44.9% Other: 0.6% http://projects.fivethirtyeight.com/2016-election-forecast/?ex_cid=rrpromo In fact, Hillary “won” by at least a 1.3 point margin: Clinton: 48.0% Other: 5.3% Trump: 46.7% http://cookpolitical.com/story/10174 Readers are guilty of selection bias; Inferring Electoral-College win from Popular-Vote win. 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 21 21

Conclusion Surveys report! Election polls predict. Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 22 Conclusion Election polls are closer to fortune telling to facts. Election polls are different (very different) from surveys! Surveys report! Election polls predict. Surveys never (almost) adjust. Election polls always adjust Polls have to adjust to match the profile of those that will vote. for (how to allocate) the undecided. the non-response bias. 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 22 22

Conclusion . 23 Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 23 Conclusion . 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 23 23

Best Predictor? Halloween Mask Sales Analyzing Numbers in the News Schield ICOTS 15 May 2008 2014 24 Best Predictor? Halloween Mask Sales www.bloomberg.com/news/videos/2016-10-13/can-halloween-masks-predict-the-winner-of-the-election. 2008SchieldNNN6up.pdf 2014-Schield-ICOTS-6up 24 24