1. 2016 US Presidential Election Why Are Polls So Wrong?
Analyzing Numbers in the News
Schield ICOTS
15 May 2008
2014
2016 US Presidential Election Why Are Polls So Wrong?
Milo Schield
Augsburg College
Minneapolis
Critical Thinking Club
Copy of these slides at:
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 1 1

2. Analyzing Numbers in the News
Schield ICOTS
15 May 2008
2014 2
2016 Aug Trump 12% Nate Silver 538 .
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 2 2

3. Analyzing Numbers in the News
Schield ICOTS
15 May 2008
2014 3
2015 Oct 22 Trump 14% Nate Silver 538 .
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 3 3

4. 2016 Nov 5 Trump 35% Nate Silver 538 . 4 Analyzing Numbers in the News
Schield ICOTS
15 May 2008
2014 4
2016 Nov 5 Trump 35% Nate Silver 538 .
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 4 4

5. Final Tally . 5 Analyzing Numbers in the News Schield ICOTS
15 May 2008
2014 5
Final Tally .
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 5 5

6. How Far Off? . 6 Analyzing Numbers in the News Schield ICOTS
15 May 2008
2014 6
How Far Off? .
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 6 6

7. Blame the Polls . 7 Analyzing Numbers in the News Schield ICOTS
15 May 2008
2014 7
Blame the Polls .
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 7 7

8. Why the anti-Trump bias?
Analyzing Numbers in the News
Schield ICOTS
15 May 2008
2014 8
Why the anti-Trump bias?
Some Plausible Explanations:
Pollsters are liberals or have a liberal bias.
Ignored statistical margin of error (plus/minus 3 points)
Ignored correlation between state margins.
Big “Undecided” up to election day. [Allocated 50/50]
Ignored time. Predicted using static analysis.
Actual election-prediction error
Selection bias by everyone.
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 8 8

9. 1. Liberal Bias . 9 Analyzing Numbers in the News Schield ICOTS
15 May 2008
2014 9
1. Liberal Bias .
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 9 9

10. 2a Not Statistically Significant Popular Vote: 11/05
Analyzing Numbers in the News
Schield ICOTS
15 May 2008
2014 10
2a Not Statistically Significant Popular Vote: 11/05 .
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 10 10

11. 2b Not Statistically Significant Electoral Votes: 11/05
Analyzing Numbers in the News
Schield ICOTS
15 May 2008
2014 11
2b Not Statistically Significant Electoral Votes: 11/05 .
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 11 11

12. 3. State Margins: Correlated
Analyzing Numbers in the News
Schield ICOTS
15 May 2008
2014 12
3. State Margins: Correlated
OH-MI: OH-WI: MI-WI: 0.86
IA-WI: IA-OH: IA-MI: 0.83
MN-OH: MN-WI: MN-MI: 0.79
PA-MN: 0.72
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 12 12

13. 4. Undecided: Twice as big
Schield ICOTS
Analyzing Numbers in the News
15 May 2008
2014 13
4. Undecided: Twice as big .
Polls normally split undecided 50-50
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 13 13

14. 4. Undecided… Already Decided for Trump
Analyzing Numbers in the News
Schield ICOTS
15 May 2008
2014 14
4. Undecided… Already Decided for Trump
Some "undecided" voters had already decided in favor of Trump, but didn't want to admit it.
Some polls showed Trump getting 0% of the black vote in Pennsylvania and Ohio.
In exit polls, Trump got about 8% of the black vote. .
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 14 14

15. Analyzing Numbers in the News
Schield ICOTS
15 May 2008
2014 15
4. Last-Week Deciders
Voters who said they were “undecided” until the election (last-week deciders) typically voted for Trump.
And they did so – by big margins! .
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 15 15

16. 5. Late-Breaking “Change” Ignored time
Analyzing Numbers in the News
Schield ICOTS
15 May 2008
2014 16
5. Late-Breaking “Change” Ignored time
Should have “predicted” next week for each state.
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 16 16

17. 6. Average of all Polls Is Not Very Accurate
Analyzing Numbers in the News
Schield ICOTS
15 May 2008
2014 17
6. Average of all Polls Is Not Very Accurate .
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 17 17

18. 6. Average of all National Polls Is Not Very Accurate
Schield ICOTS
Analyzing Numbers in the News
15 May 2008
2014 18
6. Average of all National Polls Is Not Very Accurate .
95% Margin of Error: 3.6 Points
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 18 18

19. 6. State Error is Typically More than National Error
Schield ICOTS
Analyzing Numbers in the News
15 May 2008
2014 19
6. State Error is Typically More than National Error .
95% Margin of Error: 4.8 Points
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 19 19

20. 6. Election Polls are more ‘Art” than ‘Science”
Analyzing Numbers in the News
Schield ICOTS
15 May 2008
2014 20
6. Election Polls are more ‘Art” than ‘Science”
We gave some good pollsters the same data.
They gave very different results!!!
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 20 20

21. 7. Selection Bias. National Polls were OK
Analyzing Numbers in the News
Schield ICOTS
15 May 2008
2014 21
7. Selection Bias. National Polls were OK
Nate Silver (11/08) predicted a 3.6 point margin for Hillary:
Clinton: 48.5% Johnson 5.0%
Trump: 44.9% Other: %
In fact, Hillary “won” by at least a 1.3 point margin:
Clinton: 48.0% Other: 5.3%
Trump: %
Readers are guilty of selection bias;
Inferring Electoral-College win from Popular-Vote win.
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 21 21

22. Conclusion Surveys report! Election polls predict.
Analyzing Numbers in the News
Schield ICOTS
15 May 2008
2014 22
Conclusion
Election polls are closer to fortune telling to facts. Election polls are different (very different) from surveys!
Surveys report! Election polls predict.
Surveys never (almost) adjust. Election polls always adjust
Polls have to adjust
to match the profile of those that will vote.
for (how to allocate) the undecided.
the non-response bias.
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 22 22

23. Conclusion . 23 Analyzing Numbers in the News Schield ICOTS
15 May 2008
2014 23
Conclusion .
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 23 23

24. Best Predictor? Halloween Mask Sales
Analyzing Numbers in the News
Schield ICOTS
15 May 2008
2014 24
Best Predictor? Halloween Mask Sales
2008SchieldNNN6up.pdf
2014-Schield-ICOTS-6up 24 24