ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township

First Statistical Associations Statistical Society of London (1834) Statistical Society of London American Statistical Association (1839) American Statistical Association International Statistical Institute (1885) International Statistical Institute Sir Ronald Fisher and Prasanta Mahalanobis

Anders Kiaer and the Norwegian Census of 1891 Source: Kiaer, Anders N “The Representative Method Of Statistical Surveys”, p. 44.

Fisher’s Agricultural Research Use of repeated measurements to estimate crop yields (1919) Development of randomized experiments, hypothesis testing and tests of statistical significance Publishes Statistical Methods for Research Workers in 1925

Gini, Galvani and the Italian Census of 1921 Need to make room for the 1930 census Chose 29 of 216 census districts Averages computed across districts don’t match population numbers Both purposive and random sampling considered inferior to full census tabulations Source: Feinberg, Stephen E. and Judith M. Tanur “Reconsidering Neyman On Experimentation And Sampling: Controversies And Fundamental Contributions.” Probability and Mathematical Statistics Vol. 15, pp

The Impact of the Great Depression Unemployment clearly a massive problem Questions on work developed from 1880 through 1930 censuses Population measurements needed between decennial censuses Trial Census of Unemployment Enumerative Check Census of 1937 Leads to the Current Population Survey Source: Card, David “ Origins of the Unemployment Rate: The Lasting Legacy of Measurement Without Theory”

1936 Presidential Election Literary Digest Poll –2 million responses –forecasts Republican victory Gallup Poll –300,000 responses –forecasts Democratic victory Roosevelt gets 67% of the popular vote, 98% of electoral votes

1948 Presidential Election Gallup Poll –Use of quota sampling –3,250 responses –forecasts Dewey victory Truman gets 49.5% of the popular vote, 62% of electoral votes End of quota sampling Campaign dynamics

American Association for Public Opinion Research Founded in 1947 More than 2,200 public opinion scientists, survey researchers and statisticians are members Code of Professional Ethics Transparency Initiative

Average Errors in Presidential Polls Year# of Polls# of Candidates Average Error (%) (Avg) Source: Traugott, Michael “The Accuracy of the National Preelection Polls” Public Opinion Quarterly 69(5): 649

The 2000 Election: What Went Wrong? “If we say somebody’s carried a state, you can pretty much take it to the bank.” – Dan Rather Competitive pressures on networks to call the outcome in Florida first Voter News Service has problems with following procedure for sampling and interviewing exiting voters and with computer models processing incoming results

Accuracy of the 2008 Polls Source:

Identifying the Likely Voter

Likely Voter Questions How much thought have you given to the coming presidential election? How closely have you been following news about the candidates? Do you plan to vote in the presidential election? How certain are you that you will vote? Rate your chance of voting in November on a scale of 10 to 1, with 10 being "definitely will vote" and 1 "definitely will not vote.“ How often do you follow what's going on in government and public affairs? Have you ever voted in your precinct or election district? How often would you say you vote? Do you happen to know where people in your neighborhood go to vote? In the last election, did things come up that kept you from voting or did you vote? Source: Ask the Expert: Determining Who Is a 'Likely Voter' Scott Keeter, Director of Survey Research August 29, 2012

Problems with Polling in 2012 Source: Declining Response Rates Growth of the “Cell-only” population Shifting attention to state polls after 2000 Increasing popularity of survey by “robo- call”

Declining Response Rates Source:

The Growing “Cell-Only” Population Source:

Dual Frame Estimation Source: Summary of Differences Between Dual Frame and Landline Estimates

Surveys by Robo-call Understanding the math behind a survey Sample sizeCostMargin of error Fixed cost $10,0006.9%Unit cost25 400$15,0004.9% 600$20,0004.0% 800$25,0003.5% 1,000$30,0003.1% 1,200$35,0002.8% 1,400$40,0002.6% 1,600$45,0002.5% 1,800$50,0002.3% 2,000$55,0002.2% 3,000$80,0001.8% 5,000$130,0001.4%

Rasmussen’s Methodology “While we do not currently call cell phones directly when conducting our surveys, to reach those who have abandoned traditional landline telephones, Rasmussen Reports uses an online survey tool to interview a demographically diverse panel. Just like our telephone surveys, respondents for this approach are selected on a random basis to insure the reliability of the sample. As a result, you cannot sign up to participate. “

Aggregating State Polls Nate Silver’s 538 RealClear Politics Princeton Election Consortium PollTracker Dave Leip's Atlas of U.S. Presidential Elections

RealClear Average

Nate Silver/ NY Times

Princeton Election Consortium

Important Trends Early voting Voter ID movement/Voter fraud issue – Demographic changes in the US Increasing automation/improving robo-call technology More web-based surveying