© 2010 GfK NOP Exit Polls – the Whys and Wherefores (and more importantly the Hows) Nick Moon, GfK NOP

2 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP My own experience  Extensive research to develop an exit polling technique for the BBC  Ten Parliamentary by-elections  1990 European elections  Four GB general elections – 1992 – 2005  One Greek general election  On-site observation of US Presidential Election 2000  The most recent Ukrainian Presidential Election

3 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP Opinion Polls and Exit Polls  Methods for conducting opinion polls can differ enormously from one country to another Face to face/telephone/online Random/quota  Methods for conducting exit polls are much the same the world over Ukraine is a vast country (with even worse weather) but there exit poll was almost exactly the same as ours

4 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP Cut to the Chase, Nick  Don’t just make excuses about Ukraine being a big country – tell us how you got on

5 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP While the Triumph is Still Fresh Result PollError Yanukovych Tymoshenko Tigipko Yatsenyuk Yuschenko

6 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP What is an Exit Poll?  A very expensive way of getting an idea of an election result a few hours before you know for sure  A major contribution to understanding of the electoral process  A way to tell if the election was conducted fairly (or is it?)

7 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP Types of Exit Poll  Prediction Poll Who is going to win and by how much?  Analysis Poll Why did voters vote the way they did?

8 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP An exit poll may be more than a poll  Interviewing voters This is all we do in the UK  Collection of precinct-level results as soon as they are announced A representative sample can predict the whole, quickly –In some European countries this is all that is done –In the US this is done alongside a voter survey  Analysis of partially aggregated actual results Used to fine-tune projections made using the above

9 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP Exit Poll Problems - issues  Interviewing has to be clustered by polling station Cost is a function of number of stations, not number of interviews  Voting behaviour tends to be very clustered at the polling station level Several cases in the past of >90% support for one party –And several of 100% in Ukraine Far more interviews are conducted than are actually needed, just because the interviewer might as well do as many interviews as are easily achievable  Actual results at polling station level not always available, which makes sampling less reliable US and Greece yes, UK and Ukraine no

10 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP Exit Poll Problems – issues (UK)  We are really predicting the result of over 600 separate elections, and can’t cover each one  The only local level information available on actual voting is at ward level, and only for local elections Local level turnout is far lower and behaviour often very different This makes stratification of the sample by voting is virtually impossible

11 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP Exit Poll Problems – UK solutions  Use as many polling stations as possible  Use the same polling stations as the previous election Can look at change rather than simple totals Eliminates effect of bias IF bias is constant (Big IF)

12 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP The 2005 GB Exit Poll  Conducted jointly by NOP and MORI  Conducted on behalf of both BBC and ITN  120 polling stations Most used previously by NOP or MORI  Approx 16,500 interviews in all  Just measuring claimed current vote

13 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP The Practicalities (1)  Polling runs from 7am to 10pm  Split into two equal shifts  Each has an interviewer and a “clicker”  “Clicker” uses preordained sampling interval to select respondent  Interviewer estimates voting behaviour, then asks voter to fill in an exit poll ballot  No replacement for refusals

14 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP The Practicalities (2)  Ballot designed to replicate actual one as closely as possible Has candidate names and parties In alpha order as on real ballot  Ballots collected every hour  Individual results phoned back to team of telephone interviewers  Results weighted to take into account any discrepancy between intended and actual number of interviews in each hour

15 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP The Practicalities (3)  Aggregate data at level of time by polling station produced at regular intervals Along with individual level data file  Collected by the BBC/ITN analysts  Run through various algorithms  Initial projection of result at 10pm Final projection at 11pm

16 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP And How Good Was It?

17 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP In case you’ve forgotten NOPActual result

18 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP The BBC/ITN exit poll  Forecast majority 66 seats  Actual majority 66 seats

19 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP The BBC/ITN exit poll Jeremy Paxman (paraphrased) “I don’t know why we spend all this money on the exit poll, we should just do without it” Tony King (paraphrased) “These results suggest our exit poll is overestimating Labour”

20 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP The BBC/ITN exit poll Predicted majority based on exit poll 66 seats Predicted majority based on early actual results 80 seats Final majority 66 seats

21 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP The BBC/ITN exit poll Sorry Jeremy The BBC should carry on paying for the exit poll, and stop paying for everything else

22 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP The 2010 General Election Exit Poll  Conducted jointly by GfK NOP and Ipsos MORI  BBC and ITN now joined by Sky  As close as possible to the 2005 design Changed polling places where the constituent voters or the place itself had changed significantly Replaced these with an eye to filling gaps in current design and also looking to the next election

23 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP Are exit polls measures of election fixing?  Ukraine 2004 Round 2 But many other indicators of fraud  Venezuela Questions raised about exit poll  US 2000 There almost certainly was some fraud but the exit poll is not a smoking gun

24 GfK NOPExit Polls Whys and Wherefores Nick Moon20 January 2010 © 2010 GfK NOP To conclude  Exit polls are a good way to give a good advance idea of who has won, and why, and properly conducted exit polls can act as some sort of check on electoral fraud, but as President Carter said, they should not be used as sole arbiters of the fairness of an election