Exit-poll analysis and prediction Stephen Fisher Following: Curtice, John and David Firth (2008) Exit polling in a cold climate: the BBC-ITV experience in Britain 2005, J. R. Stat. Soc. A, 171, Presentation to the NCRM-BPC Opinion Polls Conference, British Academy, 20 th January 2010

Broad research design principles: 1 Model the pattern of change across constituencies in the share of the vote since the last election – i.e. not directly estimating the results in 2010 but change since 2005, since more variance between constituencies in shares than in changes. – Also, assessing and allowing for different swings in different seats

Broad research design principles: 2 Estimate probabilities for each party winning each seat – i.e. allow for the random/unexplained variation between constituencies in the prediction Predicted number of seats for each party is the sum of the probabilities across constituencies Primary aim is to predict seat totals, not share of the vote

Infrastructure 2010 exit poll will be a joint BBC/ITN/Sky project Fieldwork by MORI and NOP, as in 2005 People contributing to the analysis: Jouni Kuha (LSE), John Curtice (Strathclyde), Clive Payne (Oxford), Rob Ford (Manchester) Debt of gratitude and computer code to David Firth

Selection of constituencies Revisit all 107 viable locations from 2005 Top up to 130 by sampling the kinds of constituencies thought to be useful and currently under-represented. – e.g. new decision to explicitly attempt to have a group of Lab-LD seats Pick the most representative polling station in the constituency for the new locations

Exit poll locations in 2005

Statistical analysis Model the change in the share of the vote since the last election Consider data with and without interviewer guesses for those who refused – In 2005 ignoring the guesses and refusals worked best Consider lots of different predictor variables – E.g. census data, market research data, strategic situation, incumbency – Expectations of geographical variation informed by practice with pre-election day polls But keep the final model simple (N=130)

Producing predictions Generate predicted probabilities for each party winning each seat from the statistical regression models of the data. – Using estimates of both explained and unexplained variance. Sum the probabilities for each party across constituencies to estimate the total number of seats for the party. In 2005 the exit-poll data suggested a Lab majority of 100 under uniform change, but the method accurately predicted 66 – Introduction of explained (regression) and unexplained (probabilistic prediction) variation both equally account for the difference between uniform change and final prediction.

Probabilistic prediction compared with the swingometer What would be the effect of allowing for unexplained variation in a swingometer estimate of the result? – Depends on the distribution of seats according to marginality. – Smoothes the relationship between predicted swing and predicted seats.

Need for probabilistic prediction in 2005

Marginal Lab-Con seats for 2010 E.g. probabilistic method would predict fewer Con seats from a 7% swing than the swingometer because it would allow for the seats immediately either side of the 7% point to split between Con and Lab, and there are more to the left than right. -But not much difference.

Simulation 1: Stability of notional 2005 results Rerun 2005 notional results but adding noise to allow probabilistic results – Seats with 05 margin <1% become 50:50 – Seats with 05 margin c.4% become 90:10 Changes the seat totals from Con 210, Lab 348 to Con 212, Lab 346 with LD unchanged. – i.e. not much change

Simulation 2: Poll projection ukpollingreport.co.uk average of polls: – Con 41, Lab 29, LD 18 – i.e. +8, -7, -5 since 2005 Uniform swing: Con majority of 44 Probabilistic projection: Con majority of 48 Very little difference

Simulation 2: Seats in the balance for the Tories under the simulation constituency wp05 sp05 pctmaj05 conprw labprw ldprw Dagenham & Rainham LAB CON Erewash LAB CON Norwich South LAB LD Bath LD CON Leeds North East LAB CON Crewe & Nantwich LAB CON Ochil & South Perthshire LAB SNP Oxford West & Abingdon LD CON Newport West LAB CON Warwickshire North LAB CON Hampstead & Kilburn LAB LD Coventry South LAB CON Dorset Mid & Poole North LD CON Argyll & Bute LD CON Telford LAB CON Berwickshire, Roxburgh & Selkirk LD CON Winchester LD CON Luton South LAB CON Brighton Pavilion LAB CON St. Austell & Newquay LD CON

Simulation 2 – Distribution of Predicted Probabilities

Particular difficulties for exit poll prediction in 2010 Boundary changes – Possible errors in both dependent and explanatory variables Expenses scandal – More MPs stepping down – Potentially more variance between constituencies Census data old (2001) High expectations from 2005!