A part of Creston Unlimited Confidential: For research purposes only British Polling Council & Market Research Society Inquiry Evaluating the ICM pre-election polls June 2015 Martin Boon

Confidential: For research purposes only2 Not good; where’s the rogue? 2015 ICM/Guardian campaign shares ICM campaign polls – April-May 2015 Base: All participants expressing an intention to vote: turnout weighted, past vote weighted & adjusted. 6% 2% 3% 1% 7%

Confidential: For research purposes only3 What happened – a failure of tried & trusted? Nope. The ICM adjustment techniques all worked pretty much as expected, and in the right direction. Raw dataDemographic weighting Past vote weighting Turnout weighting Adjustment Conservative35%32% 33%34% Labour35%38%37%36%35% Lib Dem8%7%8% 9% UKIP11%12% 11% Green3% 4% SNP/PC6% 7%6% Other1% Total99% 100%

Confidential: For research purposes only4 What was different? Voting patterns among up-weighted C1/C2 social grades directly contributed to final poll waywardness.

Confidential: For research purposes only5 Recall poll evidence -Reported 2015 vote does not match the election result, with under-estimate of the Conservative lead still very much present (Con: 34% vs Lab: 33%). Thus, late swing cannot account for our error. -Implies that sampling and weighting no longer adequate in predicting Con/Lab shares. -Vote switching: 9 in 10 did what they said they’d do – although minor and largely off-setting switching very slightly in favour of the Conservatives. -‘Shy Tory’ effect worth 1% swing and once again vindicated our ‘adjustment’. May need to be up-scaled and relative adjustments by party applied. -Recalled turnout of 86%. Despite over-statement, 10-point probability scale worked quite well, but halving the probability of 2010 non-voters did not improve voting probabilities. -Differential turnout model worth 1% swing to Conservatives. 2,914 recall interviews by telephone on 8-14 th May with people we spoke to during the campaign: polls 1-4

Confidential: For research purposes only6 Recall poll evidence Two rights do not explain the wrong. ‘Shy Tory’ & ‘Lazy Labour’ are modest & incomplete explanations of ICM’s error.

Confidential: For research purposes only7 What’s to blame? -Initial fear that sampling error was to blame remains leading suspect. -c.30,000 randomly generated telephone numbers were dialled at lest once to generate 2,000 interviews over a weekend. -The ability to reach a representative sample by telephone is now open to the same accusation as online polls: are the interested and the willing ‘different’ to the disinterested and the unwilling? -ICM includes 33% of random mobile numbers in samples. But higher levels of phone poll volatility may indicate that the solution lies elsewhere. -Weekday interviewing may have caused us to access the ‘wrong type of respondent’. Did we reach enough working people? Are orthodox data collection techniques still capable for reaching a representative sample?