The Performance of The Polls John Curtice

Why Polls Matter Can affect the extent and nature of media coverage Can affect the ability of campaigns to raise money and motivate activists Can influence the campaigns strategy and tactics Can influence whether people vote

The Challenge Regular monthly political polling in Scotland ended in Partly thanks to perceived poor performance, and partly the result of worsening newspaper finances. Thereafter mostly episodic and concentrated at election times. So companies having to estimate the attitudes of a population most have not been regularly monitoring. And for a vote intention that they have not previously attempted to estimate. While the industry is still coming to terms with the internet

The Performance - 1

The Performance - 2

A Time Trend? Based on 9 polls conducted Feb-May, 10 polls July-Sept, 8 polls Oct-Dec, and 7 polls in Jan-mid Feb and 10 polls mid-Feb- Mar. Dont Knows excluded.

House Differences Based on all polls since Sept. Dont Knows excluded

Panelbases Record

Ipsos MORIs Record

The Dont Knows Source: 1 st 2 cols: Average of all polls since September

Methodological Differences CompanyMethodPolitical Weighting/Selectio n Question PanelbaseInternet2011 Holyrood Vote; 8/10 likely to vote Was current vote; now intend to vote Sept ICMInternet2011 Holyrood VoteThink will vote Sept SurvationInternet2011 (1 st poll: 2010) & likelihood of voting Vote now TNS BMRBFace to face quota2011, incl abstention rate Intend to vote Sept YouGovInternet (18+ only)2011, + extra Lab 10, SNP 11 group Vote now Ipsos MORITelephone (RDD)None. Figures based on certain to vote Vote now

Record Panelbase – no previous polling close to election day ICM – previously all election polling done by phone Survation – new company since 2010 TNS BMRB – still using System Threes (not always successful) approach YouGov – over 10 years experience. Panellists previous vote collected on joining/after election Ipsos MORI – using same approach as for GB polls since 2008

How Their Last Poll Performed In 2011 % SNP ConstError% SNP ListError TNS BMRB YouGov Ipsos MORI* * Last Poll Conducted

The Extent and Impact of Weighting CompanyMen16-34C2DEImpact on % Yes Panelbase ICM Survation TNS BMRB YouGov * Ipsos MORI * Based on 16-39

Reported Turnout Source: Ipsos MORI (Scottish & British Polls)

Polls of The Interested? Survation & YouGov figures may include a few Other party voters

Tentative Conclusions Particularly large house effects on % Yes vote Do not simply correspond to internet vs non- internet But do correspond to experienced vs. less so Internet samples more heavily weighted But weighting etc has pushed all recent polls in a pro-Yes direction Some polls (at least) look set to be wrong – but which ones?