Polling the Leadership Debates The role of the instant polls Andrew Hawkins & Caroline Lawes 22 nd November 2010

Overview The players Methodology & sampling Challenges Results Impact Lessons learnt

The players Angus Reid ComRes/ ITV News ICM/ Guardian Populus/ The Times YouGov/ The Sun

The methodologies Across the different companies, three methodologies were used and three types of weightings. Automated telephone Online Telephone Nationally representative weighting Viewer profile weighting No weighting

Telephone ComRes: Automated Telephone Pre-recruited panel of 10,000 people, demographics collected in advance Programmed key touch survey using James Mates’s voice Quotaed sample to send out Automatic sample weighting – nationally representative Sample size of 4032, 2691, 2372 Six minutes fieldwork time ICM: CATI methodology Random sample of omnibus for recruitment. Recontacted sample of people who said they would be watching. Sample of 500 on the evening after the debate. Weighted to the profile of all people selected at random who previously stated they would be watching the debate.

Online Populus: Online Recruited through a nat rep online panel. Reminders the day before and again on the morning of the debate. Data were not weighted, but was sub sample of nat rep base. Sample of 1000 YouGov: Online Sample of people who said they would definitely or probably watch the debate, from large screener Weighted to be representative of debate viewers. Reminder s before the debates Sample of 1000 Angus Reid: Online Sample: panelists from Springboard UK panel who watched debate Fieldwork on evening and next day Weighted to be nationally representative Sample of

Audience profile Audience numbers% Share ITV Debate9,679, % Sky Debate2,212,0008.5% BBC Debate7,428, % As a UK first, a key question was who will watch the debates? Actual viewing figures: Profile % GenderSocial GroupAge MaleFemaleABC1C2DE ITV Sky BBC

Weighting In hindsight we know what the audience looked like, but each instant poll had to make a choice of whether to and how to weight. Debate 1 MaleFemale ABC1C2DE Viewing Figures45.7%54.3%19.6%16.6%16.1%16.3%31.3%60.5%39.5% ICM Viewer weighting48.9%51.1%20.2%54.6%25.2%60.0%40.0% Nationally Representative48.0%52.0%27.7%19.8%16.8%14.9%20.8%55.0%45.0% Debate 2 MaleFemale ABC1C2DE Viewing Figures51.9%48.1%29.6%18.6%19.8%15.3%16.8%73.9%26.1% ICM Viewer weighting49.0%51.0%20.2%54.6%25.2%60.0%40.0% Nationally Representative48.0%52.0%27.7%19.8%16.8%14.9%20.8%55.0%45.0% Debate 3 MaleFemale ABC1C2DE Viewing Figures47.5%52.5%24.2%15.3%16.5%15.3%28.8%65.5%35.5% ICM Viewer weighting49.0%51.0%20.2%54.6%25.2%60.0%40.0% Nationally Representative48.0%52.0%27.7%19.8%16.8%14.9%20.8%55.0%45.0%

Impact of weights Following the release of the viewing figures we reweighted the data to the actual viewer profile and the differences are given below. Who do you think won tonight’s debate? Debate 1 Nat rep weightingViewer profile weighting Cameron26%27% Clegg43% Brown20%19% Debate 2 Nat rep weightingViewer profile weighting Cameron30%32% Clegg33%32% Brown30% Debate 3 Nat rep weightingViewer profile weighting Cameron35%38% Clegg33%30% Brown26%27%

The Worm 20 participants across the two marginal constituencies of Bolton West and Bolton North East Respondents were definitely going to vote, but were undecided who for. There was a good spread of men and women, different ages and ethnic groups and different past voting intentions. Participants use an electronic keypad where they could let us know when they liked something, when they didn’t like something, or when they were neutral to what was being said.

Challenges Sampling: What is the best way to recruit, how can we be sure people are planning to watch debates, how should invitations be quotaed? Target weights: The viewer profile before the debates was unknown so how is the decision made to weight the results, or is the sample self selecting? Speed: Allowing for the over-run of the programme, yet not wanting viewers to be influenced by commentary

Results Debate 1 Angus ReidComResICMPopulusYouGov Cameron20%26%20%23%29% Clegg49%43%51%61%51% Brown19%20%19%17%19% None/ DK13%11% -2% Debate 2 Angus ReidComResICMPopulusYouGov Cameron32%30%29%37%36% Clegg33% 36%32% Brown23%30%29%27%29% None/ DK12%7%8%-3% Debate 3 Angus ReidComResICMPopulusYouGov Cameron36%35% 38%41% Clegg30%33%27%38%32% Brown23%26%29%25% None/ DK11%6%9%-2%

Additional questions After first debate, Conservatives, closely followed by Lib Dems, had best policies on immigration and this topic became increasingly important during the campaign. In debate two, expectations had been set. Indeed 47% of viewers thought that Brown performed better than expected in this debate, compared to just 21% of people for Clegg. 43% of viewers thought Clegg gave the most honest answers – compared to 29% for Cameron and 23% for Brown. After debate three, viewers were still split on who would be best to make spending cuts with about a third of people choosing each party. Source: ComRes

Changing opinion Pre debates Post first debate Post second Debate Post third debate David Cameron47%35%37% Gordon Brown30%22%26%25% Nick Clegg23%43%37%38% Source: YouGov Debate 1Debate 2Debate 3 Gordon Brown48%30%29% David Cameron44%45%49% Nick Clegg4%21%19% Don’t know4% 3% Source: Populus Most evasive and least willing to give straight answers Most like to see as PM

The impact Debates became the defining feature of the campaign Liberal Democrat surge Financial markets quivering at prospect of hung parliament

Impact on the polls Average daily voting intention figures from all polls released

Financial markets The prospect of a hung parliament caused the value of the pound to drop. 16 th April intra-day figures show the impact of the instant polls from debate one. GB Pound v. US Dollar on 16 th April 2010

Lessons Learnt Sampling: Measuring what viewer profile may look like before the event. Weighting: What is the most accurate weighting and how can the national impact be modelled? Timing: Impact of polling too early or too late Interpretation: Ensuring the results are not misinterpreted when published in the heat of an election campaign The leadership debates are likely to be a feature of all future General Elections and the polls have a vital role to play

For further information please contact: Caroline Lawes ComRes, Four Millbank, London SW1P 3JA