British Election Study Harold Clarke, David Sanders, Marianne Stewart, Paul Whiteley

Face to Face Probability Surveys - The Gold Standard? Probability Samples Yield Lists Not Achieved Samples Non-Response is a growing problem in all probability surveys (61 percent response rate for 2005 BES) respondents are unlikely to be a random subset of the list – ‘top-ups’ do not help They are very expensive (The American National Election Study in 2004 cost $5500 per voter interviewed) They are slow moving – two to three months to get the data

Face to Face Probability Surveys - Additional Issues There is a limited ability to do panel surveys – when these are required for causal inference They have limited ability to do experiments when field experiments are increasingly being utilised in social science research The costs mean that we get one study only – replication is problematic The have limited or no ability to study voters in the inter- election period where a lot of the interesting action takes place They can’t be used to study election campaigns when these are becoming more important for understanding election outcomes.

Is the Internet an Alternative? Fast responses to internet surveys – typically 95 per cent within a couple of days. This is ideal for campaign studies focusing on short-term dynamics They are very cost effective – in the BES significant savings are obtained in terms of the costs per interview Large N’s are possible – the marginal cost of adding a respondent is small Experiments are possible with large treatment groups No interviewer effects – which can be a problem when interviewers are socially homogenous BUT – they are not random samples. In the BES they are a type of quota sample

BES 2005 CORE FACE-TO-FACE PANEL SURVEY: Wave 1 Pre-election Probability Sample, Face-to-Face N= Primary Sampling Units Wave 2 Post-election Probability Sample, Face-to-Face N=4161 Including top-up, mail-back; 128 Primary Sampling Units Wave 1 Pre-campaign Baseline Survey N=7793 BES 2005 INTERNET CAMPAIGN PANEL SURVEY: Wave 2 Campaign survey 275 interviews per day for 30 days N=6068 Wave 3 Post-election Interview N=5910 Wave 4 One Year Out Interview N=6186 Waves 5, 6, 7 Annual interviews through to 2010

Comparisons of Survey Modes Election Studies are unique in allowing us to compare survey data with actual votes. Two objective measures are highly visible – turnout and vote choice If we compare in-person probability surveys with internet surveys what do we see?

Turnouts in the 2001 and 2005 BES surveys (RDD in 2001; Internet in 2005)

All Types of Survey Over-Report Turnout

How Turnout influences response rates

Vote Choice in the 2005 BES Surveys

What About Modelling the Vote? This involves a variety of different models reflecting theoretical debates in the literature One set of variables relates to social backgrounds of respondents – principally their social class Spatial Issues – issues over which the voters and parties disagree (eg Taxation and spending) Valence Issues – issues over which there is widespread agreement (eg reducing crime) but differences arise over which party can do the best job. Partisanship – the ‘Brand Loyalty’ of the parties Leadership Evaluations – Who do you like and dislike? Campaign effects

Logistic Regressions Panel A Panel B Liberal Other Predictor Variables Labour Conservative Democrat Party Age -.01**.02**.01*.02* Ethnicity -.97*** ** 2.89** Gender ** Social Class -.38*.85*** Party Identification: Conservative -1.09*** 1.56*** Labour.91*** -1.72*** -.88*** -.33 Liberal Democrat -1.51*** ***.89* Other Party -.98** *** Party Leader Affect: Blair.41*** -.50*** -.41*** -.38*** Howard -.13***.58*** * Kennedy -.31***.04.49*** -.01 Party Best on Most Important Issue: Conservative -.91*** 1.42***.08.79* Labour.80*** -.84** -.72*** -.80* Liberal Democrat -.66* -1.21*.84** -.89 Other Party * Party-Issue Proximities: Conservative -.09***.20***.07**.15*** Labour.15*** -.14*** -.12*** -.19*** Liberal Democrat -.12** ***.10 Economic Evaluations *** Party Best on Economy.94*** -1.28*** -.67*** -.42 Iraq Evaluations Emotional Reactions: Economy Iraq * NHS.13* -.24** ** Tactical Voting -.34x *.50 Constant 2.53* *** -5.73** McFadden R 2 = McKelvey R 2 = % Correctly Classified = Lambda =.68.70

Factors Affecting the Probability of Voting Labour in the 2005 Election

The Performance of Rival Labour Voting Models (Logistic Regressions) with In-Person and Internet Surveys McFadden R2 McKelvey R2 AIC BIC A. Models Estimated Using In-Person Survey Data Social Class All Demographics Economic Evaluations Issue Proximities Most Important Issue Party Identification Leader Images Composite Model B. Models Estimated Using Internet Survey Data Social Class All Demographics Economic Evaluations Issue Proximities Most Important Issue Party Identification Leader Images Composite Model

Does Mode Make a Difference? Using the Parameters of the In-Person Model to Predict Labour Voting in the Internet Model in 2005 (and Vice Versa)

Conclusions There are differences in frequency distributions of variables measured with in-person and internet surveys Sometimes the in-person version is more accurate (eg turnout) sometimes not (eg Labour vote choice) The coefficients of models of turnout and party choice are indistinguishable from each other when models are estimated with in-person and internet surveys

Internet Experiment – ‘Feedback to Respondents ’ Key Issue – Are Voter Preferences Exogenous? If they are then survey respondents should not change their preferences simply because others differ eg. Parties and leaders If they are not then voters are likely to change their preferences if they discover that others differ

Feedback to Respondents The BES campaign panel survey, N=7793 Respondents place themselves on proximity scales ‘taxation versus spending’ and ‘crime reduction versus rights of the accused’ For example: __________________________________ Raise taxes and spend more Cut taxes and spend less

Examples of FeedbackTreatments Treatment 1: Control - Respondent’s position only Treatment 2: Respondent and ‘average voter’ Treatment 3: Respondent and ‘party supporters’ Treatment 4: Respondent and ‘party leaders’ Treatment 5: Respondent and ‘party leaders with party labels’

Percentages Wanting to Move

Absolute Distances Moved on the Tax and Spend Scale

Conclusions Respondents change their preferences on issues if given feedback Cues relating to political parties are particularly likely to move them If preferences are not exogenous then spatial models of party competition are wrong Much of neo-classical economic theory is wrong too!

Summary – Advantages of Internet Surveys Huge N’s Quick & Flexible– not tied to slow-moving in-person data collection Clean administration of surveys – avoiding priming and interviewer effects Highly cost-effective Hi-tech experimentation possible (again cost effective) Panels – study dynamics in multi-wave panels easily