1 Plus ca change: barriers to innovation in survey research Patrick Sturgis, University of Southampton and National Centre for Research Methods (NCRM) Paper presented at Government Statistical Service Conference, London, July

2 NCRM Network of >10 universities offering an integrated programme of methodological research and training Quantitative, qualitative and mixed methods Online archive of papers, videos, presentations, toolkits, etc. Bursary scheme to support training attendance Find out more at

3 Key changes in survey practice Telephone interviewing RDD CAPI Electronic sample management Web surveys Mobile digital devices Attitude measurement?

4 5-point Likert agree/disagree format Probably the most widely used attitude question format in the UK and throughout the world BSA, ESS, BHPS, BCS, Taking Part, cohort studies, WVS, EVS, etc. etc…. But it has long been recognised that there are some major problems with this approach

5 ‘Working in government statistics is enjoyable’ Strongly agree/agree/neither-nor/disagree/strongly disagree

6 But there is another problem…

7 Neither/nor mid-points Perhaps the most common form of middle alternative is the ‘neither/nor’ in Likert items Usually treated analytically as implying ‘neutrality’, or being ‘in the middle’ on the issue But it is also logically consistent with having no opinion If I do not have an opinion on an issue, it is true that I am neither in favour nor opposed to the statement

8 Research Design Face-to-face CAPI interviews on the Ipsos- MORI omnibus quota survey May 2008 n=3113 Respondents randomly allocated to one of three 5-point uni-polar Likert items

9 Overall, how satisfied or dissatisfied are you with the performance of the European Commission? 1. Very satisfied 2. Fairly satisfied 3. Neither satisfied nor dissatisfied 4. Fairly dissatisfied 5. Very dissatisfied

10 How important or unimportant do you think it is it for Britain to be at the forefront of developments in nanotechnology? 1. Very important 2. Fairly important 3. Neither important nor unimportant 4. Not very important 5. Not at all important

11 On balance, the advantages of GM foods outweigh any dangers Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree

12 For those selecting ‘neither/nor’: Which of the following statements best describes why you are neither satisfied nor dissatisfied with the performance of the European Commission? 1.I don’t really have an opinion on this issue 2.I have an opinion which is right in the middle on this issue 3.Neither of the above If (3) Could you tell me what led you to say you are neither satisfied nor dissatisfied? –PROBE FULLY. For what other reasons? –RECORD VERBATIM

13 Responses can now be categorised as either: 1.Substantive (directional) response 2.Initial Don’t Know 3.Hidden Don’t Know 4.Neutral

14 Table 1 Distribution of middle alternative responses after probing

15 Reference category = initial DK

16 Why are survey question formats so resistant to change?

17 “However, you will have to balance this against the policy needs of your funders; changing the question and/or response categories will mean that the data you collect will not be comparable to other surveys against which you may be benchmarking your data”.

18 Some predictions for 2020 General population surveys will routinely be conducted using web data collection Interviewing conducted by avatars 5-point agree/disagree questions with neither/nor mid-point still widely used.

19 Some previous predicitons Frank Dobson for Mayor of London West Ham premier league champions 2010