Analysis of questionnaire 2007 Census Test Analysis of questionnaire Elizabeth McLaren Questionnaire Design & Content, 2011 Census

Overview Design of 2007 Census Test Income findings Other questionnaire findings Questionnaire timetable Questions

Design of 2007 Census Test 50% questionnaires with income questions 50% questionnaires without income questions Income at end of individual questions Included extra page of individual questions Some established 2001 topics dropped to allow new questions to be tested No relationship matrix Skewed to harder to enumerate areas

Income – setting the scene ONS recommendation on including income in 2011 based on: Evaluation of the 2007 Test; AND User requirements and relative priority of income in relation to other topics Inclusion of topics in Census also depends on: Alternative sources for information Quality of data International recommendations etc

2007 Test sources of income question

2007 Test income question

Plan for evaluating income question Primary questions: Does including income result in drop in response? What is impact on quality of response? What are cost differences eg requests for i forms? What are views of public? Supplementary questions: More people missed from households? Supporting evidence from other surveys, GROS Test, NISRA Test, 1999 Rehearsal, 1997 Test?

Assessment criteria Public acceptability and understanding Item non-response to income question Item non-response to other questions Impact on household coverage Quality of data provided Extent of requests for individual questionnaires Feedback from field staff Impact on volume of calls to helpline

Data sources Questionnaire data Census Test Evaluation Survey Field staff debriefing Helpline log

Response rates 53.3% response rate for questionnaires without income 50.6% response rate for questionnaires with income 2.7% significant difference Same trend across ETCs Note: Estimated £2 million cost for follow-up per 1% drop in response rate

Response rates for households by ETC

Findings from Test Evaluation Survey 16% refused to answer income questions 15% unhappy about answering 1 or more questions 58% unhappy about income level 48% unhappy about income sources 9% unhappy about ethnicity 2% unhappy about qualifications Comments from respondents “should be left to Revenue, nothing to do with ONS” “invasion of privacy” “irrelevant to concept of Census” “intrusive” 67% retest reliability for income level 78% retest reliability for sources of income

Other findings Non-response to income (9%) high in comparison to other topics Non-response higher for unemployed, ethnic minorities, less educated No significant adverse public reaction notified through contact centre or field staff No significant difference in requests for individual questionnaires More ‘no income’ individual questionnaires (55%) returned than ‘income’ individual questionnaires (42%) No significant difference in household undercoverage

2007 Census Test – key constraints Key constraints affecting income evaluation: Voluntary relied on public’s good will to complete a return 46.1% response rate Publicity and media coverage limited

Questions?

Second residences in 2007 Test 4% of respondents said they had a second residence A further 1% of respondents said they had a second residence outside the UK Of those who said they had a second residence: 94% entered the address 75.5% of those entered the full postcode 7.5% of postcodes were half completed 17% of postcodes were left blank

Other questionnaire findings No evidence of drop off in response due to a fourth page of individual questions High non-response (15%) to limiting long-standing illness/disability High non-response (10%) to student status question Inconsistent responses to language matrix question Confusion between usual residents and visitors Over-reporting of civil partnerships Non-response to month/year of entry 6% Year of arrival only missing 0.02% Month only missing 7%

Questionnaire timetable Cognitive question testingApril – June 2008 Omnibus Survey question testingMay Test Income Evaluation reportMay 2008 Postal TestJune Test Questionnaire Evaluation reportJune 2008 Questionnaires for 2009 Rehearsal agreedAutumn 2008 Questionnaires for 2011 Census agreedAutumn 2009 Census questionnaires approved by ParliamentSpring 2010 Census daySpring 2011

Questions?