Question and Questionnaire Design Alex Stannard
Covering letter Personalise if possible Explain purpose of questionnaire Include named contact What will happen to responses Confidentiality Publication Decision making What is in it for respondent Length Personalise – can increase response rates by 5% – 11% Decision making – telling people their views count Stamped addressed envelope Be honest about length – do not say 5 minutes when it will take 15 minutes Consider a pre-letter before sending questionnaire – very brief but says something important will arrive shortly Send thank you letters Send replacement questionnaires to non-respondents ‘we’ve not heard from you yet’ Final letter Incentives – small token incentives get better response rates Make sure everything will come out of envelop at once
Covering letter Dear Mrs Smith I am writing to ask you to take part in our Annual People Survey. The survey will only take 5 minutes and will allow us to provide better public services. Your answers are completely confidential.
Covering letter If you have questions or need help completing the questionnaire then please contact David Smith on 0111 111 111 (Monday to Friday, 9am to 5pm). Please return your completed questionnaire, in the free post envelop, by 31 May 2009. Thank you very much for your time. Yours sincerely
Questionnaire Clear instructions Sensibly ordered Font size Length Instructions – will give better results Order – will give better results, keep respondents engaged Font size – consider people with poor eyesight Length - mail questionnaire no more than 12 pages - face to face 30 to 60 minutes - web less than 15 minutes - telephone 15 to 30 minutes
Questionnaire Instructions at start Same font throughout Put age and date of birth together Spacing Have consistent write-in boxes for answers
Questions Easy to understand Completely unambiguous Are you able to communicate effectively in the local language of the Hellenic Republic? Can you speak Greek? Completely unambiguous If there is any chance a question can be misinterpreted, then it will be misinterpreted Consider literacy 2007 Scottish Parliament elections
Questions Leading questions What is the biggest issue affecting your local area? Crime for example. What is the biggest issue affecting your local area? How satisfied are you with the service? Are you satisfied or dissatisfied with the service? Nearly everyone recycles, do you recycle? Do you recycle?
Questions Where were you born? In Bishop Auckland General Hospital In County Durham In England In Great Britain In the United Kingdom
Questions What do we want to know? In which hospital? In which county? In which country?
Questions Surveys Try and keep questions as closed as possible How do you think the economy of Scotland as a whole will change over the next twelve months? It will get worse It will get better Don’t care I’ve just got a promotion so for me it will get better I hope it gets better Might improve, might get worse, it depends
Questions How do you think the economy of Scotland as a whole will change over the next twelve months? Get better Get worse Stay the same Don’t know No opinion Refused Remember don’t know, refused to answer and no opinion are different responses Order of options important – ethnicity question, mixed option
Questions Focus groups – open questions better Do you like your neighbourhood Yes, no, don’t mind What do you like about your neighbourhood and why? What do you dislike about your neighbourhood and why?
Testing Always test questionnaire and questions Review and redesign Test again
Good practice Cover letter Clear purpose Coherent, sensibly ordered and easy to follow Clear font Well spaced Not too long ‘Good’ questions
Harmonised Questions Scottish Government working on a set of harmonised questions Consultation Cognitive testing Pilot testing http://www.scotland.gov.uk/Topics/Statistics/About/SurveyHarm
Harmonised Questions Benefits Time and financial savings Allows comparisons to other surveys Minimises risk of offence Minimises risk of misunderstanding and poor data