1. 1 Questionnaire Design K Alpers, C Campese, P McKeown, V Bremer, V Prikazsky EPIET Introductory Course 2010 Lazareto, Menorca

2. 2 What is a questionnaire? A tool to –collect answers to questions –collect factual data A series of written questions in a fixed, rational order

3. 3 A well designed questionnaire Gives accurate and relevant information to your research question Minimises potential sources of bias As simple and focused as possible

4. 4 Bias Systematic difference in the response measurement Recall bias –Cases more likely to remember than controls Observer bias –Different interviewers – different interpretations –Different interpretation of similar questions Non-response bias –telephone interviews: more females, elderly

5. 5 How to reduce bias Structured questionnaire Ensure high response rate Random choice of interview partners (next birthday) Training of interviewers

6. 6 Advantages of questionnaires Can reach a large number of people Relatively easy and economic Relate directly to study question Provide quantifiable answers Relatively easy to analyse

7. 7 Disadvantages of questionnaires Provide only limited insight into a problem –the range of possible responses is limited –the question maybe misleading Varying response –Unclear question can lead to misunderstanding misinterpretation Do not allow for mistakes –must be right from the beginning –missing data hard to chase

8. 8 Types of Questionnaires Interviewer-administered –face to face

9. 9 Types of Questionnaires Interviewer-administered –face to face –telephone

10. 10 Types of Questionnaires Interviewer-administered –face to face –telephone Self-administered –by post

11. 11 Types of Questionnaires Interviewer-administered –face to face –telephone Self-administered –by post –email/Internet

12. 12 Self-administered questionnaire Advantages –cheap and easy to administer –preserves confidentiality –completed at respondent's convenience –not influenced by interviewer

13. 13 Self-administered questionnaire Advantages –cheap and easy to administer –preserves confidentiality –completed at respondent's convenience –not influenced by interviewer Disadvantages –low response –questions can be misunderstood –no control by interviewer –only literate persons –time delay (post) –e-mail/internet: need computer access/software

14. 14 Interviewer-administered questionnaire Advantages –participation of illiterate people –clarification of ambiguities –quick answers

15. 15 Interviewer-administered questionnaire Advantages –participation of illiterate people –clarification of ambiguities –quick answers Disadvantages –interviewer bias –needs more staff resources –only short questionnaires possible especially on telephone –difficult for sensitive issues

16. 16 Before starting to design a questionnaire Write a study protocol! define objectives define study questions Risk factors for being stung by jellyfish at Lazareto (Menorca), October 2010 Study protocol Prepared by cohort 16 Menorca, 7 October 2010

17. 17 Study protocol and questionnaire Definition of objectives List of variables to collect Analysis plan Study protocol Design questions Control and reading Pilot test questionnaire The question must answer your objectives The question collects appropriate information for analyses

18. 18 Use existing questionnaires As an inspiration Don´t need to re-invent the wheel Have been tested BUT: adapt to present situation!

19. 19 Questionnaire needs to be adapted to study population Know the respondents –occupation –sensitive issues –education –ethnic group –language

20. 20 What makes a well designed questionnaire? Good appearance (easy for the eye) Short and simple Relevant and logical ⇒ High response ⇒ Easier to collect to summarize to analyse

21. 21 Basic Rules On first page –Return address –Study title On all pages –Identifyer –Page numbers Numbered questions Instructions (in bold or italic)

22. 22 Structure of a questionnaire 1Interview introduction 2Identification 3Instructions on how to answer 4Questions 5Conclusion

23. 23 1 - Introduction Covering letter/ interview introduction –Who are you / you work for –Why are you investigating –Where did you obtain the respondent’s name –How and where can you be contacted –Guarantee of confidentiality –Length of interview (be honest) ⇒ Usefulness of study should be clear to all respondents

24. 24 Example of introduction Good morning, My name is....., I work for …….. You may have been already informed that a survey on risk factors for being stung by a jellyfish will be done this week in Mahon. This study has been approved by the national ethical committee. Only anonymous data will be analysed. You have been randomly selected to participate in this study. Your participation is volontary. The interview is about 10 minutes long. I will start with the first question.....

25. 25 2 - Identification Each questionnaire/interviewed person must be identified –Example :I_I I_I_I I_I_I interviewer N° district N° person N° Demographic data : age, sex … Identifiers kept separately from names (anonymous)

26. 26 3 - Instructions Guide for Interviewer/Respondent Give clear instructions about how to answer the questions –Use different fonts (e.g. italics) Give instruction which questions can be skipped -> where to jump to Example: I will ask you a few questions about your relationship with jellyfish. You will answer by yes or no If no, go to question 27

27. 27 4 - Questions Content Order Format Problems and pitfalls Coding

28. 28 Content of questions Clear focus on research question –avoid sidetracking –avoid unnecessary information

29. 29 Question order Decide on the order of items/questions –easy  difficult –general  particular –factual  abstract Group questions by topic Be aware of ordering effects Don’t put the most important items last Where to place sensitive questions?

30. 30 Starting questions –“Door-opener” –Simple –Closed format –Relevant to main subject –Non-offending –Neither demographic nor personal questions

31. 31 Format of questions Adjust to responding audience –professionals vs. public –middle class vs. prisoners Keep sentences simple and short Define key words (“fully vaccinated”) Remember options –“don’t know” –“don’t want to answer”

32. 32 Format of questions 7 - Do you like to go swimming and do you mind to be stung by jellyfish? Yes No Ask for one information at a time Answer options – Mutually exclusive and exhaustive – Vertical order of answer options

33. 33 Be accurate 8 – What is the jellyfish situation? Good Bad Versus 8 - How often did you see jellyfish during the last week? Once Twice Three times or more Never Don´t know

34. 34 Be appropriate 9 – Are jellyfish a useless nuisance? Yes No versus 9 - How often have you been stung by jellyfish? Once a day or more Once a week Less than once a week Don´t know

35. 35 Be objective 10 - Do you agree that jellyfish are annoying? Yes No versus 10 - According to you, jellyfish in Lazareto are… not at all a public health problem not a major public health problem a potential public health problem an important public health problem no opinion/don´t know

36. 36 Be simple 11- Did you see more than an average of 33 jellyfish/m 2 salt water surface on more than 3 occasions that you went swimming in the morning last week? Yes No versus 11- Have you seen jellyfish on more than 3 mornings last week? Yes No Don´t know

37. 37 Two main question formats Closed format  forced choice Yes Always No Sometimes Don’t know Never Open format  free text 12 – What did you do to avoid being stung by jellyfish? Please describe : __________________________________________ ________________________________________

38. 38 Advantages of open questions not directive detailed and unexpected answers possible –> allow exploration of issues to generate hypotheses useful for exploring knowledge and attitudes qualitative research focus groups trawling questionnaires

39. 39 Disadvantages of open questions interviewer bias time-consuming coding problems difficult to analyse difficult to compare groups

40. 40 Advantages of closed questions Straightforward response Simple Quick Less discrimination against less verbally expressive people Easy to code, record, analyse Easy to compare Easy to report

41. 41 Disadvantages of closed questions restricted number of possible answers Possible loss of additional information Compromise –insert field “others, if yes specify : __________”

42. 42 Examples for closed questions 2 - What is your age in years? ___ years 3 - How long have you stayed at Lazareto? ___ days 4 - What is your sex (gender)? Male Female 5 - Did you go swimming on 4 October 2010? Yes No Don’t know

43. 43 14 - Which of the following beaches have you visited during your stay in Menorca? Lazareto beach Cala’n Porter Rafalet Macarella SaMesquida Closed questions: Checklist

44. 44 14 - Which of the following beaches have you visited during your stay in Menorca? Lazareto beach Yes No Don´t know Cala’n Porter Yes No Don´t know Rafalet Yes No Don´t know Macarella Yes No Don´t know Sa Mesquida Yes No Don´t know Checklist

45. 45 Rating scale 15 – How often did you see jellyfish during the past weeks? Always Sometimes Seldom Never Mornings Lunchtime Evenings

46. 46 Rating scale Numerical 16 - How severe was your pain after you were stung? (please circle) 1 2 3 4 5 6 7 Not painful at all Very painful Analogue 17- How severe is your pain (put the tick on the line) 010

47. 47 Likert Scale Rensis Likert, 1903-1981 Psychometric scale Five (or more) ordered response levels 18 – Jellyfish also have the right to swim in the Mediterranean sea I strongly disagree I disagree I neither agree or disagree I agree I strongly agree

48. 48 Problems and pitfalls Avoid questions that ask two things at once - you won’t know which part people are answering: 19 - Have you seen or been stung by jellyfish? Ambiguity..... 20 - Do you swim a lot?

49. 49 Problems and pitfalls Avoid jargon/abbreviations/slang 21- Should jellyfish victims receive a PEP? (post exposure prophylaxis) Avoid not mutually exclusive options 22 - What is your age ? 16-20 20-25 25-30 35-40

50. 50 Problems and pitfalls Avoid leading questions 23 - Do you think we should forbid EPIET-fellows to go swimming in the mornings? Avoid making the questionnaire too long Typographical / spelling errors

51. 51 Coding Answers can be pre-coded Quicker and easier data entry Examples: Male 1 Ill 1 Female 0 Not ill 0 Don’t know 9 Lazareto beach 1 Harbour 2 Es Castel 3 Mahón 4

52. 52 5 - Conclusion Don’t forget to thank the interviewed persons Tell them when the results will be available and where

53. 53 Example of conclusion This is the end of the interview. Thanks for answering this questionnaire. The result of this study will be available on the website epiet.org in six months. Any others questions you wish to ask ? We are still in a very early phase of our investigation. Would you agree to be contacted again in case further questions arise? In some outbreak investigations you might want to add:

54. 54 Questionnaire validation Use or adapt existing questionnaires –validated ( and possibly harmonised ) New questionnaires –not validated –need to be tested (pilot)

55. 55 Pilot your questionnaire! Pilot with a group of people -> similar to your target subjects Highlight problems before starting –Misunderstandings –Look for alternative wording –Final polishing

56. 56 Stages in designing questionnaire Definition of objectives List of variables to collect Analysis plan Study protocol Design questions Control and reading Pilot test Questionnaire The question must answer your objectives The question collects appropriate information for analyses

57. 57 Presentation and layout Clear and consistent adequate space to answer – large font size – appropriate page breaks avoid messy layouts too many and fancy logos printing questionnaire on coloured paper may help

58. 58 Questionnaires for outbreak investigations Exploratory –Generate hypotheses –Detailed –Only some cases –Based on existing questionnaires –Open questions Analytical study –Testing hypothesis –Focus on possible vehicles

59. 59 Summary A well designed questionnaire: collects appropriate data -> to answer your research question minimises potential sources of bias -> increases the validity of the replies Will much more likely be completed