1. Questionnaire Design EPIET Introductory Course Lazareto, Menorca
6 October 2011
K Alpers
C Campese, P McKeown, V Bremer, V Prikazsky

2. What is a questionnaire?
A tool for data collection
A series of written questions in a fixed, rational order
There are other forms for data collection:
Observation
Unstructured interviews

3. A well designed questionnaire
Good appearance (easy for the eye)
Short and simple
Relevant and logical
⇒ High response
⇒ Easier to collect
to summarize
to analyse
Minimises potential sources of bias

4. Bias Systematic difference in the response measurement Recall bias
Cases more likely to remember than controls
Observer bias
Different interviewers – different interpretations
Different interpretation of similar questions
Non-response bias
telephone interviews: more females, elderly

5. How to reduce bias Structured questionnaire Ensure high response rate
Random choice of interview partners
Training of interviewers
Random choice: eg. Last birthday

6. Advantages of questionnaires
Can reach a large number of people
Relatively easy and economic
Relate directly to study question
Provide quantifiable answers
Relatively easy to analyse

7. Disadvantages of questionnaires
Provide only limited insight into a problem
the range of possible responses is limited
the question maybe misleading
Varying response
Unclear question can lead to
misunderstanding
misinterpretation
Do not allow for mistakes
must be right from the beginning
missing data hard to chase

8. Types of Questionnaires
Interviewer-administered
face to face

9. Types of Questionnaires
Interviewer-administered
face to face
telephone

10. Types of Questionnaires
Interviewer-administered
face to face
telephone
Self-administered
by post

11. Types of Questionnaires
Interviewer-administered
face to face
telephone
Self-administered
by post
Internet
Social Networks
Social Networks

12. Self-administered questionnaire
Advantages
cheap and easy to administer
preserves confidentiality
completed at respondent's convenience
not influenced by interviewer

13. Self-administered questionnaire
Advantages
cheap and easy to administer
preserves confidentiality
completed at respondent's convenience
not influenced by interviewer
Disadvantages
low response
questions can be misunderstood
no control by interviewer
only literate persons
time delay (post)
/internet: need computer access/software

14. Interviewer-administered questionnaire
Advantages
participation of illiterate people
clarification of ambiguities
quick answers

15. Interviewer-administered questionnaire
Advantages
participation of illiterate people
clarification of ambiguities
quick answers
Disadvantages
interviewer bias
needs more staff resources
only short questionnaires possible
especially on telephone
difficult for sensitive issues

16. Before starting to design a questionnaire
Write a study protocol!
define objectives
define study questions
Risk factors for being stung by jellyfish at Lazareto (Menorca), October 2011
Study protocol
Prepared by cohort 17
Menorca, 6 October 2011

17. Study protocol and questionnaire
Definition of objectives
Design questions
questions must relate to objectives
collect appropriate information for analyses
Control and reading
List of variables to collect
Pilot test
Analysis plan

18. Use existing questionnaires
As an inspiration
Don´t need to re-invent the wheel
Have been tested
BUT: adapt to present situation!

19. Questionnaires for outbreak investigations
Exploratory
Generate hypotheses
Detailed
Only some cases
Based on existing questionnaires
Open questions
Analytical study
Testing hypothesis
Focus on possible vehicles
Questionnaire needs to be adapted to the type of study

20. Questionnaire needs to be adapted to study population
Know the respondents
language
education
occupation
ethnic group
sensitive issues

21. Structure of a questionnaire
Identification
Interview introduction
Instructions on how to answer
Questions
Conclusion

22. 1 - Identification On first page On all pages
Return address
Study title
On all pages
Identifyer
Page numbers
Data protection: Identifiers kept separately from names

23. 2 - Introduction Covering letter/ interview introduction
Who are you / you work for
Why are you investigating
Where did you obtain the respondent’s name
How and where can you be contacted
Guarantee of confidentiality
Length of interview (be honest)
⇒ Usefulness of study should be clear to all respondents

24. Example of introduction
Good morning ,
My name is , I work for ……..
You may have been already informed that a survey on risk factors for being stung by a jellyfish will be done this week in Mahon. This study has been approved by the national ethical committee. Only anonymized data will be analysed. You have been randomly selected to participate in this study. Your participation is voluntary. The interview will take about 10 minutes.
I will start with the first question

25. 3 - Instructions Minimise potential sources of bias
Guide for Interviewers
Guide for Respondents in self-administered questionnaires
Which questions can be skipped => where to jump to
Selection of multiple answers possible
Use different fonts (e.g. bold or italics)
Example: I will ask you a few questions about your contacts with jellyfish.
You will answer by yes or no
If no, go to question 27

26. 4 - Questions
Content
Order
Format
Coding
Pitfalls

27. 4.1 - Content of questions Clear focus on research question
avoid sidetracking
avoid unnecessary information

28. 4.2 - Question order Decide on the order of items/questions
easy  difficult
general  particular
factual  abstract
Group questions by topic
Be aware of ordering effects
Don’t put the most important items last
Where to place sensitive questions?
Grouping:
Eg in an outbreak investigation:
Demographic information
Symptoms
Lab diagnosis
Predisposing factors
exposures

29. Starting questions “Door-opener” Simple Closed format
Relevant to main subject
Non-offending
Neither demographic nor personal questions

30. 4.3 - Format of questions Adjust to responding audience
professionals vs. public
middle class vs. prisoners
Keep sentences simple and short
Define key words (“fully vaccinated”)
Remember options
“don’t know”
“don’t want to answer”

31. Ask one thing at a time
7 - Do you like to go swimming and do you mind to be stung by jellyfish?
 Yes
 No
Ask for one information at a time
Answer options
Mutually exclusive and exhaustive
Vertical order of answer options
Otherwise you won´t know which part people are answering

32. Be accurate 8 – What is the jellyfish situation?  Good  Bad Versus
8 - How often did you see jellyfish during the last week?
 Once
 Twice
 Three times or more
 Never
Don´t know

33. Be objective 10 - Do you agree that jellyfish are annoying?  Yes  No
versus
10 - According to you, jellyfish in Lazareto are…
 not at all a public health problem
 not a major public health problem
 a potential public health problem
 an important public health problem
 no opinion/don´t know

34. Be simple
11- Did you see more than an average of 33 jellyfish/m2 salt water surface on more than 3 occasions that you went swimming in the morning last week?
 Yes
 No
versus
11- Have you seen jellyfish on more than 3 mornings last week?
 Don´t know

35. Two main question formats
Closed format  forced choice
Yes  Always 
No  Sometimes 
Don’t know  Never 
Open format  free text
12 – What did you do to avoid being stung by jellyfish? Please describe : __________________________________________________________________________________

36. Advantages of open questions
not directive
detailed and unexpected answers possible
> allow exploration of issues to generate hypotheses
useful for exploring knowledge and attitudes
qualitative research
focus groups
trawling questionnaires
Vorteile:
Nicht direktiv: Befragte werden nicht in eine Richtung gedrängt
Zur Hypothesen-Generierung: wenn mehrere Menschen ähnliche (unerwartete) Antworten haben, allgemeine Tendenz?
Abfrage von Wissen oder Einstellungen
Detaillierte, auch unerwartete Antworten möglich
Erforscht, wie Menschen über ein Problem sprechen: sprachlich
Nachteile:
Antwort Interviewer-abhängig: Interviewer schreibt nicht alles genau so hin sondern interpretiert auch hinein
Zeit- und arbeitsintensiv: Lange Antworten, Redefluss, aber auch Auswertung
Kodierprobleme
Schwierig zu analysieren: welche Einteilungen wählen?
Vergleiche zwischen Gruppen problematisch

37. Disadvantages of open questions
interviewer bias
time-consuming
coding problems
difficult to analyse
difficult to compare groups

38. Advantages of closed questions
Straightforward response
Simple
Quick
Less discrimination against less verbally expressive people
Easy to code, record, analyse
Easy to compare
Easy to report

39. Disadvantages of closed questions
restricted number of possible answers
Possible loss of additional information
Compromise
insert field
“others,
if yes specify : __________”

40. Examples for closed questions
2 - What is your age in years? ___ years
3 - How long have you stayed at Lazareto? ___ days
4 - What is your sex (gender)?
 Male
 Female
5 - Did you go swimming on 4 October 2010?
 Yes
 No
 Don’t know

41. Closed questions: Checklist
14 - Which of the following beaches have you visited during your stay in Menorca?
 Lazareto beach
Cala’n Porter
Rafalet
Macarella
Sa Mesquida

42. Checklist
14 - Which of the following beaches have you visited during your stay in Menorca?
Lazareto beach  Yes  No  Don´t know Cala’n Porter  Yes  No  Don´t know Rafalet  Yes  No  Don´t know
Macarella  Yes  No  Don´t know
Sa Mesquida  Yes  No  Don´t know

43. Rating scale
15 – How often did you see jellyfish during the past weeks?
Always Sometimes Seldom Never
Mornings    
Lunchtime    
Evenings    

44. Rating scale Numerical
16 - How severe was your pain after you were stung? (please circle)
Not painful at all Very painful
Analogue
17- How severe is your pain (put the tick on the line)

45. Likert Scale Rensis Likert, 1903-1981 Psychometric scale
Five (or more) ordered response levels
18 – Jellyfish also have the right to swim in the Mediterranean sea
 I strongly disagree
 I disagree
 I neither agree or disagree
 I agree
 I strongly agree

46. 4.4 - Coding Answers can be pre-coded Quicker and easier data entry
Examples:
Male  1 Ill  1
Female  0 Not ill  0
Don’t know  9 Don’t know  9
Lazareto beach  1
Harbour  Es Castel  3
Mahón  4

47. 4.5 - Avoid pitfalls jargon/abbreviations/slang
21- Should jellyfish victims receive a PEP?
(post exposure prophylaxis)
not mutually exclusive options
22 - What is your age ?    

48. Avoid…. Leading questions
23 - Do you think we should forbid EPIET-fellows to go swimming in the mornings?
Making the questionnaire too long
Typographical / spelling errors

49. 5 - Conclusion Don’t forget to thank the interviewed persons
Tell them when the results will be available and where

50. Example of conclusion
This is the end of the interview. Thanks for answering this questionnaire. The result of this study will be available on the ECDC website in six weeks.
Do you have any others questions you wish to ask ?
In some outbreak investigations you might want to add:
We are still in a very early phase of our investigation. Would you agree to be contacted again in case further questions arise?

51. Presentation and layout
Clear and consistent
adequate space to answer
large font size
appropriate page breaks
avoid
messy layouts
too many and fancy logos
printing questionnaire on coloured paper may help

52. Pilot your questionnaire!
Pilot with a group of people
-> similar to your target subjects
Highlight problems before starting
Misunderstandings
Look for alternative wording
Final polishing

53. Summary A well designed questionnaire: collects appropriate data
-> to answer your research question
minimises potential sources of bias
-> increases the validity of the replies
Will much more likely be completed
It is worth to spend time in the design of your questionnaire!