1. Facilitated by Elizabeth Wiredu
Questionnaire Design
Facilitated by Elizabeth Wiredu

2. Course Aim
The aim of this course is to facilitate the design of valid questionnaire to support good data collection and the subsequent analysis.

3. Learning Outcomes By the end of the course, you will be able to:
Identify the principles that will guide the design of a good questionnaire.
Learn how to put a good design together.
Examine how response types determine the choice of summaries and statistics you could generate on your data.
Critique a poorly designed questionnaire.
Create a sample good questionnaire from scratch.

4. Questionnaire What is questionnaire? When to use questionnaire?
Is validated questionnaire the solution?
Checked the literature for what others have measured before in similar studies?
Could you use a pilot interview to inform what to measure?
Using existing validated questionnaire
Using a previously validated and published questionnaire will save you time and resources;
You could use professional interviewers if necessary to collect more complex data – probe, prompt, use follow up questions, assist with writing for poor spellers.
Using standardised questionnaires, you will be able to compare your own findings with those from other studies,
You may find it easier to get published.
The Cost besides Time and Resources?
Poorly design questionnaire lead to poor quality data, sometimes ‘impossible to analyse’ data situation. No room for correction since you’ve only one opportunity with the respondent.
Lead to misleading conclusions, unreliable recommendations.
If there is no ‘off the shelf’ questionnaire available, then this lets look at how to write a bespoke questionnaire.

5. The Framework Pose the research question.
Develop and write the questionnaire, and do validity checks.
Summarise and analyse data.
Interpret and report findings.

6. Stage 1- Pose the Research Question
Funnel the research question down into specific key questions or objectives which will then be grouped as themes.
Write down the hypotheses and the conclusions you want to arrive at.
Establish the relationships, comparisons, trends, patterns or summaries you wish to make.
What data?
Work with your co-researcher to clarify this.
Do a thorough literature search for validated measures such as SF-36 questionnaire. This will increase the chance of your work to be acceptable for publication and be accepted by the ethic committee.
If you are not familiar enough with the research area and literature cannot help, use focus groups interview to find out what data to collect.
Validation
Valid questionnaire measures what it claims to measure. Refer to examples from Wording to Vague questions. Another identifier is in the areas of context. For example, An instrument developed in a different time, country, or cultural context may not be a valid measure in the group you are studying.
Reliability
Reliable questionnaires yield consistent results from repeated samples and different researchers over time. Differences in results come from differences between participants, not from inconsistencies in how the items are understood or how different observers interpret the responses. A standardised questionnaire is one that is written and administered so all participants are asked the precisely the same questions in an identical format and responses recorded in a uniform manner. Standardising a measure increases its reliability.

7. Stage 2- Develop and Write the Questions
Formulate each key question into a theme and choose which variables you must collect data on.
Decide how each variable is to be measured i.e. how to frame the question (open/closed) and its response set.
Get help from your literature reviews.

8. Stage 2- Questions Designs
Factual Information
Simple dichotomy
Simple numeric
Simple choice
Opinion and Attitude Information
Likert categorical choice
Anchor scale
Rank order
Mixed Designs
Multiple choice
Matrix design

9. Examples of Closed Questions Design
Dichotomous
Questions???
Response:
 Yes  No
 Male  Female
Simple Choice
 Exceeded expectations
 Met expectations
 Fell short of expectations
Likert Scale
 Very satisfied
 Somewhat satisfied
 Undecided
 Somewhat dissatisfied
 Very dissatisfied
Anchor Scale
1-directional design:
On a scale of 1 to 10, where 1 is virtually no pain and 10 is excruciating pain.
2-directional design:
On a scale of 1 to 5, where 1 is highly unsatisfied and 5, highly satisfied.

10. Examples of Closed Questions Design
Multiple Choice
Which of the following products do you own? (select all that applies)
 Pen drive
 Computer
 Printer
 Fax machine
 Scanner
 Digital camera
Matrices
Please rate your purchase decision based on the following characteristics
V. Important Will Consider Not Important
Reliability   
Price   
Features   
The look   
Functionality   

11. Opened Question Design Example
Open ended question…???
Do you think smoking and wound-healing are linked, and if so, how? Please write your response in the box below.

12. Stage 2- Questions Designs
Wording
Phrase your questions in positive tone
Logical Arrangement, Appearance, and Layout
Ambiguity and Imprecision
Acronyms, Jargons, Technical Language
Assumption
Memory and Knowledge
Double Questions or complex and heavy loaded questions
Leading Questions
Wording Answers before the Questions
Subjective Questions
Vague Questions

13. Reliability and Validity
Reliability. A questionnaire is considered to be reliable if the results of a study can be reproduced at different times using the same questionnaire under a similar methodology.
Use test-retest method to check reliability
Validity refers to whether the questions (questionnaire) are measuring what they are supposed to measure.
Validity checking: use pre-test
Split-half and internal consistency methods. Cronback Alpha correction usually gives a greater estimate of reliability than the split-half correlation. Nunnally’s(1978) indicated that internal consistency methods such as Cronbach’s alpha set an upper limit to a test’s reliability. Nunnally, J. C. (1978) Psychometric theory. New York; McGraw Hill. Quoted in

14. Stage 2- Sampling How much sample do you need?.
Decide on the appropriate Sampling Method(s)
Simple Random
Stratified Random
Systematic Sampling
Cluster Sampling
Quota Sampling.
Some data collection methods:
Consider the appropriate data collection method – self completion, interviewer assisted completion, postal, telephone completion, , internet hosting page.

15. Pay attention to the following during the planning stage
Have you budgeted for non-response by sending (10-40%) more to protect your specified sample size for your power calculation?
Have you carefully considered the ethics of the research, e.g. questions you should not ask? The protection of the respondents’ data?
Have you cleared access to your respondents?
Are you going to use software to summarise and analyse the responses? If so, do you know how to use the software or have you got access to support?

16. Pay attention to the following design issues:
Themes and their logical arrangement
Place a text box under each theme to ask for an overall opinion about the theme
Avoid ‘Any comment/suggestion’ question at the very end. It does not work!
Clear instructions to reduce response error
Navigation – adopt question numbering and goto jumps, etc.
Appearance, layout and sequence
Logical, inclusive/complete response sets

17. Piloting the Questionnaire
Proofreading and Editing
Pre-test piloting
Feedback prompts
Do you believe other people may have difficulty in answering this questionnaire? If so, which question in particular may pose a problem?
Given the purpose of my study outlined in the introduction section, would you suggest a question, may perhaps be added or removed to enhance the study?
How long did it take you to complete the questionnaire?
Did you read the instructions?
Were the instructions clear and neatly emphasised?
Did you object to answering any of the questions?
Was the layout of the questionnaire clear, logical and attractive?
Is there any question that the wording could be enhanced or clarify?

18. Stage 3- Summarising and Analysing the data
Summarise and organise the data
Look for common and unusual patterns, trends, differences, features in the data
Relate the data to the research question
Undertake appropriate statistical analyses of the data
Clearly communicate the relevant findings
Access the limitations of the study

19. Stage 4- Interpretation and Reporting of Findings
Relate the findings to the research question
Draw appropriate conclusions
Compare and contrast findings with those of other relevant research
Suggest future research question leading on from here.

20. Introduction to the Questionnaire
A short introduction explaining:
The purpose of your research.
How this study will benefit the respondent directly or indirectly.
State official backing or sponsorship if any.
A note of appreciation to the respondent for their contribution to your study.
Confidentiality if applicable and Data protection statement.
Contacting information if respondent wish to make a contact.
The introduction letter MUST be part of the questionnaire and also the separate covering letter.

21. Concluding Note A short concluding note containing:
A note of appreciation to the respondent for their contribution to your study
Returns address
Returns deadline

22. Frequently Asked Questions
Question: Can I use tick boxes for all the questions?
Answer: It depends on your intended statistical analysis. E.g. if you intend to do a comparative testing (Ttest/ANOVA) you will need the dependent variable to be a continuous data (scale or ordinal measure) and the factor variable, categorical data (choice response).
Question: After the data is collected, can it be manipulated to allow certain statistics?
Answer: It is possible to regroup continuous variables into categorical, or even reduce categories of a categorical variable to take advantage of certain statistics. However, it is not possible to convert categorical variable to continuous variable.

23. What Next? - Practical Session
Evaluate a sample questionnaire
Group Feedback and Discussion of the sample questionnaire
Create a questionnaire from scratch
Examine the design of your questions and the summaries or statistical analysis opportunities available.

24. How can I help you? We provide: Questionnaire design feedback.
A guide to summary statistics and inferential statistical analysis.
Assistance with reporting the statistics for publication.