1. An Overview of the Survey Research Process with a focus on questionnaire design Michael Dickson, BP, PhD Adjunct Professor Presbyterian College School of Pharmacy

2. Objectives Brief overview of the survey research process. Describe the process of getting to the survey document. – Literature review. – Protocol writing. Review key topics for questionnaire design. Present examples of validated health care survey instruments. Give an example of a recent survey research project (Professor Happe). 2

3. Overview of the Research Process Submit Abstract Research Question Literature Review Generate Hypotheses Select Study Design Select Data Source Data Collection Dissemination Analysis 3

4. Research Question Does the question have significance to your practice area? Is it a topic currently of interest (timeliness)? Can you address the question with available resources (feasibility)? Select a question that is significant, timely, and feasible because your time is limited. – Therefore invest some time in selecting the question. 4

5. Literature Review Answers to the questions about significance, and timeliness are generally found in the literature. – What have others done, when did they do it, and how did they do it? – Most literature searches are now done on readily available on-line databases. But, also consider non-indexed sources (e.g., meeting posters, dissertations, etc.). Failure to understand what is known and what is not known will waste your time and resources. 5

6. Generate Hypotheses Translate your research question into testable hypotheses. – The hypotheses determine what items will be on your survey instrument. – The hypothesis is where conceptual thinking must become concrete. – Example: What do you mean by knowledge? How will you measure the concept of knowledge? How have others measured knowledge in your domain of interest? 6

7. Written Protocol If you plan on surveying human subjects you will need a protocol (proposal) for the IRB. The protocol is a map of how you will get from concept to finished product. – Objectives (specific aims or purpose). – Research question. – Literature (what is known/not known). – Methods (hypotheses, data sources, time period covered, subject inclusion/exclusion criteria) – Analysis plan (how will hypotheses be tested). 7

8. General Survey Instrument Principles Resist the temptation to ask too much. – If a question is not directly related to your hypotheses it should not be included. Too many questions can induce respondent fatigue. Pre-test surveys your survey instrument. – Skipping this step invites disaster in the form of unusable data. Be sure that each question has only one information request. 8

9. Example of a Problem Question What is your living arrangement (type of dwelling and number of people in the household)? – This is really two questions: – In what type of dwelling did you reside during most months in 2012? Single family dwelling. Apartment building. …. – In your principal place of residence in 2012, how many other people lived with you during most months of the year? 9

10. Another Example How many times have you been seen by health care provider in the past 12 months? None. One to two. Three or more. I don’t know. ‘Health care provider’ is too vague. – Be specific, e.g., physician or nurse practitioner. Everything should be defined – respondents should not need to interpret you meaning. 10

11. Ordering of Questions Start with general questions and move toward more specific questions. Put potentially sensitive questions toward the end of the survey (e.g., income). Questions with more difficult response patterns should also be near the end. 11

12. Recall Period The literature on this is not consistent except that the more recent the recall period the more likely you are to get accurate responses. – For infrequent events the guidance is generally one week. In the last week how many times have you missed a dose of your medication? – Probably difficult to get accurate recall from most people. – For salient events a much longer recall period can be used. Have you had surgery in the last 12 months? Have you been hospitalized in the last 12 months? 12

13. Validated Instruments Many validated instruments are available for assessment of health care issues. – For example, General and disease specific quality of life scales are available at little or no cost. Benefits of validated instruments are: – Psychometrics for the scales are known. – Norms are available for comparisons. – Considerable published literature available. 13

14. Disease Specific Surveys RAQoL – Rheumatoid Arthritis Quality of Life Scale. ACT – Asthma Control Test. HIT-6 – Headache Impact Test. 14

15. Generic Validated Instruments: The SF series SF 36v2, SF12v2, SF8 – General health surveys that measure the same eight health domains. Each survey provides psychometrically-based physical component summary (PCS) and mental component summary (MCS) scores. eight health domains – Scores are calibrated so that 50 is the average score or norm. This norm-based score allows comparison among the three surveys and across the more than 19,000 studies published in the past 20 years.norm-based score 15

16. 16

17. 17

18. Rating Scales Multiple types of scaling for subject responses, such as: – Thurston scale, Guttman scale, Likert scale, etc. Most commonly used is Likert scaling. Likert scales are a method of capturing variation in some underlying phenomenon. Respondents to a Likert scale are indicating their agreement/disagreement with questions that are aspects of the domain of interest 18

19. Likert Scales Odd or even number of response categories. – Odd gives a neutral point. – Even forces a non-neutral response. Number of response categories should reflect complexity of what is being scaled. – Typical scales use 5 to 7 scaling points. Use declarative statements. Balance the number of positive and negative statements. Use a consistent response format throughout. 19

20. Likert Scales How many questions does it take to adequately scale a concept? – It depends. Acquiescence is a problem with all scales. – Respondent gives same answer regardless of the question. – Prevent by adding an equal number of positive and negative statements so results cancel each other. 20

21. Scoring a Likert Scale Likert scales have ordinal level measurement properties. – Ordinal scales do not have equal intervals among scale points. – Distance between ‘strongly agree’ and ‘agree’ can be different than distance between ‘strongly disagree’ and ‘disagree.’ – My definition of ‘agree’ and yours can be very different. 21

22. Scoring a Likert Scale Ordinal scales require non-parametric statistics for analysis. – However, some investigators report means for Likert scales – why? The Central Limit Theorem. – In general, the theory states that with a sufficiently large number of observations, the arithmetic mean for an independent variable will be normally distributed. – Parametric statistics (e.g., t-test) are appropriate for normally distributed variables. – Rule of thumb for ‘large number’ is 30 observations. 22

23. Summary Gave a quick overview of the survey research process. Described some key components of a survey questionnaire. Discussed the use of survey instruments and focused attention on validated instruments. Reviewed the basics of Likert scales. Now we have a practical example from Professor Happe. 23