QUESTIONNAIRE DESIGN: GENERAL PRINCIPLES Lu Ann Aday, Ph.D. The University of Texas School of Public Health

MEASUREMENT: Definition of Variables TYPE DESCRIPTION CONCEPTUAL DEFINITION Major concept or issue of interest, e.g., obesity OPERATIONAL Specific questions asked on the survey to obtain information on the concept VARIABLE Variable(s) constructed from the questions to actually be used in the analyses, e.g., Body Mass Index

MEASUREMENT: Levels of Measurement PROPERTY Nominal Ordinal Interval Ratio Classify X Rank Distance between points Distance from zero

MEASUREMENT: Example Concept: Obesity LEVEL/ DEFINITION Nominal Ordinal Interval OPERATIONAL Do you consider yourself overweight, underweight, or just about right? Would you say you are very overweight, somewhat overweight, or only a little overweight? About how tall are you without shoes? About how much do you weigh without shoes? VARIABLE 1=overweight 2=underweight 3=about right 1=very overweight 2=somewhat overweight 3=only a little overweight Body Mass Index (BMI)=weight divided by height, squared

MEASUREMENT: Example Concept: Family Income LEVEL/ DEFINITION Nominal Ordinal Ratio OPERATIONAL Did anyone in your family have income from wages or salaries in the past 12 months? Which of the following best describes your family’s total income from wages & salaries in the past 12 months? What was your family’s total income from wages & salaries during the past 12 months? VARIABLE 1=Yes 2=No 1=<$35,000 2=$35,000-$49,999 3=$50,000+ $ __,____/year

PRINCIPLES FOR WRITING QUESTIONS (Dillman, 2000) 1. Choose simple over specialized words. 2. Choose as few words as possible to pose the question. 3. Use complete sentences to ask questions. 4. Avoid vague quantifiers when more precise estimates can be obtained.

PRINCIPLES FOR WRITING QUESTIONS (Dillman, 2000) 5. Avoid specificity that exceeds the respondent’s potential for having an accurate, ready-made answer. 6. Use equal numbers of positive and negative categories for scalar questions. 7. Distinguish undecided from neutral by placement at the end of the scale.

PRINCIPLES FOR WRITING QUESTIONS (Dillman, 2000) 8. Avoid bias from unequal comparisons. 9. State both sides of attitude scales in the question stems. 10. Eliminate check-all-that-apply question formats to reduce primacy effects. 11. Develop response categories that are mutually exclusive.

PRINCIPLES FOR WRITING QUESTIONS (Dillman, 2000) 12. Use cognitive design techniques to improve recall. 13. Provide appropriate time referents. 14. Be sure each question is technically accurate. 15. Choose question wordings that allow essential comparisons to be made with previously collected data.

PRINCIPLES FOR WRITING QUESTIONS (Dillman, 2000) 16. Avoid asking respondents to say yes when they mean no. 17. Avoid double-barreled questions. 18. Soften the impact of potentially objectionable questions. 19. Avoid asking respondents to make unnecessary calculations.

SCALES: Definition Scales are measurement instruments that are collections of items [questions/statements] intended to reveal theoretical variables [abstract concepts] not readily observable by direct means, e.g., patient satisfaction, locus of control.

SCALES: Underlying Structure ABSTRACT CONCEPT, e.g., Unsatisfactory Patient Encounters Specific Items, e.g., series of statements regarding the nature of Patient Encounters Uniform Response Categories, e.g., scale of 1 to 5, where 1 is very typical and 5 is not at all typical of patient encounters

(Circle one response for each item.) SCALES: Example— Unsatisfactory Patient Encounters (Aday & Cornelius, 2006, NDMS, Q. 3) Very Typical 1 (Circle one response for each item.) Not at all 5 1. The patient was trying to manipulate me. 2 3 4 11. I felt that the patient didn’t believe me. 18. The patient and I seemed to come from different worlds.

SCALES: Methodological Issues ABSTRACT CONCEPT Internal consistency reliability: Verify that the specific items in the scale relate to the CONCEPT. Construct validity: Verify that the specific items in the scale define and distinguish the CONCEPT from other concepts.

SCALES: Methodological Issues Specific Items Take reading level of respondents into account in phrasing items. Avoid jargon and colloquialisms. Express only one idea in each item. Use both positively & negatively worded items, but avoid use of negatives (e.g., “not”) in negatively wording an item.

SCALES: Methodological Issues Uniform Response Categories Decide on a reasonable number of response categories, e.g., 3-10, taking concept and respondents’ ability to discriminate into account. Select meaningful labels/terms for categories, e.g., strongly agree, agree, disagree, strongly disagree; excellent, good, fair, poor.

MEASUREMENT MATRIX: National Dental Malpractice Survey (Aday & Cornelius, 2006, Table 15.1) QUESTION CONCEPT LEVEL OBJECTIVE 3 Doctor-patient communication: characteristics of unsatisfactory patient encounters (Likert scale) ordinal (interval) 10 Practice characteristics: avg. office waiting time for patient 28 Practice finances: % patients with insurance coverage interval 35 Malpractice insurance: no. of complaints 1, 2, 3 54 Demographics: gender nominal 2, 3

SURVEY ERRORS: General Principles for Formulating Questions Systematic Errors Variable Errors Questionnaire effects: under- or over-reporting Respondent effects: yeah-saying Mode effects: systematic Questionnaire effects: order & context Interviewer effects: interviewer variability Mode effects: variable Solutions to errors See Chapters Ten and Twelve. See Chapters Ten, Eleven, and Twelve. See Chapters Five and Ten. See Chapter Thirteen. See Chapter Five.

REFERENCES DeVellis, Robert F. (2003). Scale Development: Theory and Applications. Second Edition. Thousand Oaks, CA: Sage. Dillman, Don A. (2000). Mail and Internet Surveys: The Tailored Design Method. Second Edition. New York: John Wiley & Sons, Inc.