Writing Survey Questions, How to Get Responses, and What To Do With the Responses Once You Get Them! Kellee Thorburn McCrory, MPH Iowa Center for Evaluation Research The University of Iowa

What are you going to learn in 45 minutes? What is a survey? Survey questions/good and bad Increasing response rates Analyzing and organizing Time for questions at the end

Why Survey? You want to know something about a population You want to know if anything was learned (pre-post tests) You want to see if perceptions or needs have changed over time Help you to enhance/change/modify programs Helps you to understand health behaviors

Birth of a survey Know your target population Develop your survey Determine format: mail, in-person, in groups, internet, interactive voice response, self-administered, etc. Open-ended vs. close-ended questions Revise and edit questionnaire Test, test, test Cognitive interviewing Determine protocol Field test survey Implement

What makes a good survey? Clear, simple English Meaningful to the respondent Not a lot of text boxes Straight-forward questions Does not take a lot of the respondent’s time Gets it right the first time! Planning went into the development and analysis plan

The good, the bad, and the ugly

Survey Question Examples The government should force you to pay higher taxes. Taxation is the role of the government (Examples from Qualtrics, retrieved 2013)

What is your age?  0–10  10–20  20–30  30–40  40+ What is your age?  0-10  11–20  21–30  31–40     70+

How would you rate the career of legendary outfielder Joe DiMaggio? How would you rate the career of baseball outfielder Joe DiMaggio?

Give respondents an out! What is your race? What is your age? Did you vote in the last election? What are your religious beliefs? What are your political beliefs? What is your annual household income?

What is your opinion of Crazy Justin’s auto repair?  Pretty good  Great  Fantastic  Incredible  The Best Ever What is your opinion of Crazy Justin’s auto repair?  Very good  Good  Fair  Poor  Very poor

What is the fastest and most economical internet service for you? What is the fastest internet service for you? Mediacom Liberty South Slope I do not have internet service What is the most economical internet service for you? Mediacom Liberty South Slope I do not have internet service

Scales Dichotomous Yes/no; agree/disagree; true/false Three point scales extremely/moderately/not at all; too much/about right/too little Four point scale Strongly agree/agree/disagree/strongly disagree Five point scale Excellent/above average/average/below average/very poor Very good/good/fair/poor/very poor very satisfied/satisfied/neither/dissatisfied/very dissatisfied

Conventional wisdom about question wording Simple, direct, comprehensible No jargon Be specific Avoid ambiguous words Avoid double-barreled questions Avoid leading questions Include filter questions Be sure questions read smoothly Avoid emotionally-charged words Allow for all possible responses (Losch, 2003)

This is NOT how someone being surveyed should feel!

Should you survey everyone or just take a sample? There are two categories: random and nonrandom If sampling – how large should the sample be? Great sampling calculator tool (Fink, 2009)

Response Rate What is a response rate? A response rate is the percentage of people who responded to your survey Total # of surveys returned/total # sent No single rate of return is standard as it depends on your population.

Increasing Response Rates 1.Target your audience. 2.Personalize your invitations 3.Keep your invitation short 4.Make your first survey page simple - let people take the survey! 5.Be clear about privacy protections (Jensen, retrieved 2013)

6.Send reminder s 7.Consider offering incentives - gifts, prizes 8.Some people just want to share their opinion! 9.Use graphics and internet features strategically. 10.Publish your results online to survey participants.

Now what? I have piles of data! Data Organization

Data Management Data management is CRUCIAL to analysis If you are using online surveys, your data will be stored for you and usually can run crosstabs for you. If you are using paper surveys, you will need to enter data into Excel or SPSS, code your responses and analyze your data.

Coding responses If you are using a multiple-choice or yes/no survey, you will want to code your responses for easier analysis. Codes are what “speaks” to statistical software. Examples: yes=1 and no=0 Female=1; Male=2 Prefer not to answer/not applicable=99

Example of a coded survey (from the CARPS survey) 1. Are you: Female (1) Male (2) 2.How often in the last 12 months have you had 4+ drinks in one sitting? Choose one answer. Daily or almost daily (1) Four or five times a week (2) Two or three times a week (3) Two or four times a month (4) One time a month or less (5) Never (0)

Codebook A codebook is your guide to your codes and an easy way to share codes with others. Variable NameVariable LabelValues: Labels and Codes Participant CodeUNIQUEID8=digit code GenderGENDER1=female; 0=male How often per month drinks FDRINK Daily or almost daily =1; Four or five times a week =2; Two or three times a week =3; Two or four times a month=4; One time a month or less =5; Never =0

What it will look like in Excel or SPSS

Types of Analysis Quantitative data for surveys involve calculating basic descriptive statistics such as: ◦frequency, ◦distributions, ◦measures of central tendency (mode, median, and mean); and ◦measures of distribution (range, interquartile range, and standard deviation)

Ramp it up a little Crosstabs Chi Square t-test (independent sample; paired; one sample) Choosing a statistical test is reliant upon whether or not data is categorical or continuous

Guiding questions and options for methods decisions What is the purpose of the report? Who are the primary audiences for the findings? What questions will guide the report(s)? What data will answer or illuminate the report questions? What resources are available to support the report? What criteria will be used to judge the quality of the findings? (Patton, 2002)

Consider… Who is your audience? Internal? External? What is it you need to show? How do they want the results presented? PowerPoint? Written Report? What do they need? Want? What do they intend to do with the results? Interpret data for your audience Use graphics– make it visual!

Books worth spending your hard- earned money on (in my opinion) Mail and Internet Surveys: The tailored design method; Don A. Dillman Evaluating Health Promotion Programs Thomas W. Valente How to Conduct Surveys: A step by step guide. Arlene Fink Improving Survey Questions: Design and evaluation; Floyd J. Fowler, Jr.

Online Survey Software surveys/ (now part of surveymonkey) surveys/ software/ software/

Great websites calculator.asp calculator.asp

Thank you for you time and attention! For more information or questions, please feel free to contact me at kellee- or by phone at