SURVEYS Sherry Woosley & Cindy Miller

Overall Surveys

Survey Planning

Surveys should be used to:  Explore attitudes, opinions, experiences, expectations, and needs  Gather information from and about large populations  Make comparisons among subgroups of the population  Compare results from year to year  Gather statistically representative data

Surveys should not be used:  For audiences that are uncomfortable with numbers/statistics  For small numbers of participants  Without a clear understanding of the issues  When investigating issues of a sensitive or intrusive nature

Advantages  Can gather information from large numbers  Results may be generalizable to a larger population  Allow for statistical analysis that examines relationships among variables

Disadvantages  Important issues can be overlooked on surveys when the questions and responses are predetermined.  The quality of survey data is strongly dependent on the survey design.  Response rates and response bias are difficult to control.

Frequently Asked Questions – Other Big Picture Issues Surveys

FAQ – What about an on-line survey?  To decide think about:  Who is in your population?  What is in your survey? Do students care about this topic? How long is the survey?  What opportunities do you have to administer the survey?  What resources do you have? On-line surveys do NOT necessarily have lower response rates.

FAQ – How many respondents do I need?  The important issues are:  What percentage of people didn’t respond?  How well do your respondents match your non- respondents?  Do you have enough respondents to do the analysis you want? There is no magic number! In most cases, a sample size calculator does NOT adequately answer this question. There is no magic number! In most cases, a sample size calculator does NOT adequately answer this question.

FAQ – What types of analysis are appropriate? 1. Descriptives: Looking at how students responded 2. Differences: Comparing groups of students 3. Relationships: Looking at which variables may be related 4. Change: Looking longitudinally The type of analysis depends on (1) the questions being asked and (2) the statistical skills of the researcher.

FAQ – What is included in a survey report?  Need or problem and research questions  Survey characteristics  Survey methods (administration, sample, analysis)  Results  Conclusions  Implications Audience needs will determine the level of detail included in a report.

Questions / Items Surveys

Good Survey Questions  Wording should be simple, clear, direct, non- ambiguous, concrete, and uniformly understood.  There is no one “correct” response scale for all survey items.

Question Types  Open ended  Single response  Essay/paragraph  Closed  Categorical (categories)  Scales (Likert)  Rankings  Choose multiples or choose only one

Good Surveys…  Use neutral language  Generate a variety of responses  Use simple sentences (avoid compound sentences and multiple phrases)  Consist of only one question (beware of double- barreled questions)

Bad Surveys…  Ask things you already know  Ask things that your respondents cannot answer  Be too intrusive or personal  Include  universals (always, all, none)  limiters (only, just)  double negatives  abbreviations  unconventional phrases

Bad Examples – Can you identify what went wrong?  How many times did you call your parents last year?  How much time does your teacher spend preparing for class?  How often do you eat donuts and/or drink coffee?  How often have you been harassed on this campus?  How often have you participated in ACCESS?  How often are you merely late for a class?  How often do you miss class and feel bad about it afterwards?  How often do you never miss a class?

Strategies to Ensure Good Questions  Have the questions reviewed by experts  Have the questions reviewed by potential respondents  Have the questions reviewed by colleagues  Adopt or adapt questions that have been used successfully on other surveys  Pilot test surveys

Discussion Your turn…  What did you bring?  What questions do you still have?  How can you share your expertise with others?

Questions?  Thanks for attending today’s session.  More questions? Feel free to contact us. Sherry Woosley Cindy Miller