1. Survey as a Measurement Tool
Yes, I am feeling fall today.

2. What does this say about surveys and research?

3. Why will someone believe your survey results?
Validity
Does this survey measure what it is supposed to measure?
Reliability
Would you get consistent results when the characteristics have not changed?
Considerations: goals, permission, wording, demographic information requests, …

4. Essential Elements of Good Survey
Clear objective
Why are you creating this survey?
What is the ultimate goal of the research of which this survey is a measurement tool?
How will you use the data?
What decisions do you want to impact with this survey?
What are the variables that could impact answers?
Control with demographic questions
Define survey population

5. Survey Questions and Answers
Keep the survey short and simple
Make questions clear and simple (no “and”) and unbiased
Closed ended/structured questions vs. open ended questions
Consistent scale, and include a “prefer not to answer”
Use a logical ordering
Make sure answers are relevant to questions

6. Inviting Survey Respondents
Do a pilot study 10/12 and ask for feedback
Consider how random you can make it
Be clear in your request to complete the survey
Clearly state intentions for research
Include instructions
Do not ask for private information

7. Analyzing the Results Simply report Statistical Analysis
Clearly describe sample population
Graphical depictions
Statistical Analysis
Is there a statistically significant difference in answers between demographic variables?
Plan in advance of giving the survey how you will present results – this helps to drive questions

8. Common Mistakes Objective and data use not clearly planned up front
Not mutually exclusive answers (10-20, 20-30, etc.)
Strong language (forced vs required)
Could, should, might have up to a 20% difference in results
Multiple parts in question (is it fair and reasonable?)
Unbalanced scales and/or different orders (strongly agree, agree, partially agree, disagree) and reverse
Not including important demographic info
Not truly random sample

9. Measuring Validity and Reliability of a Survey
Multitrait approach: use two different measures (survey and what?)
Judgement by a panel of experts
Embed validity questions in a survey
Reliability
Inter-rater
Internal Consistency
Equivalent Forms
Test-retest