Survey Design 102 Getting Started FIE-SDFS Research & Evaluation 2003

Table Of Contents Golden Rules Plan For Success 1. Survey Contents 2. Item Development 3. Response Formats Summary

Golden Rules 1. Understand the goals of the project. 2. Keep the survey simple. 3. Field/Pilot-test the survey. 4. Consider the demographic characteristics of the respondents. 5. Visual appeal.

Golden Rule #1 Understand the Project Goals A comprehensive understanding of the project goals paves the way for successful surveying. Without that understanding, it is impossible to appropriately design and to maximize the power of the survey.

Golden Rule #2 Keep It Simple Avoid the temptation to add a couple of questions just in case you did not cover all the bases. Only ask questions that fit the scheme of your project. In this case, oftentimes, less is more. The design focus should be quality not quantity.

Golden Rule #3 Field/Pilot Test Pre-testing the survey enhances clarity, removes word meaning obscurity, establishes approximate time to completion, proper sequencing, question transitioning, and helps assure that questions are not too difficult.

Golden Rule #4 Respondent Demographic Characteristics It is important to know what and how demographic characteristics may impact the interpretation and meaning of certain questions. Variations truly exist from one setting to the next.

Golden Rule #5 Visual Appeal Appearance means everything. If the survey appears complex and difficult to follow with the eyes, respondents will be less likely to do a good job. It should appear to be short, organized, and easy to answer.

Plan For Success

Survey Content Writing the survey requires knowing what questions need to be answered, how the results be used, reporting requirements, what types of data analyses will be conducted, construct development, demographic characteristics, and response formats etc. Directions should be clear and terse. Avoid the temptation to overstate the survey intent in the directions.

Item Development Listed below are five key components to consider: 1. Meaning—whether the majority of respondents perceive the meaning intended by the survey developers. 2. Item difficulty—a question can be hard to answer even though the meaning is entirely clear, if the respondent does not have prior knowledge about the specific item construct.

Item Development 3. Respondent interest and attention. Assure the survey design and format is appealing as to pare down, brief instances of inattention by the respondents. 4. Flow, naturalness, and sequencing. Assure a natural flow and transition from one survey item to the next. 5. Determine if the survey creates a positive impression that allows candor from one question to the next. 6. If the question is not necessary, do not include it.

Item Development 7. Avoid loaded questions or the type that may put respondents on edge. Such as emotionally charged words. 8. Avoid putting two questions in one. Compound questions may lead to ambiguity in the data analysis phase. 9. Be specific. Are you seeking opinions, facts, or a first witness to an account?

Item Development 10. Avoid the use of jargon. Your respondents should not have to infer meaning. 11. Questions should not be leading. For example, smoking cigarettes cause cancer--

Response Formats Survey developers use several different types of response formats. Listed below are several of the most common response formats: 1) Multiple choice 2) Open ended 3) Rank order 4) Rating Scale--Agreement 5) To what extent 6) Forced choice 7) Closed ended

Response Formats When deciding which response type to use consider the following: a. Response type should be appropriate to the item. b. Allow a don’t know response. c. Allow an other response. d. Allow a none response. e. Allow an undecided response. f. All possibilities should be included. g. Response formats should be mutually exclusive. h. Avoid using acronyms unless they are commonplace.

Summary This presentation is one in a series of presentations designed principally for the novice survey developer. Conversely, the series may serve as a resource tool for experience developers. If you have questions or comments please call Dr. William Respress at