Creating Surveys

Research Question You should ALWAYS begin with your research question before creating specific questions on a survey. ◦ Survey’s are ideal for answering quantitative questions. You don’t want to give away your research question on a survey, or people will just tell you what you want to hear. In some cases, you can ask the question of your participants, but make sure there are other questions as well (this is called embedding)

Ask more than you need All questions should be attached to your research question/hypothesis ◦ Gender, GPA, class-level are all questions that might lead to asking a more nuanced or complex research question. But don’t ask them if they aren’t relevant When designing your survey, you can revisit your research question. But once your survey has been given out, you shouldn’t change your research question.

Freedom of Response Surveys are often made up of multiple choice options. You have to think of a predetermined set of options and give the participant the freedom to select the most appropriate. If you are unsure whether a category covers all options, add a open-ended write-in, “Other” option.

Open/closed-ended questions Open-ended questions are fill-in-the- blank. They allow the participant to write whatever they want in response ◦ Advantages: they allow the most freedom of response and provide the most precise data ◦ Disadvantages: they can be difficult to code (i.e. quantify) when you compute your final data.

Likert scale Likert scales (scale of 1-3, 1-5 or 1-7) allow a participant to select a pre-determined response that is easy for the researcher to quantify. Agreement ◦ Always disagree / sometimes disagree / neither agree nor disagree / sometimes agree / always agree Degrees ◦ Dislike / neither dislike nor like / Like / Don’t Know ◦ Never / Occasionally / Often / Always / Don’t Know

Likert scale, cont. Likert scales allow you to compare two or more variables by comparing the means (i.e. averages) of different Likert questions. ◦ When creating a character, how important is its appearance to you (never-sometimes-always) ◦ When creating a character, how important is its gender to you (never-sometimes-always) ◦ When creating a character, how important is its race to you (never- sometimes-always) Gets translated to: WoW players think more about race (M = 2.5) than they do about appearance (M = 0.95) and gender (M = 1.97) when creating a character.

Understanding your audience Most surveys fail because one of two problems, and they are both audience related: ◦ Egocentrism – egocentrism means that you assume that everybody knows the things that you know. What you assume the definition of a word is not necessarily what somebody else thinks. ◦ Leading questions – because you may be so interested in answering your research question, you inadvertently ask a leading question that assumes your audience thinks a particular way or you skip the more basic questions.

Problems The following four examples are from survey mistakes people have made in the past. What do you think the research question is for each? What is wrong with the survey? ◦ Bias, freedom of response, difficulty in coding, questions not responding to the research question? There could be more problems as well.

Example #1 Gender: Male / Female / Other GPA: <2.00 / / / / Class-level: freshman / sophomore / junior / senior What factors contributed to your selection of a laptop? price / size / features / manufacturer What kind of laptop do you have? ___________ Do you use your laptop in class? yes / no

Example #2 Grade:____________________ Major:____________________ How often do you play WoW? sometimes / often / always When do you play WoW? Morning Evening Night

Example #3 Gender:________________ GPA:___________________ World of Warcraft is the greatest game ever (yes / no) Warhammer Online was an epic fail (yes / no) Blizzard should sell gold for cash (yes / no)

Example #4 Class of: What type of character do you like the most:_______________ What type of character do you like the least:_______________ What server do you play on:_______________ Do you wish Blizzard would allow you to switch servers:_____________