Chapter 12 Survey Research

Survey Research Nonexperimental method using interviews or questionnaires to assess attitudes, activities, opinions, or beliefs. Surveys often used to assess changes in attitudes over time, to test theoretical models, and to describe and predict behavior (i.e., political polls; marketing surveys).

Survey Research (cont'd) To insure high external validity random samples should be used.

Steps in Conducting Survey Research Plan and design the survey Construct and refine the survey instrument Collect the survey data Enter and “clean” the data Analyze & interpret the data

Cross-sectional and Longitudinal Designs Cross-sectional studies involve collecting data in a single, brief time period from different populations (i.e., age groups, ethnic groups). Longitudinal studies involve collecting data at more than one point in time panel studies – type of longitudinal design in which the same individuals are surveyed multiple times over time trend study – same survey questions are asked of different samples over time (Real Clear Politics)

Link to article

Methods of Data Collection Face-to-face or personal interview advantages include ability to clear up ambiguities and higher completion rate disadvantage - expense Telephone interview less expensive than face to face and comparable data

Methods of Data Collection (cont'd) Mail questionnaires low cost but low return rate Group-administered questionnaire

Methods of Data Collection (cont'd) Electronic survey e-mail & Web-based advantages of electronic surveys cost instant access to wide audience download to spreadsheet flexible in layout—especially web-based survey

Methods of Data Collection (cont'd) Electronic survey disadvantages of electronic surveys privacy and anonymity sample may not be representative of population (volunteer sampling) Eg. Amazon’s Mechanical Turk (Mturk)

Constructing and Refining a Survey Instrument Principle 1. Write Items to Match the Research Objectives conduct literature review (don’t always need to reinvent the wheel) write items that will yield reliable and valid data (may have to construct your own measurement).

Write Items That Are Appropriate for the Respondents to be Surveyed Principle 2. Write Items That Are Appropriate for the Respondents to be Surveyed use easy-to-understand language based on reading level, culture etc. Principle 3. Write Short, Simple Questions

Avoid Loaded or Leading Questions Principle 4. Avoid Loaded or Leading Questions a loaded term is one that produced an emotional response

a leading question suggests to the respondent how they should respond When forty people were asked about headaches. Those who were asked 'Do you get headaches frequently and, if so, how often?' reported an average of 2.2 headaches per week, whereas those who were asked 'Do you get headaches occasionally and, if so, how often?' reported only 0.7 per week (Loftus, 1975).

Avoid Double-Barreled Questions Principle 5. Avoid Double-Barreled Questions double-barreled questions ask two or more things in a single question “Have you stopped beating your wife yet.” “ 95% of respondents said “Current Gun laws need to be changed”. Do you Still Support Donald Trump?

Principle 6. Avoid Double Negatives I don’t have no time Principle 6. Avoid Double Negatives I don’t have no time. - avoid negatives if you can!

Principle 7. Determine Whether Closed-Ended or Open-Ended Questions are Needed open-ended better if researcher is unsure what respondent is thinking or variable is ill-defined closed-ended are easier to code and provide more standardized data

Construct Mutually Exclusive and Exhaustive Categories Principle 8. Construct Mutually Exclusive and Exhaustive Categories mutually exclusive means that the categories do not overlap exhaustive items include all possible responses

Reasons for taking Class Class Survey. Reasons for taking Class Cards All Top 2

Consider the Different Types of Closed-Ended Response Categories Principle 9. Consider the Different Types of Closed-Ended Response Categories rating scales multichotomous (more than two choices) usually preferred ability to measure direction and strength of attitude distance between each descriptor should be the same

Anchors - descriptors places on points on a rating scale.

Should Current Gun control Law be changed? Yes  No (binary forced choice) Please rate agreement with the statements according to rating scale. Current Gun Control laws should be: 1 2 3 4 5 7 8 9 10 Greatly Kept the Greatly Strengthened Same Lessened

Consider the Different Types of Closed-Ended Response Categories Principle 9 Consider the Different Types of Closed-Ended Response Categories binary forced choice participant chooses one of pair of attitudes typically not recommended

Rankings (forces comparisons between options) - not interval scale. Checklists ( allows for more than one answer)

Principle 10. Use Multiple Items to Measure Complex or Abstract Constructs Semantic differential – scaling method in which participants rate an object on a series of bipolar rating scales.

Likert scaling (Summated Rating Scale) Likert scaling (Summated Rating Scale). Characteristics of a Likert scale: The scale contains several items. Response levels are arranged horizontally. Response levels are anchored with consecutive integers. Response levels are also anchored with verbal labels which connote more-or-less evenly-spaced gradations. Verbal labels are bivalent and symmetrical about a neutral middle Always measures attitude in terms of level of agreement/disagreement to a target statement.

Make Sure the Questionnaire is Easy to Use From Beginning to End Principle 11. Make Sure the Questionnaire is Easy to Use From Beginning to End ordering of questions positive and interesting questions first demographic questions last limit the number of contingency questions (If yes, go to) questionnaire length

Response bias social desirability bias occurs when participants respond In a way to make themselves look good reduce by insuring anonymity Include a measure of social desirability response set – tendency to respond in a specific way use even number of response categories on rating scale include multiple question types

response bias response set – tendency to respond in a specific way use even number of response categories on rating scale include multiple question types

Benevolent Sexism Scale Peter Glick and Susan Fiske (1996) developed an interesting measure called the Benevolent Sexism Scale (BSS). Its 11 items are given below: 1. No matter how accomplished he is, a man is not truly complete as a person unless he has the love of a woman. 2. In a disaster, women ought not necessarily to be rescued before men. 3. People are often truly happy in life without being romantically involved with a member of the other sex. 4. Many women have a quality of purity that few men possess. 5. Women should be cherished and protected by men. 6. Every man ought to have a woman whom he adores. 7. Men are complete without women. 8. A good woman should be set on a pedestal by her man. 9. Women, compared to men, tend to have a superior moral sensibility. 10. Men should be willing to sacrifice their own well being in order to provide financially for the women in their lives. 11. Women, as compared to men, tend to have a more refined sense of culture and good taste.

Principle 12. Pilot Test the Questionnaire until it is perfected Other Terms in Chapter 12 Population vs. Sample Convenience sampling Random sapling Sampling error Simple Random Samples vs. Representative Samples