Copyright © 2011 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin Using Survey Research

2

3 In observational research, no measures are administered to subjects In a field survey measures are administered to assess attitudes and/or behavior Survey research is purely correlational No causal inferences can be drawn Can use survey research to predict behavior You must ensure anonymity and confidentiality for participants

4 First step is to clearly define your topic Yields unambiguous responses Provide clear operational definitions for behaviors measured Keeps questionnaire focused on topic; avoid trying to do too much in a single questionnaire You should collect demographic information (e.g., age, gender, religion) Often used as predictor variables Questionnaire must also have items measuring target behavior or attitudes Often used as criterion variables

5 Open-Ended Respondents are asked to answer a question in their own words Drawbacks to open-ended items Respondent may not understand what you are asking Difficult to summarize and analyze Restricted (closed-ended) Respondents are given a list of alternatives and check the desired alternative Alternatives can be ordered or unordered Gives you control over participant’s responses

6 Partially Open-Ended An “Other” alternative is added to a restricted item, allowing the respondent to write in an alternative Rating Scale Respondents circle a number on a scale (e.g., 0 to 10) or check a point on a line that best reflects their opinions Two factors need to be considered Number of points on the scale How to label the scale (e.g., endpoints only or each point)

7

8 A Likert Scale is a five-point scale used to assess attitudes Respondents indicate the degree of agreement or disagreement to a series of statements Rating scales are used in experimental research as well as survey research

9

10 Use simple words Make the stem of a question short and easy to understand Avoid vague questions Don’t ask for too much information in one question Avoid “check all that apply” items Avoid questions that ask for more than one thing Soften impact of sensitive questions

11 Organize questions into a coherent, visually pleasing format Do not present demographic items first Use an interesting question as your first item Keep related items together (continuity) Be aware that question order can make a difference Place sensitive or objectionable items after less sensitive/objectionable items Establish a logical navigational path

12 MAIL SURVEY A questionnaire is mailed directly to participants Mail surveys are very convenient Nonresponse bias is a serious problem resulting in an unrepresentative sample May reduce nonresponse bias by: Making multiple contacts with respondents Include a small token of your appreciation (e.g., a pencil) with your survey

13 INTERNET SURVEY Survey distributed via or on a Web site survey best for relatively short surveys Web survey allows for more complex navigational paths Large samples can be acquired quickly Nonresponse bias must still be considered Biased samples are possible because of uneven computer ownership across demographic groups Results from Internet survey comparable to other methods for most nonsensitive applications Use caution when doing Internet surveys on controversial issues

14 TELEPHONE SURVEY Participants are contacted by telephone and asked questions directly Can be done by a live researcher or by using Interactive Voice Response (IVR) technology Questions must be asked carefully The plethora of “junk calls” make participants suspicious GROUP ADMINISTRATION A questionnaire is distributed to a group of participants at once (e.g., a class) Completed by participants at the same time Ensuring anonymity may be a problem

15 INTERVIEW Participants are asked questions in a face-to-face structured or unstructured format Characteristics or behavior of the interviewer may affect the participants’ responses

16 Test-Retest Reliability Requires multiple administrations of a test Attention must be paid to the intertest interval Especially problematic if Ideas being measured fluctuate with time Participants are likely to recall their responses from an earlier administration of the test Your questions are long and boring Parallel Form Reliability Essentially the same as test-retest reliability An alternate form of the test is administered the second time

17 Split-Half Reliability Reliability assessed with one administration of a test Items from one half of a test are correlated with items from the second half of a test An odd-even split is the best way to split a test Kuder-Richardson Formula (KR20) Reliability assessed with one administration of a test The formula estimates the average for all possible split-half reliabilities A KR20 of at least.75 indicates moderate reliability

18 Increase the number of items on your questionnaire Standardize the conditions under which the test is administered (e.g., timing procedures, lighting, ventilation, instructions) Make sure you score your questionnaire carefully, eliminating scoring errors Make sure items on your questionnaire are clearly written and appropriate for those who will complete your questionnaire

19 You must assess the validity of your questionnaire You can assess validity with a variety of methods Content validity Construct validity Criterion-related validity Concurrent validity Predictive validity Validity can be affected by a number of factors (e.g., method of administration, unclear questions)

20 You should obtain a representative sample The sample closely matches the characteristics of the population A biased sample occurs when your sample characteristics don’t match population characteristics Biased samples often produce misleading or inaccurate results Usually stem from inadequate sampling procedures

21 Simple Random Sampling Randomly select a sample from the population Random digit dialing is a variant used with telephone surveys Reduces systematic bias, but does not guarantee a representative sample Some segments of the population may be over- or underrepresented

22 Stratified Sampling Used to obtain a representative sample Population is divided into demographic strata A random sample of a fixed size is drawn from each stratum May still lead to over- or underrepresentation of certain segments of the population Proportionate Sampling Same as stratified sampling except that the proportions of different groups in the population are reflected in the samples from the strata

23 Systematic Sampling Used in conjunction with stratified sampling Every k th element is sampled after a randomly selected starting point Sample every fifth name in the telephone book after a random page and starting point selected, for example Cluster Sampling Used when populations are very large The unit of sampling is a group (e.g., a class in a school) rather than individuals Groups are randomly sampled from the population (e.g., ten classes from a particular school)

24 Multistage Sampling Variant of cluster sampling First, identify large clusters (e.g., school districts) and randomly sample from that population Second, sample individuals from randomly selected clusters Can be used along with stratified sampling to ensure a representative sample Nonrandom sampling revisited Surveys sometimes have nonrandom samples Surveys done on a college campus Internet survey May limit generality of survey results

25 You should try to select an economic sample Includes enough respondents to ensure a valid survey and no more Two factors are taken into account when determining necessary sample size Amount of acceptable sampling error Expected magnitude of population proportions There is a formula that is used to calculate sample size using the above parameters