SURVEY RESEARCH

Topics Appropriate to Survey Research Descriptive Exploratory Explanatory

Guidelines for Asking Questions Choose appropriate question forms. Make items clear. Avoid double-barreled questions. Respondents must be competent to answer.

Guidelines for Asking Questions Respondents must be willing to answer. Questions should be relevant. Short items are best. Avoid negative items. Avoid biased items and terms.

Guidelines for Questionnaire Construction One question per line. Use contingency questions when necessary. Format matrix questions so they are easily answered.

Guidelines for Questionnaire Construction Be aware of issues with ordering items. Include instructions for the questionnaire. Pretest all or part of the questionnaire.

Question/Response Wording Likert Scales Thermometer ratings Agree/Disagree statements. Ordered responses. Open responses. Check all that apply.

Acceptable Response Rates 50% - adequate for analysis and reporting 60% - good 70% - very good

Guidelines for Survey Interviewing Dress in a similar manner to the people who will be interviewed. Study and become familiar with the questionnaire. Follow question wording exactly. Record responses exactly. Probe for responses when necessary.

Telephone Surveys Advantages: Money and time. Control over data collection. Disadvantages: Surveys that are really ad campaigns. Answering machines.

Strengths of Survey Research Useful in describing the characteristics of a large population. Make large samples feasible. Flexible - many questions can be asked on a given topic.

Weaknesses of Survey Research Can seldom deal with the context of social life. Inflexible in some ways. Subject to artificiality. Weak on validity.

Survey Problems Reader’s Digest presidential survey of 1936; Alf Landon vs. FDR Hite Sexuality Survey – 70% of women married 5 years or more are having sex outside of marriage (4,500 completed surveys out of 100,000). Recorded phone surveys. Push & Entertainment polling: political telemarketing masquerading as a poll

Bush campaign in South Carolina asked the following: "John McCain calls the campaign finance system corrupt, but as chairman of the Senate Commerce Committee, he raises money and travels on the private jets of corporations with legislative proposals before his committee. In view of this, are you much more likely to vote for him, somewhat more likely to vote for him, somewhat more likely to vote against him or much more likely to vote against him?"

Independent Sample T-test Formula t =

Independent-samples t-tests But what if you have more than two groups? One suggestion: pairwise comparisons (t-tests)

Multiple independent-samples t-tests That’s a lot of tests! # groups # tests 2 groups = 1 t-test 3 groups = 3 t-tests 4 groups = 6 t-tests 5 groups = 10 t-tests groups = 45 t-tests

Inflation of familywise error rate Familywise error rate – the probability of making at least one Type I error (rejecting the Null Hypothesis when the null is true) Every hypothesis test has a probability of making a Type I error (  ). For example, if two t-tests are each conducted using  =.05, there is a.0975 probability of committing at least one Type I error.

The formula for familywise error rate: # groups # tests nominal alpha familywise alpha 2 groups 1 t-test.05 3 groups 3 t-tests.05 4 groups 6 t-tests.05 5 groups 10 t-tests groups 45 t-tests.05 Inflation of familywise error rate

Analysis of Variance: Purpose Are there differences in the central tendency (mean) of groups? Inferential: Could the observed differences be due to chance?

Assumptions of ANOVA Normality – scores should be normally distributed within each group. Homogeneity of variance – scores should have the same variance within each group. Independence of observations – observations are randomly selected.

Logic of Analysis of Variance Null hypothesis (H o ): Population means from different conditions are equal –m 1 = m 2 = m 3 = m 4 Alternative hypothesis: H 1 –Not all population means equal.

Logic of Analysis of Variance Null hypothesis (H o ): Population means from different conditions are equal –m 1 = m 2 = m 3 = m 4 Alternative hypothesis: H 1 –Not all population means equal.

Lets visualize total amount of variance in an experiment Between Group Differences (Mean Square Group) Error Variance (Individual Differences + Random Variance) Mean Square Error Total Variance = Mean Square Total F ratio is a proportion of the MS group/MS Error. The larger the group differences, the bigger the F The larger the error variance, the smaller the F

Logic--cont. Create a measure of variability among group means –MS group Create a measure of variability within groups –MS error

Loves StatisticsHates StatisticsIndifferent Example: Test Scores and Attitudes on Statistics

Find the sum of squares between groups Find the sum of squares within groups Total sum of squares = sum of between group and within group sums of squares.

To find the mean squares: divide each sum of squares by the degrees of freedom (2 different dfs) Degrees of freedom between groups = k-1, where k = # of groups Degrees of freedom within groups = n-k MS between = SS between /df between MS within = SS within /df within F = MS between / MS within Compare your F with the F in Table D