Week 6 Survey

Objectives Assignment review Identify the content/elements of the cover letter Identify the elements of the survey How to prepare your own survey

Announcements Introduction Assignment is graded Lit review Assignment is due today

Cover Letters for Survey Research Studies The cover letter in a survey research study is used to meet IRB requirements and Federal laws pertaining to the protection of human subjects

Areas to Address in the Cover Letter Purpose of the study Return date, process Confidentiality of Data Contact Information Name, address, telephone number of researcher Informed Consent

Informed Consent Informed Consent requires the researcher to: Inform participant of potential risks Participation is voluntary Participants can withdraw at any time without penalty Confidentiality of information is ensured Means by which confidentiality will be kept Participants are required to: sign and date the form prior to participation

Informed Consent for Surveys A signed consent form is not required in a survey study as long as: The Informed Consent information is provided in the cover letter The researcher does not require the respondent to place their name or an identifier onto the survey There are no risks to the participant

Survey Research Plan Before a survey research study can be conducted, plans should be constructed to determine: The process for implementing the study Selection of survey subjects Number of subjects to survey Anticipated response rates Costs for administering the survey Data collection and analysis

Determining the survey sample The survey sample must be representative of the total population Characteristics of subjects that could influence results must be identified and controlled The number of subjects to survey can be determined using a defined confidence interval and a margin of error Confidence intervals are calculated differently for averages, proportions, etc.

Determining the survey sample For example, let’s say a 95% confidence interval is selected by the researcher This means, if the same survey questions are administered in 100 different studies, the researcher can expect to obtain the same results 95% of the time. Using this confidence interval, a margin of error can be defined Let’s say a researcher wishes to maintain a 4% margin of error for an item. The researcher obtains a result of 60% yes and 40% no. With a margin of error of 4%, the “yes” results could really be between 64% - 56%.

Sample Size Example Formula This formula can be used for an average that follows the bell shaped curve: Z s/2 is known as the critical value, the positive value that is at the vertical boundary for the area of in the right tail of the standard normal distribution. s is the population standard deviation. N is the sample size. This formula can be used when you know the standard deviation and want to determine the sample size necessary to establish, with a known confidence level and margin of error (E), the mean value to within . As a general rule of thumb, if your sample size is greater than 30, you can replace by the sample standard deviation s.

Sample Size for a Various Types of Procedures There are a number of tables constructed to determine sample sizes, using the power of the test, and the anticipated outcome The power of a statistical test is the probability, assuming that the null hypothesis is false (i.e. an effect is significant) of obtaining a result that will allow the rejection of the null hypothesis. The tables constructed for various types of statistical procedures in which the researcher can determine the number of subjects required to find an anticipated outcome while meeting a certain power level. Using a correlation power table as an example, the table indicates that with an Alpha Level of .05, and an anticipated correlation on .80, I would need to have a minimum of 19 subjects in my study to be 99% sure of identifying a significant finding if it does exist. Note: We will cover the power of a test and Type I and Type II errors in a later class.

Determining the survey sample A minimum number of responses are needed to maintain the confidence interval and the margin of error. The number of subjects can be calculated using the desired confidence interval and margin or error. Remember, this is the number of respondents not the number of people in the sample If a researcher can expect a response rate of 20%, then they need to survey many more people to obtain the required sample size

Procedures for collecting the data How will the data be collected? Online, paper. etc Who will enter the data? Is training required (i.e.: interviews)?

Activity Cover Letter Develop a cover letter that will be attached to the survey instrument. Be sure to include vital information required by the University Institutional Review Board

Guidelines for Writing Surveys A number of resources available Survey Research by Backstrom and Hursh-Casar Survey Research Methods by Fowler

Survey Terms closed question: context effects: open question: A survey question that offers response categories. context effects: The effects that prior questions have on subsequent responses. open question: A survey question that does not offer response categories. recency effect: Over reporting events in the most recent portion of a reference period, or a tendency to select the last-presented response alternative in a list. Respondents focusing on most recent thing they heard response effects: The effects of variations in question wording, order, instructions, format, etc. on responses. screening questions: Questions designed to identify specific conditions or events.

General Guidelines Keep the survey as short as possible Ask only what you need The more white space on the survey instrument the better Typically, surveys begin with demographic items then get into specific research areas Save the more “delicate” questions for the end of the survey

Consistency Instructions should be provided for the survey and unique types of items Provide definitions if necessary The survey instrument should have like types of items grouped together Keep scales in the same direction

Closed Response Questions Agree–disagree Forced choice Ordered response categories or scales

Question Order Question order changes the context in which a particular question is asked. Prior questions can influence answers to subsequent questions through several mechanisms. For example, an obscure “monetary control bill” was more likely to be supported when a question about it appeared after questions on inflation, which presumably led respondents to infer the bill was an anti-inflation measure.

Terminology “Avoid ambiguity” is a truism of questionnaire design. Language is inherently ambiguous, and seemingly simple words may have multiple meanings. Research by Belson and others demonstrates that ordinary words and phrases, such as “you,” “children,” and “work”are interpreted very differently by different respondents.

Terminology In a national sample, respondents were randomly assigned to be asked one of two questions: 1. “Do you think the United States should allow public speeches against democracy?” 2. “Do you think the United States should forbid public speeches against democracy?” Support for free speech is greater–by more than 20 percentage points–if respondents answer question 2 rather than question 1. That is, more people answer “no” to question 2 than answer “yes” to question 1; “not allowing” speeches is not the same as “forbidding” them, even though it might seem to be the same.

Leading Questions A leading question is simply one that, either by its form or content, suggests to the subject what answer is desired or leads him/her to the desired answer (Loftus & Palmer, 1974). How fast were the cars going when they hit each other?“ In this question the word hit would be replaced with other words such as smashed, collided, bumped, and contacted. When the word smashed was used, results showed that the subject estimated the car to be traveling at a faster speed then when the word bumped was used.

Don’t Know Giving “don’t know” as an explicit response option? On one hand, this has been advocated as a way of filtering out respondents who do not have an opinion and whose responses might therefore be meaningless. On the other hand, it increases the number of respondents who say “don’t know,” resulting in loss of data.

Likert Scales There are correct and incorrect ways of setting up a Likert Scale Used to collect data about respondents' feelings or attitudes The number at one end of the scale represents least agreement, or "Strongly Disagree," and the number at the other end of the scale represents most agreement, or "Strongly Agree." Ends of a Likert Scale are opposites Label each scale item (Don’t place one at the top of the page and require the respondent to refer back to it) The minimum number of points on the scale with descriptors should be both ends and the middle (If an odd number scale)

Odd Versus Even Number Likert Scales When using an odd number of points on the scale (ie: 5 point scale), ends are opposite and the middle is “neutral” Don’t know or NA is not neutral When using an even number of points on the scale (ie: 4 point scale), end points are opposite. Even numbered scales force the respondents to make decisions.

Available Choices When asking a question with boxes to check, provide instructions as to “check one box”, check all that apply, etc. Be sure to eliminate overlapping categories How many years have your worked here? 0-1 year 1-3 years 3-5 years No possible answer What does someone check if they worked 7 years?

Thank You and Return Instructions Thank the participant Provide instruction on how to return the survey

Follow-up for non-respondents Non-respondents need follow up Send out reminders, second survey In true survey studies, a second survey study is conducted to determine if significant differences exist between those who responded to initial surveys and those who needed follow ups

Pilot Testing Before a survey instrument is ever used in a real study, it must be pilot tested!!

Planning Develop a plan for administering a short survey instrument. Determine the survey population, the procedures for collecting the data, etc. Answer the following questions: Who will be selected and how? How the survey will be distributed? What will the costs be to administer the survey? What will the timeline be? What will be done to stimulate the response rate?