EMR 6500: Survey Research Dr. Chris L. S. Coryn Lyssa N. Wilson Spring 2015

Agenda Stratified random sampling for proportions From questions to a questionnaire Budgeting survey research

Stratified Random Sampling

Estimate of Population Proportion

Example for a Population Proportion Nnn+n+ Town A Town B Rural

Example for a Population Proportion

Selecting the Sample Size for Estimating Population Proportions

Sample Size for Estimating Population Proportions

As with allocation methods for means and totals, allocation methods for stratified random samples for proportions can be equal, proportional, or optimal depending on the study’s purpose Sample Size for Estimating Population Proportions

From Questions to a Questionnaire

General Premises The design of a questionnaire should consider how to motivate the recipient to respond It should also avoid measurement errors, ranging from order effects to item nonresponse

Guidelines for Ordering Questions

Ordering Questions General guidelines 1.Group related questions that cover similar topics, and begin with questions likely to be salient to nearly all respondents 2.Choose the first question carefully 3.Place sensitive or potentially objectionable questions near the end 4.Ask questions about events in the order that they occurred 5.Avoid unintended order effects

Guidelines for Creating a Common Visual Stimulus

Visual Stimulus General guidelines 1.Establish consistency in the visual presentation of questions (across pages and screens) and use alignment and vertical spacing to help respondents organize information on the page 2.Use color and contrast to help respondents recognize the components of the questions and the navigational path through the survey 3.Visually group related information in regions through the use of contrast and enclosure 4.Use visual elements and properties consistently across questions to emphasize or deemphasize certain types of information 5.Avoid visual clutter 6.Minimize the use of matrixes and their complexity

Guidelines for Mail Questionnaires

Mail Questionnaires General guidelines 1.Determine whether keypunching or optical imaging and scanning will be used, and assess the limitations imposed on designing and processing questionnaires 2.Construct paper questionnaires in booklet formats, and choose physical dimensions based upon printing and mailing considerations 3.Decide question layout and how questions will be arranged on each page 4.Use symbols, contrast, size, proximity, and pagination effectively when designing branching instructions to help respondents correctly execute them 5.Create interesting and informative front and back pages that will have wide appeal to respondents 6.Avoid placing questions side-by-side on a page so that respondents are asked to answer two questions at once

Guidelines for Web Questionnaires

Web Questionnaires General guidelines 1.Decide whether an electronic alternative is appropriate 2.Choose how the survey will be programmed and hosted, commensurate with needs, skills, and sponsorship 3.Take steps to ensure that questions display similarity across platforms, browsers, and user settings 4.Decide how many questions will be presented on each page and how questions will be presented 5.Develop a screen format that emphasizes the respondent rather than the sponsor

Web Questionnaires General guidelines 6.Use a consistent page layout across screens and visually emphasize questions information that respondents will need to complete the survey while deemphasizing information not essential to the task 7.Do not require responses unless absolutely necessary 8.Design survey-specific and item-specific error messages to help respondents troubleshoot 9.Evaluate carefully the use of interactive features, balancing improvements in measurement with the impact on respondent burden and the implications with mixed-mode surveys

Web Questionnaires General guidelines 10.Use audiovisual capabilities sparingly, and evaluate the differential effect they have on respondents 11.Allow respondents to stop the survey and finish completing it at another time 12.Whenever possible, collect paradata that provide feedback on how respondents interact with questionnaire 13.Test the survey using a variety of platforms, connection speeds, browsers, and user-controlled settings, and test the database to ensure that items are collected and coded accurately 14.Take screenshots of each page of the final questionnaire for testing and documentation

Guidelines for Pretesting Questionnaires

Pretesting Questionnaires General guidelines 1.Obtain feedback on the draft of the questionnaire from a number of people, each of whom has specialized of some aspect of questionnaire quality 2.Conduct cognitive interviews of the complete questionnaire in order to identify wording, question order, visual design, and navigational problems 3.When the stakes are high, consider experimental evaluations of questionnaire components 4.Conduct a small pilot study with a subsample of the population in order to evaluate interconnections among questions, the questionnaire, and implementation procedures

Budgeting Survey Research

General Cost Frameworks for Survey Research Fixed-price – All costs are subsumed under a single dollar value, no matter time, effort, and other expenditures Cost-reimbursable – Reimbursement for actual expenditures

Direct Costs Personnel – Hourly staff (+7.65% FICA) – Salaried staff ( % for full benefits) – Academic staff ( % for full benefits) Travel – Local transportation, flights, per diem, lodging Other costs – Materials and supplies (e.g., office and operating supplies, telephone and conference calls, postage and shipping) – Publication and dissemination (e.g., standard printing, copying, production) – Participant costs and/or incentives – Consultants or subawards – Computer services

Indirect Costs F&A (facilities and administration) – 49% is WMU’s current HHS approved rate – 24% is WMU’s current off-campus rate (requires rental space elsewhere) – Other rates can be negotiated based on institutional policies (e.g., existing policy of funders)

Typical Budgeting Methods Per task Per day (e.g., %FTE) In budgeting, both are often necessary

Consider… A mail survey to 500 randomly selected recipients – Printing of surveys = $2.00/survey – Postage = $0.45/survey for delivery + $0.45/survey for return – Processing of surveys = 1 $18/hour – $2/recipient incentive What is the total cost for just the above, which does not include other costs?