1 Using the Web for Surveys of Medical Providers Vasudha Narayanan Presented at Third International Conference on Establishment Surveys June 21, 2007

2 AcknowledgementsAcknowledgements Co-Authors  Stephanie Fry, Jennifer Crafts & Pamela Giambo from Westat Project Officers  David Clark, William Taylor, Gladys Valentin & Mei Wang from Centers for Medicare & Medicaid Services This material was prepared by Westat Inc., under contract with the Centers for Medicare & Medicaid Services (CMS), an agency of the U.S. Department of Health and Human Services. The contents presented do not necessarily reflect CMS policy

3 OverviewOverview About the Surveys Survey Methods Questionnaire Design Issues Sampling and Contacting Providers Completes by Mode Factors Affecting Web Response Conclusions

4 About MCPSS & QIO Survey Two Surveys for Centers for Medicare & Medicaid Services (CMS) –QIO Provider Survey (2004) –MCPSS (2006 & 2007) National Sample of Medicare Providers Satisfaction Surveys Approx. 15 minute survey Multi-Mode Data Collection

5 Survey Methods Web with telephone follow-up; paper on request Telephone and paper surveys to: –Increase response rates –Reduce potential bias Importance of: –Screening –High quality mail outs –Professional Interviewers

6 Survey Methods (cont.) Field Period (12-16 weeks) –Screener –Notification of web survey –Reminder letters/ s –Hard copy questionnaire mailing/fax –Telephone non-response –Response rates (AAPOR-RR3) QIO: 90% MCPSS: 65%

7 Questionnaire Design Usability Testing Purpose of Testing –Ease of navigation –User satisfaction and perceived burden Profile of Testers Response mode bias

8 Questionnaire Design Usability Testing Issues Allow for multiple respondents –Clarity of instructions to “designate a colleague” Clarity of instruction for submitting survey Ability to skip sections and questions

9 Elements of the Final Design Allow multiple respondents for each survey Track section-level designees Track section-level status Crisp and easy to navigate web survey Seamless link between CATI & Web

10

11

12 Completes by Mode * non-response mail out of hard-copy survey in 2004 Highest Rates of Web response – Hospitals Lowest Rates of Web response – Physicians WebMail/Fax*TelephoneCompletes %21.9%45.9%17, %1.7%58.1%16, %0.9%43.9%17,747

13 Factors Affecting Web Response Percentage Web access –93% physician practices (2007) –97% other providers (2007) Hypotheses: Ease of Web access at work station Ease of survey website navigation Ability to obtain help when needed

14 ConclusionsConclusions Feasibility of a Web surveys no longer an issue Necessary factors for survey management and achieving high response rates –telephone non-response follow up essential –Frequent interface between data collection modes –Allow the respondent maximum flexibility while maintaining survey integrity

15 Contacting the Authors Vasudha Narayanan Stephanie Fry Jennifer Crafts Pamela Giambo

16 Thank you