1. 1 Using the Web for Surveys of Medical Providers Vasudha Narayanan Presented at Third International Conference on Establishment Surveys June 21, 2007

2. 2 AcknowledgementsAcknowledgements Co-Authors  Stephanie Fry, Jennifer Crafts & Pamela Giambo from Westat Project Officers  David Clark, William Taylor, Gladys Valentin & Mei Wang from Centers for Medicare & Medicaid Services This material was prepared by Westat Inc., under contract with the Centers for Medicare & Medicaid Services (CMS), an agency of the U.S. Department of Health and Human Services. The contents presented do not necessarily reflect CMS policy

3. 3 OverviewOverview About the Surveys Survey Methods Questionnaire Design Issues Sampling and Contacting Providers Completes by Mode Factors Affecting Web Response Conclusions

4. 4 About MCPSS & QIO Survey Two Surveys for Centers for Medicare & Medicaid Services (CMS) –QIO Provider Survey (2004) –MCPSS (2006 & 2007) National Sample of Medicare Providers Satisfaction Surveys Approx. 15 minute survey Multi-Mode Data Collection

5. 5 Survey Methods Web with telephone follow-up; paper on request Telephone and paper surveys to: –Increase response rates –Reduce potential bias Importance of: –Screening –High quality mail outs –Professional Interviewers

6. 6 Survey Methods (cont.) Field Period (12-16 weeks) –Screener –Notification of web survey –Reminder letters/e-mails –Hard copy questionnaire mailing/fax –Telephone non-response –Response rates (AAPOR-RR3) QIO: 90% MCPSS: 65%

7. 7 Questionnaire Design Usability Testing Purpose of Testing –Ease of navigation –User satisfaction and perceived burden Profile of Testers Response mode bias

8. 8 Questionnaire Design Usability Testing Issues Allow for multiple respondents –Clarity of instructions to “designate a colleague” Clarity of instruction for submitting survey Ability to skip sections and questions

9. 9 Elements of the Final Design Allow multiple respondents for each survey Track section-level designees Track section-level status Crisp and easy to navigate web survey Seamless link between CATI & Web

10. 10

11. 11

12. 12 Completes by Mode * non-response mail out of hard-copy survey in 2004 Highest Rates of Web response – Hospitals Lowest Rates of Web response – Physicians WebMail/Fax*TelephoneCompletes 200432.2%21.9%45.9%17,628 200640.1%1.7%58.1%16,121 200755.1%0.9%43.9%17,747

13. 13 Factors Affecting Web Response Percentage Web access –93% physician practices (2007) –97% other providers (2007) Hypotheses: Ease of Web access at work station Ease of survey website navigation Ability to obtain help when needed

14. 14 ConclusionsConclusions Feasibility of a Web surveys no longer an issue Necessary factors for survey management and achieving high response rates –telephone non-response follow up essential –Frequent interface between data collection modes –Allow the respondent maximum flexibility while maintaining survey integrity

15. 15 Contacting the Authors Vasudha Narayanan (vasudhanarayanan@westat.com) Stephanie Fry (stephaniefry@westat.com) Jennifer Crafts (jennifercrafts@westat.com) Pamela Giambo (pamelagiambo@westat.com)

16. 16 Thank you