Modifying Interviewer Strategies to Reduce Cost of Data Collection Rachael Walsh & Scott Boggess US Census Bureau European Survey Research Association, 2015 Disclaimer: The views expressed on statistical, methodological, technical, or operational issues are those of the author and not necessarily those of the U.S. Census Bureau.

Special thanks to National Center for Health Statistics, who sponsors the collection of data used in this presentation!

National Ambulatory Medical Care Survey (NAMCS) National survey of office-based physicians and community health center (CHCs) providers engaged in direct ambulatory patient care Four-month, concurrent interview periods Multiple phases – Screener, Induction, and Abstraction of medical records Paradata Collection – interviewers record info about each contact attempt

Contact History Instrument (CHI)

CHI Analysis January – October 2013 15,766 cases with >100,000 contact attempts Average of 1 to 2.6 personal visits/case (up to 24) for first portion of interview Less than 0.05% of cases refuse over the phone Scheduled appointments increase likelihood of interview completion Time of day analysis – variation in the success of personal visits and telephone attempts by time of day and day of week

CHI Applications Increasing cost, decreasing response rates Enhance operational efficiency Maintain data quality Data-driven test design Guided by previous research Telephone attempts are more cost effective

Telephone Test Protocol At least 3 telephone attempts before the 1st personal visit Complete screener and induction over the phone Call Monday, Tuesday, or Wednesday before noon Test group selection: 5 of 6 Regional Offices ½ of the areas in test group – matched to a control group with similar response rates and sample size January – April 2015 836 cases with >9,000 contact attempts

Protocol Exceptions No available number Physician prefers personal visit Physician refuses over the phone Interviewer schedules an appointment

Test Evaluation Compliance via real-time monitoring Cost Savings Weekly emails to supervisors of noncompliant interviewers Cost Savings Billable hours and number of personal visits T-tests and random intercepts to test for statistically significant differences between groups Data Quality Response rate comparison between groups

Compliance

Weekly Monitoring Automated list emailed every Thursday morning Headquarters staff emailed Field managers Field staff notified the interviewers and reported back Two errors discovered: Incorrect date stamps Provider cases spawned from CHC cases < 8% of cases were noncompliant 17% of interviewers in test group

Feedback “I feel that I am much more successful with getting doctors/office staff to cooperate because there is not such a lag time between contacting them, sampling week, and completing inductions.” “I feel that this is also a time saver and money saver for headquarters.”

Cost Savings & Data Quality

Cost Savings Results Only 2 of 5 Regional Offices saw reduction in number of personal visits in the test group Reduction from 2014 to 2015 suggests just doing a test decreased the number of personal visits Increased number of telephone attempts in the test group Test group charged fewer hours and miles than the control group Different prior to test Random intercepts findings indicated same direction, but none were statistically significant

Data Quality Results Insufficient data to test Isolating one interview period showed decreased response rates for test group Using cases closed-out prior to May 1 (all interview periods) – increased in 4 of 5 Regions Not statistically significant at the p ≤ 0.05 level when using any of the following: Random intercepts model T-tests: pooled, Satterthwaite, Cochran F-test

Conclusions Mixed results Response rates were inconclusive Just conducting the test reduced the number of personal visits No statistically significant differences between the test and control groups Response rates were inconclusive

Next Steps Personal Visit Test: Delayed Start Test: Restrict to 6 personal visits for the first portion of the interview (10 total) Delayed Start Test: Truncated interview period

Thank You! Rachael.Walsh@census.gov