1. Modifying Interviewer Strategies to Reduce Cost of Data Collection
Rachael Walsh & Scott Boggess
US Census Bureau
European Survey Research Association, 2015
Disclaimer: The views expressed on statistical, methodological, technical, or operational issues are those of the author and not necessarily those of the U.S. Census Bureau.

2. Special thanks to National Center for Health Statistics, who sponsors the collection of data used in this presentation!

3. National Ambulatory Medical Care Survey (NAMCS)
National survey of office-based physicians and community health center (CHCs) providers engaged in direct ambulatory patient care
Four-month, concurrent interview periods
Multiple phases – Screener, Induction, and Abstraction of medical records
Paradata Collection – interviewers record info about each contact attempt

4. Contact History Instrument (CHI)

5. CHI Analysis January – October 2013
15,766 cases with >100,000 contact attempts
Average of 1 to 2.6 personal visits/case (up to 24) for first portion of interview
Less than 0.05% of cases refuse over the phone
Scheduled appointments increase likelihood of interview completion
Time of day analysis – variation in the success of personal visits and telephone attempts by time of day and day of week

6. CHI Applications Increasing cost, decreasing response rates
Enhance operational efficiency
Maintain data quality
Data-driven test design
Guided by previous research
Telephone attempts are more cost effective

7. Telephone Test Protocol
At least 3 telephone attempts before the 1st personal visit
Complete screener and induction over the phone
Call Monday, Tuesday, or Wednesday before noon
Test group selection:
5 of 6 Regional Offices
½ of the areas in test group – matched to a control group with similar response rates and sample size
January – April 2015
836 cases with >9,000 contact attempts

8. Protocol Exceptions No available number
Physician prefers personal visit
Physician refuses over the phone
Interviewer schedules an appointment

9. Test Evaluation Compliance via real-time monitoring Cost Savings
Weekly s to supervisors of noncompliant interviewers
Cost Savings
Billable hours and number of personal visits
T-tests and random intercepts to test for statistically significant differences between groups
Data Quality
Response rate comparison between groups

10. Compliance

11. Weekly Monitoring Automated list emailed every Thursday morning
Headquarters staff ed Field managers
Field staff notified the interviewers and reported back
Two errors discovered:
Incorrect date stamps
Provider cases spawned from CHC cases
< 8% of cases were noncompliant
17% of interviewers in test group

12. Feedback
“I feel that I am much more successful with getting doctors/office staff to cooperate because there is not such a lag time between contacting them, sampling week, and completing inductions.”
“I feel that this is also a time saver and money saver for headquarters.”

13. Cost Savings & Data Quality

14. Cost Savings Results
Only 2 of 5 Regional Offices saw reduction in number of personal visits in the test group
Reduction from 2014 to 2015 suggests just doing a test decreased the number of personal visits
Increased number of telephone attempts in the test group
Test group charged fewer hours and miles than the control group
Different prior to test
Random intercepts findings indicated same direction, but none were statistically significant

15. Data Quality Results Insufficient data to test
Isolating one interview period showed decreased response rates for test group
Using cases closed-out prior to May 1 (all interview periods) – increased in 4 of 5 Regions
Not statistically significant at the p ≤ 0.05 level when using any of the following:
Random intercepts model
T-tests: pooled, Satterthwaite, Cochran
F-test

16. Conclusions Mixed results Response rates were inconclusive
Just conducting the test reduced the number of personal visits
No statistically significant differences between the test and control groups
Response rates were inconclusive

17. Next Steps Personal Visit Test: Delayed Start Test:
Restrict to 6 personal visits for the first portion of the interview (10 total)
Delayed Start Test:
Truncated interview period

18. Thank You!