1. Sources of Errors in Estimating Community Health Center Physicians Centers for Disease Control and Prevention National Center for Health Statistics Catharine W. Burt, Ed.D. Esther Hing, M.P.H. Division of Health Care Statistics June 3, 2008

2. Objectives  To assess coverage error among CHC physicians in the National Ambulatory Medical Care Survey (NAMCS)  To estimate bias and reliability in percentage of CHCs using electronic medical records

3. Survey Type of Data Years fielded Current sample size (approximate) National Ambulatory Medical Care Survey (NAMCS) Visits to office-based physicians 1973- 1981, 1985, 1989- present 3150 physicians 25,665 encounters NAMCS

4. National probability sample survey of office- based physicians Complex sample design 112 geographic PSUs Physicians stratified by specialty Sample of visits within physicians Sample frame: AMA and AOA masterfiles Data collected by Census Bureau NAMCS Methodology

5. Scope of the NAMCS  Physicians must be: Primarily engaged in office-based, patient care Primarily engaged in office-based, patient care Nonfederally employed Nonfederally employed Not in anesthesiology, radiology, or pathology Not in anesthesiology, radiology, or pathology

6. In-Scope NAMCS Locations  Freestanding private solo or group practice  Freestanding clinic/urgicenter  Neighborhood medical and mental health centers  Privately operated clinics  Non-Federal government clinic  Health maintenance organization  Community health center  Faculty practice plan

7. Methods  In 2006, dual sample of physicians: traditional + separate stratum of 104 CHCs  Sampling frame: NACHC and IHS  Random selection of 3 providers within each CHC Physicians and midlevel providers Physicians and midlevel providers After selection of providers, normal NAMCS procedures are followed After selection of providers, normal NAMCS procedures are followed In-person induction interviewIn-person induction interview Sample of ~30 visits during sample weekSample of ~30 visits during sample week

8. Coverage error  Compare the traditional sample of CHC physicians with the separate stratum of CHC physicians on physician characteristics Traditional Separate Stratum

9. Content bias and consistency  Use separate stratum to estimate EMR use  We have multiple providers in the sampled CHCs to answer the EMR questions  Measure the reliability of the item response with Cohen’s kappa statistic

10. Results Traditional sample (n=1,311) Separate Stratum (n=156) 6,775 (non- office- based) Traditional sample MDs who work in CHCs (n=31) 8,596 39.4% 60.6%

11. Comparison of percent distributions Physician characteristic Traditional sample - CHC Separate CHC stratum Age <35 35-44 35-44 45-54 45-54 55-64 55-64 65+ 65+ 8.0 8.047.813.219.211.917.330.729.618.9 3.5 3.5 Gender Male Female Female44.955.141.658.4 MSA status MSA MSA Non-MSA Non-MSA87.812.293.1 6.9 6.9

12. EMR use among CHC physicians No significant difference @ p=.05

13. Response consistency of physicians reporting use of electronic health record systems in their CHC Provider 2 Provider 1 EMR No EMR EMR8412 54550 134962 n=62 CHCs with multiple providers who answered the EMR items

14. The test-retest counts yield an I =“Index of inconsistency” =0.214 It can be shown that Cohen’s kappa statistic=1- I =0.786 indicating moderate reliability.

15. Summary  Separate stratum of CHCs increases sheer volume of CHC providers (reduces RSE by 23.8%) increases sheer volume of CHC providers (reduces RSE by 23.8%) Indicates little bias in key characteristics in the traditional NAMCS Indicates little bias in key characteristics in the traditional NAMCS Allows for a consistency measure of item response for EMR use Allows for a consistency measure of item response for EMR use