1. Measuring changes in physician performance: Is it necessary to adjust for patient characteristics? Hoangmai H. Pham, MD, MPH AcademyHealth Annual Meeting June 2007

2. Paradigm Incentive programs may link rewards to improvement in performance instead of, or in addition to, absolute performance  Emphasis on improvement may be less onerous than emphasis on absolute performance for providers with fewer resources, or who treat patients less likely to adhere to recommendations

3. Question But may be true with absolute performance, do non- modifiable patient factors at baseline also affect changes in physicians’ performance over time?  If so, important to adjust for these factors

4. Patient-Level Measures of Delivery of Preventive Services Diabetes care  HbA1c measurement  eye exams  urine microalbumin measurement Screening mammography Influenza vaccination

5. Data Sources  2000-2001 Community Tracking Study Physician Survey 12,406 physicians in clinical care at least 20 hours/week Clustered sample, but nationally representative Primary care physicians (PCPs) over-sampled  Complete Medicare claims for years 2000-2002 for beneficiaries treated by a CTS physician

6. Physicians Primary care physician respondents to the 2000-2001 Community Tracking Study Physician Survey who:  Had >30 Medicare patients clinically eligible for a given performance measure, and  Billed for the plurality of these patients’ E&M visits throughout 2001-2002  2,636 PCPs

7. Factors Potentially Predictive of Change in Performance Patient panel factors: age, race (B/W/Other), Medicaid eligibility, comorbidities Area demographic factors: income and education Physician and practice factors: specialty, board certification, IMG status, practice type and size, practice revenue sources, HIT to generate reminders  more modifiable?

8. Analysis  Calculate performance rates for year 2001, then for year 2002, for each physician qualifying for a given preventive service measure  OLS models of log [OR 2002 performance]  independent variable = log [OR 2001 performance]  Stepwise addition of patient panel, and area demographic factors as covariates  Repeat analyses with stepwise addition of physician/practice, patient panel, then area factors  Expected effect on change in performance = (coefficient for log [OR 2001 performance]) - 1

9. OLS Transformation Allows Modeling of Unbounded Estimates of Change Transformed - Change in performance Raw Change in performance

10. Effect of 2001 Performance on Change in Performance – OLS coefficients (1) Preventive service Unadjusted + Patient panel factors + Area factors Eye exam-0.278-0.300-0.301 HbA1c-0.224-0.240-0.242 Albumin-0.467-0.478-0.490 Mammogram-0.228-0.273-0.267 Influenza-0.205-0.226-0.227 P<0.001 for all coefficients

11. Effect of Patient & Area Factors on Change in Performance – OLS Coefficients Preventive service % black (highest quartile) Mean age % female% Medicaid eligible (highest quartile) Median zip code income % adults w 12+ yrs education Eye exam0.037-0.014-0.001-0.238***-0.024*-0.002 HbA1c-0.049-0.0690.011-0.018-0.0340.008 Albumin-0.216-0.1220.007-0.773*-0.232**0.056* Mammogram0.0120.005----0.221**0.008-0.007* Influenza-0.78*0.005-0.003**-0.0110.015*-0.001 *p<0.05, **p<0.01, ***p<0.001

12. Effect of 2001 Performance on Change in Performance – OLS Coefficients (2) Preventive service Unadjusted + Physician and practice factors + Patient panel factors + Area factors Eye exam-0.278-0.318-0.329-0.327 HbA1c-0.224-0.271-0.278-0.281 Albumin-0.467-0.507-0.512-0.520 Mammogram-0.228-0.240-0.284-0.292 Influenza-0.205-0.219-0.232-0.234 P<0.001 for all coefficients

13. Caveats  Effects of patient factors may vary: For other types of performance measures (e.g., outcomes, structural measures) For other groups of providers (e.g., specialists, hospitals) For other levels of organization (e.g., medical group, health plan) Over the lifespan of an incentive program  Incentive programs will be limited in the level of complexity they can accommodate

14. Conclusions  Patient panel and area demographic factors at baseline have modest effects on changes in physicians’ performance on preventive services over time  More consistent effects for SES than for other factors  For most services, these effects not as pronounced as those of physician/practice characteristics  more closely reflects inherent quality of care delivery, and should not always be adjusted for?

15. Looking Forward – Other Questions  How do effects of patient factors vary: For other types of performance measures ? For other groups of providers ? For other levels of organization ? (I.e., at level of organization where systematic QI takes place) Over the lifespan of an incentive program ?  What are the relationships between patient factors, provider effort, and changes in performance scores? i.e., what is the risk of “over-adjustment”?  How parsimonious can we be in selecting factors for adjustment?