Measuring changes in physician performance: Is it necessary to adjust for patient characteristics? Hoangmai H. Pham, MD, MPH AcademyHealth Annual Meeting June 2007

Paradigm Incentive programs may link rewards to improvement in performance instead of, or in addition to, absolute performance  Emphasis on improvement may be less onerous than emphasis on absolute performance for providers with fewer resources, or who treat patients less likely to adhere to recommendations

Question But may be true with absolute performance, do non- modifiable patient factors at baseline also affect changes in physicians’ performance over time?  If so, important to adjust for these factors

Patient-Level Measures of Delivery of Preventive Services Diabetes care  HbA1c measurement  eye exams  urine microalbumin measurement Screening mammography Influenza vaccination

Data Sources  Community Tracking Study Physician Survey 12,406 physicians in clinical care at least 20 hours/week Clustered sample, but nationally representative Primary care physicians (PCPs) over-sampled  Complete Medicare claims for years for beneficiaries treated by a CTS physician

Physicians Primary care physician respondents to the Community Tracking Study Physician Survey who:  Had >30 Medicare patients clinically eligible for a given performance measure, and  Billed for the plurality of these patients’ E&M visits throughout  2,636 PCPs

Factors Potentially Predictive of Change in Performance Patient panel factors: age, race (B/W/Other), Medicaid eligibility, comorbidities Area demographic factors: income and education Physician and practice factors: specialty, board certification, IMG status, practice type and size, practice revenue sources, HIT to generate reminders  more modifiable?

Analysis  Calculate performance rates for year 2001, then for year 2002, for each physician qualifying for a given preventive service measure  OLS models of log [OR 2002 performance]  independent variable = log [OR 2001 performance]  Stepwise addition of patient panel, and area demographic factors as covariates  Repeat analyses with stepwise addition of physician/practice, patient panel, then area factors  Expected effect on change in performance = (coefficient for log [OR 2001 performance]) - 1

OLS Transformation Allows Modeling of Unbounded Estimates of Change Transformed - Change in performance Raw Change in performance

Effect of 2001 Performance on Change in Performance – OLS coefficients (1) Preventive service Unadjusted + Patient panel factors + Area factors Eye exam HbA1c Albumin Mammogram Influenza P<0.001 for all coefficients

Effect of Patient & Area Factors on Change in Performance – OLS Coefficients Preventive service % black (highest quartile) Mean age % female% Medicaid eligible (highest quartile) Median zip code income % adults w 12+ yrs education Eye exam ***-0.024* HbA1c Albumin *-0.232**0.056* Mammogram ** * Influenza-0.78* ** * *p<0.05, **p<0.01, ***p<0.001

Effect of 2001 Performance on Change in Performance – OLS Coefficients (2) Preventive service Unadjusted + Physician and practice factors + Patient panel factors + Area factors Eye exam HbA1c Albumin Mammogram Influenza P<0.001 for all coefficients

Caveats  Effects of patient factors may vary: For other types of performance measures (e.g., outcomes, structural measures) For other groups of providers (e.g., specialists, hospitals) For other levels of organization (e.g., medical group, health plan) Over the lifespan of an incentive program  Incentive programs will be limited in the level of complexity they can accommodate

Conclusions  Patient panel and area demographic factors at baseline have modest effects on changes in physicians’ performance on preventive services over time  More consistent effects for SES than for other factors  For most services, these effects not as pronounced as those of physician/practice characteristics  more closely reflects inherent quality of care delivery, and should not always be adjusted for?

Looking Forward – Other Questions  How do effects of patient factors vary: For other types of performance measures ? For other groups of providers ? For other levels of organization ? (I.e., at level of organization where systematic QI takes place) Over the lifespan of an incentive program ?  What are the relationships between patient factors, provider effort, and changes in performance scores? i.e., what is the risk of “over-adjustment”?  How parsimonious can we be in selecting factors for adjustment?