Physician Performance Measures: Like It Or Not? Elizabeth A. McGlynn, Ph.D. Associate Director, RAND Health October 26, 2006

Many Purchasers Are Demanding Cost and Quality Metrics on Physicians High/Low High/High Low/Low Low/High Efficient Effective

Employers Are Using Effectiveness & Efficiency Metrics In Several Ways Public reporting – information to help people make more cost-conscious decisions Pay-for-performance – financial rewards to providers with better performance Tiering – differential co-payments tied to provider performance Selective contracting – networks defined based on provider performance

Current Approaches to Effectiveness Measurement “Leading indicators” One measure at a time Condition-specific aggregates/composites Multiple measures on the same population with the same health problem Multi-condition/comprehensive index Multiple measures across multiple conditions including preventive care

Current Approaches to Efficiency Measurement Episode-based Claims are aggregated to summarize all care for a chronic condition, acute event, prevention Measure is cost or resources used per episode Population-based Patient population is weighted by burden of disease Measure is cost per risk-adjusted patient year

Data Sources for Measuring Performance Available sources include: Administrative (claims) data Manual abstraction of medical records Surveys of patients and doctors Inspection of office practice Extraction of data from electronic medical records Board certification/Maintenance of certification Each of these sources has strengths and weaknesses No single source is adequate to address all questions

Most Existing Approaches to Measuring Physician Performance Use Claims Data Data are readily available and impose less burden on providers But they have some significant problems Generally available one payer at a time Information availability driven by the benefit package Financial incentives affect coding practices Physician practice patterns confounded with patient behavior Pressure to deliver answers driving widespread use of these methods

Some Challenges in Measuring Physician Performance

Physicians See Multiple [Different] Patients MD1 PT3 PT2 PT1 MD2 PT5 PT4 PT3 So, representing the variety of practice matters: Adequate information

A Market Basket of Indicators May Be Necessary to Reflect the Variety of Practice

What You Measure May Affect the Conclusions You Draw

Different Measures Will Produce Different Results for Some Physicians

Patients See Multiple Providers PT1 MD3 MD2 MD1 PT2 PT3 PT4 PT5 PT6 PT9 PT8 PT7 Hosp A Hosp B So, determining who is “responsible” matters Attribution

Three Classes of Attribution Rules Major MD 30% of visits for an episode 30% of visits overall Principal MD 50% of visits for an episode 50% of visits overall Anchor MD Physician who first saw patient for this problem (started the episode)

Comparison of Number of Physicians with Efficiency Scores By Method of Attribution

MDs Included In Efficiency Scoring Generate the Majority of Claims

Physicians Have Multiple Contracts Medicare MD3 MD2 MD1 Harvard Pilgrim HNE BCBSMA Tufts Fallon Medicaid Oxford So, putting the pieces together matters: Aggregation

Few Physicians Can Be Evaluated Using Single Indicators from One Payer

Physicians Practice in Different Systems So, understanding the organizational context matters: Fair comparisons

Little Routine Information Available on Physician Practice Setting Taking organizational context into account is challenging because of data limitations Using practice location may be misleading Shared space vs. shared practice Rationale for constructing scores at group level Increase sample size Demonstrate value of integrated medical groups Avoid scores at the physician level Relatively little known about within vs. between group variation

Putting Effectiveness & Efficiency Together

Little Relationship Found Between Effectiveness & Efficiency High Low

For Most Applications, Categorical Assignments Are Used To illustrate some of the challenges, we compare: Principal condition physician Quartiles Statistical testing Anchor physician In a recent analysis, we found that these rules agreed on assignment of episodes to physicians just 59% of the time

Data Set For This Analysis 8686 physicians who filed one or more claims with a health plan during a two year period Include only those with a score on either: Efficiency (minimum of 10 episodes) or Effectiveness (minimum 10 eligibility events) Used two methods for assigning categories Statistical testing (80% confidence interval) Quartile scores (combining middle quartiles to create the average group)

Principal Condition MD & Quartiles Unknown Quality Low Quality Average Quality High Quality Efficiency 5838 190 305 170 Best Efficiency 119 90 239 62 Average Efficiency 415 151 317 138 Worst Efficiency 164 91

Principal Condition MD & Statistical Testing Unknown Quality Worst Quality Average Quality Best Quality Efficiency 5838 95 457 113 Best Efficiency 29 14 87 21 Average Efficiency 664 128 874 196 Worst Efficiency 5 1 19 3

Anchor MD & Statistical Testing Unknown Quality Worst Quality Average Quality Best Quality Efficiency 5793 103 526 139 Best Efficiency 36 19 82 12 Average Efficiency 637 112 798 173 Worst Efficiency 10 4 31 9

Summary A number of methodological issues arise in creating efficiency and effectiveness scores at the physician level We need to better understand the implications of these methodological choices Because the data on which the scores are based were not intended for this purpose, feedback loops and data quality improvement are essential But, the world isn’t going to wait for us to get the methods perfect…