1. Physician Performance Measures: Like It Or Not?
Elizabeth A. McGlynn, Ph.D.
Associate Director, RAND Health
October 26, 2006

2. Many Purchasers Are Demanding Cost and Quality Metrics on Physicians
High/Low
High/High
Low/Low
Low/High
Efficient
Effective

3. Employers Are Using Effectiveness & Efficiency Metrics In Several Ways
Public reporting – information to help people make more cost-conscious decisions
Pay-for-performance – financial rewards to providers with better performance
Tiering – differential co-payments tied to provider performance
Selective contracting – networks defined based on provider performance

4. Current Approaches to Effectiveness Measurement
“Leading indicators”
One measure at a time
Condition-specific aggregates/composites
Multiple measures on the same population with the same health problem
Multi-condition/comprehensive index
Multiple measures across multiple conditions including preventive care

5. Current Approaches to Efficiency Measurement
Episode-based
Claims are aggregated to summarize all care for a chronic condition, acute event, prevention
Measure is cost or resources used per episode
Population-based
Patient population is weighted by burden of disease
Measure is cost per risk-adjusted patient year

6. Data Sources for Measuring Performance
Available sources include:
Administrative (claims) data
Manual abstraction of medical records
Surveys of patients and doctors
Inspection of office practice
Extraction of data from electronic medical records
Board certification/Maintenance of certification
Each of these sources has strengths and weaknesses
No single source is adequate to address all questions

7. Most Existing Approaches to Measuring Physician Performance Use Claims Data
Data are readily available and impose less burden on providers
But they have some significant problems
Generally available one payer at a time
Information availability driven by the benefit package
Financial incentives affect coding practices
Physician practice patterns confounded with patient behavior
Pressure to deliver answers driving widespread use of these methods

8. Some Challenges in Measuring Physician Performance

9. Physicians See Multiple [Different] Patients
MD1
PT3
PT2
PT1
MD2
PT5
PT4
PT3
So, representing the variety of practice matters:
Adequate information

10. A Market Basket of Indicators May Be Necessary to Reflect the Variety of Practice

11. What You Measure May Affect the Conclusions You Draw

12. Different Measures Will Produce Different Results for Some Physicians

13. Patients See Multiple Providers
PT1
MD3
MD2
MD1
PT2
PT3
PT4
PT5
PT6
PT9
PT8
PT7
Hosp A
Hosp B
So, determining who is “responsible” matters
Attribution

14. Three Classes of Attribution Rules
Major MD
30% of visits for an episode
30% of visits overall
Principal MD
50% of visits for an episode
50% of visits overall
Anchor MD
Physician who first saw patient for this problem (started the episode)

15. Comparison of Number of Physicians with Efficiency Scores By Method of Attribution

16. MDs Included In Efficiency Scoring Generate the Majority of Claims

17. Physicians Have Multiple Contracts
Medicare
MD3
MD2
MD1
Harvard
Pilgrim
HNE
BCBSMA
Tufts
Fallon
Medicaid
Oxford
So, putting the pieces together matters:
Aggregation

18. Few Physicians Can Be Evaluated Using Single Indicators from One Payer

19. Physicians Practice in Different Systems
So, understanding the organizational context matters:
Fair comparisons

20. Little Routine Information Available on Physician Practice Setting
Taking organizational context into account is challenging because of data limitations
Using practice location may be misleading
Shared space vs. shared practice
Rationale for constructing scores at group level
Increase sample size
Demonstrate value of integrated medical groups
Avoid scores at the physician level
Relatively little known about within vs. between group variation

21. Putting Effectiveness & Efficiency Together

22. Little Relationship Found Between Effectiveness & Efficiency
High
Low

23. For Most Applications, Categorical Assignments Are Used
To illustrate some of the challenges, we compare:
Principal condition physician
Quartiles
Statistical testing
Anchor physician
In a recent analysis, we found that these rules agreed on assignment of episodes to physicians just 59% of the time

24. Data Set For This Analysis
8686 physicians who filed one or more claims with a health plan during a two year period
Include only those with a score on either:
Efficiency (minimum of 10 episodes) or
Effectiveness (minimum 10 eligibility events)
Used two methods for assigning categories
Statistical testing (80% confidence interval)
Quartile scores (combining middle quartiles to create the average group)

25. Principal Condition MD & Quartiles
Unknown
Quality
Low Quality
Average Quality
High Quality
Efficiency
5838
190
305
170
Best Efficiency
119 90
239 62
Average Efficiency
415
151
317
138
Worst Efficiency
164 91

26. Principal Condition MD & Statistical Testing
Unknown
Quality
Worst Quality
Average Quality
Best Quality
Efficiency
5838 95
457
113
Best Efficiency 29 14 87 21
Average Efficiency
664
128
874
196
Worst Efficiency 5 1 19 3

27. Anchor MD & Statistical Testing
Unknown
Quality
Worst Quality
Average Quality
Best Quality
Efficiency
5793
103
526
139
Best Efficiency 36 19 82 12
Average Efficiency
637
112
798
173
Worst Efficiency 10 4 31 9

28. Summary
A number of methodological issues arise in creating efficiency and effectiveness scores at the physician level
We need to better understand the implications of these methodological choices
Because the data on which the scores are based were not intended for this purpose, feedback loops and data quality improvement are essential
But, the world isn’t going to wait for us to get the methods perfect…