Comparison of Physician Rankings on Performance Quality Composites in the Care of Hypertensive Patients AcademyHealth, June 9, 2008 Washington, DC Weifeng Weng, Gerald K. Arnold, Eric S. Holmboe, Rebecca S. Lipner

ABIM and Maintenance of Certification (MOC)  ABIM certifies physicians in internal medicine and its subspecialties  Certification is time-limited: 10 year duration  Renew certificates through MOC program  MOC requires demonstration of –Professionalism –Lifelong Learning –Cognitive Expertise –Practice performance

Practice Improvement Module (PIM TM ) Performance Report Improvement Chart audit Patient survey Impact plan do study act Practice survey Based on Picker patient and CAHPS surveys Based on Wagner’s Chronic Care Model & IHI’s Idealized Office Design Evidence-based guidelines

Research Question  In P4P programs –Clinical measures dominate –Patient survey and Practice system survey measures used less frequently –Typically, rewards awarded by relative ranking –Typically not all three data streams used  Do physician performance rankings (and rewards) vary considerably when different combinations of the three data streams are used?

Methods  Physician database with patient-level data  Standardized composite scores for the three data streams (1) Chart audit (2) Patient survey (3) Practice systems  Super-composite scores: Combine composites  Examine changes in physician rankings

Physician and Patient Samples  659 Physicians –Mean Age: 44 (SD = 6.4), 26% female –61% general internists, 39% subspecialists (largely nephrologists and cardiologists) –29% in solo practice  Patients –Chart audit: 13,096 patients, age 18-75, 51% male –Patient survey: 14,913 patients, age 18-75, 53% male

Chart Audit Individual Measures Measures Physician levelICC N for R m =0.85 Mean (SD)Rho95% CI Blood pressure control0.52 (0.19) to LDL control0.55 (0.20) to Complete lipid profile0.82 (0.22) to Urine protein test0.81 (0.26) to Annual serum creatinine test0.88 (0.16) to DM co-morbidity documentation or screen test0.93 (0.15) to Aspirin therapy for eligible patients0.74 (0.25) to Smoking status and cessation advice and treatment0.96 (0.08) to Counseling for diet and physical activity0.89 (0.18) to Outcome variables are risk adjusted for co-morbidity conditions: BP control <130/80 for pts with dm or stroke co-morbidities, <140/90 for rest. LDL control <100 for pts with major risks, <130 for pts with other risks, <160 for the rest.

Patient Survey Individual Measures Measures Physician levelICC N for R m =0.85 Mean (SD)Rho95% CI Overall hypertension care0.88 (0.12) to Encouraging/answering questions0.83 (0.14) to Providing information on Medication side effects0.73 (0.17) to Providing information on foods to eat & avoid0.60 (0.17) to Providing information on taking medication properly0.86 (0.13) to

Practice System Individual Measures Measures (# of measures)MeanSDAlpha Information Management (27) Patient activation & communication (28) Access & communication with patients (7) Safety & efficiency (11) Practice team (8) Consultation & referral (4) Practice system survey of 89 questions

Distribution of Physician Performance Composite Scores C+P+S C+S C+PSystem (S) Patient (P) Chart (C)

Correlations among Composites ChartPatientSystemC+PC+SC+P+S Chart (C) Patient (P) System (S) C+P C+S C+P+S1.00

Percent who change rankings by more than one quartile* Baseline: Chart * One quartile counts for 164 rank positions Ranks better than chartRanks worse than chart

Percent who change rankings by more than two quartiles* * Two quartile counts for 329 rank positions Baseline: Chart Ranks better than chartRanks worse than chart

Examples of Extreme Discordance of Performance – more than three quartiles Ranks and (z scores) PhysicianChart (C)Patient (P)System (S)C+PC+SC+P+S (1.3)(-1.2)(1.1)(0.08)(2.4)(1.2) (-3.6)(0.23)(1.7)(-3.4)(-1.9)(-1.7) Rank 1 = Best; Rank 659 = Worst

Conclusions  Measuring multiple dimensions in the quality of patient care is complex –Very moderate correlations among three data streams –Rankings change considerably depending on combinations  A profile that incorporates more than one aspect of patient care tells a different story than any one of them alone

Limitations and Future Research  Self-report data for chart and system data  Participants are volunteers  Need more robust risk adjusters  Investigate other analytic approaches for combining individual measures into composites  Investigate stability of pass/fail decisions