1. VA Quality Measurement: Is There a “Halo Effect?” Steven M.Asch MD MPH VA Sepulveda HSR&D COE RAND Health Geffen School of Medicine at UCLA

2. VA Performance Measurement Dates back to 1995 ~30 mostly chart-based process measures, varying evidence support Facility-based primary care sampling Feedback, public reporting Regional managers have financial incentives

3. VA Improves 12/13 Leading Indicators 1995- 2000 Jha NEJM 348:22 2003

4. VA Beats Medicare 12-1 in 2000 Jha NEJM 348:22 2003

5. What About Overall Quality? Leading indicator systems –Focus QI efforts, but –Vulnerable to gaming, resource misallocation What about overall quality? How does VA overall quality compare to the community?

6. Overall Process Quality Measured by QATools Literature reviews conducted to develop evidence-explicit indicators 5 Delphi expert panels convened to evaluate recommended indicators 348 clinically detailed process indicators for 26 clinical areas Example: Patients with pre-existing coronary disease with an LDL > 130mg/dl should begin diet or drug therapy within 3 months.

7. Example Clinical Areas General Medical Conditions BPH (4) Depression (14) Diabetes (13) Dyspepsia/PUD (8) Headache (21) Low back pain (6) Orthopedic conditions (10) Preventive care (35) STDs (26) Oncologic Conditions Colorectal cancer (12) CA pain and palliation (3) Prostate cancer (6) Cardiopulmonary Conditions Asthma (25) Congestive heart failure (36) COPD (20) Coronary artery disease (37) Hyperlipidemia (7) Hypertension (26) Pneumonia (5) Stroke/TIA (10)

8. Study Sites SEATTLE ORANGE COUNTY PHOENIX LITTLE ROCK INDIANAPOLIS CLEVELAND GREENVILLE MIAMI NEWARK BOSTON SYRACUSE LANSING VISN 22 VISN 11

9. % Receiving Indicated Care

11. Is There A “Halo Effect?” Does the VA advantage exist only in specific processes subject to VA performance management? Does it extend to clinically related conditions or areas? Does it extend to clinically unrelated areas? P. Gaugin-”Self Portrait with a Halo”

12. 3 Mutually Exclusive Summary Scores  In VA measurement set (26 indicators)  Not in VA measurement set, but in same condition (178 indicators)  Neither (167 indicators)

13. Adjusted % Receiving Indicated Care VA Measures Neither

14. Conclusions VA is improving on leading indicators VA beats Medicare on leading indicators VA beats community on broader measures “Halo effect” of performance measurement diminishes with clinical distance

15. Wins: Performance measurement Losses: ????? Ties: Acute care, eye exams, COPD Errors: Hoping for too big a halo

16. Questions

17. Adjusted RR of Indicated Care 1.22* 2.04* 2.74*

18. Sampling: Men>35 with 2 Visits/yr CommunityVA SourceRandom digit dial Admin data Response rate48%97% N992597 Study period1/98-12/99 Sites12 cities26 facilities