1. Informing for Improvement Report cards, performance measures and quality indicators – why bother? Richard Hamblin Center for Health Studies Group Health Cooperative of Puget Sound

2. Why publish report cards? How do we encourage consumerism? Well, one thing you do is you make sure people understand their options… –G.W Bush, Minneapolis, August 22, 2006 Our choice information pages …help you make a choice that best suits you.” –UK Department of Health choice website

3. But… “when this information is published only a minority are aware of it; of those, most do not understand it, trust it or use it…” –Gwyn Bevan 2005

4. My objectives Understand this contradiction Group health provides a unique population –Chronic conditions –My Group Health What is the likely use of such information? What does this imply about presentation and dissemination? More enthusiastic users? }

5. How might it work Pressure on providers Patients as consumers – choosing the best providers Informed and empowered patients – getting the best from their providers Which of these two is most likely to work?

6. Survey Written survey tool mailed to 600 respondents My GH non-user no chronic condition n = 150 My GH user no chronic condition n = 150 My GH user diabetes n = 150 My GH non-user diabetes n = 150 My GH non-user n = 300 My GH user n = 300 Diabetes n = 300 no chronic condition n = 300

7. Survey 22 written questions to test pre-determined hypotheses –Patients with long-term conditions are more interested –My Group Health users are more interested –Less satisfied patients are more interested –Information more likely to be used for boosting confidence than changing provider Data collection February to April 2007

8. Responses Ethnicity similar across groups MyGH users wealthier and better educated Diabetics generally lower income

9. Lots of interest 11 point scale (0-10) used to report interest MeanMedian Weighted total7.78 My Group Health User7.98 My Group Health Non User7.49 Diabetes7.49 No chronic condition7.99

10. But little prior knowledge or use No meaningful differences between the groups

11. What do I do with this then? the continuum of potential use Active and immediate None Passive/ uncertain Prospective Active Consumerist Active informed patient “I would not use” “Understand better how my doctor rates” “Choose doctor when entering health plan” “Boost confidence to discuss things I don’t understand or agree with.” “Change doctor inside current plan”

12. Proportion of respondents citing different uses for data (forced choice of one use) For all groups “understand better” is a significantly greater proportion than any other Use versus self-reported interest does not vary (except for the would not use group)

13. Active users only Surprising result – expected a greater proportion in the active section Active uses only – significant results (p<0.05) “Boost confidence” a more common response, but very small numbers 922Diabetes* 821MyGH* Change doctorBoost confidencen

14. Key result Just because people are interested in the information doesn't mean that they are going to use it to make choices tomorrow

15. How do patients want data presented? Individual measures vs an overall service rating Individual physicians vs hospital/practice Benchmarks/expected performance vs rankings * (p<0.01) 76.1%*57.8%45.9%Total BenchmarksIndividual physicians Individual measures % of responders

16. How does satisfaction affect interest? Are satisfied patients less interested in having information about quality? Test 1: Correlation of interest scale with CAHPS satisfaction scale 0.001No CC 0.001Diabetes 0.011NMyGH 0.002MyGH 0.000Total r2r2 Correlation between interest and satisfaction ratings

17. How does satisfaction affect interest? Test 2: Comparison of interest scale with specific CAHPS attributes of patient- focused care Mean interest scores by regularity of CAHPS attributes 7.97.5Time 7.57.7Respects 7.77.6Listens 7.7 Explains Not alwaysAlways

18. CONCLUSIONS Many prior expectations were wrong Interest in performance information uniformly high Prior knowledge and use of report cards uniformly low Contrary to expectations, little difference between groups No relationship between satisfaction with doctor and desire for information about quality

19. CONCLUSIONS The importance of “better understand” Doesn't have to be “used” to be “useful” Possible interpretations –Information as a resource Reassurance Accountability –Understanding as a precursor of action

20. CONCLUSIONS Large majority in all groups favoured comparisons with benchmarks rather than ranked performance Consistent with the “use” finding

21. Policy implications – consumerisms’ weaknesses and an alternative approach Publishing information about quality will not necessarily encourage choice Not because the data are badly presented but because most patients don’t prioritise choice A different goal of trust and understanding of quality of service

22. Policy implications – unresolved next steps Balancing measurement of different things: –Clinical process –outcome –experience How to set external benchmarks? (e.g. NQF process) How to determine the “normal range”? (e.g. outliers, composites)