1. From information to improvement: evidence from UK & US 27 April 2010 Gwyn Bevan

2. Information  improvement  Three pathways & evidence from US Change Selection Reputation  Evidence from UK  Questions

3. Selection & Change Results Aware of differences Knowledge about performance Consumers able to choose Knowledge about process & results Management & Professionals Implement change Motivation Adapted from Berwick et al. 2003

4. Evidence from two systematic reviews  Limited evidence Industry of performance assessment Rigorous evalution  Cardiac Surgery Reporting System (CSRS)  6 other US systems  Change Weak evidence  Selection No evidence providers respond to threat of patients using information as consumers Ineffective in US  likely to work in UK? Sources: Marshall et al (2000) & Fung et al (2008)

5. CSRS (2001): CABG outliers good & poor outcomes Source: http://www.health.state.ny.us/statistics/diseases/cardiovascular/ RAMR Rates

6. The Clinton’s selection pathway (2001 CSRS data available)  sudden onset of chest pains & shortness of breath (September 2004) small hospital near home cardiologists @ Westchester Medical Center quadruple bypass @ Columbia-Presbyterian Source:http://www.nytimes.com/2004/09/06/health/06hosp.html?scp=1&sq= Clinton%20Surgery%20Puts%20Attention%20on%20Death%20Rate&st=cse

7. CSRS (2001): CABG outliers good & poor outcomes Source: http://www.health.state.ny.us/statistics/diseases/cardiovascular/ RAMR Rates Columbia Presbyterian Westchester Medical Center

8. CSRS: pathways  Change  mediocre / below-average performance: failed to use rich performance data  Selection  outliers with good / poor performance: no changes in market share  Reputation  outliers with poor performance: ‘naming & shaming’  galvanised to improve Source: Chassin (2002)

9. 3 rd Pathway: Reputation  Characteristics i.ranking ii.published: widely disseminated iii.easily understood: performance good or poor? iv.future report: performance improved?  Paradox information not used by patients to switch from poor to good hospitals managers of poor hospitals respond to repair perceived damage to public reputation not market share Sources: Hibbard et al. (2003, 2005)

10. Controlled experiment in Wisconsin ChangeSelectionReputation public-report  private-report  no-report   Summary indices adverse events Deaths & complications) General: surgery / nonsurgery Specific: cardiac, maternity, & hip/knee  System characteristics Ranking, easily understood, followed up Source: Hibbard et al. (2003, 2005)

11. Wisconsin: pathways  Change: Private report  little effort to improve quality  Selection: Public report  no anticipated changes in market share  no actual changes in market share  Reputation: Public report  significantly greater efforts to improve quality because of concerns over reputational damage Source: Hibbard et al. (2003, 2005)

12. Information  improvement  Three pathways & e vidence from the US  Evidence from UK Changing policy mix in England Natural experiment Evidence  Questions

13. Changing policy mix in England  1997-99: Selection  Change ‘internal market’  ‘third way’  2000-02: Reputation star ratings (2001 – 05)  2002 - : Reputation & Selection Reputation: star ratings  healthcheck (2006) Selection: FTs & ISTCs, PbR, patient choice, & World Class Commissioning

14. UK’s natural experiment ChangeReputationSelection England 1991-97 1997-99 2000-02 2002 -    Devolved countries 1999 -  Reward failure  Abolish p/p split

15. England: Selection  Change 1997 manifesto pledge  waiting lists Numbers waiting elective admissions (England) (‘000s)

16. Star rating: Reputation ‘naming & shaming’ zero stars  ‘devastating …hit right down to the workforce – whereas bad reports usually hit senior management upwards …nurses demanding changing rooms.. because being accosted in streets’ Source: Mannion et al (2005) 2001:the dirty dozen

17. England: Change  Reputation Numbers waiting elective admissions (England) (‘000s) Star ratings published

18. Funding of UK’s natural experiment

19. Natural experiment: Change & Reputation No/‘000 waiting > 6 months for elective hospital admission

20. Natural experiment: Change & Reputation Source: Connolly et al (2010) Funding and Performance of Healthcare Systems in the Four Countries of the UK before and after Devolution. The Nuffield Trust. % waiting list < 13 weeks for hospital admission (March 2008)

21. Natural experiment: Change & Reputation No/‘000 waiting > 3 months GP  1st outpatient appointment

22. England: Change  Reputation Change Reputation % Ambulance response times to life-threatening emergencies < 8 minutes (Target 75%) Source: Bevan & Hamblin (2009)

23. Natural experiment: Change & Reputation Star ratings published Target  Source: Bevan & Hamblin (2009) % Ambulance response times to life-threatening emergencies < 8 minutes

24. Barber (2007) Instruction to Deliver  Awful  adequate Command & control  public not satisfied  have to keep flogging the system  Adequate  good / great quasi market & consumer choice  innovation from self- sustaining systems

25. The impact of the NHS market: An overview of the literature*  No good evidence reforms produced beneficial outcomes classical economic theory predicts of markets  provider responsiveness to patients & purchasers  large-scale cost reduction  innovation in service provision  NHS incurs transaction costs of market without benefits * Brereton & Vasoodaven (2010) http://www.civitas.org.uk/nhs/download/Civitas_LiteratureReview_NHS_market_Feb10.pdf

26. System reform package & commmissioning No

27. Questions  Change Devolution  worse performance in devolved countries?  Selection Purchaser / provider split? ‘World Class Commissioning’? Patient choice?  Reputation Executive  democratic accountabilty?

28. References  Audit Commission & Healthcare Commission. Is the treatment is working?  Auditor General for Wales (2005) NHS waiting times in Wales. Volume 2 - Tackling the problem.  Bevan (2006) Setting Targets for Health Care Performance: lessons from a case study of the English NHS. National Institute Economic Review, 197, 67-79.

29. References  Bevan (2010) Approaches and impacts of different systems of assessing hospital performance. Journal of Comparative Policy Analysis, 2010, 12(1 & 2): 33 – 56  Bevan & Hamblin (2009) Hitting and missing targets by ambulance services for emergency calls: impacts of different systems of performance measurement within the UK. Journal of the Royal Statistical Society (A), 172(1):1-30.  Bevan & Hood (2006) Have targets improved performance in the English NHS? British Medical Journal, 332, 419-422.  Bevan & Hood (2006) What’s Measured is What Matters: Targets and Gaming in the English Public Health Care System. Public Administration 84, 3, 517-38.  Chassin (2002) Achieving and sustaining improved quality: Lessons from New York State and cardiac surgery. Health Affairs, 21(4), 40-51.

30. References  Fung et al (2008) Systematic review: The evidence that publishing patient care performance data improves quality of care. Annals of Internal Medicine, 148, 111- 123.  Hibbard (2008) What can we say about the impact of public reporting? Inconsistent execution yields variable results. Annals of Internal Medicine, 148, 160 - 161.  Hibbard et al (2003) Does publicizing hospital performance stimulate quality improvement efforts? Health Affairs, 22 (2), 84-94.  Hibbard et al (2005) Hospital performance reports: impact on quality, market share, and reputation. Health Affairs, 24(4), 1150-60.

31. References  House of Commons Health Committee (2010) Commissioning. (Fourth Report of Session 2009–10, Volume I) HC 268-I. London: The Stationery Office Limited.  Mannion et al (2005) Impact of star performance ratings in English acute hospital organisations. Journal of Health Services Research & Policy, 10 (1), 18–24.  Mannion & Goddard (2001) Impact of published clinical outcomes data: case study in NHS hospital organisations. BMJ, 323, 260 – 263.

32. References  Mannion & Goddard (2003) Public disclosure of comparative clinical performance data: lessons from the Scottish experience. Journal of Evaluation in Clinical Practice, 9 (2), 277–286  Marshall et al (2000) The Public release of performance data: What do we expect to gain? A review of the evidence. Journal of the American Medical Association, 283(14), 1866-1874.