1. Does transparency improve quality? lessons learnt from cardiac surgery BCIS meeting 2006 Ben Bridgewater SMUHT

2. History of cardiac surgical audit Cardiac surgery register since 1977

3. UK database since 1994 History of cardiac surgical audit

4. Cardiac surgery register since 1977 UK database since 1994 Dr Foster/The Times 2001 History of cardiac surgical audit

5. Cardiac surgery register since 1977 UK database since 1994 Dr Foster/The Times 2001 Named unit mortality SCTS 2001 History of cardiac surgical audit

6. Cardiac surgery register since 1997 UK database since 1994 Dr Foster/The Times 2001 Named unit mortality SCTS 2001 SCTS individual ‘standards’ 2003

7. Cardiac surgery register since 1997 UK database since 1994 Dr Foster/The Times 2001 Named unit mortality SCTS 2001 SCTS individual ‘standards’ 2003 Guardian named surgeon data 2005 –Freedom of Information Act History of cardiac surgical audit

8. Cardiac surgery register since 1977 UK database since 1994 Dr Foster/The Times 2001 Named unit mortality SCTS 2001 SCTS individual ‘standards’ 2003 Guardian named surgeon data 2005 Healthcare commission named surgeon data 2006

9. History of cardiac surgical audit Cardiac surgery register since 1997 UK database since 1994 Dr Foster/The Times 2001 Named unit mortality SCTS 2001 SCTS individual ‘standards’ 2004 Guardian named surgeon data 2005 Healthcare commission named surgeon data 2006

10. History of cardiac surgical audit Cardiac surgery register since 1997 UK database since 1994 Dr Foster/The Times 2001 Named unit mortality SCTS 2001 SCTS individual ‘standards’ 2004 Guardian named surgeon data 2005 Healthcare commission named surgeon data 2006

11. History of cardiac surgical audit Cardiac surgery register since 1997 UK database since 1994 Dr Foster/The Times 2001 Named unit mortality SCTS 2001 SCTS individual ‘standards’ 2004 Guardian named surgeon data 2005 Healthcare commission named surgeon data 2006

12. History of cardiac surgical audit Cardiac surgery register since 1997 UK database since 1994 Dr Foster/The Times 2001 Named unit mortality SCTS 2001 SCTS individual ‘standards’ 2004 Guardian named surgeon data 2005 Healthcare commission named surgeon data 2006

18. Issues Has public accountability improved quality?

19. Issues Has public accountability improved quality? Is there now a culture of ‘risk-averse’ behaviour?

20. Has public accountability improved quality?

21. Mortality significantly higher than average – Dr Foster Mortality significantly lower than average – Healthcare commission Has public accountability improved quality?

22. Risk adjusted mortality – National data – isolated CABG Increased predicted risk Decreased observed mortality

23. Hawthorn effect New York state database Pennsylvania report cards SCTS database Northern New England Cardiovascular study group VA database NW regional audit project 1997 to 2001 Public disclosure No disclosure

24. Collecting and using data improves the quality of outcomes

25. Why is public reporting important? Because it has driven data collection and use Clinicians managers support staff professional organisations

26. Is there now a culture of risk averse behaviour?

27. Newsnight survey of UK cardiac surgeons 2000 –80% surgeons in favour of public accountability –90% felt that high risk cases would be turned down –Only 6% felt that available algorithms adjusted appropriately for risk See also Burack 1999, Schneider and Epstein 1996, Narins 2005

28. Existing data Little ‘hard’ statistical data investigating the influence of public accountability on cardiac surgical practice NY experience suggests conflicting data –Hannan 1996 –Dranove 2003

29. Is there risk averse behaviour in the UK? Very difficult to measure surgical ‘turndowns’ If there was significant risk averse behaviour you would expect to see a decrease in the number of high risk cases coming to surgery Complex issues with respect to surgical case mix due to PCI developments

30. Northwest data 1997 to 2005 25,730 patients under 30 surgeons –Isolated CABG alone Observed and predicted mortality Number of low risk, high risk and very high patients each year 2 time periods –1997 to 2001 – prior to public disclosure –2001 to 2005 – post public disclosure

31. Results Significant decrease in observed mortality Significant increase in overall predicted mortality Significant decrease in risk adjusted mortality

32. Results Significant decrease in observed mortality Significant increase in overall predicted mortality Significant decrease in risk adjusted mortality

33. Is there now a culture of risk averse behaviour? No overall effect May be transient or individual effects Important that this is ‘mopped up’

34. Is there now a culture of risk averse behaviour? What is perceived by someone as risk- averse behaviour is perceived by another as good clinical decision making

35. Is there now a culture of risk averse behaviour? What is perceived by someone as risk- averse behaviour is perceived by another as good clinical decision making Transparency may have focussed the multidisciplinary team on optimising treatment strategies for individual patients

36. Risk adjustment ‘No model is perfect – some are useful’

37. Risk adjustment ‘No model is perfect – some are useful’ Need clarity around ‘fit for purpose’

38. Risk adjustment ‘No model is perfect – some are useful’ Need clarity around ‘fit for purpose’ Arguments about models can paralyse developments

39. Risk adjustment ‘No model is perfect – some are useful’ Need clarity around ‘fit for purpose’ Arguments about models can paralyse developments Model ‘drift’ –Calibration and weightings

40. Risk adjustment ‘No model is perfect – some are useful’ Need clarity around ‘fit for purpose’ Arguments about models can paralyse developments Model ‘drift’ –Calibration and weightings Progress will be too slow for some and too quick for others

42. Good Luck!