How to prove that you are being useful - a guide to system level metrics in urgent and emergency care Dr Ian Sturgess Partner, NHS Interim Management and Support Senior Clinical Lead, Emergency Care Intensive Support Team

A whole system perspective Preventative/ Predictive care Disease management Managed populations Alternatives to acute admission settings Alternative access for diagnosis Alternative settings for therapy Alternative sites for discharge Alternative sites for readmission Health Promotion General Practice & GP OOH Community Support Ambulance Service & GP OOH A+E MAU/SAU/ Short Stay Focus on CDM and more effective responses to urgent care needs – ACS condition management Clear operational performance framework and integrated in to primary care Improved integration with primary care responders Front load senior decision process incl primary care Redesign to left shift LOS Inpatient Wards Optimise ambulatory emergency care Information flow converting the unheralded to the heralded Discharge Process

“Quality begins with intent, which is fixed by management.” W. E. Deming, Out of the Crisis, p.5

System Level Improvement? How Do You Know? Be clear about your aim statement – with a definable system level improvement metric(s) – how much by when and how measured. Measure some key ‘Process’ metrics. Ignore balancing metrics at your peril! Use Statistical Process Control effectively

Clarify what your aim is. Promote healthy living and independence. Attendance and admission avoidance at times of acute care need. Reducing length of stay and readmissions by effective early supported discharge End of Life care All are amenable to having a definable ‘aim statement’ with a linked system level impact metric, a process metric and a balancing metric.

The Three Faces of Performance Measurement Aspect Improvement Accountability Research Aim Improvement of care Comparison, choice, reassurance, spur for change New knowledge Methods: Test Observability Tests are observable No test; merely evaluate current performance Test blinded or controlled tests Bias Accept consistent bias Measure and adjust to reduce bias Design to eliminate bias Sample Size “Just enough” data, small sequential samples Obtain 100% of available, relevant data “Just in case” data Flexibility of Hypothesis Hypothesis flexible, changes as learning takes place No hypothesis Fixed hypothesis Testing Strategy Sequential tests No tests One large test Determining if a Change is an Run charts or Shewhart control charts No change focus Hypothesis, statistical tests (t-test, F-test, chi square), p-vlaues Confidentiality of the Data Data used only by those involved with improvement Data available for public consumption and review Research subjects’ identities protected

Focus on the Vital Few! There are many things in life that are interesting to know. It far more important, however, to work on those things that are essential to quality than to spend time working on what is merely interesting! The challenge, therefore, is to be disciplined enough to focus on the essential (or vital few) things and set aside those things that might be interesting but trivial!

System Levels Example Nursing Services Macrosystem Divisions Mesosystem Macrosystem Microsystem Nursing Services Divisions Frontline Units Example System Levels Source: Bojestig, Jonkoping CC Sweden

Building a Cascade of Measures Outcome - system level eg admissions, death, harm, Institutionalisation etc L 1 System L 2 Process + Outcome Board & CEO L 3 Service Line L 4 Process (+ Outcome) Microsystems: Units, Depts L 5 Individual Physician & Patient Process Metrics Adapted from Lloyd & Caldwell

Appropriate Utilization of Resources at the End-of-Life Drivers Leading in Patient Safety 30th October-2nd November 2007Leading Improvement in Patient Safety 30th October – 2nd November 2007 Appropriate Utilization of Resources at the End-of-Life Utilization Measures (last six months of life) Hospital days ICU days Physician visits Hospital Care Coordination of Care Patient and Family Support Provider Supply Appropriate use of intensive hospital services (ICU care) Identification of patient severity and wishes with respect to end of life care Timely referral to palliative care / hospice options Identification of provider responsible for coordination Handoff management Execution of a shared treatment plan (all providers and patient and family) Assist patient and family to establish goals and intention Preparation of family caregivers to cope with exacerbation 24 hour access to appropriate services Drivers Secondary Drivers Lehigh SWVMC CAMC Memorial Hermann (Sepsis + CHF) Availability of providers Availability of resources NHS Institute for Innovation and Improvement and Institute fir Healthcare ImprovementNHS Institute for Innovation and Improvement and NHS Institute for Healthcare Improvement

Managing the Streams Identify the stream Number of patients Short stay Sick specialty Sick frail Complex Allocate early to teams skilled in that stream 250 Short stay – manage to the hour Maximise ambulatory care 200 Clarity of specialty criteria Specialty case management plan at Handover – no delays Green bed days vs red bed days Number of patients 150 Minimise handover Decompensation risk Early assertive management Green bed days vs red bed days 100 Complex needs – how much is decompensation? Detect early and design simple rules for discharge 50 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 Length of stay (days)

The road to ruin: Capacity plans and contracts based on average past activity Fail to deliver required activity to meet demand Income less than expected Guarantee waiting times beyond emergency and elective targets Increased staff overtime & waiting list initiatives Cost cutting initiatives Fail to account for variation in demand Reduces effective capacity Increased costs Increased variations in capacity variation in capacity +

Impact – Beds occupied – Total Objective – Hard Red Lines Aim – Reduce Acute beds occupied to SPC mean of 600 or less + reduce crude in-hospital mortality rate by 10% + a fall in SCHMI by 31st March 2012 Process measure – The whole system action plan etc etc ie holding the system to account not just the acute sector. Balancing – Deliver a decrease in Long term care ie more patients returning to live at home. No increase in 30 day re-admission rate

Trust NEL Admissions and Discharges Aim – Reduce emergency admissions – by 20 by 31st March 2012 Processes – RAT in A+E, 10 Care improvements, improved EoL care etc Balancing - prevent any increase in institutional care

Zero LOS Discharges - Trust Excl paediatrics, midwifery and obstetrics Aim – Increase zero LOS Process – deliver AEC Balancing – Reduce overall NEL admissions

2 midnights or less LOS Discharges - Trust Aim – Increase short stay discharges Process – deliver AEC + short stay review process Balancing – Reduce overall NEL admissions

In-Patients with LOS 14 days or more - Trust Aim – Reduce I/P with LOS 14 + to 75 or less by 31st March 2012 Process – Early identification of at risk group, CGA, early supported discharge schemes Balancing – no increase in institutional care – aim for a reduction in over 75s in Long term Care

Background to Statistical Process Control (SPC) Introduced by Walter Shewhart (Bell Telephone Laboratories 1924) The method was exported to Japan in the 1950s, where it was successfully applied in industry. SPC techniques “demonstrate the simplicity and power of control charts at guiding their users towards appropriate action for improvement”. 1 1. Mohammed MA, Cheng KK, Rouse A, Marshall T. Bristol, Shipman, and clinical governance: Shewhart's forgotten lessons. Lancet 2001; 357(9254):463-467

Can you identify the flaws in the following “dashboard?”

Given two different numbers, one will always be bigger than the other! Something very important! Last month This What action is appropriate?

What does this data tell us? All data needs to be put into context.

Variation in a system is normal 1 The variation is caused by factors that are inherent in the system over time They affect all outcomes This is ‘common cause’ variation or The causes are ‘unassignable’ Common cause variation can be reduced by tackling things that affect the process all the time 1. Roberts T. Understanding variation [Online] July 2005 [cited July 2009]; [17 Pages] Available from: http://www.evidencebasedpractice.org.uk/documents/presentations/spc_TEBPCJune2005.ppt

Some variation may not be normal 1 The factors are not present in the process all the time They do not affect everybody They arise because of specific circumstances This is ‘special’ or ‘assignable’ cause variation. 1. Roberts T. Understanding variation [Online] July 2005 [cited July 2009]; [17 Pages] Available from: http://www.evidencebasedpractice.org.uk/documents/presentations/spc_TEBPCJune2005.ppt

Two types of SPC chart If you want to compare different individuals, units or hospitals etc over a single time period, a ‘funnel chart’ may be helpful If you want to compare a single individual, unit or hospital over different time periods, a ‘time chart’ may be helpful

Anatomy of an SPC ‘Funnel Chart’ Likely Common Cause Variation Likely Special Cause Variation Practices with higher or lower than average admissions may be explained by a variety of factors Upper 3SD Confidence Interval Lower 3SD Confidence Interval Non Elective admissions / 100 Patients Upper 2SD Confidence Interval Lower 2SD Confidence Interval Mean List Size Example data for illustrative purposes only

The Improvement Process 10 20 30 40 50 60 70 80 90 100 week 1 week 3 week 5 week 7 week 9 week 11 week 13 week 15 week 17 week 19 week 21 week 23 week 25 week 27 Time Special causes present - unpredictable Process predictable Process improvement Service improvement is a science and should be tackled in a methodical way. You can’t just go and have a go, how do you know I the system is stable and ripe for an improvement. How do you know if what you did made a difference.

3 Dangers to Beware Of… Reacting to special cause variation by changing the process Ignoring special cause variation by assuming “its part of the process” Do not compare more than one process