1. How to prove that you are being useful - a guide to system level metrics in urgent and emergency care
Dr Ian Sturgess
Partner,
NHS Interim Management and Support
Senior Clinical Lead,
Emergency Care Intensive Support Team

2. A whole system perspective
Preventative/
Predictive care
Disease management
Managed populations
Alternatives to acute admission settings
Alternative access for diagnosis
Alternative settings for therapy
Alternative sites for discharge
Alternative sites for readmission
Health Promotion
General Practice & GP OOH
Community Support
Ambulance Service & GP OOH
A+E
MAU/SAU/
Short Stay
Focus on CDM and more effective responses to urgent care needs – ACS condition management
Clear operational performance framework and integrated in to primary care
Improved integration with primary care responders
Front load senior decision process incl primary care
Redesign to left shift LOS
Inpatient
Wards
Optimise ambulatory emergency care
Information flow converting the unheralded to the heralded
Discharge Process

3. “Quality begins with intent, which is fixed by management.”
W. E. Deming, Out of the Crisis, p.5

4. System Level Improvement? How Do You Know?
Be clear about your aim statement – with a definable system level improvement metric(s) – how much by when and how measured.
Measure some key ‘Process’ metrics.
Ignore balancing metrics at your peril!
Use Statistical Process Control effectively

5. Clarify what your aim is.
Promote healthy living and independence.
Attendance and admission avoidance at times of acute care need.
Reducing length of stay and readmissions by effective early supported discharge
End of Life care
All are amenable to having a definable ‘aim statement’ with a linked system level impact metric, a process metric and a balancing metric.

6. The Three Faces of Performance Measurement
Aspect
Improvement
Accountability
Research
Aim
Improvement of care
Comparison, choice, reassurance, spur for change
New knowledge
Methods:
Test Observability
Tests are observable
No test; merely evaluate current performance
Test blinded or controlled tests
Bias
Accept consistent bias
Measure and adjust to reduce bias
Design to eliminate bias
Sample Size
“Just enough” data, small sequential samples
Obtain 100% of available, relevant data
“Just in case” data
Flexibility of
Hypothesis
Hypothesis flexible, changes as learning takes place
No hypothesis
Fixed hypothesis
Testing Strategy
Sequential tests
No tests
One large test
Determining if a
Change is an
Run charts or Shewhart control charts
No change focus
Hypothesis, statistical tests (t-test, F-test, chi square),
p-vlaues
Confidentiality of
the Data
Data used only by those involved with improvement
Data available for public consumption and review
Research subjects’ identities protected

7. Focus on the Vital Few!
There are many things in life that are interesting to know. It far more important, however, to work on those things that are essential to quality than to spend time working on what is merely interesting!
The challenge, therefore, is to be disciplined enough to focus on the essential (or vital few) things and set aside those things that might be interesting but trivial!

8. System Levels Example Nursing Services Macrosystem Divisions
Mesosystem
Macrosystem
Microsystem
Nursing
Services
Divisions
Frontline
Units
Example
System Levels
Source: Bojestig, Jonkoping CC Sweden

9. Building a Cascade of Measures
Outcome - system level eg admissions,
death, harm, Institutionalisation etc
L 1
System
L 2
Process + Outcome
Board & CEO
L 3
Service Line
L 4
Process (+ Outcome)
Microsystems: Units, Depts
L 5
Individual
Physician & Patient
Process Metrics
Adapted from Lloyd & Caldwell

10. Appropriate Utilization of Resources at the End-of-Life Drivers
Leading in Patient Safety 30th October-2nd November 2007Leading Improvement in Patient Safety
30th October – 2nd November 2007
Appropriate Utilization of Resources at the End-of-Life
Utilization Measures (last six months of life)
Hospital days
ICU days
Physician visits
Hospital Care
Coordination of Care
Patient and Family Support
Provider Supply
Appropriate use of intensive hospital services (ICU care)
Identification of patient severity and wishes with respect to end of life care
Timely referral to palliative care / hospice options
Identification of provider responsible for coordination
Handoff management
Execution of a shared treatment plan (all providers and patient and family)
Assist patient and family to establish goals and intention
Preparation of family caregivers to cope with exacerbation
24 hour access to appropriate services
Drivers
Secondary Drivers
Lehigh
SWVMC
CAMC
Memorial Hermann (Sepsis + CHF)
Availability of providers
Availability of resources
NHS Institute for Innovation and Improvement and Institute fir Healthcare ImprovementNHS Institute for Innovation and Improvement and NHS Institute for Healthcare Improvement

11. Managing the Streams Identify the stream Number of patients
Short stay Sick specialty Sick frail Complex
Allocate early to teams skilled in that stream
250
Short stay – manage to the hour
Maximise ambulatory care
200
Clarity of specialty criteria
Specialty case management plan at
Handover – no delays
Green bed days vs red bed days
Number of patients
150
Minimise handover
Decompensation risk
Early assertive management
Green bed days vs red bed days
100
Complex needs – how
much is decompensation?
Detect early and design
simple rules for discharge 50 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59
Length of stay (days)

12. The road to ruin: Capacity plans and contracts
based on average past activity
Fail to deliver
required activity
to meet demand
Income less
than expected
Guarantee waiting
times beyond emergency
and elective targets
Increased staff overtime
& waiting list initiatives
Cost cutting
initiatives
Fail to account for
variation in demand
Reduces
effective
capacity
Increased costs
Increased
variations
in capacity
variation in capacity +

13. Impact – Beds occupied – Total Objective – Hard Red Lines
Aim – Reduce Acute beds occupied to SPC mean of 600 or less + reduce crude in-hospital mortality rate by 10% + a fall in SCHMI by 31st March 2012
Process measure – The whole system action plan etc etc ie holding the system to account not just the acute sector.
Balancing – Deliver a decrease in Long term care ie more patients returning to live at home. No increase in 30 day re-admission rate

14. Trust NEL Admissions and Discharges
Aim – Reduce emergency admissions – by 20 by 31st March 2012
Processes – RAT in A+E, 10 Care improvements, improved EoL care etc
Balancing - prevent any increase in institutional care

15. Zero LOS Discharges - Trust
Excl paediatrics, midwifery and obstetrics
Aim – Increase zero LOS
Process – deliver AEC
Balancing – Reduce overall NEL admissions

16. 2 midnights or less LOS Discharges - Trust
Aim – Increase short stay discharges
Process – deliver AEC + short stay review process
Balancing – Reduce overall NEL admissions

17. In-Patients with LOS 14 days or more - Trust
Aim – Reduce I/P with LOS 14 + to 75 or less by 31st March 2012
Process – Early identification of at risk group, CGA, early supported discharge schemes
Balancing – no increase in institutional care – aim for a reduction in over 75s in Long term Care

18. Background to Statistical Process Control (SPC)
Introduced by Walter Shewhart (Bell Telephone Laboratories 1924)
The method was exported to Japan in the 1950s, where it was successfully applied in industry.
SPC techniques “demonstrate the simplicity and power of control charts at guiding their users towards appropriate action for improvement”. 1
1. Mohammed MA, Cheng KK, Rouse A, Marshall T. Bristol, Shipman, and clinical governance: Shewhart's forgotten lessons. Lancet 2001; 357(9254):

19. Can you identify the flaws in the following “dashboard?”

21. Given two different numbers, one will always be bigger than the other!
Something
very important!
Last
month
This
What action
is appropriate?

22. What does this data tell us?
All data needs to be put into context.

23. Variation in a system is normal 1
The variation is caused by factors that are inherent in the system over time
They affect all outcomes
This is ‘common cause’ variation or
The causes are ‘unassignable’
Common cause variation can be reduced by tackling things that affect the process all the time
1. Roberts T. Understanding variation [Online] July 2005 [cited July 2009]; [17 Pages] Available from:

24. Some variation may not be normal 1
The factors are not present in the process all the time
They do not affect everybody
They arise because of specific circumstances
This is ‘special’ or ‘assignable’ cause variation.
1. Roberts T. Understanding variation [Online] July 2005 [cited July 2009]; [17 Pages] Available from:

25. Two types of SPC chart
If you want to compare different individuals, units or hospitals etc over a single time period, a ‘funnel chart’ may be helpful
If you want to compare a single individual, unit or hospital over different time periods, a ‘time chart’ may be helpful

26. Anatomy of an SPC ‘Funnel Chart’
Likely Common Cause
Variation
Likely Special Cause Variation
Practices with higher or lower than
average admissions may be explained
by a variety of factors
Upper 3SD Confidence Interval
Lower 3SD Confidence Interval
Non Elective admissions / 100 Patients
Upper 2SD Confidence Interval
Lower 2SD Confidence Interval
Mean
List Size
Example data for illustrative purposes only

27. The Improvement Process10 20 30 40 50 60 70 80 90
100
week 1
week 3
week 5
week 7
week 9
week 11
week 13
week 15
week 17
week 19
week 21
week 23
week 25
week 27
Time
Special causes present -
unpredictable
Process predictable
Process improvement
Service improvement is a science and should be tackled in a methodical way. You can’t just go and have a go, how do you know I the system is stable and ripe for an improvement. How do you know if what you did made a difference.

28. 3 Dangers to Beware Of…
Reacting to special cause variation by changing the process
Ignoring special cause variation by assuming “its part of the
process”
Do not compare more than one process