Measurement for Improvement 27 October 2011 Measurement for Improvement Insert name of presentation on Master Slide Presenter: Mike Davidge

Measurement for improvement 2 Measurement for improvement “Measurement is for improvement not judgement.” D. Berwick Don Berwick – CEO of the IHI in America. Today is about using data for improvement, the majority of the NHS data that we collect is used for judgement and benchmarking ourselves against each other.

The measurement journey Research Improvement The measurement journey The rest of this talk is about the measurement journey. We’ll kick off with the type of measurement we want to use. What sort of measurement There are 3 types of measurement – which are we talking about? Let’s look at one we are very familiar with in our day to day work of managing our organisations – measurement for judgement. Talk through the 2 examples. I make no apologies for using examples from the past. Good examples are like good jokes, the old ones are always the best! Judgement Measurement for Improvement

The 3 reasons for measurement Source: Robert Lloyd IHI 2006 based on Solberg et all 1998 Why is measurement for improvement different? What are the characteristics then of this type of measurement? Bring up list of aspects and go through them. Now introduce (click) those for judgement. Let’s bring our clinical colleagues in now. They often work to a different measurement paradigm – measurement for research. Click to bring up characteristics. Why is M4I different? [don’t bring up table yet, leave to summarise]. Measuring the right things in the right way. Let’s deal with the right way first. Measurement for Improvement

The 3 reasons for measurement Why is measurement for improvement different? Source: Robert Lloyd IHI 2006 Measurement for Improvement

Measurement for improvement 6 Measurement for improvement It’s not audit Mike Davidge Don Berwick – CEO of the IHI in America. Today is about using data for improvement, the majority of the NHS data that we collect is used for judgement and benchmarking ourselves against each other.

% medicines reconciled in a Medical Admissions Unit Pharmacy included Form printed Letter from CDs Form piloted

Model for Improvement Remember this model from yesterday?

7 Steps to measurement 1 Decide aim 2 Choose measures 3 Define measures Slide 13 – 7 steps to measurement Say that measurement doesn’t just happen. It requires a well defined process. The diagram shows the 7 steps necessary to get measurement to work for you. Move to next slide For reference Step 1 – Decide your aim Step 2 – Choose your measures Step 3 – Confirm how to collect and display your data Step 4 – Collect your baseline data Step 5 – Analyse and present your data Step 6 – Meet to decide what it is telling you Step 7 – Repeat steps 4 to 6 each month or more frequently 7 Repeat steps 4-6 6 Review measures 4 Collect data 5 Analyse & present

Would you be able to describe your aim in a couple of sentences? Step 1 Decide aim 1 Decide aim 2 Choose measures 3 Define measures Slide 14 - Step 1: Decide your aim They have just done this as part of the groupwork but also link to Vision session 7 Repeat steps 4-6 6 Review measures 4 Collect data 5 Analyse & present Would you be able to describe your aim in a couple of sentences?

Step 2 Choose measures 2 Choose measures 1 Decide aim 2 Choose measures (late starts/finishes) Patient Experience & Outcomes Safety & Reliability Efficiency & Value Leadership & High Performing Teams Clinical Incidents Pain score % Theatre Utilisation Delays Training & Development Staff Survey Cancellations Turnaround Time Satisfaction Readmissions Staff absence % Patients Complication free in Recovery Exceptions from ‘ time out ’ checklist % correct equipment to hand Average time patient starved turnover 2 Choose measures There are two tools to help you choose measures Process Maps Driver Diagrams or cause and effect diagrams 3 Define measures Slide 15 - Step 2: Choose your measures Again they have done this as part of groupwork 7 Repeat steps 4-6 6 Review measures 4 Collect data 5 Analyse & present

Step 3 Define measures 1 Decide aim 2 Choose measures An operational definition is a description, in quantifiable terms, of what to measure and the steps to follow to measure it consistently Are we measuring the same thing? 3 Define measures Slide 15 - Step 2: Choose your measures Again they have done this as part of groupwork 7 Repeat steps 4-6 6 Review measures 4 Collect data 5 Analyse & present

Measure checklist 2 sides of A4 Page 1: Why important? Who owns? Definitions? Goals? Do your thinking up front and avoid mistakes Slide 21 – Define measure Explain that a clear definition is essential so everyone knows what the measure is and how to interpret the results To help you we have devised a measures checklist. It comes in two sections Section 1 – Define your measure Section 2 – Agree your measurement process Talk them through section 1 It is only a guide and not meant to be comprehensive Section on goals – do you want to set any?

Collect Decisions, decisions What - All patients or a sample? 1 Decide aim Decisions, decisions What - All patients or a sample? Who – what role? How – hospital system or audit? When – Real time or retrospective? 2 Choose measures 3 Define measures Slide 20 – Define measures (5 minutes for slides 20-22) Explain they will need to look in detail at each measure to decide how and what to collect, what analysis is needed etc. This means planning steps 4 to 6 for each measure 7 Repeat steps 4-6 6 Review measures 4 Collect data 5 Analyse & present

Analyse 1 Decide aim “The type of presentation you use has a crucial effect on how you react to data” Summarise Visualise 2 Choose measures 3 Define measures Slide 20 – Define measures (5 minutes for slides 20-22) Explain they will need to look in detail at each measure to decide how and what to collect, what analysis is needed etc. This means planning steps 4 to 6 for each measure 7 Repeat steps 4-6 6 Review measures 4 Collect data 5 Analyse & present

Complaints Here’s some data, what would you do with it?

How might we report these numbers? And what conclusions might we draw? Yearly total as numbers Quarterly average as numbers Quarterly average as bar chart

We have 2 quarterly data points - is this an improvement? Higher is better

Are we assuming something like this?

But it could be like this ...

Or this ...

Or this!

As a run chart

You will use these 99% of the time How am I doing? Where’s the problem?

Groupwork: How will you analyse? For the measure or measures you have just defined, complete the Analyse section of the checklist Note any issues you will need to resolve outside the workshop People you need to contact Specialist help you might need

Review measures It is a waste of time collecting and analysing your data if you don't take action on the results 1 Decide aim 2 Choose measures 3 Define measures Slide 18 – Group discussion Report & review (10 minutes for slides 18-19) Explain they are now going to decide where the review meeting(s) will happen As a whole group, use a flip chart (the facilitator may want to scribe at this point) and discuss how they will use the measures information: i) how it will be reviewed and communicated – e.g. in a theatre user group, newsletter, theatre display board ii) when will you review these – for each of the methods identified in (i) , how frequent will you do this Show an example of a review meeting as set out in slide 19. 7 Repeat steps 4-6 6 Review measures 4 Collect data 5 Analyse & present

7 Steps to measurement You may not get it right first time! You may need several attempts to get it right for you 1 Decide aim 2 Choose measures 3 Define measures Slide 16 – an iterative process Explain the PDSA symbol represents the several tests you might need to do to get it right 7 Repeat steps 4-6 6 Review measures 4 Collect data 5 Analyse & present