Measurement for Improvement 18 March 2008 Mike Davidge

Measurement for Improvement The prologue What actually is the question?

Measurement for Improvement What are we trying to accomplish? How will we know that a change is an improvement? What changes can we make that will result in the improvements that we seek ? Model for improvement ActPlan StudyDo aims measurements interventions

Measurement for Improvement How does measurement help us? By showing the variation that lives within our data (and therefore our processes) By indicating whether the way we react to that variation is taking us in the right direction Source: Robert Lloyd IHI 2006

Measurement for Improvement The measurement journey

Measurement for Improvement Why are we measuring? Research Judgement Improvement The answer to this question will guide our entire measurement journey

Measurement for Improvement Measurement for judgement % waiting over 4 hours in A&E England by week 2003/04 target The week Trusts were measured for performance ratings

Measurement for Improvement Outpatient waiting times

Measurement for Improvement The 3 reasons for measurement Source: Robert Lloyd IHI 2006 Why is measurement for improvement different?

Measurement for Improvement Take two numbers

Measurement for Improvement Take two numbers 1229

Measurement for Improvement We have 2 quarterly data points - is this an improvement? Higher is better

Measurement for Improvement Are we assuming something like this?

Measurement for Improvement But it could be like this...

Measurement for Improvement Or this...

Measurement for Improvement Or this!

Measurement for Improvement Case study: Housing repairs Example courtesy of John Seddon in ‘Freedom from Command & Control’

Measurement for Improvement The scenario A Local Authority manages a large stock of housing They manage repairs using 2 measures  Budget  Time to repair ‘Time to repair’ is a government mandated BVPI (best value performance indicator) measured in %  % emergencies completed in 24 hrs  % urgents completed in 7 days  % routines completed in 28 days The LA achieving all BVPI’s

Measurement for Improvement Hunky dory? What does the customer want? Repair done quickly and properly How do we know we are delivering this? Let’s measure the end to end time – from request to job complete But there’s a problem – the IT system couldn’t provide the data – they only had the BVPI percentages so had to track each job manually

Measurement for Improvement SPC chart showing true performance

Measurement for Improvement Not so hunky dory … Special causes not different – system unstable Achieve targets by distorting the data & system  Ignore if tenant is out  Reclassifying emergencies as urgents  Splitting a job into several, each one opened & closed Got a problem? Have a management restructure!  Actually destabilised the system even more

Measurement for Improvement So what next? They understood they had a problem They looked at the process and discovered it was broke  40% ‘failure’ demand at the call centre  Tradesmen scheduling work to maximise earnings  Queuing to get the right materials Led to redesigning the process Within weeks all repairs completed within 8 days

Measurement for Improvement They didn’t know they had a problem until they measured the right thing in the right way!

Measurement for Improvement And that reminds me of … “What gets measured gets managed” “even when it’s pointless to measure and manage it and even if it harms the purpose of the organisation to do so.”

Measurement for Improvement The 3 reasons for measurement Source: Robert Lloyd IHI 2006 Why is measurement for improvement different?

Measurement for Improvement What are we trying to accomplish? How will we know that a change is an improvement? What changes can we make that will result in the improvements that we seek ? Model for improvement ActPlan StudyDo aims measurements interventions Source: IHI Save 1000 lives and 50,000 episodes of harm HSMR and Global Trigger Tool 6 Content Areas national

Measurement for Improvement What are we trying to accomplish? How will we know that a change is an improvement? What changes can we make that will result in the improvements that we seek ? Model for improvement ActPlan StudyDo aims measurements interventions Use the measures and interventions for each of the content areas local

Measurement for Improvement How to measure? CAR ollect – the right data in a sustainable way nalyse – the right measures presented in a helpful way eview – whether your changes are having the desired effect Start collecting the data and using it for local decision making

Measurement for Improvement The ‘How’ questions How many of your project leads can:  Collect the relevant data  Analyse data effectively (create run charts)  Review it regularly (interpret charts and make the right decisions) How much time do your Information analysts have to support them? How many of the content area measures can be obtained via your corporate IT systems?

Measurement for Improvement In summary Measures follow aims – clarity here means you know what 42 means Measurement for improvement is different from the other types of measurement we are used to Plot your data over time to understand current situation and track changes Ensure your project teams have the right measurement skills Re-align your Information department to provide longer term sustainability