“Measures that matter” Marie-Claire Leese Improvement Advisor marie-claire.leese@gov.scot Twitter: @MCL40 @scotgovCYPIC Intro – Hello my name is and intro everyone else Today you will leave here with an understanding of The critical importance of the relationship between your improvement aim and your measures How the model for improvement hinges on a good measurement plan Types of measures Live examples from colleagues who have created a meaningful measurement plan A song that will be ringing in your ears for the next 24 hours – who said data and measures were boring? Health & Safety – Fire Drill Toilets Tweet – Using #CYPIC Link to the session via ‘insert title of app’ “Measures that matter”

Measures Who is just beginning to use Imp methodology and embarking on a measure plan Who has projects running with a measurement plan? Who is comfortable creating a measurement plan?

Why are you measuring? Research? Improvement? Accountability? Measures can bring out all sorts of emotions depending on why you are measuring. Our most common measures are for accountability – anxiety, fear of being judged For the purposes of today we are talking about improvement. These can bring feelings of positivity, progress, “that we are doing something” If you link your improvement work to day to day work that you have accountability for it can be very powerful and enabling Improvement? The answer to this question will guide your entire quality measurement journey!

Looking at data can help you see the “why” Why are you focussing on this …

Strictly spoiler alert!!! Before we go on I want to give you a key message.

Looking at data over time is critical Looking at data over time is critical. It builds up a picture and tells a story. The dangers of snap shot are highlighted in the next set of pictures. Sat evening Susan and Kevin start the Tango More of the dance – starts to build up the picture of what type of dance this is – sharp movement, facial expressions – tells us more Kevins happy but Susan not looking too happy – It looks like Tess is consloing her?! Snap shot picture post dance – not looking good. Susan looks unhappy, Kevin slightly unsure of what has been said The results are in – they are through. Thank goodness we looked at this data over time and didn’t stop collecting at the last data point! We could have picked up the newspaper and just seen the last picture but we would have missed the beauty of the dance and the anticipation of the results.

The Model For Improvement The Thinking Part The Doing Part The model is split into two parts. The first being the thinking part and the second being the doing part. We will now take you through the two distinct parts in some detail with practical exercises to get everyone used to using the model and be able to describe and direct others in terms of our support function as a Unit. .

Who is this? Who can name a famous song of hers that I might want to share with you to help you bring some joy to measurement?

How will I know that change is an improvement?

We have all agreed that we have data to look at – go and look at it and be curious about it. Then form a good question to understand “WHY” something is happening. Create an aim around how you will answer that question. Caution – anything you are going to measure must be connected to a wider question. In improvement terms this is something we spend a lot of out time on because …

In addition if you don’t set a clear aim, you will lose clarity of purpose

The components of a good aim…

Improvement is not just about measurement However… without measurement you will never be able to answer the question! Always focus on how will I know if my prediction was correct? Did that intervention work?

If you don’t create a measurement plan you will fly blind If you don’t create a measurement plan you will fly blind. Things may get better or worse but you won’t be able to say if it was because of something you did.

How do you know it was what you did that made the difference How do you know it was what you did that made the difference? You want to say with a degree of confidence that it was because of your change idea. It might well be that the differences in %’s here are just about the particular cohorts of pupils. Always be curious and always know your population! By only looking at the data on an annual basis it will take a long time to tell if you are improving things.

Let’s talk about measures! Don’t ask “what will we measure?” Ask “what do we need to know?” then see out if you can measure it. We won’t ask what will we measure What will we ask? What do we need to know and then How will I know What do we need to know????

Balancing Measures Outcome Measures Process Measures Is the young person getting the right outcome? Is the system working as planned? What about the bigger picture? Balancing Measures Looking at the system from different dimensions. Does improving one thing cause problems elsewhere? Outcome Measures Are we making things better? Are we on track to achieve our Aim? Process Measures Are we doing the right things at the right time, every time? Is the process reliable? Process measures are about the how The outcomes measures are about the impact, the so what The balancing measures are the unintended consequences and can be negative or positive Dave Williams, IHI

Create a measurement plan that matters If your aim was an increase in attainment of Pulling the data together in a dashboard format

Your measures are there to keep you on track Your measures are there to keep you on track. They are your feedback mechanism to tell you if what you are doing is making a difference. THEY BRING SIMPLICITY AND CLARITY TO A COMPLEX PROBLEM

Sometimes you measure things that are unrelated to the aim- interesting – but unrelated to the aim. This means your focus will slip and you will go down other routes that will take you away from what you are trying to accomplish! Can anyone identify with this???? What happened???

Measures Embrace your data, don’t be afraid of it - they will tell a very rich story.

Thank you Intro – Hello my name is and intro everyone else Health & Safety – Fire Drill Toilets Tweet – Using #CYPIC Link to the session via ‘insert title of app’