Scottish Improvement Skills Planning measurement SIS: Group C Measurement: Module – Planning measurement: Facilitator

System of Profound Knowledge For this session we are in this part of the System of Profound Knowledge: Understanding Variation. Aims To create operational definitions for your project To create a measurement plan Subsidiary aim depending on how challenging participants find this module: To be able to distinguish between different types of measure (count, %, and rate) and explain how to define a % and rate Key messages Importance of planning measurement and everyone who may collect data having a shared understanding of each measure and how to collect the data Importance of defining measures early in the project. Timing 60 mins minimum for the whole module Deming 2000

Planning measurement By the end of this session you will be able to: Create operational definitions for your measures that others can follow reliably to collect data Create a measurement plan for your improvement work. Learning outcomes: Lead Facilitator read out or ask participants to read Recap from Introduction to Measurement – elicit 3 types of measure and what they are: Outcome – voice of the customer Process – voice of the system Balancing – unintended consequences

How big is your banana? DISCOVERY Aim Participants to be able to explain the importance of a clear, shared understanding of what and how teams are measuring. Key messages We need everyone in our team to be clear about what and how we are measuring in our improvement work. Timing 25 minutes Materials For each team: Banana Ruler Paperclip String – must be long enough to go end to end twice Paper – 2 sheets Pen/pencil And: Bananas online countdown timer (a mobile phone timer doesn’t work as well as it isn’t as loud, and people tend to ignore it) Two sets of numbers on paper or card 1 to 5 or 6 (ie to label the teams who first measure, then second measure – 2 different colours may help) Flip chart sheet prepared with numbers of teams (initial then other team measuring the same banana). Allow plenty of space to write several measures eg if they measured around as well as end to end. Lead facilitator Divide the participants into teams. No more than 5-6 teams as the key messages can easily be covered in debriefing on this quantity of examples. Set up countdown timer. Support facilitators Facilitator A: give out materials Facilitator B: Give each team a number (otherwise they forget later who has which team’s banana) Instructions Measure the size of your banana. Write out instructions for another team to follow so they can repeat the measurement process. Time limit: 7 minutes Use whatever pieces of kit you like and decide what measurement(s) to take. On a separate piece of paper note down the size of the banana - do not share this with anyone at this point. After 7 minutes they stop. Faciiltator pass the banana and measurement instructions to the next team i.e. 1 pass to 2 , 2 pass to 3 etc The next team measures the ‘new‘ banana according to instructions, and writes the measurements down on the instruction sheet. Give a 4 minute time limit, using the same countdown timer. If all teams finish before 4 minutes have elapsed, cancel the timer and move on to the debrief. Debrief: Lead Facilitator Note down both measurements for each banana on a flipchart, noting where there are differences in measurement for the same banana. Facilitate discussion about these differences. Support facilitators must have an opportunity to feed in to this discussion, as it is likely that they will have noticed things the Lead Facilitator may have missed.

Operational definitions Define the specific components of the measure If it is an average, specify the calculation for deriving the average If it is a score (eg patient satisfaction score), describe how the score is derived. If it is a percentage or a rate, specify the numerator and denominator Describe any special equipment needed Describe the criteria to determine concepts such as ‘accurate’ or ‘complete’. PRACTICE Aim Participants distinguish between outcome, process and balancing measures Participants generate alternative measures for the same outcome or process Key messages There is usually more than one way we could measure the same thing Range of criteria for selecting a measure eg historical data or benchmarks, ease of collection. ease of standardisation What to include in an operational definition (possibly including the difference between a percentage and a rate Timing 15 minutes Materials Case study wall cards: Aim, drivers and change ideas already on the wall Measures Blu tac or Velcro Lead Facilitator Picking up from issues arising from bananas re operational definitions, do participants have anything to add to this slide, or any questions? Facilitators Depending on the number of participants and facilitators, decide in advance which case studies will be used. All facilitators decide which measures they will use first (it is not necessary to use all the cards, if time is short); make sure at least one example of each of the following is used: count, percentage, rate. Each facilitator go to one of the case studies, with the Measures cards and blutac/Velcro For each measure card, ask participants to match it to one of the components of the driver diagram and change ideas, and put the measure on the wall beside the relevant component. Then elicit: - whether the measure is O, P or B - any issues with defining it - logistical issues eg equipment required, calibration, resources/workload - possible alternatives to this measure; in which case, which measure might be most suitable and why And a brief discussion on stratification If all is going well, highlight (elicit if possible), the difference between a percentage and a rate. If participants are struggling, consider whether or not it will be helpful to introduce this distinction.

Project work: measurement planning Pick one of your measures Create an operational definition Consider exclusions and stratifiers Provide outline data collection method PROJECT Aim Participants become familiar with a standard measurement plan template. Participants create at least one measurement plan for their project Timing 20 minutes Materials Measurement plan examples QIHub measurement plan template QIHub data collection plan template Consider having participants do this on laptops with preloaded templates Lead facilitator Pick one of the measures that you have so far outlined for your project. Referring to the examples, as well as the case studies on the wall: Write a detailed operational definition. Best to do this on the back of the sheet, to ensure the message is clear that more space is needed for this than the hard copy template allows. Complete the other sections of the plan. Facilitators Monitor the room. In particular check that operational definitions have sufficient detail. Be ready to answer questions about other sections of the plan eg stratifiers, exclusions, sampling. Options for early finishers: Draft a measurement plan (starting with the operational definition) for another project measure Look at the data collection plan template (in particular the guidance notes), which includes more logistical issues, and start drafting.

Planning measurement: summary Aim To briefly recap the session content: To support a sense of learning and accomplishment To aid memory of the session later An opportunity for participants to ask any outstanding questions from any part of the session Timing 2 – 5 minutes, depending on time available Lead facilitator Elicit key messages from the module eg What’s the purpose of a measurement plan? (everyone has the same understanding of what you are measuring and how) What do you need to include? How many measurement plans do you need? (as many as you have measures) When do you create measurement plans? (for any new measure, asap to ensure it’s the right measure) Who should be involved? (not just you eg people who may ‘own’ historical data, people who will be collecting data, people who can help with equipment for measurement) Planning measurement: summary