Quality Improvement Indicators and Targets 12/5/2018 4:33 AM Quality Improvement Indicators and Targets

A Family of Measures Ideally, every quality improvement project should include: At least one Outcome Measure – allows one to determine if the goal of the project is achieved At least one Balancing Measure - allows one to determine if there are unintended consequences resulting from the project At least one Process Measure for each Change Idea – allows one to determine the extent to which the change ideas are becoming the new way of providing service delivery for all clients

What is an Outcome Measure? Outcome Measures /[Big Dots] Answer “so what?” (Why are we delivering service or what are we ultimately trying to achieve for clients?) Voice of the Client; what payers care about Some will be organization-specific (within the control of a single organization); others will be cross-sector (contributed to by more than one sector) Example: Organization-specific: Access: Wait Time for Service Client-centred: Client Satisfaction with Care Coordination Cross-sector: Integrated: Repeat Visits to the Emergency Department

What is a Process Measure? And when are these defined? Process Measures Provide information about the extent to which a practice/intervention has been implemented; voice of the process. These get defined when the action plan or change ideas are generated. Example 1: Outcome Measure: Wait Time from Referral to Service Initiation Change Idea: Standard Work – Intake Staff prioritize client appointments for assessment within 48 hours of referral Process Measure: % Client Calls Made to Schedule Assessments Within 48 Hours of Referral Example 2: Outcome Measure: Client Satisfaction with Care Coordination Change Idea: Assign a Care Coordinator to clients assessed as high risk/complex condition Process Measure: % of Complex Clients Assigned a Care Coordinator

What is a Balancing Measure? Balancing Measures Measure unintended consequences that could result from implementing the idea(s) for change. Example 1: Outcome Measure: Decreased length of service delivery Idea for Change: Provide education on self-management and provide fewer weeks of service delivery Balancing Measure: % Clients on service with a CTAS 4 or 5 ED Visit within 30 days of service delivery termination Example 2: Outcome Measure: Overall Client Experience Idea for Change: Ask clients prior to leaving their house if there is anything else you can do for them today Balancing Measure: Increased Staff Perception of Workload

Identify whether each of the following best fits the description of an Outcome, Process or Balancing Measure… Measure Outcome? Process? Balancing % Clients Who Are Very Satisfied with Continuity of Service Delivery % of Clients Who Received an Intake Call Within 24 Hours of Referral Number of Medication Errors % Clients Involved in a Team Meeting to Discuss Goals for Service % of Staff Who Report an Increase in Workload % of Clients Receiving Home Support Who Visit the ED within 2 Weeks of Discharge from Service % Clients Readmitted to Hospital % of Clients With an Improvement in ADL Capability % Clients engaged in Teach Back to Encourage Self Management Skills

Types of Measures PDSA Measure(s) Project OUTCOME Measure(s) Project BALANCING Measure(s) [Remember these are also outcome measures with a different purpose) Process Measures linked to individual Change Ideas Mix of quantitative and qualitative, but primarily qualitative Purpose is to assess the need for tweaking the change idea to ensure: . Feasibility . Acceptability Baseline data not required. Only time you might expect to have data is if the change idea is already in place but uptake has not been good & your improvement is an improved approach to gaining uptake Eg. Have 30% uptake & you require 90-100% Ideally, want baseline data to determine whether the changes you are making are resulting in Improvement Preferably: 12 months of data for baseline

Frequency of Data Collection/Data Points Duration of Data Collection Measurement Measure Type Baseline Data ? Frequency of Data Collection/Data Points Likely Data Source Duration of Data Collection Outcome Measure – Linked to Project Ideally, 12 months of data Monthly Routinely collected data, if possible Baseline through to end of project – and possibly beyond at Senior Levels for Sustainability Balancing Measure – Linked to Project Baseline through to end of project Process Measure – At least one for each Change Idea Usually none Weekly Almost always manual (charts or tick sheets) To completion of implementation of the Change Idea; periodic audit for local sustainability PDSA Measure – usually a few for each cycle of testing for each Change Idea (multiple PDSA cycles per change idea) Definitely none With every cycle Always manual and through feedback/interview Only for the duration of the PDSA cycle test

Making manual data collection as easy as possible Tick sheets at point of service Time from Referral to Service Initiation Date Referred Date Service Started # Days PSW: _______________ # Magnets on Fridge Date Client # On Fridge (Y/N) Other forms of progressive data collection 7 4 16 27 1

Elements to consider when collecting data…. Purpose for Measure Indicator Name Numerator (include inclusions/ exclusions) Denominator (include inclusions/ Data Source Sampling (if applicble) Frequency of Reporting Who will collect? Who will report? Purpose for Measure Indicator Name Numerator (include inclusions/ exclusions) Denominator (include inclusions/ Data Source Sampling (if applicble) Frequency of Reporting Who will collect? Who will report? Process Measure for Change Idea on Fridge Magnets % Clients Posting Magnets to Fridge # Clients with Magnets for Contact Information on Fridge # Clients in Living Unit 1 Manual Data Collection N/A (every client visited) Weekly PSW’s (during regular visits); weekly data sheets Project Team Lead

Target Setting: For Outcome Indicators Setting Indicator Targets…. as much art as science Based on: Organization’s own experience Other organizations in the top 10th percentile Best in class Theoretical best (e.g. “0 defects”) “Half life” – 50% increases/decreases over a few years Benchmarks in the literature Key is finding a balance between an inspirational “stretch goal” and ensuring staff are not demoralized by a target that is not within reach

Target Setting: Process Indicators Should always be 90-100% because these indicators provide information on the extent to which the change ideas have been fully implemented (to all staff, for all clients); If these indicators are not at 90-100%, we can’t expect to impact our Outcome Indicators

Family of Measures – Example Outcome Measure Balancing Measure PDSA Testing begins here PDSA Testing begins here Process Measure Process Measure

Determine which of the following targets are appropriate for the indicator Indicator: % Clients Engaged in Teach Back to Encourage Self Management Skills Target: 50% 75% 95% Not sure

Determine which of the following targets are appropriate for the indicator Indicator: Number of Medication Errors Resulting in Harm Target: 0% 5% 90% Not sure

Determine which of the following targets are appropriate for the indicator Indicator: % Increase in Time Required to Complete Intake Target: 90% 0% 5% Not sure

To summarize… Ideally, every quality improvement project should include: At least one Outcome Measure – allows one to determine if the goal of the project is achieved; there are guidelines that help to set these targets At least one Balancing Measure - allows one to determine if there are unintended consequences resulting from the project; generally the target should be zero or as low as possible since it is an outcome that is not desirable At least one Process Measure for each Change Idea – allows one to determine the extent to which the change ideas are becoming the new way of providing service delivery for all clients; target should always be 90-100% Data can come from routinely collected sources for Outcome Measures; process measure data generally requires manual data collection; always develop simple, real time methods for data collection to decrease staff burden Never skip measurement – without it, we will never know if the changes we are making are resulting in improvement!