1. Quality Improvement Indicators and Targets
12/5/2018 4:33 AM
Quality Improvement Indicators and Targets

2. A Family of Measures
Ideally, every quality improvement project should include:
At least one Outcome Measure – allows one to determine if the goal of the project is achieved
At least one Balancing Measure - allows one to determine if there are unintended consequences resulting from the project
At least one Process Measure for each Change Idea – allows one to determine the extent to which the change ideas are becoming the new way of providing service delivery for all clients

3. What is an Outcome Measure?
Outcome Measures /[Big Dots]
Answer “so what?” (Why are we delivering service or what are we ultimately trying to achieve for clients?)
Voice of the Client; what payers care about
Some will be organization-specific (within the control of a single organization); others will be cross-sector (contributed to by more than one sector)
Example:
Organization-specific:
Access: Wait Time for Service
Client-centred: Client Satisfaction with Care Coordination
Cross-sector:
Integrated: Repeat Visits to the Emergency Department

4. What is a Process Measure? And when are these defined?
Process Measures
Provide information about the extent to which a practice/intervention has been implemented; voice of the process. These get defined when the action plan or change ideas are generated.
Example 1:
Outcome Measure: Wait Time from Referral to Service Initiation
Change Idea: Standard Work – Intake Staff prioritize client appointments for assessment within 48 hours of referral
Process Measure: % Client Calls Made to Schedule Assessments Within 48 Hours of Referral
Example 2:
Outcome Measure: Client Satisfaction with Care Coordination
Change Idea: Assign a Care Coordinator to clients assessed as high risk/complex condition
Process Measure: % of Complex Clients Assigned a Care Coordinator

5. What is a Balancing Measure?
Balancing Measures
Measure unintended consequences that could result from implementing the idea(s) for change.
Example 1:
Outcome Measure: Decreased length of service delivery
Idea for Change: Provide education on self-management and provide fewer weeks of service delivery
Balancing Measure: % Clients on service with a CTAS 4 or 5 ED Visit within 30 days of service delivery termination
Example 2:
Outcome Measure: Overall Client Experience
Idea for Change: Ask clients prior to leaving their house if there is anything else you can do for them today
Balancing Measure: Increased Staff Perception of Workload

6. Identify whether each of the following best fits the description of an Outcome, Process or Balancing Measure…
Measure
Outcome?
Process?
Balancing
% Clients Who Are Very Satisfied with Continuity of Service Delivery
% of Clients Who Received an Intake Call Within 24 Hours of Referral
Number of Medication Errors
% Clients Involved in a Team Meeting to Discuss Goals for Service
% of Staff Who Report an Increase in Workload
% of Clients Receiving Home Support Who Visit the ED within 2 Weeks of Discharge from Service
% Clients Readmitted to Hospital
% of Clients With an Improvement in ADL Capability
% Clients engaged in Teach Back to Encourage Self Management Skills

7. Types of Measures PDSA Measure(s) Project OUTCOME Measure(s)
Project BALANCING
Measure(s)
[Remember these are also outcome measures with a different purpose)
Process Measures linked to individual Change Ideas
Mix of quantitative and qualitative, but primarily qualitative
Purpose is to assess the need for tweaking the change idea to ensure:
. Feasibility
. Acceptability
Baseline data not required.
Only time you might expect to have data is if the change idea is already in place but uptake has not been good
& your improvement is an improved approach to gaining uptake
Eg. Have 30% uptake & you require %
Ideally, want baseline data to
determine whether the changes
you are making are resulting in
Improvement
Preferably:
12 months of data for baseline

8. Frequency of Data Collection/Data Points Duration of Data Collection
Measurement
Measure Type
Baseline Data ?
Frequency of Data Collection/Data Points
Likely Data Source
Duration of Data Collection
Outcome Measure – Linked to Project
Ideally, 12 months of data
Monthly
Routinely collected data, if possible
Baseline through to end of project – and possibly beyond at Senior Levels for Sustainability
Balancing Measure – Linked to Project
Baseline through to end of project
Process Measure – At least one for each Change Idea
Usually none
Weekly
Almost always manual (charts or tick sheets)
To completion of implementation of the Change Idea; periodic audit for local sustainability
PDSA Measure – usually a few for each cycle of testing for each Change Idea (multiple PDSA cycles per change idea)
Definitely none
With every cycle
Always manual and through feedback/interview
Only for the duration of the PDSA cycle test

9. Making manual data collection as easy as possible
Tick sheets at point of service
Time from Referral to Service Initiation
Date Referred Date Service Started # Days
PSW: _______________
# Magnets on Fridge
Date Client # On Fridge (Y/N)
Other forms of progressive data collection 7 4 16 27 1

10. Elements to consider when collecting data….
Purpose for Measure
Indicator Name
Numerator
(include inclusions/
exclusions)
Denominator (include inclusions/
Data Source
Sampling (if applicble)
Frequency of Reporting
Who will collect?
Who will report?
Purpose for Measure
Indicator Name
Numerator
(include inclusions/
exclusions)
Denominator (include inclusions/
Data Source
Sampling (if applicble)
Frequency of Reporting
Who will collect?
Who will report?
Process Measure for Change Idea on Fridge Magnets
% Clients Posting Magnets to Fridge
# Clients with Magnets for Contact Information on Fridge
# Clients in Living Unit 1
Manual Data Collection
N/A (every client visited)
Weekly
PSW’s (during regular visits); weekly data sheets
Project Team Lead

11. Target Setting: For Outcome Indicators
Setting Indicator Targets…. as much art as science
Based on:
Organization’s own experience
Other organizations in the top 10th percentile
Best in class
Theoretical best (e.g. “0 defects”)
“Half life” – 50% increases/decreases over a few years
Benchmarks in the literature
Key is finding a balance between an inspirational “stretch goal” and ensuring staff are not demoralized by a target that is not within reach

12. Target Setting: Process Indicators
Should always be % because these indicators provide information on the extent to which the change ideas have been fully implemented (to all staff, for all clients);
If these indicators are not at %, we can’t expect to impact our Outcome Indicators

13. Family of Measures – Example
Outcome Measure
Balancing Measure
PDSA Testing begins here
PDSA Testing begins here
Process Measure
Process Measure

14. Determine which of the following targets are appropriate for the indicator
Indicator: % Clients Engaged in Teach Back to Encourage Self Management Skills
Target:
50%
75%
95%
Not sure

15. Determine which of the following targets are appropriate for the indicator
Indicator: Number of Medication Errors Resulting in Harm
Target: 0% 5%
90%
Not sure

16. Determine which of the following targets are appropriate for the indicator
Indicator: % Increase in Time Required to Complete Intake
Target:
90% 0% 5%
Not sure

17. To summarize…
Ideally, every quality improvement project should include:
At least one Outcome Measure – allows one to determine if the goal of the project is achieved; there are guidelines that help to set these targets
At least one Balancing Measure - allows one to determine if there are unintended consequences resulting from the project; generally the target should be zero or as low as possible since it is an outcome that is not desirable
At least one Process Measure for each Change Idea – allows one to determine the extent to which the change ideas are becoming the new way of providing service delivery for all clients; target should always be %
Data can come from routinely collected sources for Outcome Measures; process measure data generally requires manual data collection; always develop simple, real time methods for data collection to decrease staff burden
Never skip measurement – without it, we will never know if the changes we are making are resulting in improvement!