Learning Collaborative Webinar: March 25, 2015 9/18/2018 8:17 PM Learning Collaborative Webinar: March 25, 2015

Agenda for Today Time Topic 12:00 – 12:10 am Overview of Collaborative: 2 Roadmaps 12:10 am – 12:25 pm Deep dive on 2 areas of challenge: 4 types of measures PDSA Scale 12:25 – 1:00 pm Team Reflections: Key accomplishment(s) Key challenge(s) Ability to apply the learning to a new project 1:00 – 1:10 pm Expectations for October 2015 1:10 – 1:15 pm Our Reflections 1:15 – 1:30 pm Q & A

Learning Collaborative 9/18/2018 8:17 PM Learning Collaborative Session 1 Nov 12 ACTION PERIOD 1 Session 2 Dec 16 Reinforce Learning from Nov 12 Creative Thinking & Change Ideas PDSA Measurement using Run Charts Leadership Communication Tools ACTION PERIOD 2 Session 3 Jan 21 Micro-System Sustainability Change Management Team presentations ACTION PERIOD 3 Webinar & Individual Team Call/Email feedback) Submit: Partial Project Charter Fishbone & Pareto Process /EBD Map OCSA Conference FALL 2015 Team Storyboard Project Charter Problem Statement Aim Statement Family of Measures Diagnostic Tools, including EBD Team Roles Expectations Executive Sponsor Session Oct. 24 Personal support and homemaking Services provided in supportive housing Meals on Wheels Transportation programs Respite and adult day programs Other services Webinar 1.-2 Calls Submit: Revised Charter Driver Diagram Run Chart – Outcome 1 PDSA Worksheet Webinar 2 Calls Submit: Revised Charter Revised Driver Diagram Run Chart – Outcome, Process PDSA Worksheets

Project Design Design/ Preparation Diagnostic Solution Generation What are we trying to accomplish? What changes can we make that will result in improvement? How will we know that a change is an improvement? Design/ Preparation Diagnostic Solution Generation Testing/ Implementation Spread & Sustainability Context: Design in Factors for Success Project Charter Problem Identification Team Composition & Roles Aim Statement Family of Measures Cause & Effect Diagram 5 Whys Pareto Chart Process Map Diagnostic Components of EBD Change Concepts/ Change Ideas Lateral Thinking Driver Diagram Rapid Cycle Improvement: PDSA Cycles Run Charts Sustainability Change Management Source: Paula Blackstien-Hirsch

2 Areas of Challenge Identified

3 Types of Measures PDSA Measure(s) Project OUTCOME Measure(s) Project BALANCING Measure(s) [Remember these are also outcome measures with a different purpose) Process Measures linked to individual Change Ideas Mix of quantitative and qualitative, but primarily qualitative Purpose is to assess the need for tweaking the change idea to ensure: . Feasibility . Acceptability Baseline data not required. Only time you might expect to have data is if the change idea is already in place but uptake has not been good & your improvement is an improved approach to gaining uptake Eg. Have 30% uptake & you require 90-100% Ideally, want baseline data to determine whether the changes you are making are resulting in Improvement Preferably: 12 months of data for baseline

Frequency of Data Collection/Data Points Duration of Data Collection Measurement Measure Type Baseline Data ? Frequency of Data Collection/Data Points Likely Data Source Duration of Data Collection Outcome Measure – Linked to Project Ideally, 12 months of data Monthly Routinely collected data, if possible Baseline through to end of project – and possibly beyond at Senior Levels for Sustainability Balancing Measure – Linked to Project Baseline through to end of project Process Measure – At least one for each Change Idea Usually none Weekly Almost always manual (charts or tick sheets) To completion of implementation of the Change Idea; periodic audit for local sustainability PDSA Measure – usually a few for each cycle of testing for each Change Idea (multiple PDSA cycles per change idea) Definitely none With every cycle Always manual and through feedback/interview Only for the duration of the PDSA cycle test

Making manual data collection as easy as possible Tick sheets at point of service Time from Referral to Service Initiation Date Referred Date Service Started # Days PSW: _______________ # Magnets on Fridge Date Client # On Fridge (Y/N) Other forms of progressive data collection 7 4 16 27 1

Elements to consider when collecting data…. Purpose for Measure Indicator Name Numerator (include inclusions/ exclusions) Denominator (include inclusions/ Data Source Sampling (if applicble) Frequency of Reporting Who will collect? Who will report? Purpose for Measure Indicator Name Numerator (include inclusions/ exclusions) Denominator (include inclusions/ Data Source Sampling (if applicble) Frequency of Reporting Who will collect? Who will report? Process Measure for Change Idea on Fridge Magnets % Clients Posting Magnets to Fridge # Clients with Magnets for Contact Information on Fridge # Clients in Park Assisted Living Program Manual Data Collection N/A (every client visited) Weekly PSW’s (during regular visits); weekly data sheets Project Team Lead

Family of Measures – Example Outcome Measure Balancing Measure PDSA Testing begins here Process Measure Process Measure

Measures PDSA Measure: Use of Client Profile by Volunteers Did the use of the profile add time to the visit? Visit Start Time Visit End Time Visit Length (min) 1st visit 2nd visit 3rd visit If it is not adding time, and there are no more changes, these measures can be stopped! Are there changes to the profile that would make it more useful __________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Scale of PDSAs Thoughts? How might you scale it down? Situation: 1 assisted living unit 2 supervisory staff over 2 teams of PSW’s Scale of PDSA Proposed for Change Idea 1: 2 supervisory staff with 2 teams of PSW’s Test with clients for one month Thoughts? How might you scale it down?

Tips to think about when designing PDSA cycles How small can I make the test? Can I involve only one staff member? Can I initially keep the testing to staff on the improvement team? Can I involve only a few clients? Even better, can I design a simulation prior to testing it on clients? Can I try it initially for one day? One week? One day programme? If the next cycle requires tweaking, based on the prior cycle: Use the same size test as the prior one, or if you scale up, make it as modest an increase as possible REMEMBER….THIS IS AS MUCH ABOUT CHANGE MANAGEMENT AS IS IT ABOUT ULTIMATELY IMPLEMENTING IDEAS THAT WILL PRODUCE GOOD OUTCOMES.

Key Accomplishments, Challenges & Future Application

Accomplishments Staff Engagement and Empowerment A very rewarding experience to draw upon the skills, input and expertise of a cross-functional team (including various admin staff, nursing staff, volunteer, Board, and client). Our organization has not assembled this type of robust team before and the benefits have prompted us to incorporate this approach in future projects. Increased interaction with the team from a brainstorming, root-cause analysis perspective. Engaging the staff in this way has uncovered a great deal of information and perspectives from the team that we didn’t otherwise have. Soliciting team feedback on a regular basis has improved our communication and made the team more aware of the [need] to continuously improve the quality of our work. Ability of the core team (that attended the sessions) to engage the rest of the team in the project and share the learning on improvement tools .  This created a sense of enthusiasm and is helping to move the project forward.

Accomplishments (cont) Achieving Results The tests are working; we are seeing improvements - wait times are decreasing! Although we haven’t collected enough data to demonstrate improvement in our outcome measure, we are receiving qualitative feedback from volunteers saying that the change ideas have had a positive impact both on engaging their clients and also reducing the volunteer’s own anxiety (an unanticipated benefit). Tools/Methods Provide a New Way of Working Learning the tools and knowing how to improve the quality and effectiveness of service delivery, especially as it relates to intake assessment. Completion of multiple PDSA’s has been significant - has not only contributed to an improved process, but also educated us about how to move forward with ideas in a manner that ensures that our decisions are evidence-based. Using the quality improvement tools at our all staff meetings and testing change ideas based on the feedback received.

Accomplishments (cont) Redesigned Materials We’ve Used for a Long Time The project has caused us to re-examine and redesign the standardized client experience surveys we’ve relied on, to include more questions that capture the “voice” of the client for services that have a personal impact on the client. Through team brainstorming using a Fishbone diagram and the “5 Whys” , we’ve redesigned and reworded the client information we’ve been distributing to better meet he needs of the seniors population we serve.

Challenges Volume of Work Volume of work, particularly during the Christmas/holiday timeframe when deliverables and timelines were very tight and many staff were away or unavailable. Workload: we cannot afford to back-fill employees to free up time to work on QI, so it is an add-on, with very tight deadlines and going over the Holidays was stressful BUT… the tight deadlines helped to keep us on track! Finding time to focus on this project in the depth it deserves., given other projects also in motion and due to weather-related disruptions. Maintaining Momentum with Slow Cycle Times The ‘quick’ in ‘quick tests of change’ has not happened – we have no control of when and how many clients request an admission to ADS; as we wait to collect the data, it’s hard to remain focused and maintain momentum.

Challenges (cont) Discipline and Patience to Stay True to and Trust the Process/Methodology It seems difficult and time consuming to go through different layers and administrative procedures before a change or even a test can be implemented at an organizational level. Being dependent on volunteers for data collection for some of the change ideas can be time consuming; requires a lot of reminders to collect it in a timely manner.   In order to succeed, consistently remembering to follow a standardized process: Identify Problem (root cause)>Develop a Change Idea>Test a Change Idea>Implement a Change Idea. Many would want to jump from identifying root cause to implementing a change in one step. In order to be effective, we must first take time to examine the root causes of the problem and test the change on a small scale. Change Management Challenging to get the team’s buy in and engagement. There are currently staff/team morale issues that are acting as barriers to the work that we are trying to accomplish.

Ability to apply the knowledge to a new area Yes, But Further Support Would be Helpful We know the theory of quality improvement, but need more practice to apply the various tools and models to new projects. Could benefit from the availability of consultative services. While we still have much to learn, I believe we can apply our learning to new projects.  Our depth of knowledge will increase over time, but we feel comfortable enough to move forward with use of the basic tools. Yes, Look Forward to Doing So We definitely feel equipped to apply the quality improvement tools and methodologies learned to other projects in the organization and are looking forward to having the opportunity to do so. Our team feels equipped to meet on a regular basis in order to identify potential problems in service delivery. We would apply the same method of analysis before embarking upon change.

Ability to apply the knowledge to a new area (cont) Yes, We’ve Already Applied It To New Areas! We all feel confident in our ability to apply what we have learned to new projects.  In fact, our Executive Sponsor and a team member are already applying the model/tools learned to a new project in another department. Yes, we have formed a Quality Improvement Committee that includes participants from the Learning Lab, which will add value going forward. We decided to change the format of our teams’ work plan format and will use project charters in future and will run PDSA ‘s (instead of our previous work plan format). We now approach change & improvement differently; we are asking different and more questions; we are engaging more people in the discussion; we appreciate the importance and value of collecting and analyzing the data, as well as timely communication regarding change. Most of the team feels equipped to apply the tools again, and actually, we are currently using them in some other projects and it is going well!

Our Reflections: Structural Components that facilitated success 9/18/2018 8:17 PM Session 1 Nov 12 ACTION PERIOD 1 Session 2 Dec 16 Reinforce Learning from Nov 12 Creative Thinking & Change Ideas PDSA Measurement using Run Charts Leadership Communication Tools ACTION PERIOD 2 Session 3 Jan 21 Micro-System Sustainability Change Management Team presentations ACTION PERIOD 3 Webinar & Individual Team Call/Email feedback) Submit: Partial Project Charter Fishbone & Pareto Process /EBD Map OCSA Conference FALL 2015 Team Storyboard Project Charter Problem Statement Aim Statement Family of Measures Diagnostic Tools, including EBD * Team Roles Expectations * Executive Sponsor Session Oct. 24 Personal support and homemaking Services provided in supportive housing Meals on Wheels Transportation programs Respite and adult day programs Other services Webinar 1.-2 Calls Submit: Revised Charter Driver Diagram Run Chart – Outcome 1 PDSA Worksheet Webinar 2 Calls Submit: Revised Charter Revised Driver Diagram Run Chart – Outcome, Process PDSA Worksheets *

What would we do differently? Longer action periods to ensure two calls per action period – take holidays into consideration More content on change management/communication planning A site visit would be ideal Integrate in more sharing: Learning sessions vs webinars Learn from the evaluations!

Learning Collaborative Sessions: Content Covered Assessment/Description Definition 1.0 Project Identified and Team Formed Project and target population identified; team formed based on scope of project 1.5 Project Defined Team is meeting; the following items have been completed on the Project Charter: Problem Statement, Project Scope, Aim, Measures (Outcome and Balancing); if baseline data are available, starting to gather baseline for outcome measure 2.0 Current State Understood and Root Causes of Problem Identified   Team is/has employed diagnostic tools to assess current state and identify root causes of the problem 2.5 Change Ideas Developed based on Theory of Change Team is/has developed change ideas using evidence and experience linked to root causes and has identified appropriate process measures; driver diagram developed 3.0 Testing of Changes Initiated, but No Improvement Yet Change ideas using PDSA cycles initiated, but it is still too early to demonstrate change on process measures 3.5 Early Improvement of Process Measures, but Not Yet Significant Some improvement demonstrated on process measures, but not yet at a level of significance 4.0 Significant Improvement on Process Measures, Early Change in an Outcome Measure High fidelity on change ideas with significant change in process measures; early change demonstrated on at least one outcome measure, but not yet at a level of significance; team is preparing for local implementation of the changes 4.5 Significant Change in Outcome Measure(s) There is significant change in at least one outcome measure; team is actively engaged in ensuring local sustainability of results and readying the change ideas for spread 5.0 Sustained Improvement The team has demonstrated sustained change in at least one outcome measure and is either involved in spreading the change or has handed off the responsibility to a new team Learning Collaborative Sessions: Content Covered 9/18/2018 8:17 PM Personal support and homemaking Services provided in supportive housing Meals on Wheels Transportation programs Respite and adult day programs Other services