Measuring the Effectiveness of Change Rosa West PhD, LMHC, LMFT How’re We Doing? …..measuring the effectiveness of change Rosa West PhD, LMHC, LMFT This product was supported by the Florida Department of Children and Families Substance Abuse and Mental Health Program Office funding. Rosa West PhD, LMHC, LMFT www.westcarewellness.com

Leading Successful Change Requires: Leadership Staff Buy-In Resources Commitment R. West PhD

Change is Hard Information or Data is invaluable to leading successful change R. West PhD

R. West PhD

Good Data Provides… Enhanced services Greater likelihood of finding a solution Stronger Support & Clarity Starting point! R. West PhD

So Why Measure? We can use data to create change by measuring what matters! R. West PhD

Types of Measures Outcome Measures Process Measures Balancing Measures Tells whether changes made lead to improvement Outcome Measures Tells whether a specific process change has been accomplished and if it is having intended effect Process Measures Used to determine if changes to improve one part of the system is or isn’t causing problems in another part Balancing Measures R. West PhD

Outcome Measures Reducing Appointment times Average length of time in days between the day a patient requests an appointment and the next available appointment 14 day avg reduced to 3 day avg R. West PhD

Process Measures Patient Continuity Percentage of visits the patient makes to their own clinician vs. another provider R. West PhD

Continuity of Care Formula Formula: Number of Patients of Provider X that were seen by Provider X, divided by the total number of Patients for Provider X that have been seen. 94 of Dr. Smith’s patients were seen in clinic this month. 78 patients were seen by Dr. Smith. 16 of Dr. Smith’s patients were seen by another provider. 78 / 94 = .829 .829 X 100 = 82.9% Continuity R. West PhD

Balancing Measures How would you rate the length of time waiting for today’s visit? R. West PhD

When Implementing Change… It’s important to assess readiness in instituting change R. West PhD

Ready… Set… Go!!! R. West PhD

Change Readiness Diagnostic: Risk Assessment Rate each from 1 to 5 with 1 = very low, 2 = low, 3 = medium, 4 = high, 5 = very high. Totals 15 - 34 points Watch Out! 35 - 55 points Keep a Close Eye on Things. 56 - 75 points You/ Your Organization are Very Likely to Successfully Implement the Change Program. Risk Factor 1: Adequacy of Risk Management Process Risk Factor 2: Adequacy of Change Program Definition Risk Factor 3: Effectiveness of Change Management Process Risk Factor 4: Adequacy of Sponsorship and Resources Risk Factor 5: Adequacy of Communication and Involvement Risk Factor 6: Range of Linked / Consequential Actions Identified Risk Factor 7: Coherence in the Sequencing of Linked Actions Change Readiness Diagnostic: Risk Assessment

Change Readiness Diagnostic: Risk Assessment Rate of Return Factor 1: Extent and Timing of Benefits Rate of Return Factor 2: Change Program Budgets Economy Rate of Return Factor 3: Extent to which Project Time, Specifications and Costs are Managed Rate of Return Factor 4: Degree of Focus on Business Results Latent Opportunity Factor 1: Program Scope Latent Opportunity Factor 2: Linking Change Drivers Through Actions to Performance Latent Opportunity Factor 3: Appropriateness of Benchmark Targets Latent Opportunity Factor 4: Quality of the Benchmarking Process

Mobilizing Commitment Stakeholder Analysis/Commitment Chart Grumpy Grumpy Snooty Snooty Grouchy Grouchy Slouchy Slouchy Steps: 1. Plot where individuals currently are with regard to desired change (O = current). 2. Plot where individuals need to be (X = desired) in order to successfully accomplish desired change identify gaps between current and desired. 3. Indicate how individuals are linked to each other, draw lines to indicate an influence link using an arrow () to indicate who influences whom. 4. Plan action steps for closing gaps. Can use Resistance Analysis tool.

Include as member of Change Team Mobilizing Commitment Resistance Analysis Include as member of Change Team Job Security Grumpy Ambiguous Roles High ? Med ? Snooty Structure Grouchy Slouchy

Collecting Baseline Data Measuring the Effectiveness of Change Collecting Baseline Data Before you begin any process improvement project you need to collect baseline data. Keep this in mind “How will you know how far you gone if you don’t know where you were?” (click to the next slide) Rosa West Ph.D. Rosa West PhD, LMHC, LMFT www.westcarewellness.com

Measuring the Effectiveness of Change Aims & Goal Setting Reduce Waiting Times Reduce No Shows Increase Admissions Increase Continuation After you conduct your initial walk through want to make sure you identify your aim. The Aim that you choose to focus on should be selected based on the insights gained from your walkthrough or aligned with the priorities of your CEO and/or Executive sponsor. A quick review of the NIATx aims (click to the next slide) Rosa West Ph.D. Rosa West PhD, LMHC, LMFT www.westcarewellness.com

An Example: Reducing Waiting Times Measuring the Effectiveness of Change An Example: Reducing Waiting Times Let’s consider an example. Although we may not actually see this standing in front of our desk, this is the vision; particularly this gentlemen staring at watch wondering when it’s his turn. I think this is a concern for many of us as we want to make sure we provide timely service (click to next slide) Rosa West Ph.D. Rosa West PhD, LMHC, LMFT www.westcarewellness.com

Measurement R. West PhD

Measuring the Effectiveness of Change Key Principles… What gets measured gets done! Change is virtually impossible when we do not measure our performance. R. West PhD Rosa West PhD, LMHC, LMFT www.westcarewellness.com

Measuring the Effectiveness of Change Key Principles… It’s important to choose the most appropriate means of measuring the change Change is virtually impossible when we do not measure our performance. R. West PhD Rosa West PhD, LMHC, LMFT www.westcarewellness.com

When Choosing Measures: Relevance: does it have a significant, demonstrable relation to the goals/objectives? Reliable: is the measure consistent? Will we get the same results after repeating the measure? Availability: are the data necessary for measuring the change available at a reasonable cost? R. West PhD

If You have the Wrong Problem, You will get the Wrong Solution. Problem exploration is the first stage in selecting the change and establishing measures However…… Exploration may be limited because you think: The assumptions are known (although assumptions can be so ingrained you fail to notice them) Complexity means this will be hard to solve You know what you want The problem is obvious (although perception is flawed) R. West Ph.D.

Considerations in Selecting Measures Measuring the Effectiveness of Change Considerations in Selecting Measures Re-evaluating existing measures (minimize influence of): Fear Ownership What-Ifs Estimates Staff/Managers may be fearful of measuring performance. What will the data show? How will the findings effect us? R. West PhD Rosa West PhD, LMHC, LMFT www.westcarewellness.com

Considerations in Selecting Measures Measuring the Effectiveness of Change Considerations in Selecting Measures It’s important to measure processes and results Measuring processes can provide information about the effectiveness of the process and measure the ultimate results of these process. R. West PhD Rosa West PhD, LMHC, LMFT www.westcarewellness.com

Considerations in Selecting Measures Measuring the Effectiveness of Change Considerations in Selecting Measures Measures Should Foster Goal-Driven Teamwork R. West PhD Rosa West PhD, LMHC, LMFT www.westcarewellness.com

ESTABLISH BASELINE RESULTS Plan: Change is driven by the action steps established and measurements set in place to assess performance. Hold the Gain Study Act (implement) Do Plan Standardize ESTABLISH BASELINE RESULTS R. West PhD

Measuring Effectiveness of Change Measuring the Effectiveness of Change Measuring Effectiveness of Change What change was most important? What changes worked? Before What changes resulted in an improvement? During What didn’t work? We need to make sure we measure the effectiveness of our change by asking ourselves multiple questions… By collecting data before, during, and after your organization’s change cycle, you can measure the progress you have made towards achieving your goals and the impact of your change cycle. (click to next slide) After R. West PhD www.NIATx.com Rosa West PhD, LMHC, LMFT www.westcarewellness.com

6-Step Process of Measuring Change Define Measures Establish Aims Collect Baseline Data Ask Questions Consistently Collect Data Chart Progress R. West PhD

Measuring the Effectiveness of Change Define Measures Define your measures before you start the change process Definition MUST be clear to measure change successfully Defining measures does not have to be overly complicated…keep it simple with tally sheet, this something quick an easy anyone can use that yields the same results as any other measure. Or..and excel spreadsheet. Click to next slide R. West PhD Rosa West PhD, LMHC, LMFT www.westcarewellness.com

Measuring the Effectiveness of Change Define Measures Defining measures does not have to be overly complicated…keep it simple with tally sheet, this something quick an easy anyone can use that yields the same results as any other measure. Or..and excel spreadsheet. Click to next slide Tally Sheet R. West PhD Rosa West PhD, LMHC, LMFT www.westcarewellness.com

Excel Spreadsheet R. West PhD

Collecting Data Be specific (clarity) What is “one day”? Do PDSA on data collection method Check data periodically during collection process to ensure procedures are consistent R. West PhD

Collecting Data Be mindful of potential variations i.e. sampling today may not yield same sample than during holidays R. West PhD

Measuring the Effectiveness of Change Baseline Data BEFORE Before you change anything you must collect baseline data. Baseline data shows you where you are today! It allows you to measure the impact of any changes that are made. It also provides a before and after picture of a situation. Now let’s go back to our original example…(Click to next slide) AFTER R. West PhD Rosa West PhD, LMHC, LMFT www.westcarewellness.com

An Example: Reducing Waiting Times Measuring the Effectiveness of Change An Example: Reducing Waiting Times To collect baseline data, each day have the office assistant document the number of days until the next available appointment. (Data can be collected for 1 week, 2 week, etc) Once we have our data we can establish our aim…click to next slide Record the number of days to the next available appointment R. West PhD Rosa West PhD, LMHC, LMFT www.westcarewellness.com

Measuring the Effectiveness of Change Establish Aims Establish a clear aim. Reduce wait time from 21 days to 15 days Be flexible when establishing your goals. And keep in mind, that goals can be too ambitious or not ambitious enough. Set incremental goals to allow you to celebrate your successes along the way. Set a realistic goal that still challenges your organization to improve. Once you achieve your goal, set a new one! Click to next slide. R. West PhD www.NIATx.com Rosa West PhD, LMHC, LMFT www.westcarewellness.com

Consistently Collect Data Measuring the Effectiveness of Change Consistently Collect Data Continue to collect data on the number of days to the next available appointment as you implement changes. This is a critical part of the change process. You may already have measure in place to track & collect data. However, you may need to establish measures to collect data relevant to the changes you intend to make. Click to next slide R. West PhD www.NIATx.com Rosa West PhD, LMHC, LMFT www.westcarewellness.com

Measuring the Effectiveness of Change Chart Your Progress WAIT TIMES Pre-Change Baseline =21 days 20 One graph One Message 15 Excel Chart Number of days 10 5 Overtime you will collect both pre-change & post change data. Make sure you share this data with others in your organization. An effective means of sharing this information would be through an excel chart. Any visual aids you use should be simple and to the point. R. West PhD Rosa West PhD, LMHC, LMFT www.westcarewellness.com

Measuring the Effectiveness of Change Chart Your Progress Pre-Change Baseline =21 days WAIT TIMES 20 17 days 15 10 Number of days 5 Once the change was implemented there was almost an immediate reduction in wait time, with a 4 day drop in in wait time within one week. You can also track a steady decline in wait time over the course of the change project. R. West PhD Rosa West PhD, LMHC, LMFT www.westcarewellness.com

Measuring the Effectiveness of Change Ask Questions What does the information tell me about changes in my organization? If change is successful, the information should tell you which information was useful in helping you meet your goals. Remember to ask questions throughout the change process. What is the information telling me about change in my organization? www.NIATx.com R. West PhD Rosa West PhD, LMHC, LMFT www.westcarewellness.com

Celebrate your successes!!! R. West PhD

Warning!!! Do not collect too much data Do not focus on too many measures Avoid analysis paralysis! www.NIATx.com R. West PhD

Tips for Success Identify the question you want answered. Build measurement into daily workflow. Post data where everyone can see it. Create team ownership of the data. Use data to drive decision-making. R. West PhD

Final Comments/Questions? R. West PhD