Fundamentals of Data Interpretation for Quality Improvement An Overview of Online Modules Mary A. Dolansky, PhD, RN Mark E. Splaine, MD, MS

Goals  Share why QSEN developed this course of five modules  Interact with course content by Hearing an overview Working through case examples  Learn how to access the online modules 2

Agenda  Introduction (Mary – 5 minutes)  Interactions with Module Content (Mark) Balanced Measures (25 minutes) Data Over Time (25 minutes) Run Charts (25 minutes)  Summary (Mary – 10 minutes) 3

Background  Observation about need  Desire to make content easily available  Ability to do entire course or select module(s) of interest  Self-paced  Experiential learning by using templates  Assessment and CE credit 4

Modules 1. Developing Measures for Quality Improvement 2. Refresher on Critical Math Concepts for Quality Improvement 3. The Value of Monitoring Data Over Time 4. Creating and Using Run Charts 5. Introduction to Statistical Process Control Charts 5

Module Contents  Overview  Brief activity  Case interaction  Summary  Resources/tools  Assessment 6

Module 1 Developing Measures for Quality Improvement

Aims Developing Measures for Quality Improvement  Appreciate the value of using balanced measures in quality improvement work  Construct a value compass of measures for a specific clinical condition  Recognize the difference between a conceptual and an operational definition 8

Model for Improvement 9 Three questions... …coupled with an approach for testing change. Langley GJ, et. al. The Improvement Guide (2 nd Edition), What are we trying to accomplish? (Aim) How will we know that a change is an improvement? (Measures) What change can we make that will result in improvement? (Change) Plan DoStudy Act (Measures)

Approach to Developing Measures  Know what question you are trying to answer to achieve your aim  Develop a set of possible measures  Choose one (or a couple) of measures  Define the measures and develop a method for collecting the data  Understand your process 10

Balanced Measures & The Value Compass 11 Biologic or Clinical Functional Status Cost Satisfaction Against Need  Distribute measures around the compass  Consider both process and outcome measures  Value = Quality / Cost Measures of Quality Measures of Cost

Generic Value Compass Physical Mental Social/Role Perceived Well-being Functional Status & Quality of Life Satisfaction Against Need Health Care Delivery Perceived Health Benefit Biologic/Clinical Status Mortality Morbidity Costs Direct Medical Indirect Social

Diabetes Value Compass Functional Status Satisfaction Clinical Costs Medications Clinic visits Admissions ER visits Hemoglobin A1C Medication changes Emotional functioning Days missed from work Patient’s perspective 13

Measure Definitions  Conceptual Brief statement describing a variable of interest Tells what you want to measure  Operational A clearly specified method for reliably sorting, classifying or measuring a variable Tells how a variable should be measured 14

Diabetes Examples MeasureConceptual Definition Operational Definition HbA1c Measure of average blood sugar control Lab test ordered on all diabetic patients every three months; results of last test entered into diabetic flowsheet before office visit by medical assistant; if test not up-to-date, patient is called by medical assistant and asked to have test done before next visit. Emotional Function Patient’s self- assessment of depression symptoms Survey (PHQ9) given to patient upon check-in to appointment by receptionist; medical assistant computes score for completed survey and enters value into diabetes flowsheet; done every six months for all diabetic patients. ER Visits Patient use of Emergency Room for care Medical record review by a patient care technician confirms date of and reason for any Emergency Room visit in the previous six months; results of review entered into the diabetes flowsheet prior to office visit; done prior to every office visit for all diabetic patients. 15

Case #1: Improving Heart Attack Care  Part 1: Balanced Measures Place measures around the value compass  Part 2: Operational Definitions Develop a definition for one of your measures We will debrief when you have finished both parts 16

Summary  Attention must be given to selecting, defining and collecting data for any measure  This always sounds easier than it actually is, so one should leave time to do this well  Practice is helpful, and even more useful when applied to something specific on which you are working 17

Module 3 Displaying Data Over Time and Types of Variation

Aims Displaying Data Over Time & Types of Variation  Understand different options for visual display of data  Recognize the differences between a display of aggregated data and a display of data over time  Interpret a display of data over time  Recognize the difference between non- random (special cause) and random (common cause) variation 19

Diabetics & Flu Shots 20

21 Time Plot  A graph of data in time order  Often kept to identify if and when problems appear (proactive)  Also used to see trends over time (reflection)  Especially helpful when you implement a change to follow the results

Diabetes Time Plot Day Blood Sugar Before Bed Reading 22

Diabetes Time Plot Day Blood Sugar Before Bed Reading 23 Goal Diet Change

Diabetes Stratified Time Plot Day Blood Sugar Morning ReadingBefore Bed Reading 24

Case #2: Meeting the Needs of Hospitalized Patients  Part 1: Time Plot with Target Review and interpret two data displays  Part 2: Stratified Time Plot Review and interpret one data display We will debrief after each part 25

26 Types of Variation  Random or Common Cause  Non-Random or Special Cause

27 Random or Common Cause Variation  Typically due to a large number of small sources of variation Example: Variation in arrival time of a patient might include: weather, vehicle problems, parking issues  Usually requires a deep understanding of the process to change

28 Non-Random or Special Cause Variation  Something not part of the process all the time; arises from special circumstances Example: Patients arrive late for appointments due to a bus strike or major traffic accident  Usually best uncovered when monitoring data in real time (or close to that)

29 How to React to Variation  Dealing with each type of cause of variation requires a different approach Non-Random or Special Cause Track down and understand Eliminate if undesired Replicate or design in desired Random or Common Cause Change through disciplined improvement efforts Focus on process

30 Summary  Visual display of data helps to tell the story inherent in a set of data  There are many options for visual displays; often helpful to use more than one  Time plots display data over time  Often helpful to annotate your chart Use text boxes, arrows, goal level, etc.  Two major types of variation Actions to take are different for each types

Module 4 Understanding and Interpreting Run Charts

Aims Understanding and Interpreting Run Charts  Recognize the differences between a run chart and other displays of data over time  Understand and demonstrate the application of rules for interpreting run charts  Identify the presence or absence of non- random variation and interpret its meaning  Formulate actions based on the interpretation of run charts 32

33 Anatomy of a Run Chart Variable “x” Time Center line is MEDIAN

Run Chart Days Fasting Blood Sugar (mg/dl) Median 34

How to React to Variation ActionProcess result Process with only random variation Not satisfied with result: redesign process to get a better result Reduce variation : make the process even more predictable or reliable Process with non-random variation Identify the cause: If positive, then can it be replicated or standardized. If negative, then cause needs to be eliminated Target the causes - to get the process predictable 35

17  The presence of a shift in the process A run that is too long (6 or more consecutive points on one side of the median) A “run” is one or more consecutive points on the same side of the median  The presence of a trend 5 or more consecutive points increasing or decreasing  The presence of too much or too little variability (see next slide for details) Too few or too many runs (depends on number of points on the chart) Non-Random Variation Signals on Run Charts 36 Perla, Provost, and Murray. BMJ Qual Saf. 2011;20:46-51

Source: Perla, Provost, and Murray. BMJ Qual Saf. 2011;20:

A Run is a point or group of consecutive points that fall on one side of the median Days How to Count Runs 38 Fasting Blood Sugar (mg/dl)

Run Chart Example Days 39 Fasting Blood Sugar (mg/dl)

Questions for You 1. What does the red line on the graph represent? 2. How many runs are there? 3. How many shifts do you see? 4. How many trends are in the data? 5. How many rules of special cause variation (special cause signals) are met in this run chart? 6. What is your interpretation of the chart? 40

Case #3: Improving Post-op Surgical Care  Part 1: Interpret a Run Chart Before a change is implemented  Part 2: Interpret a Run Chart To assess the effect of implementing a change We will debrief after each part 41

Summary  Understanding the type of variation (non- random and random) can help monitor, adjust and improve processes.  Studying variation with run charts can offer insights about possible cause of that variation and offer clues to the design of change. 42

Questions and Discussion  What thoughts do you have that we could address? 43

Summary  Modules are available on QSEN website Can select individual module(s) or the entire course  Education credit is provided if learner completes the module assessment with satisfactory score  We welcome your feedback on any of the module content 44