1. Tutorial on Risk Adjusted P-chart Farrokh Alemi, Ph.D.

2. Have Changes Led to Improvement? Common cause variation (changes in outcomes because of chance) is everywhere. Decision makers often mistakenly attribute positive outcomes to their own skills and negative outcomes to others, while in reality both could be a chance outcome

3. Why Chart Data? To discipline intuitions To communicate data in vivid graphical ways

4. Purpose of Risk Adjustment Purpose of p-chartPurpose of risk- adjusted p-chart To detect if the process has improved beyond historical levels. Assumes historically we have been serving the same type of patients as now To detect if the process has improved beyond what can be expected from patient conditions

5. Data Needed Data collected over time Risk (expected outcomes) for each patient Outcomes for each patient The purpose is to improve not to get so lost in measurement to loose sight of improvement.

6. What Is Risk? A patient’s condition or characteristics that affects the expected outcomes for the patient A severity index used to predict patient outcomes Clinicians’ consensus regarding expected outcomes Patient’s self rating of expected outcomes

7. MI Patients Over 8 Months in One Hospital Observed mortality during this time period Number of cases Expected probability of mortality for case 8 in time period 1. Estimated from severity indices or experts’ consensus.

8. Elements of a Control Chart X axis shows time Y axis shows probability of adverse events Observed rates are plotted against time sequence Upper control limit is a line drawn so that points above it are rare to be observed by mere chance Lower control limit is a line drawn so that points below it are rare to observe by mere chance Lets take a look

9. An Example of P-chart Upper control limit Lower control limit Observed rate

10. Steps in Creating P- chart for Mortality 1. Check assumptions 2. Calculate observed rates and plot them 3. Calculate expected rates and plot them 4. Calculate expected deviation 5. Calculate control limits and plot them 6. Interpret findings 7. Distribute chart and interpretation

11. Step One: Check Assumptions We are examining discrete events that either happen or do not happen, e.g. mortality among MI patients, falls among nursing home patients, error in medical record entry, etc. The event is not rare, meaning the probability of it occurring exceeds 5% for each time period. Observed events are independent from each other. The probability of the event occurring does not change over time. This assumption is violated if one patient’s outcomes affects the outcomes for others, e.g. when dealing with infectious diseases.

12. Step Two: Calculate Observed Rates and Plot P i = Mortality rate in time period “i” O i = Mortality in period “i” n i = Number of cases in time period “i” P i = O i / n i

13. Observed Mortality Rates for All Time Periods Plot of mortality rates

14. Plot of the Observed Rates Time period 7 and 3 seem different but don’t rush to judgment. Wait, until you see control limits of what could have been expected.

15. Step Three: Calculate Expected Mortality E ij = Expected mortality of case ‘j’ in time period “i” E i = Expected mortality for time period “i” E i = (  j=1,…,n i E ij ) / n i Sample calculation: E 1 = (.18+.88+.33+.29 +.14+.24+.15+.04)/8

16. Expected Mortality Rates for All Time Periods

17. Plot of Expected Mortality Plotting expected mortality helps interpret the observed rates but does not settle the question of whether differences are due to chance.

18. Step Four: Calculate Expected Deviation E ij = Expected mortality of case ‘j’ in time period “i” D i = Standard deviation of expected mortality in time period “i”, called by us as the Expected Deviation D i = (  j=1,…,n i E ij (1-E ij )) 0.5 / n i See sample calculation

19. Expected Deviation for Time Period 1 Expected deviation A B C D

20. Results: Expected Deviations for All Time Periods

21. Step Five: Calculate Control Limits UCL i = Upper control limit for time period “i” LCL i = Lower control limit for time period “i” t = Constant based on t-student distribution UCL i = E i + t * D i LCL i = E i - t * D i Where for 95% confidence intervals: Degrees of freedom 95% student t value 72.37 82.31 92.26 102.23

22. Calculation of Control Limits for Time Period 1 UCL =.28 + 2.37 *.13 =.59 LCL =.28 –2.37 *.13 = -.03 t-value Negative limits are set to zero as negative probabilities are not possible

23. Results: Control Limits for All Time Periods

24. Plot Control Limits UCL LCL

25. Step Six: Interpret Findings There are no points above UCL. There is one point below LCL. In time period 3, mortality is lower than what can be expected from patient’s conditions. All other time periods are within expectations, even time period 7 with its high mortality rate is within expectation.

26. Step Seven: Distribute Control Chart Include in the information: How was severity measured and expected mortality anticipated? Why are assumptions met? What does the control chart look like? What is the interpretation of the findings?

27. Index of Content Click on the Slide You Wish to Review 1. Check assumptions Check assumptions 2. Calculate and plot observed mortality Calculate and plot observed mortality 3. Calculate expected mortality Calculate expected mortality 4. Calculate expected deviation Calculate expected deviation 5. Calculate and plot control limits Calculate and plot control limits 6. Interpret findings Interpret findings 7. Distribute control chart Distribute control chart