1. Presenting Data & Standardized Infection Ratios Tim Wiemken PhD MPH CIC Assistant Professor of Medicine Assistant Director of Epidemiology and Biostatistics University of Louisville School of Medicine, Division of Infectious Diseases Clinical and Translational Research Support Center tim.wiemken@louisville.edu Phone: 502.852.4627 www.ctrsc.net

2. Pre-Presentation Comment Do not collect any data you do not plan to use (unless your boss tells you to).

3. Pre-Presentation Comment What constitutes ‘use’? 1.Displaying somewhere other than just a meeting 2.Preliminary to a follow-up to collect data that will be used 3.Education 4.Targeted prevention 5.Surveillance (only if you educate with it!)

4. Overview What makes a good report? How to choose a chart Components of good charts Tips for reports Standardized Infection Ratios

5. Overview What makes a good report? How to choose a chart Components of good charts Tips for reports Standardized Infection Ratios

6. What Makes a Good Report? The catch all answer – it depends! There are some key ideas you should think about when preparing any report

7. What Makes a Good Report? Create reports that can be USED, not just looked at. …more later

8. What Makes a Good Report? 1.How much time do you have?

9. What Makes a Good Report? 2.Who is your audience? How sophisticated are they? What do they already know? Remember that people think and speak differently

10. What Makes a Good Report? 3.Why are you giving the presentation? To show data? (I hope not!) To provide the status of something? To prove your worth? To affect change? To get someone fired?

11. What Makes a Good Report? Each of these helps you decide one key piece of information: What do you NEED to present?

12. What Makes a Good Report? A few rules: 1.Never present more than you need to present.

13. What Makes a Good Report? A few rules: 1.Never present more than you need to present. 2.Never present less than you need to present.

14. What Makes a Good Report? A few rules: 1.Never present more than you need to present. 2.Never present less than you need to present. 3.Keep it as short as possible.

15. What Makes a Good Report? Therefore, focus on what is necessary to meet your goals.

16. What Makes a Good Report? Yeah, so smarty pants, what is necessary?

17. What Makes a Good Report? IT DEPENDS!

18. What Makes a Good Report? Although what is needed depends on how much time you have, your audience, and your goal: 1.It is rarely OK to present raw data by itself:

19. What Makes a Good Report? Infection Prevention and Control Report, May 2012: CLABSI: 5 VAP: 4 CAUTI: 7 MRSA: 10

20. What Makes a Good Report? Infection Prevention and Control Report, May 2012: CLABSI: 5 VAP: 4 CAUTI: 7 MRSA: 10

21. What Makes a Good Report? Infection Prevention and Control Report, May 2012: This is only slightly better.

22. What Makes a Good Report? Although what is needed depends on how much time you have, your audience, and your goal: 2.It is rarely OK to present significant amounts of text –E.g. policies, procedures, etc.

23. What Makes a Good Report? 2.It is rarely OK to present significant amounts of text –No one wants a six page policy handed to them in a meeting to review. –It will not get reviewed (at least not well). –Send these out early… and many times.

24. What Makes a Good Report? The Holy Order of Data Quality (worst to best). 1.Text 2.Raw numbers 3.Raw rates 4.Tables of chronological numbers or rates 5.Charts (hierarchy of charts later…) 6.Charts with #4 7.Charts with #1, #3, and more fun statistics

25. Overview What makes a good report? How to choose a chart Components of good charts Tips for reports Standardized Infection Ratios

26. Choosing a Chart Charts –Numbers alone are bad

27. Choosing a Chart Charts –Visual depiction is easier to quickly comprehend

28. Choosing a Chart Charts –Visual depiction is easier to quickly comprehend –Choosing the correct chart is extremely important

29. Choosing a Chart Charts –Visual depiction is easier to quickly comprehend –Choosing the correct chart is extremely important –Some charts show trends or comparisons in data

30. Choosing a Chart Charts –Visual depiction is easier to quickly comprehend –Choosing the correct chart is extremely important –Some charts show trends or comparisons in data –The best chart does not have to be explained

31. Choosing a Chart Most common/useful types of charts 1.Pie 2.Bar 3.Line 4.Radar

32. Choosing a Chart Most common/useful types of charts 1.Pie

33. Choosing Charts Pie –Less appropriate than one would think –Categorical data that comprise of portions that add to 100% –Never use 3D pie charts –Do not use a pie chart if you have a lot of categories

34. Choosing Charts Pie –Do not use a pie chart if you have a lot of categories

35. Choosing Charts Pie “Use a pie chart when you don’t have anything to say” - Dr. Julio Ramirez “Use a pie chart when you don’t have anything to say” - Dr. Julio Ramirez

36. Pie Charts

37. No!

38. Choosing a Chart Most common/useful types of charts 2.Bar

39. Choosing Charts Bar –Snapshots of data [rates in one month] –Comparing data across different categories –Never use 3D charts unless you have a 3 rd dimension!

40. Bar Charts

41. No!

42. Bar Charts Is it ever appropriate to use 3D charts?

43. Yes! (Really?)

44. Choosing a Chart Most common/useful types of charts 3.Line

45. Choosing a Chart Line –Longitudinal data (data over time) –Contiguous points

46. Choosing a Chart Line –Longitudinal data (data over time) –Contiguous points –Rates, Counts, etc. by Month/Quarter/Year Run charts Statistical Process Control charts

47. Choosing a Chart Run charts –Use when you have few time periods (e.g. <25 months).

48. Choosing a Chart Run charts –Anatomy Center Line / Median –represents the median of all of the data points. X-axis –represents the time period of interest (days, weeks, months, quarters, years). Y-axis –represents the scale of the plotted data points (e.g. rate or count of infection). Data points – the actual data values.

49. Choosing a Chart Run charts Y-axis (Rate) X-axis (Month) Center Line (Median) Data Points (<25)

50. Choosing a Chart Run charts –Use Use to identify when the data are different than you expect (for better or worse) through detecting abnormal variation

51. Choosing a Chart Run charts –Rules for abnormal variation 1.Seven or more consecutive points on either side of the Center Line (median). 2.Five or more consecutive points increasing or decreasing. 3.Fourteen or more consecutive points alternating up and down.

52. Choosing a Chart Run charts Rule 1 7 points below median Rule 2 5 consecutive points increasing

53. Choosing a Chart Statistical Process Control (SPC) charts –Use when you have many time periods (e.g. ≥25 months). –These are much better than run charts.

54. Choosing a Chart SPCcharts –Anatomy Center Line / Mean–represents the average of all of the data points. X-axis –represents the time period of interest (days, weeks, months, quarters, years). Y-axis –represents the scale of the plotted data points (e.g. rate or count of infection). Data points – the actual data values. Standard deviation lines (control limits) – represent 1, 2 or 3 standard deviations on each side of the center line

55. SPC Chart Components MeanMean

56. +1 SD -1 SD

57. SPC Chart Components +2 SD -2 SD

58. SPC Chart Components +3 SD -3 SD Upper Control Limit (UCL) Lower Control Limit (LCL)

59. Types of SPC Charts Choosing the correct chart can be difficult - it is also very important. Control limits are calculated differently for each chart....although they are marked in standard deviations, the formulas vary...

60. Types of SPC Charts

61. Choosing a Chart SPC charts –Use Same as Run Charts, except Identify different types of variation

62. Variation Common Cause - ‘ In Control’ Special Cause - ‘Out of Control’

63. Rules to Detect Special-Cause Variation one point above or below 3SD two of three points above/below 2SD four of five points above/below 1SD eight points in a row on either side of the mean trends of 6 points in a row increasing or decreasing fifteen points in a row within 1SD fourteen points in a row alternating up and down eight points in a row outside of 1SD

64. Variation Common Cause - ‘ In Control’ Special Cause - ‘Out of Control’

65. A Special SPC Chart Often it is useful to provide reports that identify the ‘time since the last event’. This motivates and provides some competition

66. A Special SPC Chart Rare events make terrible charts. Rates for rare events do not show you much.

67. A Special SPC Chart You can combine the benefits of the ‘time between events’ and the issues with rare events into an SPC g Chart.

68. C chart (number of events) g chart (time between events)

69. Choosing a Chart Most common/useful types of charts 4.Radar (Spider)

70. Choosing Charts Radar –Rarely appropriate – confusing –May work for showing multiple percentages in one small chart – Bar chart may still be better.

71. Radar Charts

72. Overview What makes a good report? How to choose a chart Components of good charts Tips for reports Standardized Infection Ratios

73. Components of Good Charts Good charts tell the complete picture of what they contain

74. Components of Good Charts Good charts tell the complete picture of what they contain Ideally, someone who is not familiar with the data being presented should be able to easily and quickly understand what the chart is representing

75. Components of Good Charts Never use 3D charts unless you have a 3 rd dimension

76. Components of Good Charts The scale used for charts (that have a scale) should begin at zero and end near the highest value –Changing the axis scale can tell a drastically different story to those that are just glancing

77. Same Data, Different Scale What would you say about these two services? MRSA colonization rate by service

78. Same Data, Different Scale MRSA colonization rate by service How about now?

79. Overview What makes a good report? How to choose a chart Components of good charts Tips for reports Standardized Infection Ratios

80. Tips for Reports Title must be very specific –Include person (organism/disease state/compliance measure, etc.), place and time Example: –Hospital-associated Methicillin-resistant Staphylococcus aureus Isolates: Hospital X, Acute Care ICU, January 2007-January 2010

81. Tips for Reports Data definitions should be included as a footnote or on an introductory page.

82. Tips for Reports Data definitions should be included as a footnote or on an introductory page. Example: Clostridium difficile –Definition for numerator Are you measuring toxin only? Antigen? By PCR? On liquid stool only? –Definition for denominator Patient-days? Admissions?

83. Example introductory slide for MRSA rates Hospital-associated Methicillin- resistant Staphylococcus aureus (MRSA) Case of MRSA (Numerator): A case of MRSA was defined as a new and unique, hospital-associated (isolated >48 hours after admission), microbiological isolate from a patient admitted to hospital x during the month of interest without a prior history of MRSA. Patient days (Denominator): The denominator for the calculation of the rate of MRSA was defined as the number of patient-days for hospital x during the month of interest, regardless of risk status. Rate: (Numerator / Denominator) * 1,000

84. Tips for Reports For charts, you should also include a formula for how the rate was calculated.

85. Tips for Reports For charts, you should also include a formula for how the rate was calculated. –# Unique, hospital-associated MRSA isolates divided by # Patient-days of care X 1,000

86. Tips for Reports For charts, you should also include a formula for how the rate was calculated. –# Unique, hospital-associated MRSA isolates divided by # Patient-days of care X 1,000 –Be specific. Isolates, Infections, Colonizations, etc.

87. Tips For Reports Include the appropriate chart!

88. Tips for Reports Raw data is important to be able to understand what is in the chart and to verify the data.

89. Tips for Reports Raw data is important to be able to understand what is in the chart and to verify the data. Some people just like the numbers.

90. Tips for Reports Raw data is important to be able to understand what is in the chart and to verify the data. Some people just like the numbers. Raw data should include numerator, denominator and standardized rate.

91. Tips for Reports Raw data is important to be able to understand what is in the chart and to verify the data. Some people just like the numbers. Raw data should include numerator, denominator and standardized rate.

92. Tips for Reports Including aggregated data for quick comparisons is also a good idea. Yearly average rates may work. When possible, include P-values to detect statistically significant differences (rate comparison example to come..)

93. Tips for Reports Always include a description of the assessment of the chart and your plans.

94. Tips for Reports Always include bubbles or call-out boxes that adequately describe dates and components of any interventions.

95. Tips for Reports You may use a bubble to describe if your surveillance definition changes.

96. Tips for Reports You may use a bubble to describe if your surveillance definition changes. A better idea is to just start over.

97. Hospital-Associated Methicillin-resistant Staphylococcus aureus Isolates: Hospital X, MICU, January 2006 – January 2010 # HA MRSA Isolates divided by # Bed-days of Care Assessment: Process is in statistical control. Plan: Continue surveillance activities.

98. Tips for Reports Some reports must be text, although every attempt to transform into a graphic should be made.

99. Tips for Reports Some reports must be text, although every attempt to transform into a graphic should be made. Keep text to a minimum with summary information presented.

100. Tips for Reports Some reports must be text, although every attempt to transform into a graphic should be made. Keep text to a minimum with summary information presented. Email out early and often.

101. Tips for Reports Another option is a dashboard. This may be particularly useful for providing back to staff to post in their work areas. After all – a report that isn’t used for something other than looking pretty is not particularly worth your time!

102. Tips for Reports

103. KHA has a wonderful template for you!

104. Overview What makes a good report? How to choose a chart Components of good charts Tips for reports Standardized Infection Ratios

105. Standardized Infection Ratio SIR A way to standardize/risk adjust rates for better comparability across institutions

106. Standardized Infection Ratio SIR Why are rates not comparable? –Different types of institutions have different patient populations –These patients have different risks. –Therefore, comparing a University Hospital MICU to a 4 bed Cardiac Unit in Wyoming is not useful.

107. Standardized Infection Ratio SIR SIR takes into account the types of patients seen in various different types of units or wards in a healthcare facility.

108. Standardized Infection Ratio SIR Compares the number of infections you have to the number you expect to see, based on some benchmark rate.

109. Standardized Infection Ratio SIR –SIR>1 : Worse than expected numbers –SIR <1 : Better than expected numbers –SIR = 1 : As expected

110. Standardized Infection Ratio Advantages –Adjusts based on benchmarked rates –Adjusts for various locations [ICU vs Ward] or types of surgeries, etc. –Excellent for comparing low numbers [removes high variation] –Greater precision for adjustment Disadvantages –Statistical bias = may not make appropriate comparisons –Utility beyond SSI is relatively unstudied –Wide understanding is just not there. ICHE April 2006, 27(4): 427-429. ICHE January 2005, 26(1):8-9.

111. SIR for CLABSI 1.Get observed numerator as usual

112. SIR for CLABSI 1.Get observed numerator as usual 2.Get number of patient-days (line-days, vent-days, etc) as usual

113. SIR for CLABSI 1.Get observed numerator as usual 2.Get number of patient-days (line-days, vent-days, etc) as usual 3.Get NHSN benchmark rate from the relevant publication

114. SIR for CLABSI 1.Get observed numerator as usual 2.Get number of patient-days (line-days, vent-days, etc) as usual 3.Get NHSN benchmark rate from the relevant publication 4.Multiply benchmark rate by the number of patient days (line days, ventilator-days, etc)= expected number of infections

115. SIR for CLABSI Divide your observed number by the expected number = SIR!

116. SIR for CLABSI Example 1.Get observed number of CLABSIs as usual: 5

117. SIR for CLABSI Example 1.Get observed number of CLABSIs as usual: 5 2.Get number of line-days as usual: 1,234

118. SIR for CLABSI Example 3.Get NHSN benchmark rate from the relevant publication Can compare to different percentile rates if that is your goal!

119. SIR for CLABSI Example 1.Get observed number of CLABSIs as usual: 5 2.Get number of line-days as usual: 1,234 3.Get NHSN benchmark rate from the relevant publication: 1.9 4.Multiply benchmark rate by the number of line-days and divide by 1,000 = expected number of infections: (1.9 *1,234) / 1,000 = 2.35

120. SIR for CLABSI Example Divide your observed number by the expected number = SIR!: Observed (5)/ Expected (2.35) = 2.13

121. SIR for CLABSI Example SIR = 2.13 You are doing worse than expected given the benchmark.

122. SIR for CLABSI This method calculated an SIR for one particular unit. You can calculate an overall SIR that adjusts for multiple types of units. This makes it more comparable to other institutions. Lets see how this works…

123. SIR for CLABSI Example Using Free Software http://innovationsforglobalhealth.com/ipstat/

124. Standardized Infection Ratio and SPC SIRs can be plotted on an XmR chart and may be more sensitive to detect special causes (very much controversy here!)

125. Final Thoughts Charts are good – use them whenever possible.

126. Final Thoughts Charts are good – use them whenever possible. Knowing your audience is important – don’t assume they know what you know or are interested in what you are interested in.

127. Final Thoughts Charts are good – use them whenever possible Knowing your audience is important – don’t assume they know what you know or are interested in what you are interested in. SIRs may be useful in some situations

128. Final Thoughts Make sure everyone knows how to read your charts POST YOUR REPORTS WHEREVER YOU CAN

129. More time? Example of how to make a control chart using Excel.control chart Calculating P-values to compare ratescompare rates You can download this file at: http://innovationsforglobalhealth.com/files/Stat_Tests_v13.xls

130. Questions? Thanks!