1. CUSP 4 MVP – VAP Improving Care for Mechanically Ventilated Patients Data 3: Introduction to Objective Outcome Measures ARMSTRONG INSTITUTE FOR PATIENT SAFETY AND QUALITY Johns Hopkins University March 4, 2015

2. 2 CUSP 4 MVP – VAP: Improving Care for Mechanically Ventilated PatientsData 3: Introduction to Objective Outcome Measures CUSP 4 MVP - VAP Comprehensive Unit-based Safety Program for Mechanically Ventilated Patients and Ventilator-Associated Pneumonia

3. 3 CUSP 4 MVP – VAP: Improving Care for Mechanically Ventilated PatientsData 3: Introduction to Objective Outcome Measures Polling Question Who is on the call? IP – infection preventionist RN – registered nurse RT – respiratory therapist PT – physical therapist OT – occupational therapist MD – medical doctor Quality improvement/ patient safety professional Healthcare executive Educator National project team Other 

4. Objective Outcomes Michael Klompas MD, MPH, FRCPC, FIDSA Harvard Medical School, Harvard Pilgrim Health Care Institute, and Brigham and Women’s Hospital, Boston, USA CUSP for Mechanically Ventilated Patients March 4, 2015 What are they? Why do they matter?

5. Disclosures Grant funding from the US Centers for Disease Control and Prevention Honoraria from Premier Healthcare Alliance for lectures on VAP surveillance

6. Ventilator-associated pneumonia Affects ~5-10% of ventilated patients Increases ICU length of stay by ~4-7 days Increases hospital length of stay by ~14 days Crude mortality rate 30-50% Attributable mortality 8-12% Adds ~$10-50,000 to cost of hospital stay CMS 1533-P, 2007 Safdar et al, Crit Care Med 2005; 33:2184 Tejerina et al, J Crit Care 2006; 21:56 Muscedere et al, J Crit Care 2008;23:5-10 Eber et al, Arch Intern Med 2010;170:347-353 Nguile-Makao et al, Intensive Care Med 2010;36:781-9 Beyersmann et al, Infect Control Hosp Epidemiol 2006;27:493

7. Diagnostic Criteria for VAP High Temp Low Temp High WBC Low WBC Low P:F Ratio Increased vent settings Purulent secretions Gram stain neutrophils New Antibiotic Start Infiltrate CDC Old Definition ✓✓✓✓✓✓✓ CDC New Definition ✓✓✓✓✓✓✓ HELICS Criteria ✓✓✓✓✓ ACCP Criteria ✓✓✓✓✓ Clinical Pulmonary Infection Score ✓✓✓✓✓✓✓ Johanson’s Criteria ✓✓✓✓ Ego et al. Chest 2015;147:347-355

8. Impact of Diagnostic Criteria on VAP Prevalence Prospective surveillance, 1824 patients, Tertiary Med-Surg Unit, Belgium Ego et al. Chest 2014;ePub ahead of print

9. All VAP Signs Subjective, Non-Specific, or Both The core clinical signs associated with VAP: Radiographic opacities Fever Abnormal white blood cell count Impaired oxygenation Increased pulmonary secretions

11. Interobserver agreement in VAP surveillance 7 IP 1 (11 VAPs) IP 2 (20 VAPs) 3 3 0 1 7 5 IP 3 (15 VAPs) Klompas, AJIC 2010:38:237 Kappa = 0.40 50 ventilated patients with respiratory deterioration

12. 6 Case Vignettes Presented to 43 Surveyors Crit Care Med 2014;42:497

13. Physician Agreement Poor Series of 84 ICU patients with abnormal chest x-rays and purulent sputum Evaluated by 7 physicians for VAP “True diagnosis” established by histology or quantitative bronchoscopy cultures 32% found to have VAP Physicians disagreed on presence or absence of VAP in 35/84 (42%) of patients The “best” doc missed 28% of true VAP’s The “worst” doc missed 50% of true VAP’s Both labeled ~20% of patients without VAP as having VAP Fagon et al, Chest 1993; 103:547-53

14. Accuracy of clinical diagnosis of VAP Relative to 253 autopsies 80% 100% Sensitivity / Positive Predictive Value 60% 40% 20% 0% Positive Predictive Value Tejerina et al., J Critical Care 2010;25:62 Sensitivity Loose definition: Infiltrate and 2 of temp / wbc / purulence Strict definition: Infiltrate and 3 of temp / wbc / purulence

15. Accuracy of quantitative BAL cultures Relative to histology 80% 100% Sensitivity / Positive Predictive Value 60% 40% 20% 0% Positive Predictive Value Kirtland, Chest 1997;112:445 Fabregas, Thorax 1999;54:867 Chastre, Am Rev Respir Dis 1984;130:924 Torres, Am J Resp Crit Care Med 1994;149:324 Marquette, Am J Resp Crit Care Med 1995;151:1878 Papazian, Am J Resp Crit Care Med 1995;152:1982 Sensitivity

16. Implications for prevention

17. from doctorrw.blogspot.com

18. The VAP Prevention Paradox VAP Rates Vent LOS ICU LOS Hospital LOS Death Regular oral care with chlorhexidine Silver-coated endotracheal tubes Head-of-bed elevation Crit Care 2009;13:315

19. Silver-Coated Endotracheal Tubes VAP Rates and Outcomes VAP Incidence VAPs per 100 Patients 4.0 5.0 3.0 2.0 1.0 0 8.0 10.0 6.0 2.0 0 4.0 Conventional ETTsSilver coated ETTs Lengths of Stay (days) Vent days ICU days Hospital days 6.0 12.0 JAMA 2008;300:805

20. Routine Oral Care with Chlorhexidine Meta-analysis suggests CHG lowers VAP rates Risk Ratio 0.72 (0.55-0.94) Lancet Infectious Dis 2011;11:845

21. Routine Oral Care with Chlorhexidine Impact on mean duration of mechanical ventilation: NONE! Impact on ICU length-of-stay NONE! Impact on mortality POSSIBLE INCREASE!!! RR 1.13 (0.99 to 1.28) JAMA Internal Med 2014;174:751

22. Why the Paradox? “VAP” as measured includes a multitude of conditions, ranging from benign to serious  Less serious: bacterial colonization superimposed upon atelectasis / CHF / barotrauma  More Serious: pneumonia, ARDS, pulmonary infarction, others Most prevention measures work by decreasing oral secretions or bacterial colonization  Circularity between mechanism of action and diagnostic criteria  Preferential impact on less serious events leading to lower perceived VAP rates, smaller impact on more serious events For any given intervention, difficult to know if primary impact is on less serious or more serious events since both count as “VAPs” Klompas, Critical Care 2009;13:315

23. Critical Care Medicine 2013;41:2467-2475

24. Ventilator -associated conditions (VAC) Rise in daily minimum PEEP ≥3cm or FiO2 ≥20 sustained ≥2 days after ≥2 days of stable or improving daily minimum PEEP or FiO2

25. http://www.cdc.gov/nhsn/VAE-calculator

26. Impact of VAEs on length-of-stay Controlled for time to VAE, age, sex, unit, comorbidities, severity of illness. All comparisons are to patients without VAE (control). Control VAC *** IVAC *** Possible VAP *** Probable VAP *** Control VAC *** IVAC *** Possible VAP *** Probable VAP *** Days Infect Control Hosp Epidemiol 2014;5:502-510

27. Impact of VAEs on mortality Odds Ratio or Hazard Ratio 110520.5 USA – 3 centers PLoS ONE 2011;6:e18062 USA – 8 centers Crit Care Med 2012;40:3154 Canada – 11 centers Chest 2013;144:1453 Netherlands – 2 centers Am J Resp Crit Care Med 2014;189:947 USA – 2 centers Crit Care Med 2014;ePub USA – 1 center Infect Control Hosp Epidemiol 2014;5:502 VAEVAP

28. The VAP Prevention Paradox VAP Rates Vent LOS ICU LOS Hospital LOS DeathVAEs Regular oral care with chlorhexidine ? Silver-coated endotracheal tubes ? Head-of-bed elevation ? Crit Care 2009;13:315

29. The Upshot VAP rates are unreliable outcomes VAE rates likely reliable but still very new. Unclear how best practices will impact them. If we want to know whether a prevention measure really works or not, we have to look at objective outcomes such as: duration of mechanical ventilation ICU length-of-stay hospital length-of-stay mortality

30. Summary VAP diagnosis is subjective and non-specific Inconsistent association between VAP and patient outcomes Many interventions purportedly lower VAP rates but no impact on patient outcomes: “the VAP Prevention Paradox”. Makes VAP an unreliable outcome VAEs more objective and consistently predict adverse outcomes… but the definitions are still very new, very little data so far on how prevention strategies impact VAE rates Implication: need to look at objective outcomes if we want to be sure that CUSP 4 MVP is helping our patients

31. Michael Klompas (mklompas@partners.org) Thank you!

32. CUSP4MVP – VAP Comprehensive Unit-based Safety Program for Mechanically Ventilated Patients- Ventilator Associated Pneumonia Objective Outcomes Registry

33. Overview of the Objective Outcomes Data Entry and Data Upload Review of Objective Outcomes Performance Monitor Review of Network Performance and Reports Page Objective Outcomes Registry – Training Overview

34. Home Page Site can be accessed directly at: https://armstrongresearch. hopkinsmedicine.org https://armstrongresearch. hopkinsmedicine.org Option to login on or browse. Some pages only available with log-in The “Return to Armstrong Institute – Home” will take you back to the Armstrong Institute’s homepage Clicking on Project will take you to the Project Home Page

35. Tools that Focus on Eliminating Ventilator- Associated Pneumonia CUSP4MVP-VAP Home Page

36. My Projects Shows all the available projects and will highlight the one(s) you are participating in You will be prompted to login before accessing this page Clicking on the project will take you to the CUSPMVP- VAP Project page Mary Schmidt

37. CUSP4MVP-VAP My Tools Page Ability to Search by Tool Allows you to quickly access various tools your network(s) are registered for. Coordinating Entities will have the ability to view Tools registered to their Children networks. Mary Schmidt CUSP4MVP-VAP Test Network Tool Name Network which Data will be Entered for Search by Tool Name or by Network Name

38. Manage/View CUSP4MVP-VAP National Tools CUSP4MVP-VAP Project Page Displays your data entry tools Your access to a tool is determined by your role - whether you are a member or an administrator. For example, only administrators will see the “Manage” button for HSOPS. Displays the Network entering data. Ability to click on network name to navigate to network Network Admins see the Manage Button Faith Memorial ICU as Jane Tester

39. My Networks View Networks Shows all the Networks that you are associated with Ability to search by name of Network Mary Schmidt

40. Enter data by accessing the Events Tab Sort previous entries by Column Titles Sort by Filter Criteria as well Ability to Edit or Delete entries Objective Outcomes Add New Event Faith Memorial ICU as Jane Tester

41. Objective Outcomes – Data Entry

42. Help icons available for each data entry field. Data entry grid opens in modal overlay. Date validation enabled on form.

43. Objective Outcomes Add New Event Enter data by uploading a spreadsheet- very convenient! Faith Memorial ICU as Jane Tester

44. Objective Outcomes Add New Event via Upload Faith Memorial ICU

45. Objective Outcomes Add New Event via Upload

46. Objective Outcomes Reports Faith Memorial ICU as Jane Tester My Rate will show your performance rate as well as an indicator of whether “Higher is Better” or “Lower is Better”

47. Improvement Opportunities display the number of events, which if eliminated improve your measure value Faith Memorial ICU as Jane Tester Objective Outcomes Reports

48. View Chart Interactive chart that allows selection of your comparators and time periods Ability to download PDF Hover over data point to display your rate and the comparators Ability to turn comparators ON and OFF

49. My Networks My Network Performance Network Admins will see the measures from the Tools for each of the networks they are associated with Reports available for download

50. My Networks My Reports Allows you to run all reports from a single interface. Reports are also available within their specific data entry tool. Tools, Networks and Reports are displayed based upon what you are associated with

51. Recommended System Requirements Firefox 17.0+, Internet Explorer 8.0+, Safari 4.0+ Adobe Acrobat 5.0+ (For downloading reports) Excel 2000 + (For downloading reports) Broadband connection with 500+ Kbps

52. Support Technical Support Contact Information: cusp4mvp@jhmi.edu

53. Thank You! Questions?

54. Next Steps CUSP 4 MVP – VAP Improving Care for Mechanically Ventilated Patients

55. 55 CUSP 4 MVP – VAP: Improving Care for Mechanically Ventilated PatientsData 3: Introduction to Objective Outcome Measures Next Steps: Homework Determine information sources for the Objective Outcome Measures Collect and enter Objective Outcome Measures data into the data portal Continue collecting and entering VAE Surveillance data

56. 56 CUSP 4 MVP – VAP: Improving Care for Mechanically Ventilated PatientsData 3: Introduction to Objective Outcome Measures Mark Your Calendar: Upcoming Content Webinars For current schedule of upcoming project webinars, visit https://armstrongres earch.hopkinsmedi cine.org/cusp4mvp/ schedules.aspx

57. 57 CUSP 4 MVP – VAP: Improving Care for Mechanically Ventilated PatientsData 3: Introduction to Objective Outcome Measures CUSP 4 MVP – VAP Website Visit: https://armstrongrese arch.hopkinsmedicine.org/cusp4mvp.aspx https://armstrongrese arch.hopkinsmedicine.org/cusp4mvp.aspx

58. 58 CUSP 4 MVP – VAP: Improving Care for Mechanically Ventilated PatientsData 3: Introduction to Objective Outcome Measures What Can I Find on the CUSP 4 MVP – VAP Website? Education materials –Daily Care Toolkits –SAT/SBT Protocol –SAT/SBT Literature Review –SAT/SBT Fast Fact Sheet Exposure Receipt Assessment tool CUSP Tools and Guides Archive of webinars led by subject matter experts https://armstrongresearch.hopkinsmedicine.org/cusp4mvp.aspx

59. 59 CUSP 4 MVP – VAP: Improving Care for Mechanically Ventilated PatientsData 3: Introduction to Objective Outcome Measures