1. Adapting AHRQ Patient Safety Indicators to QIO Data Jocelyn Andrel, MSPH Charles P. Schade, MD, MPH Patricia Ruddick, RN, MSN

2. Outline of Presentation What are AHRQ Patient Safety Indicators? How can you use QIO data to get them? What are their characteristics in one state? How can you share them with hospitals? What do one state’s hospitals think of them? How do they relate to other evidence about safety in a state’s hospitals?

3. AHRQ Patient Safety Indicators What they are How to compute them

4. AHRQ Patient Safety Indicators: Background Early 1990s Developed by the Agency for Healthcare Research and Quality (AHRQ) to measure the safety of hospital care using administrative inpatient discharge data. The Indicators screen for problems that patients experience as a result of exposure to the healthcare system.

5. Concept of PSIs Based on conditions that clearly reflect medical error (foreign body left in) Based on conditions that could reflect medical error (PE or DVT) Not based on underlying comorbidities

6. Steps to determine PSIs 1.Define the concepts and the evaluation framework 2.Search the literature to identify potential PSIs 3.Develop a candidate list of PSIs 4.Review the PSIs 5.Evaluate the PSIs using empirical analysis

7. Limitations Some events don’t show up in discharge data –Adverse drug reactions –Medical events –Psychiatric events. Administrative data may not address finer detail Patient Safety Indicators should be used to prompt investigation into areas where the hospital could potentially improve quality of care

8. PSIs Accidental puncture or laceration Complications of Anesthesia Death in low mortality DRGs Decubitus Ulcer Failure to Rescue Foreign body left in during procedure Iatrogenic pneumothorax Postoperative hemorrhage or hematoma Postoperative hip fracture

9. PSIs continued Postoperative physiologic and metabolic derangement Postoperative pulmonary embolism or DVT Postoperative respiratory failure Postoperative sepsis Postoperative wound dehiscence Selected infections due to medical care Transfusion reaction Plus 4 Obstetric measures not addressed here

10. Converting ISAT data General Instructions from AHRQ –1. The data must be in SAS –2. You may have to recode specific data elements to match what is used in the software. Fortunately, conversion of the ISAT file to comport with the AHRQ input requirements is fairly simple

11. Conversion Elements Creating/Formatting Variables –Age –Length of Stay –Create variables for the number of diagnoses and the number of procedures –Set payor to the code for Medicare –Format Hospital codes, Race, Sex, Key, Hospital ID, DRG, Admission Source, Admission Type –Rename Diagnosis and Procedure codes Major Diagnostic Codes from the HSE Claims Lookup Table

12. And then… The ISAT file is ready to be input into the AHRQ Patient Safety Indicator programs

13. AHRQ Patient Safety Indicators: Results in a Single State

14. Methods Adapted standard output (psp3 table at hospital level) to a graphic display and comparative report Generated histograms of hospital performance on each indicator for 2000- 2002 Generalized code to run with any state’s data as input

15. Results: Distribution of Hospitals Some indicators appeared normally distributed Some were highly skewed, with outliers Some appeared bimodal

16. AHRQ Risk Adjusted PSI Rate Failure to Rescue WV Hospitals, 2002 Death rate in discharges with potential complications of care, e.g., pneumonia, DVT/PE, sepsis, acute renal failure, shock/cardiac arrest, GI hemorrhage/acute ulcer. Definition

17. AHRQ Risk Adjusted PSI Rate Selected Infections Due To Medical Care WV Hospitals, 2002 Discharges with ICD-9-CM code of 999.3 or 996.62 in any secondary diagnosis field excluding immunocompro- mised and cancer Definition

18. AHRQ Risk Adjusted PSI Rate Post-Operative Sepsis WV Hospitals, 2002 Elective surgical discharges with ICD-9-CM code for sepsis in any secondary diagnosis field excluding immunocompromised and cancer Definition

19. Results: Statewide Values Over 3 Years We also used the following format for the tabular report to individual hospitals Most indicators based on small numerators statewide and appeared to show statistical fluctuation from year to year Failure to rescue declining? Postop sepsis and DVT/PE increasing?

23. Report to Hospitals Calendar year 2002, with offer of other years’ results Tabular (see previous) and graphical format Explanatory letter, definitions of indicators Mailed to hospital patient safety contact or HCQIP contact Asked for feedback on report contents and utility

25. Patient Safety Indicators: Implications for WVMI’s Patient Safety Project

26. Specific Goals of the WV Patient Safety Project Establish a system of confidential reporting for medical errors and near misses Stimulate reporting of such events by developing a non-punitive response system Provide feedback of surveillance data at appropriate levels of aggregation Educate consumers of healthcare about patient safety guidelines

27. Comparing PSI Data to the Patient Safety Data Purpose: 1. Ascertain the usefulness of the PSI data in hospitals in West Virginia 2. Compare the data received from the PSI data to the data received from the Patient Safety Project 3. Explore further opportunities for quality improvement projects

28. PSI/Patient Safety Data Study CEOs and Quality Improvement staff from 41 acute care West Virginia hospitals received: 1.Information letter 2.Patient Safety Indicator definitions 3.Table which showed the actual number of specific incidences of each PSI (2002), crude and adjusted rates, and comparative percentiles of all hospitals in the state combined 4.Graphical representation of the data presented in the table 5.Brief questionnaire on the usefulness of the graph and tables

29. Patient Safety Questionnaire Feedback on the Patient Safety Indicator Reports Please take a minute or two to tell us your reaction to the enclosed reports. Your responses will be kept confidential and used only for evaluating this project. 1.Please check the box that most closely describes your role in the hospital  Quality improvement staff  Patient safety staff  Medical staff  Clinical nursing staff  Administration  Other ________________

30. 2. Please circle the number indicating the extent to which you agree or disagree with each statement, where: 5 = strongly agree 4 = agree 3 = indifferent 2 = disagree 1 = strongly disagree If a question is not applicable to your situation, please leave it blank. Strongly agree...strongly disagree a. The patient safety indicator reports were easy to understand 5 4 3 2 1 b. The graphic report was easier to use than the tabular report 5 4 3 2 1 Patient Safety Questionnaire, cont.

31. c. The tabular report provided more information than the graphic report 5 4 3 2 1 d. My hospital’s indicator results, compared with the state’s rates, are about what I would have expected 5 4 3 2 1 e. I want to share the report with colleagues in my hospital 5 4 3 2 1 f. I need additional information about one or more of the indicators 5 4 3 2 1 Patient Safety Questionnaire, cont.

32. 3. Please tell us anything you liked about the reports: 4. Please let us know of anything you did not like about the reports: 5. Finally, please let us know any questions you’d like answered about the reports: Patient Safety Questionnaire, cont.

33. Results (14/41 questionnaires returned) 54321 a. The PSI reports were easy to understand28%42%14% b. The graphic report was easier to use than the tabular report 35%28%14% 7% c. The tabular report provided more information than the graphic report 21%28%35%7%14% d. My hospital’s indicator results, compared with the state’s rates, are about what I expected 25% 33%17% e. I want to share the report with colleagues in my hospital 35%28%21%7% f. I need additional information about one or more of the indicators 50%7%14%21%7%

34. Likes/dislikes about the PSI Reports Likes: Good overview of our results Serves as a step for further analysis Great idea-shared this with Department of Medicine Graphs were self-explanatory Dislikes: Leaves many questions unanswered Need more current information Would like to set up and run on their own

35. Examples Patient Safety IndicatorPatient Safety Event Complications of anesthesiaAdverse Clinical Event -sedation management -complication/monitoring Decubitus ulcersAdverse Clinical Event -skin integrity -decubitus Foreign body left in during procedure Adverse Clinical Event -operative/invasive procedure -instrument/needle/sponge count Infection due to medical careAdverse Clinical Event -infection May have to search several fields to find coordinating PSI

36. Conclusions Data captured from PSIs may best be used to investigate potential patient safety problems when hospitals compare PSIs to the coordinating medical error on the incident reporting tool since: Some events don’t show up in discharge data that are captured in the incident reporting tool, e.g. –Adverse drug reactions –Administrative events –Fall events –Employee events –Visitor events PSI data is more general and may have to use several fields in the incident reporting tool to capture complete PSI data

37. WVMI plans to: Compare 2003 PSI data with data from Web-based incident reporting tool for hospitals that are part of the WV Patient Safety Project –Unable to compare 2002 PSI data with patient safety data since the hospitals participating in the Patient Safety Project did not start until middle of 2002; and indicators do not correspond exactly. –Provide this information to each participating hospital in order that they will be able to compare their reporting rates to PSI data

38. Source AHRQ patient safety indicator programs http://www.qualityindicators.ahrq.gov Conversion routines and hospital output code: –jandrel@wvmi.org –cschade@wvmi.org WVMI’s Patient Safety Project –pruddick@wvmi.org