1. REFERENCES & ACKNOWLEDGMENTS Repurposing Decision Support Data to Detect Prescribing Errors—An Application for Quality Measurement David L. Chin, Ashley S. Trask, Victoria T. Johnson, Michelle H. Wilson, Brittanie Neaves, Andrea Gojova, Heejung Bang, Patrick S. Romano Center for Healthcare Policy and Research, (Chin and Romano) and Dept. of Pharmacy (Gojova and Trask) and Div. of General Internal Medicine–Dept. of Internal Medicine (Johnson, Neaves and Romano) and Div. of Biostatistics– Dept. of Public Health Sciences (Bang) University of California, Davis; Dept. of Internal Medicine, Santa Clara Valley Medical Center; San Jose, CA (Wilson) TOPICS FOR DISCUSSION Why: Clinical Decision Support (CDS) data can be used to measure prescribing errors in the hospital. “Real-time” quality measures have not been described. Applications: Measure physician-level quality of care. (future) Provide real-time monitoring, dashboard reporting for QI. Generalizability: This approach could be generalized in any hospital equipped with a computerized physician order entry (CPOE) and CDS. 1.Preventing Medication Errors: Quality Chasm Series. The National Academies Press; 2007. Acknowledgments: The project was supported by the NIH National Center for Advancing Translational Sciences, through grant number UL1 TR000002 and linked award TL1 TR000133. Support was also provided by grant number 1T32HS022236-01 from the Agency for Healthcare Research and Quality (AHRQ) through the Quality Safety Comparative Effectiveness Research Training (QSCERT) Program. AcademyHealth acknowledges the Agency for Healthcare Research and Quality (AHRQ) for its support: Grant U13 HS19564-01.The authors acknowledge Rebecca Davis for her assistance in reviewing the literature, Terry Vierra and Michael Fong for interpreting and making use of the raw data. For more information contact dlchin@ucdavis.edu or visit www.edm-forum.org LESSONS LEARNED Algorithm development and validation required multiple iterations to understand data structure and limitations. For example, upon chart review we found allergy alerts did not correlate with errors. CDS systems are highly customized, reducing the generalizability of source code and potential for interoperability. Although repurposed EHR data can provide many opportunities to conduct research, multiple gaps exist between the acquisition and interpretation of raw data, and operationalizing data for research. Despite these limitations, alert data taken from a CDS can be used to generate useful information: Physician-level real-time quality measurement. Medication order safety profiling. Hospital quality surveillance, monitoring in real-time. Conclusions: We present a systematic and reproducible tool to measure quality real-time in the hospital. Further work is needed to expand this tool to detect other prescribing errors from CDS data and validation in multiple clinical settings. METHODS and RESULTS Background: Medication errors harm 1.5 million patients per year, yet EHR- based method to detect prescribing errors have not been reported. Despite significant improvements in quality of care, medication prescribing errors are still common in hospitals. No methods exist to measure real-time hospital processes or outcomes. Goals of Project: Repurpose clinical data to separate true prescribing errors from non-error noise. Develop a method to measure real-time quality events that is systematic and reproducible. Create a new method to simultaneously measure patient and provider-level elements related to quality of care from EHR data. BACKGROUND and MOTIVATION Setting: One 619-bed academic medical center hospital. Data Sources and Analysis: Medication Prescribing Alerts, Medication orders—Clarity/SQL. Analyses— SAS 9.3. Algorithm Development*: 6,079,783 medication orders prescribing alerts (n=147,420) were generated by a knowledge- based CDS (Epic) beginning Mar 2008 for 45 months. Constructed the prescribing error detection algorithm (PEDA) by identifying prescribing error candidates that had high potential to cause harm. One pharmacist and one physician independently manually reviewed a stratified random sample of events from dose, allergy, drug-drug interaction categories. Dose and drug-drug interaction alerts were included in algorithm. Allergy alerts were not associated with prescribing errors (false positive rate >80%). PEDA Performance: Dose: PPV= 96%; NPV = 32% Drug-drug interaction: a blinded validation was not feasible. Prescribing Error Rates: Hospital Setting ICU: 19.8 errors/100 alerts Internal Medicine: 11.7 errors/100 alerts Emergency Dept: 12.9 errors/ 100 alerts Surgery: 12.2 errors/100 alerts Physician Type: Intern: 14.2 errors/100 alerts Resident: 10.7 errors/ 100 alerts Fellow: 33.2 errors/ 100 alerts Faculty: 16.6 errors/ 100 alerts * Exclusions include pediatric and outpatient orders. Alert criteria are defined by First Databank. Figure 2: Data stream and development processes Dose (n=19,616 ) Exclusion Criteria by Alert Characteristic: Not physician Pediatric order Removed (order withdrawn) Other category (Pregnancy, Drug-Drug, etc.) Inpatient Alerts (n = 147,420) Errors (n=9,965) Non- errors (n=1,223) Error Decision Drug-Drug Interaction (n-21,403) Errors (n=2,243) Non-errors (n=19,160) Error Decision Exclusion Criteria by Order Characteristic: Opioids Benzodiazepines Medication Orders (n = 6.0M) CDS Figure 1: Example of prescribing alert Error definitions: Dose: if the order exceeded the maximum daily or the maximum single dose, by ≥ 20%**. (Opioids, benzodiazepine-class and inhaled steroid medications were excluded). ** Exceptions: NSAID threshold was defined as 13%. Drug-drug interaction: if two drugs ordered were duplicative therapy or from the same class.