1. Resident Research Conference November 20, 2013 Alex Bryant Physician handling of medication order alerts

2. Overview What are electronic order alerts? Rationale for study Objectives and Hypothesis Design and scope Findings Discussion Future directions and quality improvement Questions and acknowledgements

3. A familiar frustration?

4. Why is the EMR alerting my orders? Part of CMS “Meaningful Use” criteria for EMR  All practice groups who meet MU criteria get incentive pay  If no meaningful use of EMR by 2015, reimbursement is cut  Requirements include CPOE, clinical decision support, allergy and drug interaction checking at medication entry Electronic order entry with medication checking reduced errors and adverse drug events (ADEs) 1  Serious ADE rates dropped from 0.7% to 0.1%  Caveat: multiple simultaneous interventions Bates et al. JAMA 1998;280:1311-16.

5. Are order checks preventing patient harm? Extremely difficult to answer  Alerts now integral in most commercial EMRs  Multiple confounders, difficult to detect ADEs Providers strongly dislike intrusive alerting 2  Is this intervention still beneficial, or just distracting? Horn et al. Am J Health-Syst Pharm 2013;70:905-9.

6. Override rates: a proxy for relevance Alert override rates serve to gauge clinical utility  Alerts are meant to be sensitive, not specific  Still, 50-90% of alerts are overridden  No change in override rates despite 10+ years of QI effort  Partly attributed to “alert fatigue” and information overload 80-90% of alerts overridden at our VA in 2006 3 Lin et al. JAMIA 2008; 15:620-6.

7. Surely the UW can do better? Ongoing effort to improve alert relevance since 2008  Implemented many usability changes from the literature Panel of MDs, RNs, pharmacists, IT staff meets monthly to review alert and override statistics  Integrate expert opinions on interaction risk and feedback from practicing physicians  Low-risk or irrelevant interactions are removed from alerting, or downgraded so that only Pharmacy sees them Shouldn’t alerts be more relevant and accepted now?

8. Objectives and Hypotheses Objective: Analyze critical medication alert override rates and associated factors at UWMC & HMC Hypotheses Rates will be lower than historical norms, including those at our VA, due to ongoing improvement efforts Physicians who see more alerts will be more likely to override due to “alert fatigue”

9. Capturing physician behavior Required reason Alert type (only “critical” interactions alerted) OVERRIDE Override gets logged with time, drugs, patient and provider/team

10. MD orders MD override required for any interactions of “critical” severity Pharmacy releases RPh sees all interactions (severe or not), may call MD or hold order Nurse administers Monitor for ADE Medication order processing Order and alert logged

11. Design and Scope Retrospective observational study of all medication orders and “critical” alerts at HMC and UWMC All providers ordering June 10 – June 13 2013 (96h) Filtered to include only physician-entered orders  461 unique MDs saw alerts during this period No observation of behavior outside of alerts

12. Alert data breakdown CategoryTotal % of alerts Override number Override rate Medication orders 18354 Unique alerts2455100 %228092.9 % Interaction type Drug-drug115347.0 %109795.1 % } p < 0.0001 Drug-allergy130253.0 %118390.9 % Physician level Attending48019.6 %45494.6 % } p = 0.11 Resident197580.4 %183092.5 % Hospital HMC120048.9 %111192.3 % } p = 0.25 UWMC125551.1 %117593.5 %

13. High alerting and override rates We’re alerting 13% of orders (2455/18354)  Compare with 2.5% (VA 2006), up to 20% in other studies But higher volume is probably not the reason drug- drug alerts are overridden 95% of the time  Average MD sees only 1 alert per day  < 5 % of MDs see more than 4 alerts per day  MDs with more alerts were not more likely to override

14. Short term “alert fatigue” not significant

15. Why are override rates still so high? Neither hospital site nor training level matter Most interaction pairs are overridden every time  Including antithrombotics, antipsychotics, sedatives, analgesia  Almost no drug triggers have < 80% override  Reglan + antipsychotics a notable exception  Antibiotics usually have < 90% override, though some higher Allergy alerting data are only slightly better  Due to known inaccuracies in allergy charting?  Even exact allergy matches (9%) have 75% override  Hard to believe such a large override rate is appropriate

16. Provider specified override reason says little Vast majority of us choose “Provider approved” None of the options gives much information for QI 18% of drug-drug alerts had an “Allergy” reason!  Inappropriate selection suggests many users are not even reading the alert window

17. In summary Override rates remain high despite best QI efforts It’s not a problem of pure alert fatigue, but relevance Suspect that providers are ignoring the alert window Large proportion of interactions are always ignored All to suggest that medication order alerts are NOT preventing ADEs at point of entry ...but we can’t prove this without a much more difficult study!

18. How can we improve from here? Improve override reasons and allergy charting? Restriction of alert triggers with 100% override?  i.e. not alerting antipsychotic interactions to Psych MDs  Limited by liability concerns  Cutting down alert volumes alone won’t increase acceptance Can we encourage conscious processing of alerts?  Provide specific risk information in alert window  Suggest alternate therapy

19. Questions? Many thanks to:  Tom Payne for his mentorship  Joe Smith for data and technical assistance  Grant Fletcher, John Horn, and Paul Sutton for their insights Publishing soon!