1. INTERVENTIONS TO REDUCE DIAGNOSTIC ERRORS IN AMBULATORY CARE Mark Graber, MD, Stephanie Kissam, MPH, Hardeep Singh, MD, MPH, Asta Sorensen, MA, Nancy Lenfestey, MHA, Elizabeth Tant, BA, Ken LaBresh, MD, and Kerm Henriksen, PhD Collaborators: RTI International, Northport Veterans Affairs Medical Center, Michael E. DeBakey Veterans Affairs Medical Center and the Houston VA HSR&D Center of Excellence AHRQ Action I Master Task Order Mechanism, Contract Number HHSA290200600001 Task 8

2. SPONSORED BY AHRQ

3. PROJECT GOALS  Perform a comprehensive literature review of interventions that could reduce diagnostic errors  Identify and pilot test an intervention targeting diagnostic errors in an ambulatory care setting

4. BLUNT end SHARP end Patient’s Clinical Course SYSTEM MD Communication, coordination, training, policies, procedures Cognitive FRAMEWORK FOR ADVERSE EVENTS

5. ETIOLOGY OF DIAGNOSTIC ERRORS

6. METHODS Handpicked articles:  Non-medical databases (business, psychology, military, engineering)  Recommendations from experts Analysis :  All articles reviewed by one of three health service researchers  Any questionable inclusions reviewed by collaborating physicians PubMed database search: 2000 – 2010

7. INCLUSION CRITERIA  Articles describing tested interventions to reduce error in medical diagnostic settings  Studies demonstrating outcome measures in the field of diagnostic errors  Articles providing a theoretical basis on how to reduce diagnostic errors (from any field)

8. EXCLUSION CRITERIA  Studies describing inter-rater or observer variation  Articles describing validations of screening instruments, tests, case reports, or techniques to enhance diagnosis  Articles describing screening instruments, tests, or technology aides  Studies reporting diagnostic error frequency; etiology; or assessments of provider satisfaction, preference, or acceptance of interventions

9. RESULTS Total number of articles:949 Articles meeting inclusion criteria:157 Tested interventions:37 Cognitive:32 System-related:5 Engaging patients:0 Suggested interventions:120

10. Hardeep Singh, MD MPH Houston HSR&D Center of Excellence, Michael E. DeBakey VA Medical Center INTERVENTIONS TO REDUCE DIAGNOSTIC ERRORS IN AMBULATORY CARE SYSTEMS INTERVENTIONS Result of collaboration between RTI International, Northport Veterans Affairs Medical Center, Michael E. DeBakey Veterans Affairs Medical Center and the Houston VA HSR&D Center of Excellence AHRQ Action I Master Task Order Mechanism, Contract Number HHSA290200600001 Task 8. RTI International is a trade name of Research Triangle Institute

11. SYSTEMS FACTORS  Communication and coordination of care issues (transitions)  Teamwork/Supervision  Technology/equipment related issues  Organizational features  Safety culture  Policy, processes and procedure related issues  Leadership, management, or personnel problems  Inadequate resources or available expertise  Training issues

12. History, exam or ordering diagnostic tests for further work-up Patient-Provider Encounter Ordered tests either not performed or performed/interpreted incorrectly Diagnostic Tests Problems with follow-up of abnormal test results or scheduling of follow-up visits Follow-up and Tracking Lack of appropriate actions on requested consultation or communication breakdown from consultant to referring provider Referrals Delay in seeking care or adherence to appointments Patient Related LOOKING FOR INTERVENTIONS IN THESE “PROCESS” DIMENSIONS…

13. GENERAL RESULTS  Only 1 of 5 controlled study;  2 were only post-test evaluations  All effective  Most interventions in the literature were “conceptual”  Lack of standardization in process or outcome measures

14. PATIENT–PROVIDER ENCOUNTER Change the process of care delivery  Form designated trauma response team in ER  Conduct comprehensive reexamination in ER Establish educational programs (suggested only)  Reinforce history-taking skills  Provide teamwork training in medical setting Perno JF, et al. (2005) Howard J, et al. (2006) 2 tested interventions

15. DIAGNOSTIC TESTS  Implementation of Picture Archiving and Communication System (PACS) for radiology images Weatherburn, G et al. (2000) One tested intervention

16. FOLLOW-UP AND TRACKING Improving delivery of test results through electronic means Other suggested interventions:  Establish criteria for communication of abnormal test results  Standardize steps involved in the flow of test result information  Improve management and presentation of test result data  Use an ER manager to monitor radiology test results reporting  Create processes to ensure easy retrieval of test result information  Develop highly structured hand-off processes that are performed systematically 2 tested interventions Singh, H,et al. (2009) Poon, EG, et al. (2002)

17. suggested only PATIENTS Notify patients of test results  Address patient preferences for receiving test results  Communicate normal test results  Use computerized test results management tool  Designate patient navigator No tested interventions

18. PATIENTS Provide patient access to test results  Use online portal  Provide access to entire medical record Improve patient-clinician communication  Consider cognitive limitations when taking patient history  Consider communication strategies to optimize patient understanding of medical information Increase patient engagement in health care  Involve patients to ensure the follow-up of test results

19. GENERAL INTERVENTIONS (NO SPECIFIC DIMENSION)  Manage error-producing conditions (suggested only)  Provide education on error-producing conditions like fatigue  Address work–related conditions that could produce boredom, time pressure, etc.  Establish systematic tracking of diagnostic error in organization (suggested only)  Downstream feedback

20. CONCLUSIONS – SYSTEM ISSUES  Limited literature on systems interventions that reduced diagnostic error in ambulatory care  Empiric data only for 3/5 dimensions of diagnostic process  Many interventions well conceptualized but poorly operationalized as “testable” interventions  Much discussion of methods to notify patients of test results, but little focused on abnormal results  Health IT potential and workflow related issues

21. CONCLUSIONS – SYSTEM ISSUES  Gaps in tested interventions aimed at patients  Efficacy of patient and family engagement in preventing or reducing diagnostic error?  Multiple organizations and experts advocate for patient engagement in patient safety, yet limited studies successfully do so  No studies report actual interventions engaging patients and families in the process of making medical diagnoses.

22. OPEN DISCUSSION – SYSTEM ISSUES Question How and when can we effectively engage patients and families in diagnostic error reduction?

23. Mark L Graber MD FACP VA Medical Center, Northport NY & SUNY Stony Brook INTERVENTIONS TO REDUCE DIAGNOSTIC ERRORS IN AMBULATORY CARE COGNITIVE INTERVENTIONS Collaborators: RTI International, Northport Veterans Affairs Medical Center, Michael E. DeBakey Veterans Affairs Medical Center and the Houston VA HSR&D Center of Excellence AHRQ Action I Master Task Order Mechanism, Contract Number HHSA290200600001 Task 8.

24. COGNITIVE ERRORS

25. Most cognitive errors involve breakdowns in synthesizing the available data, due to …..  faulty context assumptions  premature closure  the inherent shortcomings of heuristic (intuitive) thinking  affective biases and environmental factors that detract from optimal conditions: distractions, fatigue, stress, workload

26. INTERVENTIONS TO REDUCE COGNITIVE ERROR Improve clinical reasoning Get help Increase medical knowledge and expertise

27. INCREASE KNOWLEDGE & EXPERTISE  Increase training time & events to increase experience (3 tested interventions)  Use simulation to provide compacted experience (1 tested intervention)  Increase feedback to improve calibration and reduce overconfidence (3 tested interventions)

28. IMPROVE CLINICAL REASONING  Improve evidence-based medicine skills, normative decision-making skills  Improve intuitive decision-making  Teach heuristics & biases  Use de-biasing techniques; improve metacognition  Reflective practice; checklists; be comprehensive, consider the opposite suggested only No tested interventions

29. GET HELP  Increase consultation, second opinions, fresh eyes  Use decision support tools; increase access to medical knowledge (web access, texts, info buttons) 10 tested interventions 12 tested interventions

30. CONCLUSIONS - COGNITIVE FACTORS  A broad array of ideas for interventions (N=157), but few tested (N=37)  Gaps:  Most interventions apply to diagnostic specialties (radiology, pathology, laboratory), not the ED or PC  Tests have been done under artificial conditions  Learning assessed only in the short term  Tools developed aren’t used

31. SUGGESTED PROJECT: CHECKLIST(S)  Checklists are ideal in dealing with COMPLEXITY  Checklists can combine system-based, patient- based, and cognitive interventions  Checklists are HOT

32. A GENERAL CHECKLIST  Obtain YOUR OWN, COMPLETE medical history  Perform a FOCUSED and PURPOSEFUL physical examination  Generate some initial hypotheses and differentiate these with appropriate questions, examination, or diagnostic tests  Pause to reflect – Take a diagnostic “time out”:  Was I comprehensive ?  Did I consider the inherent flaws of heuristic thinking ?  Was my judgment affected by any other bias ?  Do I need to make the diagnosis NOW, or can I wait ?  What’s the worst case scenario ? What are the ‘don’t miss’ entities ?  Embark on a plan, but acknowledge uncertainty and ENSURE A PATHWAY FOR FOLLOW-UP

33. A SYNDROME-SPECIFIC CHECKLIST CHEST PAIN  MI  PE  Pneumonia  Pericarditis  Musculoskeletal  Gerd  Herpes Zoster  Pleurisy  Aortic stenosis  Tumors – lung, lymphoma, mediastinum  Spinal cord compression  Esophageal spasm  Psychiatric

34. OPEN DISCUSSION – COGNITIVE ISSUES Question Question 2 – What would it take to convince frontline providers to use a checklist ? Question Question 3 - Will they help reduce diagnostic errors, or are we better off just trusting our initial (intuitive) diagnoses ? Question 1 – Which would be more effective – a GENERAL checklist, or SYNDROME SPECIFIC checklists ?

35. OTHER QUESTIONS?  Measurement of diagnostic errors?  How to evaluate quality of clinical reasoning?  How do you teach this stuff?

36. ACKNOWLEDGMENTS: AHRQ, RTI, VA