HIT Hazard Manager: for Proactive Hazard Control James Walker MD, Principal Investigator, Geisinger Health System Andrea Hassol MSPH, Project Director, Abt Associates September 10, 2012 AHRQ Contract: HHSA i,#14

Accident Analysis

Patient Harm Analysis (e.g., RCA) Accident Analysis

Near-Miss Analysis

Patient Harm Near Miss Analysis (e.g., RCA) Near-Miss Analysis

Proactive Hazard Control

”Un-Forced” HIT-Use Error Error in Design or Implementation Interaction between HIT and other healthcare systems Proactive Hazard Control Care-Process Compromise? Identifiable Patient Harm? Patient Harm No Adverse Effect Near Miss Hazard in Production No Adverse Effect Yes HIT-Related Hazards Yes No Use-Error Trap Hazard Identified? Hazard Resolved? HIT-Use-Error Trap

Proactive Hazard Control: A Case Pre-implementation Analysis: New CPOE cannot interface safely with the existing best-in-class pharmacy system. Pre-implementation Analysis: New CPOE cannot interface safely with the existing best-in-class pharmacy system. Replace the pharmacy system with the one that is integrated with the CPOE: Expensive 9-month delay Replace the pharmacy system with the one that is integrated with the CPOE: Expensive 9-month delay Years later, David Classen studied 62 HER implementations and concluded that CPOE and pharmacy systems from different vendors can never be safely interfaced. Years later, David Classen studied 62 HER implementations and concluded that CPOE and pharmacy systems from different vendors can never be safely interfaced.

The Hazard Ontology Why a standard language (ontology) for HIT hazards? To decrease the cost and increase the effectiveness of hazard control. Example: Much of the budget of the Aviation Safety Information Analysis and Sharing (ASIAS) system is devoted to normalizing data—because every airline uses a different ontology and can’t afford to change.

Health It Hazard Manager – AHRQ ACTION Task Order Design & Alpha-Test (266 hazards) Design & Alpha-Test (266 hazards) – Geisinger Beta-Test (Website) Beta-Test (Website) – Geisinger – Abt Associates – ECRI PSO Beta-Test Evaluation Beta-Test Evaluation – Abt Associates – Geisinger

Hazard Manager Beta-Test 7 sites: integrated delivery systems, large and small hospitals, urban and rural – Usability (individual interviews) – Inter-rater Scenario Testing (individual web or in- person sessions) – Ontology of hazard attributes (group conference) – Usefulness (group conference) – Automated Reports (group conference) 4 vendors offered critiques All-Project meeting: 6 test sites, 4 vendors, AHRQ, ONC, FDA

HIT Hazard Manager 2.0 Demo

Hazard Ontology Discovery: when and how the hazard was discovered; stage of discovery Discovery: when and how the hazard was discovered; stage of discovery Causation: usability, data quality, decision support, vendor factors, local implementation, other organizational factors Causation: usability, data quality, decision support, vendor factors, local implementation, other organizational factors Impact: risk and impact of care process compromise; seriousness of patient harm Impact: risk and impact of care process compromise; seriousness of patient harm Hazard Control: control steps; who will approve and implement the control plan Hazard Control: control steps; who will approve and implement the control plan

Beta-Test Analytic Methods Content analysis of 495 Short Hazard Descriptions Content analysis of 495 Short Hazard Descriptions Frequencies of hazard ontology factors: combinations often selected together; factors never selected Frequencies of hazard ontology factors: combinations often selected together; factors never selected Inter-rater differences in entries of mock hazard scenarios/vignettes Inter-rater differences in entries of mock hazard scenarios/vignettes Suggestions from testers to improve ontology clarity, comprehensiveness, mutual exclusivity Suggestions from testers to improve ontology clarity, comprehensiveness, mutual exclusivity Content analysis of “Other Specify” entries Content analysis of “Other Specify” entries

Example: Unforced User Error Unforced User Error was the second most frequently chosen factor (79 hazards). Unforced User Error was the second most frequently chosen factor (79 hazards). In 55 instances, another factor was also chosen: In 55 instances, another factor was also chosen: Usability Data Quality CDS Software Design Other Org. Factors * Multiple selections possible Inter-rater testing revealed differing attitudes about the role of health IT in preventing user errors. Inter-rater testing revealed differing attitudes about the role of health IT in preventing user errors.

Ontology Revision: “Use Error”

Hazard Manager Benefits

Value: Care-Delivery Organizations Prior to an upgrade, learn about hazards others have reported. Prior to an upgrade, learn about hazards others have reported. Identify hazards that occur at the interface of two vendors’ products. Identify hazards that occur at the interface of two vendors’ products. Control hazards proactively. Control hazards proactively. Estimate the risk hazards pose and prioritize hazard-control efforts. Estimate the risk hazards pose and prioritize hazard-control efforts. Inform user-group interactions with vendors. Inform user-group interactions with vendors. Protect confidentiality. Protect confidentiality.

Value : HIT Vendors Identify the 90% of hazards that their customers do not currently report. Identify the 90% of hazards that their customers do not currently report. Learn which products interact hazardously with their own. Learn which products interact hazardously with their own. Prioritize hazard control efforts. Prioritize hazard control efforts. Identify hazards early in the release of new versions. Identify hazards early in the release of new versions. Preserve confidentiality. Preserve confidentiality.

Value : Policy Makers Identify and categorize common hazards that occur at the interface of specific types of products (e.g., pharmacy and order entry). Identify and categorize common hazards that occur at the interface of specific types of products (e.g., pharmacy and order entry). Move hazard identification earlier in the IT lifecycle (especially prior to production use). Move hazard identification earlier in the IT lifecycle (especially prior to production use). Monitor the success of hazard control in reducing health IT hazards and decreasing their impact on patients. Monitor the success of hazard control in reducing health IT hazards and decreasing their impact on patients.

For more information: Beta-Test Final Report available on AHRQ website: healthit.ahrq.gov/HealthITHazardManager FinalReport