The Importance of Creating an Ontology-Specific Consensus Before a Markup- Based Specification of Clinical Guidelines Erez Shalom 1, Yuval Shahar 1 Eitan Lunenfeld 2, Meirav Taieb-Maimon 1, Ohad Young 1, Guy Bar 2, Susana B.Martins 3, Laszlo Vaszar 3, Yair Liel 2, Avi Yarkoni 2, Mary K.Goldstein 3, Akiva Leibowitz 2, and Tal Marom 4 1 The Medical Informatics Research Center, Ben Gurion University, Beer Sheva, Israel 2 Soroka Medical Center, Ben Gurion University, Beer Sheva, Israel 3 Veterans Administration Palo Alto Heath Care System, Palo Alto,CA 4 E.Wolfson Medical Center, Holon, Israel
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006
The Required Infrastructure to support automation of GLs
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 What are the essential steps for the overall guideline specification process?
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 A Methodology for GL Specification & for its Evaluation The methodology is embedded within the Digital Guideline Library (DeGeL) architecture Creating an Ontology-Specific Consensus (OSC) is a crucial, mandatory step before markup 1.
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 The Importance of an Ontology- Specific Consensus (OSC) An OSC is a document that describes the core objectives and means of the GL Refers to the knowledge roles of the chosen (target) specification ontology Enhances accuracy of the mark-up Decreases variability during mark-up and during application of the GL
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 The Consensus-Creation Methodology Clinical consensus – the pathway of the GL KE +EP add procedural knowledge semantics to the pathway (e.g parallel order between the regimens) Ontology-Specific Consensus KE+EP add declarative knowledge to the pathway (e.g filter condition to the main root of the GL)
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 The second stage in forming an Ontological consensus
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 The third stage in forming an Ontological consensus
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Results (I): Procedural Complexity of the GLs
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Results (II): Quality of markups EPs can perform markup! High completeness: 91% of the plans and 97% of the KRs were recreated, compared to Gold Standard markup Variable correctness, but proportion of perfect scores significantly high (P<0.05) Several clusters of KRs can be formed by level of difficulty to structure Creating an OSC is crucial for achieving high quality
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Results (III): Types of errors The differences in the total numbers of errors between the three GLs were highly significant in a proportion test (P<0.001)
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Conclusions o Markup is feasible by EPs! o The more detailed and structured the OSC was, the lower the total number of errors committed by the EPs for each KR o The need for a graphical OSC-forming graphical tool
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Summary The need for gradual GL specification Creating an ontology-specific consensus as a first step Using a well defined methodology for the overall process to increase the quality of the markups For more information:
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Questions?
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 The second stage in forming an Ontological consensus
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Creation of OSC – example: Textual Source in the PID Guideline
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 The first stage in forming an Ontology-Specific consensus
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Results (III): Procedural Complexity of the GLs The Subjective Qualitative Ranking
AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Results (I): The Guidelines Pelvic Inflammatory Disease (PID) Chronic Obstructive Pulmonary Disease (COPD) Hypothyroidism (HypoThyrd)