1. The Importance of Creating an Ontology-Specific Consensus Before a Markup- Based Specification of Clinical Guidelines Erez Shalom 1, Yuval Shahar 1 Eitan Lunenfeld 2, Meirav Taieb-Maimon 1, Ohad Young 1, Guy Bar 2, Susana B.Martins 3, Laszlo Vaszar 3, Yair Liel 2, Avi Yarkoni 2, Mary K.Goldstein 3, Akiva Leibowitz 2, and Tal Marom 4 1 The Medical Informatics Research Center, Ben Gurion University, Beer Sheva, Israel 2 Soroka Medical Center, Ben Gurion University, Beer Sheva, Israel 3 Veterans Administration Palo Alto Heath Care System, Palo Alto,CA 4 E.Wolfson Medical Center, Holon, Israel

2. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006

3. The Required Infrastructure to support automation of GLs

4. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 What are the essential steps for the overall guideline specification process?

5. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 A Methodology for GL Specification & for its Evaluation The methodology is embedded within the Digital Guideline Library (DeGeL) architecture Creating an Ontology-Specific Consensus (OSC) is a crucial, mandatory step before markup 1.

6. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 The Importance of an Ontology- Specific Consensus (OSC) An OSC is a document that describes the core objectives and means of the GL Refers to the knowledge roles of the chosen (target) specification ontology Enhances accuracy of the mark-up Decreases variability during mark-up and during application of the GL

7. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 The Consensus-Creation Methodology Clinical consensus – the pathway of the GL KE +EP add procedural knowledge semantics to the pathway (e.g parallel order between the regimens) Ontology-Specific Consensus 1 2 3 4 KE+EP add declarative knowledge to the pathway (e.g filter condition to the main root of the GL)

8. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 The second stage in forming an Ontological consensus

9. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 The third stage in forming an Ontological consensus

10. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Results (I): Procedural Complexity of the GLs

11. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Results (II): Quality of markups EPs can perform markup! High completeness: 91% of the plans and 97% of the KRs were recreated, compared to Gold Standard markup Variable correctness, but proportion of perfect scores significantly high (P<0.05) Several clusters of KRs can be formed by level of difficulty to structure Creating an OSC is crucial for achieving high quality

12. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Results (III): Types of errors The differences in the total numbers of errors between the three GLs were highly significant in a proportion test (P<0.001)

13. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Conclusions o Markup is feasible by EPs! o The more detailed and structured the OSC was, the lower the total number of errors committed by the EPs for each KR o The need for a graphical OSC-forming graphical tool

14. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Summary The need for gradual GL specification Creating an ontology-specific consensus as a first step Using a well defined methodology for the overall process to increase the quality of the markups  For more information: erezsh@bgu.ac.il http://medinfo.ise.bgu.ac.il/medlab

15. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Questions?

16. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 The second stage in forming an Ontological consensus

17. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Creation of OSC – example: Textual Source in the PID Guideline

18. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 The first stage in forming an Ontology-Specific consensus

19. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Results (III): Procedural Complexity of the GLs The Subjective Qualitative Ranking

20. AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006 Results (I): The Guidelines  Pelvic Inflammatory Disease (PID)  Chronic Obstructive Pulmonary Disease (COPD)  Hypothyroidism (HypoThyrd)