1. A Graphical Framework for Specification of Clinical Guidelines at Multiple Representation Levels Erez Shalom, B.Sc. and Yuval Shahar, M.D., Ph.D. Medical Informatics Research Center Department of Information Systems Engineering Ben Gurion University, Beer Sheva, Israel

2. The need for Automation of Clinical Guidelines Following Clinical guidelines (GLs) has been shown to improve the quality of medical care Automatic support provides:  Visual specification  Search and retrieval  Application of a GL  Retrospective quality assurance However: Most GLs are text based and inaccessible at the point of care

3. The Required Infrastructure A machine-comprehensible GL representation language Support for a gradual structuring of the GL (from text to an executable code) Runtime GL application and QA tools

4. D The Structuring Process The Guideline as a tree of plansThe Guideline as a text document Involves 2 main types of knowledge: Procedural knowledge – e.g. plan for administer the two medications in parallel In parallel D D Declarative knowledge - e.g. the eligibility criteria “Is drug available?” Plan 1 Plan 1.1 Plan 1.2 P PP

5. Several Unresolved Issues Unclear division of the responsibility between the knowledge engineer (KE) and the Expert physician (EP) The transformation into a formal representation is often done in one step Lack of a detailed methodology for the overall process Usually the structuring methodology is focused on only one GL-specification language/ontology

6. Expert Physician Knowledge Engineer The hybrid representation model Gradually structuring the GL using increasingly formal representation levels  Implemented as part of DeGeL [Shahar et al, JBI 2004]  Used with the URUZ GL markup tool

7. URUZ (I): Specification of declarative knowledge Expert physician Selects “filter condition” knowledge role

8. URUZ (cont’d) : Specification of procedural knowledge Expert physician decomposing the GL into tree of plans

9. The URUZ evaluation: Insights Structuring guidelines by expert physicians, in principle is feasible Creating a ontology-specific consensus is an essential preliminary step! A comprehensive methodology was developed for the overall process An complete evaluation model and tool was developed and used

10. The URUZ Evaluation: Conclusions Interface is cumbersome and not user friendly (average Scalable Usability Scale: 47.5 [over 50 is user friendly]) The initial procedural structuring should be independent of any specification language There is a need for customized graphical widgets  especially for the procedural knowledge Additional representation levels should be supported We now know what users need: GESHER - A Graphical Framework for Specification of Clinical Guidelines at Multiple Representation Levels

11. The GESHER Architecture

12. The GESHER’s Main Features  User friendly graphical client application  Support specification at multiple representation levels  Support to multiple specification languages (GL ontologies)  Access centralized resources such as DeGeL [Shahar et al, JBI 2004] and a knowledge base

13. GESHER: Semi-Structured Level

14. GESHER(I) : Semi-Formal Representation Level

15. GESHER(II) :Semi-Formal Widgets

16. The Expression Builder – Acquisition of Semi-Formal Expressions

17. Ongoing Evaluation of URUZ and GESHER Three GLs in different domains were used :  Pelvic Inflammatory Disease (PID)  Chronic Obstructive Pulmonary Disease (COPD)  Hypothyroidism For each GL:  Collaborative consensus was formed with Expert physician and knowledge engineer  Three markups performed by experts physicians  One of the markups which was build with knowledge engineer and Expert physician was used as Gold Standard  7 different questionnaires were filled by EPs and KEs participated in the evaluation

18. Several Conclusions For making an ontology-specific consensus, understanding the Asbru semantics, as well as their own knowledge, helped EPs more than knowing the different DEGEL tools EPs understand better intuitive KRs such as Actors, level of evidence or clinical context, rather than Asbru’s more semantically oriented KRs such as eligibility criteria, or control structures such as switch-case.

19. Summary The need for gradual GL specification  Making an ontology-specific consensus as first step  Use a well defined methodology for the overall process Current tool : URUZ Improved graphical framework: GESHER Both tools are now ongoing a formal evaluation Future work:  Finish evaluations  Develop formal widgets  Creation and use of graphical templates for GL specification

20. Acknowledgments All Medical informatics research center members Our colleagues at Stanford and VA hospital: Drs. Mary Goldstein, Susana Martins, Lawrence Basso, Herbert Kaizer, Laszlo Tudor Vaszar Our colleagues at Soroka’s university medical center : Prof. Eitan Lunenfeld, Dr. Avi Yarkoni, Dr. Guy Bar, Prof. Yair Liel and Dr. Tal Merom NLM award No LM-06806 Contact info : erezsh@bgu.ac.il Visit our web site : http://medinfo.ise.bgu.ac.il/medlab/

21. Questions?

22. Consensus Formation: The Basis for The Structuring Process EPs create a Clinical consensus : EPs and KE adds procedural knowledge : EP+KE add declarative knowledge : Ontology- specific consensus Doxycycline Metronidazole Order :parallel Filter condition for drug : Is patient not sensitive to Doxycycline and the drug available? Give Doxycycline 100 mg orally or IV every 12 hours Plus Metronidazole 500 mg IV every 8 hours

23. Methodology of the structuring process Before Markup:  Make Consensus  Read consensus and GL sources  Have some training with the tool Perform Structuring according to consensus and GL sources Evaluate the markup

24. Why this gradegradecategorynum completeness This is best markup * ( 1.3.1 and 1.3.3 are checked) 1 identical to GS1.2.1 We assume that missing content is wrong * (1.3.1 and 1.3.3 cannot be checked) exist in GS and missing in markup 1.2.2 Generally don't care * ( should check for correctness) 0 exist in markup and missing in GS 1.2.3 Most of the content complete * ( should check for correctness) 0 exist in markup partially compared to GS 1.2.4 content not complete at all * ( should check for correctness) - 1 Not worsing patient outcome 1 correct clinically1.3.1 correctness Not worsing patient outcome 0 incorrect clinically1.3.2 worse patient outcome Not worsing patient outcome 1 correct according to Asbru semantics 1.3.3 Not worsing patient outcome 0 incorrect according to Asbru semantics 1.3.4 worse patient outcome

26. Question : In what way did each of the following subjects help you to make an ontological consensus ? (-3 / 3 scale) AverageEP3EP2EP1Subject 2.67332 Your expertise 2.33223 Asbru KR ’ s – Declarative Part 2.33232 Reading the guideline sources before making consensus 2.00033 Knowing the multiple representation level model 2.00222 Asbru KR ’ s – Procedural part 2.00303 Having more than one source 1.33031 Ontology 0.67020 DeGeL 0.67011 URUZ -main interface 0.67020 IndexiGuide 0.33001 URUZ – plan body wizard 0.33010 Vaidurya 0.33100 Vocabulary server 0.00000 Spock

27. Question: How easy was it for you to understand each of the following knowledge Roles? (-3/ 3 scale ) AverageEP4EP3EP2EP1Knowledge Role 33333 Actors 33333 A process still to be defined 2.752333 Level of evidence 2.752333 Strength of recommendation 2.752333 Clinical context 2.53313 Abort Condition 2.53313 Complete Condition 2.52323 Simple Action 2.252331 Filter Condition 1.5133 Reactivate condition 1.5233-2 Subplans – sequential order 1.25332-3 If-then -else 1.252210 Repeating plan 1.25233-3 Subplans – any order 1221 Suspend condition 1133-3 Subplans – parallel order 0.75212-2 Guideline knowledge 0.533 -2 Intentions Overall - outcome

28. AverageEP3 EP2EP1Knowledge Role 0.5230-3 Plan activation 0.2532-3 unordered – Subplans 022-2 Intentions Overall - process 022-2 Intentions intermediate - outcome 011 Setup Condition -0.50-2 Intentions intermediate - process -0.53 -3 Switch case Question: How easy was it for you to understand each of the following knowledge Roles? (-3/3) Cont..

29. Consensus example – procedural part

30. 1.1 Patient Treatment and evaluation

31. Top level - PID ?Level of evidence ?Strength of recommendation Doctors - Gynecologist clinicsActors ER, ward, OR,Clinical context To Cure PID Or substantial clinical improvement Intentions Overall - outcome (suspected pid) and (sexually active female)Filter Condition ?Set Up Condition Elimination of diagnosis of PID Or patient diedAbort Condition No further treatment (IV or Per Os or surgery) or follow up is neededComplete Condition Consensus example – declerative part

32. Hospitalization Level of evidence Strength of recommendation Doctors - ward's GynecologistActors ward or operation roomClinical context To treat severe PIDIntention intermediate process Severe PID Or failure of oral treatment Or tubo-ovarian abscess Or Surgical emergencies or (mild pid and (pregnant or HIV positive)) Filter Condition Set Up Condition Abort Condition discharge from hospitalComplete Condition

33. Digital Electronic Guideline Library) DeGeL)

34. Sample GL Modeling Methods Knowledge acquisition toolMethod A Protégé-based interface EON and SAGE, Prodigy, GLIF ArrezoPROforma GEM-CutterGEM "CG_AM" graphical interfaceGLARE NEWGUIDEGUIDE Asbru-View, GMT, Stepper, URUZ, GESHER Asbru