A Graphical Framework for Specification of Clinical Guidelines at Multiple Representation Levels Erez Shalom, B.Sc. and Yuval Shahar, M.D., Ph.D. Medical Informatics Research Center Department of Information Systems Engineering Ben Gurion University, Beer Sheva, Israel

The need for Automation of Clinical Guidelines Following Clinical guidelines (GLs) has been shown to improve the quality of medical care Automatic support provides:  Visual specification  Search and retrieval  Application of a GL  Retrospective quality assurance However: Most GLs are text based and inaccessible at the point of care

The Required Infrastructure A machine-comprehensible GL representation language Support for a gradual structuring of the GL (from text to an executable code) Runtime GL application and QA tools

D The Structuring Process The Guideline as a tree of plansThe Guideline as a text document Involves 2 main types of knowledge: Procedural knowledge – e.g. plan for administer the two medications in parallel In parallel D D Declarative knowledge - e.g. the eligibility criteria “Is drug available?” Plan 1 Plan 1.1 Plan 1.2 P PP

Several Unresolved Issues Unclear division of the responsibility between the knowledge engineer (KE) and the Expert physician (EP) The transformation into a formal representation is often done in one step Lack of a detailed methodology for the overall process Usually the structuring methodology is focused on only one GL-specification language/ontology

Expert Physician Knowledge Engineer The hybrid representation model Gradually structuring the GL using increasingly formal representation levels  Implemented as part of DeGeL [Shahar et al, JBI 2004]  Used with the URUZ GL markup tool

URUZ (I): Specification of declarative knowledge Expert physician Selects “filter condition” knowledge role

URUZ (cont’d) : Specification of procedural knowledge Expert physician decomposing the GL into tree of plans

The URUZ evaluation: Insights Structuring guidelines by expert physicians, in principle is feasible Creating a ontology-specific consensus is an essential preliminary step! A comprehensive methodology was developed for the overall process An complete evaluation model and tool was developed and used

The URUZ Evaluation: Conclusions Interface is cumbersome and not user friendly (average Scalable Usability Scale: 47.5 [over 50 is user friendly]) The initial procedural structuring should be independent of any specification language There is a need for customized graphical widgets  especially for the procedural knowledge Additional representation levels should be supported We now know what users need: GESHER - A Graphical Framework for Specification of Clinical Guidelines at Multiple Representation Levels

The GESHER Architecture

The GESHER’s Main Features  User friendly graphical client application  Support specification at multiple representation levels  Support to multiple specification languages (GL ontologies)  Access centralized resources such as DeGeL [Shahar et al, JBI 2004] and a knowledge base

GESHER: Semi-Structured Level

GESHER(I) : Semi-Formal Representation Level

GESHER(II) :Semi-Formal Widgets

The Expression Builder – Acquisition of Semi-Formal Expressions

Ongoing Evaluation of URUZ and GESHER Three GLs in different domains were used :  Pelvic Inflammatory Disease (PID)  Chronic Obstructive Pulmonary Disease (COPD)  Hypothyroidism For each GL:  Collaborative consensus was formed with Expert physician and knowledge engineer  Three markups performed by experts physicians  One of the markups which was build with knowledge engineer and Expert physician was used as Gold Standard  7 different questionnaires were filled by EPs and KEs participated in the evaluation

Several Conclusions For making an ontology-specific consensus, understanding the Asbru semantics, as well as their own knowledge, helped EPs more than knowing the different DEGEL tools EPs understand better intuitive KRs such as Actors, level of evidence or clinical context, rather than Asbru’s more semantically oriented KRs such as eligibility criteria, or control structures such as switch-case.

Summary The need for gradual GL specification  Making an ontology-specific consensus as first step  Use a well defined methodology for the overall process Current tool : URUZ Improved graphical framework: GESHER Both tools are now ongoing a formal evaluation Future work:  Finish evaluations  Develop formal widgets  Creation and use of graphical templates for GL specification

Acknowledgments All Medical informatics research center members Our colleagues at Stanford and VA hospital: Drs. Mary Goldstein, Susana Martins, Lawrence Basso, Herbert Kaizer, Laszlo Tudor Vaszar Our colleagues at Soroka’s university medical center : Prof. Eitan Lunenfeld, Dr. Avi Yarkoni, Dr. Guy Bar, Prof. Yair Liel and Dr. Tal Merom NLM award No LM Contact info : Visit our web site :

Questions?

Consensus Formation: The Basis for The Structuring Process EPs create a Clinical consensus : EPs and KE adds procedural knowledge : EP+KE add declarative knowledge : Ontology- specific consensus Doxycycline Metronidazole Order :parallel Filter condition for drug : Is patient not sensitive to Doxycycline and the drug available? Give Doxycycline 100 mg orally or IV every 12 hours Plus Metronidazole 500 mg IV every 8 hours

Methodology of the structuring process Before Markup:  Make Consensus  Read consensus and GL sources  Have some training with the tool Perform Structuring according to consensus and GL sources Evaluate the markup

Why this gradegradecategorynum completeness This is best markup * ( and are checked) 1 identical to GS1.2.1 We assume that missing content is wrong * (1.3.1 and cannot be checked) exist in GS and missing in markup Generally don't care * ( should check for correctness) 0 exist in markup and missing in GS Most of the content complete * ( should check for correctness) 0 exist in markup partially compared to GS content not complete at all * ( should check for correctness) - 1 Not worsing patient outcome 1 correct clinically1.3.1 correctness Not worsing patient outcome 0 incorrect clinically1.3.2 worse patient outcome Not worsing patient outcome 1 correct according to Asbru semantics Not worsing patient outcome 0 incorrect according to Asbru semantics worse patient outcome

Question : In what way did each of the following subjects help you to make an ontological consensus ? (-3 / 3 scale) AverageEP3EP2EP1Subject Your expertise Asbru KR ’ s – Declarative Part Reading the guideline sources before making consensus Knowing the multiple representation level model Asbru KR ’ s – Procedural part Having more than one source Ontology DeGeL URUZ -main interface IndexiGuide URUZ – plan body wizard Vaidurya Vocabulary server Spock

Question: How easy was it for you to understand each of the following knowledge Roles? (-3/ 3 scale ) AverageEP4EP3EP2EP1Knowledge Role Actors A process still to be defined Level of evidence Strength of recommendation Clinical context Abort Condition Complete Condition Simple Action Filter Condition Reactivate condition Subplans – sequential order If-then -else Repeating plan Subplans – any order 1221 Suspend condition Subplans – parallel order Guideline knowledge Intentions Overall - outcome

AverageEP3 EP2EP1Knowledge Role Plan activation unordered – Subplans Intentions Overall - process Intentions intermediate - outcome 011 Setup Condition Intentions intermediate - process Switch case Question: How easy was it for you to understand each of the following knowledge Roles? (-3/3) Cont..

Consensus example – procedural part

1.1 Patient Treatment and evaluation

Top level - PID ?Level of evidence ?Strength of recommendation Doctors - Gynecologist clinicsActors ER, ward, OR,Clinical context To Cure PID Or substantial clinical improvement Intentions Overall - outcome (suspected pid) and (sexually active female)Filter Condition ?Set Up Condition Elimination of diagnosis of PID Or patient diedAbort Condition No further treatment (IV or Per Os or surgery) or follow up is neededComplete Condition Consensus example – declerative part

Hospitalization Level of evidence Strength of recommendation Doctors - ward's GynecologistActors ward or operation roomClinical context To treat severe PIDIntention intermediate process Severe PID Or failure of oral treatment Or tubo-ovarian abscess Or Surgical emergencies or (mild pid and (pregnant or HIV positive)) Filter Condition Set Up Condition Abort Condition discharge from hospitalComplete Condition

Digital Electronic Guideline Library) DeGeL)

Sample GL Modeling Methods Knowledge acquisition toolMethod A Protégé-based interface EON and SAGE, Prodigy, GLIF ArrezoPROforma GEM-CutterGEM "CG_AM" graphical interfaceGLARE NEWGUIDEGUIDE Asbru-View, GMT, Stepper, URUZ, GESHER Asbru