July 6, th International Protégé Conference Reasoning in a Tutoring System: Transforming Knowledge to Teaching. Olga Medvedeva Center for Pathology Informatics, University of Pittsburgh

July 6, th International Protégé Conference Outline Our approach for teaching visual diagnosis General system architecture Knowledge representation in different tutor modules Pluses and minuses of our system

July 6, th International Protégé Conference Medical KB Training System Challenge Problems –Medical knowledge is complex and dynamic –Errors in KB can cause serious problems –Demands on extendibility and maintenance of large KBS Requirements –Combine knowledge representation and flexible instructional system –Adaptive for new observables and unique strategies –Reusable and modular

July 6, th International Protégé Conference Intelligent Tutoring Systems Paradigm  ITS strive to replicate a method of teaching and learning exemplified by a one-on-one human tutoring interaction  Model Tracing ITS guide user through problem space, can correct each small intermediate reasoning step  Cognitive Tutors based on ACT_R theory of learning proceduralize declarative knowledge in the rules( step instructions )

July 6, th International Protégé Conference Student Module Pedagogic Knowledge Interface Expert Model Allow correct steps Correct errors Give hints on next step Collect data on what student does Make predictions on what student knows Provide data for pedagogic decision making Canvas for problem solving Make goals visible Case sequence When to intervene How to intervene Intelligent Tutor System Structure

July 6, th International Protégé Conference Disadvantages of ITS Paradigm Developed for highly procedural domains Not designed for large complex dynamic declarative knowledge Domain specific production rules knowledge representation Maintenance is difficult and time consuming Knowledge modification alter the rules

July 6, th International Protégé Conference SlideTutor Characteristics SlideTutor – a system to teach visual classification problem solving in Pathology Similar to other medical diagnostic tasks –Combination of search, identification, interpretation –Well characterized diagnostic reasoning in medical domains –Some areas are highly algorithmic, some – not –Both empirical and theoretical work can guide the development –Combination of heuristic classification and deductive/inductive reasoning is the best foundation for classification problem-solving.

July 6, th International Protégé Conference SlideTutor Approach Combine the aspects and methodology of both KBS and ITS to create a general framework for teaching decision-making process for classification problems in Dermopathology using UPML Component Mode approach. –Extract and modularize all expert and pedagogic declarative knowledge into ontologies => make domain task neutral –Reuse PSM by tutor procedural rule based system => make system domain neutral –Preserve all of the major pedagogic components associated with Cognitive Tutors in ontologies and rules => add significant flexibility to pedagogic model

July 6, th International Protégé Conference SlideTutor General Architecture

July 6, th International Protégé Conference Domain Model Set of ontologies that express relationships between evidence and disease concepts Uses Motta’s parametric design approach ( slightly extended by adding attributes to features ) Similar disease and evidence representation –Hierarchical structure with multiple inheritance for diseases –Set of evidences represent set of diseases –Both can occur multiple times in different sets

July 6, th International Protégé Conference Feature – Domain KB – Case Relationship

July 6, th International Protégé Conference Task Model Models the abstract structure of the Dynamic Solution Graph (DSG) – a directed acyclic graph Represents possible relationships in the domain knowledge that are pertinent for reasoning –Identifying region –Identifying and refining a set of features –Triggering one or more hypothesis –Creating a differential diagnosis –Finding features that distinguish between the hypotheses –Defining that critical feature is absent –Linking supportive features to a particular hypothesis –Accepting some hypotheses as diagnosis Direction of DSG is defined by an order of some steps in task (deftemplate task (slot type) (multislot parent) (slot role) (slot required) (slot priority))

July 6, th International Protégé Conference JessTab Extensions Added UserFunctions load-jdbc-project - load db project disposep - dispose current Protégé Modified code –Preserve class hierarchy structure –Multiple inheritance (MAIN::NEUTROPHILS (is-a NEUTROPHILS) (is-a-name "NEUTROPHILS") (OBJECT ) (has-parents "INFLAMMATORY INFILTRATE") (feature_name "isolated neutrophils"))

July 6, th International Protégé Conference Dynamic Solution Graph Generates path through problem state based on combination of Domain, Task and Case models Dynamic – no predefined solution – each cycle generates the current problem state and all valid next steps Contains a set of abstract PSM that allow to add/delete/update nodes and arcs Path through the problem is defined by a consequence of student actions Behavior structures encapsulate node type specific response to a triggered event Supports forwards and backwards reasoning

July 6, th International Protégé Conference DSG Implementation (deftemplate node (slot type (type STRING)) (multislot property_name ) ;; e.g “ name ” “ x ” “ y ” “ z ” (multislot property_value) (slot internal_id (type STRING)) (slot state (default " INITIAL ")) ;; INITIAL, IDENTIFIED (slot input_value) ;; easy match with useraction input slot (slot external_id (default nil)) ;; id of a corresponding object on user side (slot is_goal (default FALSE)) (slot is_from_case (default FALSE))) ;; node can not delete if came from case Node reflects the semantic meaning of fact Correct student action must match all of the node properties State indicates that step was performed by user or not Interpretation of action is left to the instructional layer Special node type – Cluster node – expresses integrated relation between a specific group if nodes and nodes outside it

July 6, th International Protégé Conference

July 6, th International Protégé Conference DSG Cognitive Values Enables rapid feedback Provides a method for stepping forward in the model to generate next-step hints Supports intermediate solution and revision Determines general classes of errors and allows pedagogic model to remediate them Provides flexibility in tutor response Reusable, because domain and pedagogically independent

July 6, th International Protégé Conference Instructional Layer Pedagogic Model –Explanation of a particular student error and rich next-step hints upon student request –Delivered messages contain context-specific text accompanied by the pointers to the places of interest on the user side –Determines the most appropriate error from the error list generated by the DSG as a response to incorrect student action based on the state of student model –Hierarchical hints from general to most specific and directive Pedagogic Task – represents the goal of the instructional process

July 6, th International Protégé Conference Case-Focused Interface Local view of the problem

July 6, th International Protégé Conference Knowledge-Focused Interface Global view of the problem (use SpaceTree cs.umd.edu)

July 6, th International Protégé Conference Conclusion Preserved essential characteristics of CT Utilized KB for modeling knowledge across the system components Modular and flexible set of frames and methods to teach classification problem solving Limitation – deterministic approach –No support for probabilistic relationship between evidence and hypothesis –No attempt to model all evidence combinations or incomplete evidence – No reasoning under uncertainty

July 6, th International Protégé Conference Acknowledgements NLM 1 R01 LM (Crowley, PI) Rebecca Crowley, Pathology Informatics Eugene Tseytlin, Pathology Informatics Elizabeth Legowski, Pathology Informatics Gerish Chavan, Pathology Informatics Maria Bond, Pathology Informatics

July 6, th International Protégé Conference More details at Demo Session Integrating Protégé into an Intelligent Medical Training System –Ontologies –Knowledgebase Validation Tool –Case Authoring Protégé plug-in –Dynamic Solution Graph –Protocol Filter Query –SlideTutor –DinoTutor