KANAL: Knowledge ANALysis Jihie Kim Yolanda Gil USC/ISI www.isi.edu/expect/rkf/

Role of Knowledge Analysis in SRI Team To point out to the Interaction Manager what additional K needs to be acquired or what existing K needs to be modified To guard the knowledge server from invalid statements entered by the user

Approach: Using Interdependency Models Relating different pieces of Knowledge among themselves and to the existing KB (e.g., how different pieces of knowledge are put together to generate an answer) Successfully used in checking problem-solving K in EXPECT (Gil & Melz 96; Kim & Gil 99)

Current Focus: Checking Process Models Verification checks: model is correct (e.g., no steps missing Validation checks: model is as user intended (e.g., alert user of impossible paths) KANAL Interaction Manager UI KM Interaction Plans for fixing errors

Validating Complex Process Models Lambda Virus Invasion 2 … … Transcribe Assemble Enter … Replicate Arrive … Integrate Divide Disintegrate Circularize Synthesize Copy

Describing Process Models (Composed Concepts) Each individual step has Preconditions, Add-list, Delete-list Links among the steps Decomposition links between steps and substeps Disjunctive alternatives Temporal links … VirusInvasion substeps Enter Integrate Synthesize . . . disjunction

Checks on Process Models All the steps are properly linked (substep, nextstep, disjunctive nextstep, conjunctive nextstep, …) All the preconditions of each step are satisfied during the simulation All the expected effects can be achieved There are no unexpected effects There are no impossible paths . . .

Current Focus: Dynamic Checks Simulation (or symbolic execution) results show how substeps of the process model are related each other (Interdependency Model) Perform various kinds of checks unachieved preconditions expected/unexpected effects disjunctive branches loops causal links redundancies unordered steps … : Implemented

Checking Unachieved Preconditions During simulation, collect unachieved preconditions by tracing failed expressions Suggest fixes Add a step that can achieve the condition Add ordering constraints between the failed step and another step that undid the condition Delete the step . . . VirusInvasion Failed Precondition: Virus near Cell Proposed Fixes: Add an Arrive step Add a Move step . . . Enter Integrate

Checking Effects Compute the effects by simulation Suggest fixes for unachieved expected effects Add steps that can achieve the effect Add ordering constraints between effect adding steps and effect deleting steps Check unexpected effects After VirusInvasion ProteinCoat of the virus broken  Achieved DNA of the virus has replicates  Unachieved <Proposed Fixes> Add a Replicate step Add a Divide step

Checking Disjunctive Branches Inform all the combinations of alternatives so that the user can check if some are impossible KANAL can simulate and highlight disjunctive paths

Example: Lambda Virus Invasion (From Alberts ECB Chapter 9) Enter Circularize Integrate Divide Disintegrate Synthesize Replicate disjunction Arrive <Paths Simulated> Path1: Arrive1  Enter2  Circularize3  Integrate4  Divide5  Disintegrate6  Synthesize7  Replicate8 Path2: Arrive1  Enter2  Circularize3  Synthesize7  Replicate8

Example: Conjunctive Branches Life cycle of a virus (from Alberts ECB Chapter 9) Transcribe Arrive Enter Conjunction Assemble Replicate <Simulation sequence> Arrive1  Enter2  Trascribe3  Replicate4  Assembly5 Arrive1  Enter2  Replicate4  Trascribe3  Assembly5

Checking Loops Enter Circularize Integrate Divide Disintegrate Synthesize Replicate disjunction Arrive <Loops Found> Loop1: Arrive1  Enter2  Circularize3  Integrate4  Divide5  Disintegrate6  Synthesize7  Replicate8 Arrive1 Loop2: Arrive1 Enter2  Circularize3  Synthesize7  Replicate8  Enter1 Loop3: Divide5 Divide5

Checking Causal Links Describe which step enables (or disables) a given step Enter Circularize Integrate Divide Disintegrate Synthesize Replicate disjunction Arrive <Causal Links> Arrive1 enables Enter2 by achieving “Virus near Cell” Integrate4 enables Disintegrate6 by achieving “Virus DNA integrated with chromosome”

Fixing Problems: Using Interaction Plans Interaction Plan: describes how to proceed with the user interaction direct what to do next based on the results from K Analysis KANAL’s dialogue for fixing errors is implemented with interaction plans Will be integrated with the Interaction Manager

Keeping Track of Interaction History ... Choose what to simulate choose model: VirusInvadesCell choose substep to test: VirusInvadesCell Simulate model VirusInvadesCell simulate-steps-&-find-failed-events ask-to-fix-failed-event: (failed preconditions of Enter) propose-fixes-for-failed-event ask-what-to-fix-for-failed-event : ((the location of (the patient of Enter)) = (the space-near of (the agent of Enter))) ask-how-to-fix-failed-event (add Arrive before Enter)

Future Extensions (I): Static Checks Let user pose questions about various features of the process model to test the model KANAL will maintain test suites Users pick from sample query templates example: retrieving role values, part-of relations, type definitions,.. Users may specify their expected results Users may vary the initial situations to start from Explanation or trace of the answer to a query show how different pieces of K are used to generate the answer (Interdependency Model)

Future Extensions (II) Exploiting history and evolution of Interdependency Models (for both simulations and queries) Example: Check what tests were correctly answered before Using heuristics to focus K analysis Example: when invalid results are obtained, KANAL will use a divide-and-conquer strategy and check intermediate results to find the sources of the problem Testing with different initial states and different arguments

Future Extensions (III) Interdependency Models for problem solving knowledge EKCP Build on past work on EXPECT

Using KANAL for Intelligent Tutoring Systems ITSs can acquire domain knowledge from human instructor and use simulations to refine the knowledge (Johnson et al 2000, Scholer et al 2000, Angros et al 99) We are exploring the use of KANAL to check and analyze the domain models while it is being built

Knowledge Authoring Environment for Tutoring Systems (current) Instructor Student Demonstration Final Model (Lessons) Steve Agent Initial Model Refined Model Experimenter Library of actions Domain Simulator

Knowledge Authoring Environment for Tutoring Systems (future) Instructor Student Demonstration Editor Final Model (Lessons) Steve Agent Initial Model Refined Model Experimenter KANAL Library of actions Domain Simulator