1 KANAL: Knowledge ANALysis Jihie Kim Yolanda Gil USC/ISI

2 USC INFORMATION SCIENCES INSTITUTE KANAL Role of Knowledge Analysis in SRI Team To point out to the Interaction Manager what additional K needs to be acquired or what existing K needs to be modified To guard the knowledge server from invalid statements entered by the user

3 USC INFORMATION SCIENCES INSTITUTE KANAL Approach: Using Interdependency Models Relating different pieces of Knowledge among themselves and to the existing KB (e.g., how different pieces of knowledge are put together to generate an answer) Successfully used in checking problem- solving K in EXPECT (Gil & Melz 96; Kim & Gil 99)

4 USC INFORMATION SCIENCES INSTITUTE KANAL Current Focus: Checking Process Models Verification checks: model is correct (e.g., no steps missing Validation checks: model is as user intended (e.g., alert user of impossible paths) UI Interaction Manager KM KANAL Interaction Plans for fixing errors

5 USC INFORMATION SCIENCES INSTITUTE KANAL Validating Complex Process Models Enter Arrive Lambda Virus Invasion 2 Transcribe Replicate Circularize IntegrateDivideDisintegrate SynthesizeCopy … … Assemble … …

6 USC INFORMATION SCIENCES INSTITUTE KANAL Describing Process Models (Composed Concepts) Each individual step has Preconditions, Add-list, Delete-list Links among the steps Decomposition links between steps and substeps Disjunctive alternatives Temporal links … VirusInvasion substeps Enter Integrate Synthesize... disjunction

7 USC INFORMATION SCIENCES INSTITUTE KANAL Checks on Process Models All the steps are properly linked (substep, nextstep, disjunctive nextstep, conjunctive nextstep, …) All the preconditions of each step are satisfied during the simulation All the expected effects can be achieved There are no unexpected effects There are no impossible paths...

8 USC INFORMATION SCIENCES INSTITUTE KANAL Current Focus: Dynamic Checks Simulation (or symbolic execution) results show how substeps of the process model are related each other (Interdependency Model) Perform various kinds of checks unachieved preconditions expected/unexpected effects disjunctive branches loops causal links redundancies unordered steps … : Implemented

9 USC INFORMATION SCIENCES INSTITUTE KANAL Checking Unachieved Preconditions During simulation, collect unachieved preconditions by tracing failed expressions Suggest fixes Add a step that can achieve the condition Add ordering constraints between the failed step and another step that undid the condition Delete the step... VirusInvasion Enter Integrate... Failed Precondition: Virus near Cell Proposed Fixes: Add an Arrive step Add a Move step

10 USC INFORMATION SCIENCES INSTITUTE KANAL Checking Effects Compute the effects by simulation Suggest fixes for unachieved expected effects Add steps that can achieve the effect Add ordering constraints between effect adding steps and effect deleting steps Check unexpected effects After VirusInvasion ProteinCoat of the virus broken Achieved DNA of the virus has replicates Unachieved Add a Replicate step Add a Divide step

11 USC INFORMATION SCIENCES INSTITUTE KANAL Checking Disjunctive Branches Inform all the combinations of alternatives so that the user can check if some are impossible KANAL can simulate and highlight disjunctive paths

12 USC INFORMATION SCIENCES INSTITUTE KANAL Example: Lambda Virus Invasion Path1: Arrive1 Enter2 Circularize3 Integrate4 Divide5 Disintegrate6 Synthesize7 Replicate8 Path2: Arrive1 Enter2 Circularize3 Synthesize7 Replicate8 (From Alberts ECB Chapter 9) EnterCircularize IntegrateDivideDisintegrate SynthesizeReplicate disjunction Arrive

13 USC INFORMATION SCIENCES INSTITUTE KANAL Example: Conjunctive Branches Arrive1 Enter2 Trascribe3 Replicate4 Assembly5 Arrive1 Enter2 Replicate4 Trascribe3 Assembly5 Life cycle of a virus (from Alberts ECB Chapter 9) Enter Transcribe Assemble Replicate Conjunction Arrive

14 USC INFORMATION SCIENCES INSTITUTE KANAL Checking Loops Loop1: Arrive1 Enter2 Circularize3 Integrate4 Divide5 Disintegrate6 Synthesize7 Replicate8 Arrive1 Loop2: Arrive1 Enter2 Circularize3 Synthesize7 Replicate8 Enter1 Loop3: Divide5 Divide5 EnterCircularize IntegrateDivideDisintegrate SynthesizeReplicate disjunction Arrive

15 USC INFORMATION SCIENCES INSTITUTE KANAL Checking Causal Links Describe which step enables (or disables) a given step Arrive1 enables Enter2 by achieving Virus near Cell Integrate4 enables Disintegrate6 by achieving Virus DNA integrated with chromosome EnterCircularize IntegrateDivideDisintegrate SynthesizeReplicate disjunction Arrive

16 USC INFORMATION SCIENCES INSTITUTE KANAL Fixing Problems: Using Interaction Plans Interaction Plan: describes how to proceed with the user interaction direct what to do next based on the results from K Analysis KANALs dialogue for fixing errors is implemented with interaction plans Will be integrated with the Interaction Manager

17 USC INFORMATION SCIENCES INSTITUTE KANAL Keeping Track of Interaction History... Choose what to simulate choose model: VirusInvadesCell choose substep to test: VirusInvadesCell Simulate model VirusInvadesCell simulate-steps-&-find-failed-events ask-to-fix-failed-event: (failed preconditions of Enter) propose-fixes-for-failed-event ask-what-to-fix-for-failed-event : ((the location of (the patient of Enter)) = (the space-near of (the agent of Enter))) ask-how-to-fix-failed-event (add Arrive before Enter)

18 USC INFORMATION SCIENCES INSTITUTE KANAL Future Extensions (I): Static Checks Let user pose questions about various features of the process model to test the model KANAL will maintain test suites Users pick from sample query templates example: retrieving role values, part-of relations, type definitions,.. Users may specify their expected results Users may vary the initial situations to start from Explanation or trace of the answer to a query show how different pieces of K are used to generate the answer (Interdependency Model)

19 USC INFORMATION SCIENCES INSTITUTE KANAL Future Extensions (II) Exploiting history and evolution of Interdependency Models (for both simulations and queries) Example: Check what tests were correctly answered before Using heuristics to focus K analysis Example: when invalid results are obtained, KANAL will use a divide-and-conquer strategy and check intermediate results to find the sources of the problem Testing with different initial states and different arguments

20 USC INFORMATION SCIENCES INSTITUTE KANAL Future Extensions (III) Interdependency Models for problem solving knowledge EKCP Build on past work on EXPECT

21 USC INFORMATION SCIENCES INSTITUTE KANAL Using KANAL for Intelligent Tutoring Systems ITSs can acquire domain knowledge from human instructor and use simulations to refine the knowledge (Johnson et al 2000, Scholer et al 2000, Angros et al 99) We are exploring the use of KANAL to check and analyze the domain models while it is being built

22 USC INFORMATION SCIENCES INSTITUTE KANAL Knowledge Authoring Environment for Tutoring Systems (current) Demonstration Library of actions Domain Simulator Experimenter Initial Model Refined Model Final Model (Lessons) Instructor Steve Agent Student

23 USC INFORMATION SCIENCES INSTITUTE KANAL Knowledge Authoring Environment for Tutoring Systems (future) Editor Demonstration Library of actions Domain Simulator Experimenter KANAL Initial Model Refined Model Final Model (Lessons) Instructor Steve Agent Student