Explanation in GILA 2 Stanford -> RPI McGuinness, Ding January 15, 2008

Motivation Improve trust in recommendations from GILA components Support evaluation – Understand why GILA makes suggestions – Identify which prior knowledge is (re-)used – Identify which learned knowledge is learned and (re-)used – Summarize usage of external interaction information steps Support internal trust and reuse – Identify which component suggested what and why – Identify/ propagate dependencies

Provided Knowledge runtime input prior input Learned Knowledge Final Output Flow of GILA Knowledge expert execution trace ontological knowledge constraint-violation pstep list as the final solution Learning & practice performance context knowledge facts embedded in the input problem problem/solution constraints conflict priority

GILA Knowledge KnowledgeCategoryProducerConsumerOWL class Execution trace ProvidedExpertMRE, ILRsgilcore:ExecutionTrace Input ACO problem ProvidedBlueforceMRE, ILRsgilaco:ProblemACO Context knowledge ProvidedContext service MRE, ILRsPhase II Ontological knowledge ProvidedExpertMRE, ILRsaco:* Constraint LearnedCLILRsgilcore:Constraint Constraint Violation LearnedSCMRE, ILRsgilaco:ViolationStatement An ACO problem state LearnedMREILRsgilaco:ProblemACO Cost of an ACO problem state LearnedDTLMRE, ILRs?gilaco:CostStatement Credit/Blame Assignment LearnedSC + MREILRsPhase II Conflicts in an ACO problem state Learned4DMREgilcore:Conflict Order of conflicts LearnedILRsMREPhase II Intersection of conflicts Learned4DMREgkst:IntersectionDetails Pseudo expert trace LearnedMRE, ILRsILRsgilcore:ExecutionTrace Final solution LearnedMRE-DMPstepManagergilcore:PstepList

Phase II Objectives Show usage of provided knowledge – Expert execution trace – Ontological knowledge Show usage of learned knowledge – Constraints – Blame (constraint violation) assignments Show (abstracted) problem-solving trace (with dependencies) – problem, learner, solution, …., final solution

Approach Ontology Representation – Reference knowledge at appropriate level of granularity – Provide interlingua for learners’ knowledge usage Extract some knowledge from KB Derive new knowledge from existing knowledge Computational Components – Run time API for sharing knowledge of explanations – API for extracting, summarizing and visualizing problem- solving and knowledge usage trace utilizing Inference Web approach

Example: Blame-assignment #vstmt192 (gilaco:ViolationStatement) Degree of importance of violation How severe the violation is Confidence #constraint 7 (gilcore:Constraint) #pstep261 (gilcore:Pstep) #problem26 (gilaco:ProblemTrySafetyConstraint) #solution28 (gilaco:SolutionTrySafetyConstraint) Safety Checker MRE-DM #solution23 (gilaco:SolutionResolveConflict) #solution26 (gilaco:SolutionResolveConflict) X-ILR DT-ILR …

Learned knowledge Provided knowledge Example: Learn and Solve Based on correspondence, santi Nov 30, 2007 Solution 5647Case 101 PSTEP 8859 PSTEP 8860 PSTEP 8861 ExecutionTrace 1878 generarted from learned from learned from learned from in confidence confidence Performance Step 129 Learning Step 126 CBL ILR Has conclusion Has antecedent Has conclusion Has antecedent match case and solve Learn new case Has learner Use rule

Directions Determine appropriate granularity for representation and propagation (initially coarse level moving to finer granularity where required) Design appropriate primitives for phase II topics – context, credit/blame, orderings, priorities, … Focus on dependencies initially (supporting explanations showing usage of prior knowledge, external interaction- gained info, use and re-use of learned knowledge, similarity knowledge, adaptation knowledge…) Design GILA-appropriate explanation templates exploiting our explanation interlingua Present knowledge provenance summaries (with follow up options)

PSTEP Manager Fujitsu Song

PSTEP Manager in Phase 1 SWebAD PSTEP Manager (PMAN) PSTEP List Translate into SWebAD command Backup PSTEP information Execution Generate execution trace Known Issues – PMAN is widely used to check whether a proposed solution is correct or not, each check takes long time. – Most failures are due to some errors which can be checked before execution and avoided (e.g. NaN maximum altitude, minimum altitude > maximum altitude, etc.) – The knowledge may not be discovered from the expert trace (since the expert does not make this type of mistakes), but can be learned from the execution result reported by SWebAD during the execution time. PSTEP Manager in Phase 1 is a simple execution engine and it hides execution related details from other modules

PSTEP Manager in Phase 2 STBMCS PSTEP List Translate into STBMCS command Backup PSTEP information Generate execution trace Early Error Detection Execution Request STBMCS Error Learner Constraint knowledge repository PSTEP List Optimization New Modules Existing Modules PSTEP Manager (PMAN) STBMCS Error learner learns new knowledge from error report of STBMCS (previously SWebAD) Share constraint knowledge with 4DCL Use constraint knowledge to detect errors of proposed solution before real execution Share with 4DCL/R Error Report

Extra

Files/WWW Toolkit Proof Markup Language (PML) CWM (NSF TAMI) JTP (DAML/NIMD) SPARK (DARPA CALO) UIMA (DTO NIMD Exp Aggregation) IW Explainer/ Abstractor IWBase IWBrowser IWSearch Trust Justification Provenance N3 KIF SPARK-L Text Analytics IWTrust provenance registration search engine based publishing Expert friendly Visualization End-user friendly visualization Trust computation Semantic Discovery Service (DAML/SNRC) OWL-S/BPEL Framework for explaining question answering tasks by abstracting, storing, exchanging, combining, annotating, filtering, segmenting, comparing, and rendering proofs and proof fragments provided by question answerers. Inference Web Infrastructure McGuinness, Ding, Pinheiro da Silva, Chang, Fikes, Glass, Zeng