Yolanda Gil Jihie Kim Jim Blythe Surya Ramachandran

Slides:



Advertisements
Similar presentations
1 KANAL: Knowledge ANALysis Jihie Kim Yolanda Gil USC/ISI
Advertisements

Modelling with expert systems. Expert systems Modelling with expert systems Coaching modelling with expert systems Advantages and limitations of modelling.
Computer Science CPSC 322 Lecture 25 Top Down Proof Procedure (Ch 5.2.2)
Verification and Validation
Semantics Static semantics Dynamic semantics attribute grammars
Testing and Quality Assurance
KOS and the Conduct of Science© Straits Knowledge 2011 Knowledge Organisation Systems as Enablers to the Conduct of Science Patrick Lambe.
Knowledge Acquisitioning. Definition The transfer and transformation of potential problem solving expertise from some knowledge source to a program.
Knowledge Acquisition CIS 479/579 Bruce R. Maxim UM-Dearborn.
Approaches to Typing Programming Languages Robert Dewar.
CS 330 Programming Languages 09 / 16 / 2008 Instructor: Michael Eckmann.
Cs164 Prof. Bodik, Fall Symbol Tables and Static Checks Lecture 14.
1 USC INFORMATION SCIENCES INSTITUTE Tailor, 6/20/04 TAILOR*: Modifying Calo’s Procedure Knowledge through Instruction Jim Blythe, Yolanda Gil, Jihie Kim.
1 USC Information Sciences Institute Jihie Kim Yolanda Gil Jim Blythe Intelligent Systems Division USC/Information Sciences Institute
1 USC INFORMATION SCIENCES INSTITUTE Modeling and Using Simulation Code for SCEC/IT Yolanda Gil Jihie Kim Varun Ratnakar Marc Spraragen USC/Information.
1.eCognition Overview. 1 eCognition eCognition is a knowledge utilisation platform based on Active Knowledge Network technology eCognition covers the.
February 3rd, 2010 KS BRMS. Discalaimer The GUI for the BRMS is currently not running, and was developed using a outdated framework so fixing is not an.
Mathematical Modeling and Formal Specification Languages CIS 376 Bruce R. Maxim UM-Dearborn.
Interpreting Dictionary Definitions Dan Tecuci May 2002.
1 USC INFORMATION SCIENCES INSTITUTE TEMPLE meeting, July 2000 TEMPLE: TEMPLate Enhancement through Knowledge Acquisition Yolanda Gil Jim Blythe Jihie.
Detailed design – class design Domain Modeling SE-2030 Dr. Rob Hasker 1 Based on slides written by Dr. Mark L. Hornick Used with permission.
Building Tools by Model Transformations in Eclipse Oskars Vilitis, Audris Kalnins, Edgars Celms, Elina Kalnina, Agris Sostaks, Janis Barzdins Institute.
1 USC INFORMATION SCIENCES INSTITUTE TEMPLE meeting, July 2000 Specifying Planning Objectives Yolanda Gil Jim Blythe Jihie Kim Surya Ramachandran
ENM 503 Lesson 1 – Methods and Models The why’s, how’s, and what’s of mathematical modeling A model is a representation in mathematical terms of some real.
1 USC INFORMATION SCIENCES INSTITUTE CALO, 8/8/03 Acquiring advice (that may use complex expressions) and action specifications Acquiring planning advice,
Proactive Acquisition Dialogues Jihie Kim Yolanda Gil
ISBN Chapter 3 Describing Semantics -Attribute Grammars -Dynamic Semantics.
1 USC INFORMATION SCIENCES INSTITUTE CAT: Composition Analysis Tool Interactive Composition of Computational Pathways Yolanda Gil Jihie Kim Varun Ratnakar.
1 USC, INFORMATION SCIENCES INSTITUTE An integrated environment for KA An Integrated Environment for Knowledge Acquisition Jim Blythe
Machine Learning Chapter 5. Artificial IntelligenceChapter 52 Learning 1. Rote learning rote( โรท ) n. วิถีทาง, ทางเดิน, วิธีการตามปกติ, (by rote จากความทรงจำ.
1 USC INFORMATION SCIENCES INSTITUTE Expect: COA Critiquing PSM EXPECT: A User-Centered Environment for the Development and Adaptation of Knowledge-Based.
1 Analysing system-user cooperation in KADS H. P. de Greef and J. A. Breuker, Department of Social Science Informatics, University of Amsterdam Knowledge.
Eurostat 4. SDMX: Main objects for data exchange 1 Raynald Palmieri Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, October.
SLICK: Proactive Acquisition Dialog Jihie Kim Yolanda Gil Varun Ratnakar.
1 USC INFORMATION SCIENCES INSTITUTE EXPECT TEMPLE: TEMPLate Extension Through Knowledge Acquisition Yolanda Gil Jim Blythe Information Sciences Institute.
Modelling the Process and Life Cycle. The Meaning of Process A process: a series of steps involving activities, constrains, and resources that produce.
1 USC INFORMATION SCIENCES INSTITUTE Gil & Kim Interactive Knowledge Acquisition Tools: A Tutoring Perspective Yolanda Gil Jihie Kim USC/Information Sciences.
WonderWeb. Ontology Infrastructure for the Semantic Web. IST Project Review Meeting, 11 th March, WP2: Tools Raphael Volz Universität.
KANAL (Knowledge ANALysis) Jihie Kim Jim Blythe Yolanda Gil
WonderWeb. Ontology Infrastructure for the Semantic Web. IST WP4: Ontology Engineering Heiner Stuckenschmidt, Michel Klein Vrije Universiteit.
On Abductive Equivalence Katsumi Inoue National Institute of Informatics Chiaki Sakama Wakayama University MBR
What is this? SE-2030 Dr. Mark L. Hornick 1. Same images with different levels of detail SE-2030 Dr. Mark L. Hornick 2.
Artificial Intelligence Knowledge Representation.
Data Mining What is to be done before we get to Data Mining?
KANAL (Knowledge ANALysis) Status Jihie Kim Yolanda Gil Jim Blythe Varun Ratnakar
Mechanisms for Requirements Driven Component Selection and Design Automation 최경석.
Knowledge Representation Techniques
Theories in Formation (TIF)
KANAL: Knowledge ANALysis
Action Editor Storyboard
Muneo Kitajima Human-Computer Interaction Group
Typical Person :^) Fall 2002 CS/PSY 6750.
Can SNOMED CT be harmonized with an upper-level ontology?
KANAL: Knowledge ANALysis
Ch. 4 – Semantic Analysis Errors can arise in syntax, static semantics, dynamic semantics Some PL features are impossible or infeasible to specify in grammar.
Architecture Components
Object oriented system development life cycle
Next Step #2: Acquisition Dialogue
Verification and Validation
THEORY IN EDUCATIONAL RESEARCH
Deriving Acquisition Principles from Tutoring Principles
Verification and Validation
ece 627 intelligent web: ontology and beyond
Usability Techniques Lecture 13.
Typical Person :^) Fall 2002 CS/PSY 6750.
TEMPLE: TEMPLate Enhancement through Knowledge Acquisition
Ling 566 Oct 14, 2008 How the Grammar Works.
Evaluating alternative anthrax production processes using a generic PSM: A worked example Yolanda Gil Jim Blythe Jihie Kim Surya Ramachandran USC/ISI.
University of Manchester
Presentation transcript:

Yolanda Gil Jihie Kim Jim Blythe Surya Ramachandran Knowledge Analysis Yolanda Gil Jihie Kim Jim Blythe Surya Ramachandran 6/19/2000 USC/INFORMATION SCIENCES INSTITUTE

USC/INFORMATION SCIENCES INSTITUTE Kinds of K Analysis Validation Static and dynamic checks to detect errors Testing with examples Dynamic checks, but done through execution/ground facts Verification (comply with spec) showing explanations to user facilitating browsing and viewing presenting analogies 6/19/2000 USC/INFORMATION SCIENCES INSTITUTE

Validation: Checks and Error Categories 1) Definition check parsing errors, if any are possible missing basic required parts of definitions (e.g., process step not linked to any others) 2) Static checks within a k unit undefined terms illegal domain/range for relation duplicates or overlaps with existing k unit 3) Dynamic checks (i.e., in reasoning) unsatisfied conditions/missing method invalid result unusable consequent/unused method 4) Clean up potential generalizations compilation composite definitions 6/19/2000 USC/INFORMATION SCIENCES INSTITUTE

Validation: Some Example Tools CHIMAERA KI/KM EXPECT V&V (SEEK) -------------------------------------------------------------------------------------------------------------------------------------------- K UNIT frame components psk rules (conj) Definition checks parsing errors, if interface allows them struct editor missing basic required parts of definitions simple incompl for psk OTHERS Static checks within a k unit undefined terms acronym expansion undefined c/i/r inconsistent definition slot value/type clash unsatisfied constr invalid expr ambivalent rules duplicates or overlaps with existing k unit redundant types psk exists redundant, subsumed OTHERS definition cycle Dynamic checks (i.e., in reasoning) (TRIPLE) missing k unit missing psk unusable consequent invalid result invalid result type unusable k unit unused psk unsatisfiable rule Clean up checks potential generalizations abduction compilation EBL based composite definitions OTHERS taxonomic analysis circularity Example-based checks NOTE: REFINERY will do checks that are similar to CHIMAERA, and will include "inferential checks" (they would be a kind of dynamic checks) and example-based checks. NOTE: Many other systems do similar error detection (TAQL, ASPEN). NOTE: semantic constraints can be explicitly used to detect errors. Validation: Some Example Tools 6/19/2000 USC/INFORMATION SCIENCES INSTITUTE

USC/INFORMATION SCIENCES INSTITUTE Dynamic Checks Dynamic checks can be done with: views ps goals explanations prediction analogy ... 6/19/2000 USC/INFORMATION SCIENCES INSTITUTE

Knowledge Analysis: Beyond Error Detection 3 main functions: Error detection Diagnosis of error source Suggested fixes This means that: detecting errors is just the beginning the other 2 functions require having more context 6/19/2000 USC/INFORMATION SCIENCES INSTITUTE

USC/INFORMATION SCIENCES INSTITUTE Knowledge Units an axiom, a concept, a psm, a component, a cmap, ... whatever it is Checks can be performed by iterating bottom up Info associated with the k units source info (provided by the IM) where & how often & how well it's been used/tested interdependency model: links to other k units links within a ps episode role mapping (e.g., for a component mapping) 6/19/2000 USC/INFORMATION SCIENCES INSTITUTE

USC/INFORMATION SCIENCES INSTITUTE Why Heuristics Suppose the system generates lots of questions to the user: does not assume without checking does not generalize without checking double checks any possible connection, deduction, etc. considers it needs to know everything Quantity not necessarily correlated with quality 6/19/2000 USC/INFORMATION SCIENCES INSTITUTE

Basic Principles to Focus Knowledge Analysis 1) Principle of practical validation (PPV) Invalid/incomplete statements are more likely to appear in k fragments that have not been exercised by using them to solve problems or answer questions 2) Principle of experiential context (PEC) Invalid/incomplete statements are more likely to appear in k fragments where limited prior knowledge (theories, components, models, etc.) can be or has been brought to bear 3) Principle of local consistency (PLOC) Inconsistencies are more likely to appear in k fragments that have not been defined and/or cannot be viewed in proximity (spatial, temporal, representational, or inferencial) by the user 6/19/2000 USC/INFORMATION SCIENCES INSTITUTE

USC/INFORMATION SCIENCES INSTITUTE 6/19/2000 USC/INFORMATION SCIENCES INSTITUTE

Entry VS Analysis Checks Knowledge analysis will become more challenging as we strive to make knowledge entry easier Needs to be defined in terms of capabilities of other components Knowledge entry Less constrained in format Less constrained by BG K Less constrained in structure Knowledge analysis More room for error More discrepancies with KS Less generalized/principled 6/19/2000 USC/INFORMATION SCIENCES INSTITUTE

Instrumentation: Issues Suppose the system generates lots of questions to the user: does not assume without checking does not generalize without checking double checks any possible connection, deduction, etc. considers it needs to know everything Quantity not necessarily correlated with quality 6/19/2000 USC/INFORMATION SCIENCES INSTITUTE

Example Inconsistencies and Knowledge Gaps [Kim&Gil, AAAI-99] Guide the design and creation of a new KB element (e.g., a method) Find dependencies within the KB element based on representation language Ex: new method uses a role that has inadequate domain and range or is undefined Ex: new method has a variable with no declared type Find if the new method fits in principle with existing knowledge Ex: new method has same capability as a previously defined method Detect missing knowledge Find undefined methods given the newly created ones Ex: the new method has a subgoal that cannot be achieved by any existing methods Propose initial version of new methods to add Ex: propose a capability and a result type based on the unmatched subgoal Fitting pieces together Find user defined and yet unused methods Ex: method not used to achieve any subgoals Propose potential uses of an unused method in other methods Ex: new method can almost match another method’s subgoal 6/19/2000 USC/INFORMATION SCIENCES INSTITUTE