Using Non-Taxonomic Knowledge to Improve Semantic Matching Peter Yeh July 22, 2003

Talk Outline Introduction Analysis of Existing Techniques Our Approach Initial Evaluation Proposed Work

Introduction Many AI tasks require determining whether two knowledge representations encode the same knowledge.

Information Retrieval Match queries with documents. Q: “A car with a bumper made of gold.” Car BumperGold has-part material Car Gold material Car Acme Produce agentobject Car BumperGold has-part material A: “Acme makes a car made of Gold.”

Knowledge Acquisition Match new knowledge with existing knowledge. KB KB: Are you trying to encode a conversion? MicrobePollution agentobject DestroyCreatecauses Food result MicrobePollution agentobject DestroyCreatecauses Food result New Knowledge Conversion next-event agentobject DestroyCreate result agent subevent Entity Existing Knowledge

Rule-based Classification Match rule antecedents with working memory. For example, Course of Action (COA) critiquing. Attack Delay Military-Unit agent object causes Military-Unit IF THEN AttackBlock Delay Advance Artillery-Unit Armor-Unit agent object agent causes Pattern COA Attack Delay Military-Unit agent object causes Military-Unit “This COA has a rating of good for enemy maneuver engagement.”

The Core Problem Solving this matching problem is hard because multiple encodings of the same knowledge rarely match exactly. Representations don’t match exactly because: –Expressive Ontology. –Knowledge is encoded by different sources. –Knowledge being encoded is complex.

Types of Mismatches Informal examination of a knowledge-base containing: –Patterns. –COAs. Knowledge-base was built by two Subject Matter Experts (SMEs) participating in DARPA’s RKF project. Looked for cases of mismatch.

Types of Mismatches (cont.) Taxonomic Differences “an armored brigade engaging an armored battalion.”

Types of Mismatches (cont.) Taxonomic Differences Equivalent Alternatives “One military unit attacking another unit.”

Types of Mismatches (cont.) Taxonomic Differences Equivalent Alternatives Omissions “Mechanized infantry brigade engaging mechanized infantry battalion.”

Types of Mismatches (cont.) Taxonomic Differences Equivalent Alternatives Omissions Granularity “Support attack occurs before main attack.”

Analysis of Existing Techniques Analogy Inexact Matching Semantic Matching Conceptual Indexing Ontology Merging

Analogy Analogy: mapping of knowledge from a base domain to a target domain. Structure Mapping Engine (Forbus et. al. 89): –Maps relational knowledge (mappable systems). –Systematicity Principle used to select best analogy. Analogy based on common generalizations (Leishman 92) –Maps both relational knowledge and object attributes. –Prefers minimal common generalization.

Analogy: Structure Mapping Engine Attack Delay Military-Unit agent object causes Military-Unit agent object causes AttackBlock Delay Advance Artillery-Unit Armor-Unit agent object agent causes agent object agent causes agent object agent causes agent object agent causes agent object agent causes agent object agent causes AttackBlock Artillery-Unit Armor-Unit agent object agent causes Block Delay Artillery-Unit Armor-Unit agent object agent causes Attack Delay Military-Unit agent object causes Military-Unit

Inexact Matching Inexact Matching: tries to address mismatches between representations Graph Editing (Tsai et. al. 83, Shapiro and Haralick 81, Messmer et. al. 93, Wolverton et. al. 2003) –Uses edit distance parameters. –Similarity based on shortest sequence of edits. Partial Matching –Does not require representations to be isomorphic. –Similarity based on amount of structural overlap. –Minimal Common Supergraph (Bunke et. al. 2000) and Maximal Common Subgraph (Bunke and Shearer 98).

Inexact Matching: MCS Attack Delay Military-Unit agent object causes Military-Unit AttackBlock Delay Advance Artillery-Unit Armor-Unit agent object agent causes Block Delay Artillery-Unit Armor-Unit agent object agent causes AttackBlock Artillery-Unit Armor-Unit agent object agent causes Attack Delay Military-Unit agent object causes Military-Unit

Semantic Matching Semantic Matching: uses knowledge to match representations. Projection: –Uses taxonomic knowledge. –Ontoseek (Guarino et. al. 99) and ELEN (Huibers et. al. 96). Projection+: Projection alone is too restrictive –  -projection (Genest and Chein 97). –Common generalization, graph splitting, regular expressions (Fargues 92, Buche et. al. 2000, Martin et. al. 2001). Semantic Overlap –Maximal Joins and Generalizations (Myaeng 92, Poole et. al. 95). –Shared Semantic Structures (Zhong et. al. 2002).

Semantic Matching: Semantic Overlap Attack Delay Military-Unit agent object causes Military-Unit AttackBlock Delay Advance Artillery-Unit Armor-Unit agent object agent causes AttackBlock Delay Advance Artillery-Unit Armor-Unit agent object agent causes Attack Delay Artillery-Unit Armor-Unit agent object Attack Delay Artillery-Unit Armor-Unit agent object Attack Delay Military-Unit agent object causes Military-Unit

Conceptual Indexing Conceptual indexing: how to organize and index knowledge. Requires so form matching. Generalization hierarchy (Bournard et. al. 95, Ellis 92, Levinson 82, Woods 97). –Knowledge indexed by common generalizations. –Generalizations organized hierarchically by subsumption relationships. –Retrieve Most Specific Subsumer (MSS) of a query. Match procedure is similar to Projection - suffers the same problems.

Ontology Merging and Translation Ontology Merging: merge multiple ontologies built by different sources –Chimaera (McGuinness et. al. 2000) –SMART (Noy and Musen 99). Ontology Translation: translates a representation from one language to another –Ontomorph (Chalupsky 2000). Goals are different but share some of the same problems.

Our Approach The goal of this research is to solve the matching problem. We believe existing semantic approaches can be extended with additional knowledge to significantly improve matching. What kinds of additional knowledge? –Transformations Handle mismatches. Improve matching. –Not taxonomic knowledge.

Our Approach (cont.) Generality and domain-independence. –Want additional knowledge (e.g. Transformations) to be useful across domains. We believe domain-independence is possible given a reusable domain-neutral upper ontology. –Contains a small set of general concepts. –SMEs use this upper ontology to build KBs on specialized topics (e.g. chemistry, biology, battle space planning). –No training in logic or knowledge representation.

Illustration of Our Framework Transformations Ontology KB KE SME/KE KB can be viewed as a domain-specific matcher (e.g. match symptoms to diseases). Domain-independent KB for the task of matching.

Our Prototype Extend semantic matchers with transformations. Apply transformations in a forward-chaining manner. Use existing techniques for reasoning with Conceptual Graphs (Corbett et. al. 99, Salvat et. al. 96, Willems 95): –Projection. –Unification. –Graph rules. Two caveats because existing techniques lead to promiscuous matches.

Transformations that Retains Semantics Buyobject agent origin Car Person: Y Person: X Buyobject agent origin Car Person: Y Person: X CarLike Person: X object agent Projection CarLikeobject agent Driving-Licensepossesses Buyobject agent origin Car Person: John Person: Bob Driving-Licensepossesses Buyobject agent origin Car Person: John Person: Bob CarSell Person object agent recipientPerson

Transformations that Retains Semantics Buyobject agent origin Car Person: Y Person: X Driving-Licensepossesses Buyobject agent origin Car Person: John Person: Bob CarSell Person object agent recipientPerson Buyobject agent origin Car Person: Y Person: X CarSell Person: Y object agent Driving-Licensepossesses Buyobject agent origin Car Person: John Person: Bob Sellobject agent Car Person: John Sellobject agent CarSell Person: Y object agent

Rule Applicability Buyobject agent origin Car Person: Y Person: X CarSell Person: Y object agent Driving-Licensepossesses Buyobject agent origin Car Person: John Person: Bob Sellobject agent Driving-Licensepossesses Buyobject agent origin Car Person: John Person: Bob Buyobject agent origin Car Person: Y Person: X Buyobject agent origin Car Person Driving-Licensepossesses

Rule Applicability Driving-Licensepossesses Buyobject agent origin Car Person: John Person: Bob CarSell Person object agent recipientPerson Buyobject agent origin Car Person: Y Person: X CarSell Person: Y object agent Driving-Licensepossesses Buyobject agent origin Car Person: John Person: Bob Sellobject agent Buyobject agent origin Car Person: Y Person: X CarSell Person: Y object agent

Enumerating Transformations Transformations derived from our domain-neutral upper ontology. Enumerated all ways that a relation can be legally used to encode information in a conceptual graph. Considered whether the same information can be expressed differently. Enumeration was possible because: –Small upper ontology. –Each concept had well-defined semantics.

Transformations Enumerated We were able to enumerate about 300 transformations. Resulting transformations fall into three general categories: –Transitivity –Part Ascension –Transfers Through

Transformations Enumerated (cont.) relationTransitivePart AscensionTransfers Through causesX-subevent, resulting-state caused-byXsubevent-ofresulting-from defeats--- defeated-by-subevent-ofcaused-by enablesX-causes, resulting-state, subevent enabled-byXsubevent-ofcaused-by, resulting-from inhibits-subevent-ofresulting-state inhibited-by-subevent-ofcaused-by, resulting-from by-means-ofX-- means-by-whichX-- prevents-subevent-of- prevented-by-subevent-ofcaused-by, resulting-from resulting-state--causes resulting-from---

Example: Our Approach Attack Delay Military-Unit agent object causes Military-Unit AttackBlock Delay Advance Artillery-Unit Armor-Unit agent object agent causes agent object Military-Unit Delay causes Attack Military-Unit Attack Military-Unit Delay Military-Unit Delay 1: 2: 3: 4: 5: agent object causes Attack Delay A: B: C: D: E: Advance agent Block Artillery-Unit Block Armor-Unit object agent BlockArmor-Unit BlockDelay causes Armor-Unit F: G: H: I: l 1 =l 1 = {(1,A)} M = {}{(1,A)}

Example: Our Approach {(1, A)}, {(3,C)}, {(4,D)}, {(5,E)} } M = { A B CD E F G H I AttackBlock Delay Advance Artillery-Unit Armor-Unit agent object agent causes Attack Delay Military-Unit agent object causes Military-Unit A B CD E F G H I AttackBlock Delay Advance Artillery-Unit Armor-Unit agent object agent causes Attack Delay Military-Unit agent object causes Military-Unit A B CD E F G H I AttackBlock Delay Advance Artillery-Unit Armor-Unit agent object agent causes Attack Delay Military-Unit agent object causes Military-Unit A B CD E F G H I AttackBlock Delay Advance Artillery-Unit Armor-Unit agent object agent causes Attack Delay Military-Unit agent object causes Military-Unit A B CD E F G H I AttackBlock Delay Advance Artillery-Unit Armor-Unit agent object agent causes Attack Delay Military-Unit agent object causes Military-Unit

Example: Our Approach AttackBlock Delay Advance Artillery-Unit Armor-Unit agent object agent causes Attack Delay Military-Unit agent object causes Military-Unit Transformations Action causes Action causes Action causes Action causes

Example: Our Approach Transformations Action causes Action causes AttackBlock Delay Advance Artillery-Unit Armor-Unit agent object agent causes Attack Delay Military-Unit agent object causes Military-Unit causes AttackBlock Delay Advance Artillery-Unit Armor-Unit agent object agent causes Attack Delay Military-Unit agent object causes Military-Unit causes

Initial Evaluation Used our matcher in an application in the domain of battle space planning (DARPA's RKF Project). The task is to analyze COAs. Battle space ontology built by extending our upper ontology. Two military analysts used this ontology to build KBs containing: –Patterns. –COAs. Our matcher matched the patterns to COAs.

Example Output

Experiment 1 Evaluates our first hypothesis. –How significant is the improvement? Compared our matcher to: –Maximal Common Subgraph (MCS). –Semantic Search Lite (SSL). Methodology: –300 domain-neutral transformations; 80 domain-specific transformations. –Matched the patterns to the COAs. –A pattern matches a COA if the match score meets or exceeds a pre-specified threshold. –Used metrics of precision and recall.

Experiment 1: Precision

Experiment 1: Recall

Experiment 2 Initial evaluation of our second hypothesis. –Assesses the domain independence of using transformations. Limited - conducted in only one domain, but can still offer some insight. Methodology: –Divided transformations into 2 groups (domain-neutral vs. domain-specific). –Used domain-neutral transformations to construct DN –Used domain-specific transformations to construct DS –Everything else is the same as Experiment 1.

Experiment 2: Precision

Experiment 2: Recall

Proposed Work More Comprehensive Evaluation. Use background knowledge. Incorporate indexing to make matching more efficient.

Comprehensive Evaluation Evaluate our approach in several applications in four domains. Four data sets: –Chemistry (Halo). –Biology (RKF). –Battle Space Planning (RKF). –Office Procedures (EPCA). Three Applications: –Elaboration: Chemistry and Office Procedures. –Question Answering: Biology and Battle Space. –Plan Evaluation: Battle Space and Office Procedures.

Background Knowledge Background Knowledge. Can be used to normalize new knowledge at acquisition time via a join (Mineau et. al. 93). Idea can be applied to matching. –Increase similarity. Two problems: –When should a join be performed? –How to better control the join? Ontology BlockMove object prevents Military-Unit Block

Background Knowledge Background Knowledge. Can be used to normalize new knowledge at acquisition time via a join (Mineau et. al. 93). Idea can be applied to matching. –Increase similarity. Two problems: –When should a join be performed? –How to better control the join? BlockMove object prevents Military-Unit AttackBlock object Military-Unit causes object AttackMove object prevents Military-Unit BlockMove object prevents Military-Unit Attack object Military-Unit Attack Military-Unitobject Move object Military-Unit Move object Military-Unit

Indexing Need indexing to make matching more efficient. A common technique is a generalization hierarchy –Overhead for storage can be expensive. –Finding the MSS can also be expensive. We intend to study: –How to index knowledge by content? –Other index structures that are more parsimonious.