June 11-13, 2002AQUA Question-Answering System1 AQUA: AQUAINT Question Answering System Project Progress Report SAIC, San Diego KSL, Stanford

June 11-13, 2002AQUA Question-Answering System2 Project Summary

June 11-13, 2002AQUA Question-Answering System3 Collaboration with NMSU Team This spring, NMSU has been added as an AQUAINT contractor Synergies between the NMSU Team and the SAIC Team have led to a collaborative effort for AQUAINT

June 11-13, 2002AQUA Question-Answering System4 SAIC-KSL-NMSU Collaborative System QUESTION NL Query Interlingua Query KIF Query KIF Answer Interlingua Answer NL Answer ANSWER NMSU Query Processor SAIC Interlingua  KIF Translator KSL Java Theorem Prover SAIC KIF  Interlingua Translator NMSU NL Generator

June 11-13, 2002AQUA Question-Answering System5 Key Tasks—KSL Providing knowledge tools for team Ontolingua Knowledge Server (OKS) Background ontologies and lexicons JTP deductive answer determination system Developing new methods for Answer determination from large, complex KBs Using interoperating special-purpose reasoners KB partitioning so that most reasoning occurs within partitions Using Semantic Web representation languages (e.g., DAML+OIL) Providing understandable explanations for deduced answers sourcesconversionsReasoning steps, data sources, reasoning method, data conversions, … Use-specific and user-specific customization Detecting and resolving conflicts in KBs likelyProactive background testing for both likely and provable conflicts annotateInteractive tools for helping analyst correct or annotate conflicts

June 11-13, 2002AQUA Question-Answering System6 Progress to Date—KSL Developing JTP – An answer determining reasoner Developing a suite of special-purpose reasoners E.g., for determining time-dependent answers Reasoner for QA from ontological knowledge (in DAML+OIL) Produces and caches reasoning results during KB loading Will be able to accept a series of queries, without waiting for answers Usability improvements to JTP For KB development and query-answering testing and debugging Support for rapid reloading and changing of developing KBs E.g., Checkpointing and “untell” Hierarchical presentation of reasoning explanations DQL – Standard query language and QA protocol for DAML+OIL Basic framework for client/server deductive query answering Answers generated a batch at a time Formal semantics Applicable to other representation languages Being developed jointly with the DAML language design committee Will be implemented for JTP

June 11-13, 2002AQUA Question-Answering System7 Key Tasks—SAIC Ontolingua Translator Automatic translation of information from NMSU team into KIF knowledge representations Includes dynamic semantic alignment of TMR Ontology and Ontolingua Ontology Includes inferring relations that span multiple sentences Includes flagging relations with document contexts (time, location) Pre-reasoning to pre-answer known useful questions Modular, interchangeable KB systems to determine specific types of relations that are likely to be highly relevant and useful Current system is event-based, but architecture valid for other types of representations OKS Interface Transfer, load, and store KIF representations into Ontolingua KBs Maintain DB of translated documents with info about contents of each Defines what documents are relevant to specific queries Reduces burden of KB partitioning –Each document has a separate KB file –Query TMR matched to contents of known documents to determine relevant KBs –Only KBs for relevant documents loaded into JTP to determine the answer Source Credibility Assessment Develop historical records of source reliability for sources Assessment of estimated source credibility for incoming knowledge Develop methodologies for dynamic changes to credibility ratings

June 11-13, 2002AQUA Question-Answering System8 SAIC-KSL-NMSU Collaborative System QUESTION NL Query Interlingua Query KIF Query KIF Answer Interlingua Answer NL Answer ANSWER NMSU Query Processor SAIC Interlingua  KIF Translator KSL Java Theorem Prover SAIC KIF  Interlingua Translator NMSU NL Generator

June 11-13, 2002AQUA Question-Answering System9 Converting Semantic Info to Knowledge Ontologies (and ontological philosophies) differ between Ontolingua and TMR SAIC dynamically aligns the two ontologies as part of the knowledge formation process Must be dynamic process since both TMR and Ontolingua ontologies are actively changing during system development An automated mechanism for translating information in TMR into KIF relations is needed. TMR breaks text into smallest pieces SAIC must re-unify the pieces to produce more meaningful knowledge representations Next slide shows a typical example of this problem

June 11-13, 2002AQUA Question-Answering System10 Re-Unification of Knowledge TMR includes separate references for this sentence fragment for: united-states-soldier soldier-human-adult united-states-human-adult We re-unify this into a single definition that captures that these are all the same and plural. (defobject united-states-soldier (instance-of united-states-soldier person) (has-country united-states-soldier united-states) (member-of united-states-soldier us-army-special-forces)) (defobject group-of-united-states-soldier (group group-of-united-states-soldier united-states-soldier) (cardinality group-of-united-states-soldier 36)) SAMPLE TEXT: About 36 US Special Forces troops started a month of anti-terrorism training…

June 11-13, 2002AQUA Question-Answering System11 KIF Formulations from Narratives Often, useful text sources are narratives rather than unrelated compilations of facts Story comprehension must extend across sentences to the entire text of the narrative SAIC’s experience developing knowledge representations from narratives in HPKB and RKF programs provides unique and powerful capability in this area Our “event descriptor templates” provide relations needed to generate a series of event descriptors for a narrative These allow us to answer highly complex questions about complicated, real-world situations Developed and proven successful in HPKB Program Not trying to generate all possible relations from the text—only those relations that are in the event descriptors (i.e., known useful relations)

June 11-13, 2002AQUA Question-Answering System12 Why Event Templates? Using event templates dramatically decreases the work load on JTP for each query Distributes the analysis across multiple small reasoners specialized to answer specific types of questions Pre-query analysis anticipates common questions that may be asked about this document and pre-determines the answers automatically JTP’s set of multiple reasoners includes forward chaining, which may add other relations at load time rather than waiting for a specific query against the events Result should be to dramatically improve answer response time for many queries Also provides extensibility of the system because each set of relations is handled by a separate, modular KB reasoner Support for other types of inputs than narratives may replace the mini-reasoners but doesn’t change the architecture

June 11-13, 2002AQUA Question-Answering System13 Progress to Date Application of general-purpose rules that apply across a broad spectrum of instances Initial processing of event basics in place Identification of type of event, agent performing event, basic identification of object/agent acted on in event Subject-verb-object Automatic definition of specific objects from text Places, people, groups Including cardinality of groups if available in original text Location and time of event Determination (from raw text dateline, etc.) of event context location and event context time All events in this context are tagged relative to event-context- location and event-context-time

June 11-13, 2002AQUA Question-Answering System14 Automatic Knowledge Representation Status (cont.) More sophisticated event processing Interests relations General rules about actions supporting interests of agents performing them Citizenship relations Action-object events Dealing with verbs that are objects of actions in action-object-is relations Example: “pretending to do something”  action-object-is “doing something” Representation is an event for “doing-something-event” and a separate event for “pretending-the-doing-something-event” Object re-unification Recognition of previously referred to objects (from other sentences or within longer sentences) as the same object Pronoun dereferencing Multiple phrasings of the same object

June 11-13, 2002AQUA Question-Answering System15 Status of Knowledge Representation System Preliminary demo of automatic translation from TMR to KIF is up and running Our demonstration system is limited at the moment Limited ontology Limited relations Limited lexical terms The Demo System is improving quickly in its capabilities; in the meantime…

June 11-13, 2002AQUA Question-Answering System16 Demonstration System