Knowledge Management Challenges for Question Answering Vinay K. Chaudhri SRI International White Paper Co-authors: Ken Barker (UT), Tom Garvey (SRI), Ken Murray (SRI), Bruce Porter (UT), Tomas Uribe (SRI)

Outline  Interest in QA  Similarities and differences to AQUAINT  Comments on AQUAINT

Interest in QA  Co-chaired AAAI Symposia Question Answering Systems (Fall’99) Mining Answers from Text and KBs (Spring 2002)

Interest in QA  Associated with DARPA projects High Performance Knowledge Bases Knowledge bases by knowledge engineers Rapid Knowledge Formation Knowledge bases by domain expert

Rapid Knowledge Formation Piece of knowledge Knowledge Entry Tool Knowledge Base Solve Problems

Rapid Knowledge Formation Piece of knowledge Knowledge Entry Tool Knowledge Base Solve Problems Text Answer questions

Rapid Knowledge Formation Piece of knowledge Knowledge Entry Tool Knowledge Base Solve Problems Text Answer questions Biology textbook Questions at the back of the book

Rapid Knowledge Formation Piece of knowledge Knowledge Entry Tool Knowledge Base Solve Problems Text Answer questions Biology textbook Questions at the back of the book We never work with text while answering questions

Rapid Knowledge Formation Piece of knowledge Knowledge Entry Tool Knowledge Base Solve Problems In the head of a human - Course of Action - Critiquing knowledge Simulation

Similarities and Differences  Similarities Knowledge can come from text QA as an evaluation technique  Differences Reducing the cost of hand crafted KB construction Logical representations for masses

Comments on AQUAINT  Breadth  Synergy  Evaluation

Comments on AQUAINT Breadth  Impressed by breadth Free Text Media Language Structured Data

Comments on AQUAINT Breadth  Impressed by breadth  Large amount of text is fundamental Free Text Multi-media Multi-lingua Structured Data

Comments on AQUAINT Synergy  Numerous synergies by combining free text with structure Handcrafted KBs (Cyc, Wordnet, Framenet,..) Learning from Text (ISI, LCC,..) Annotations (Time markup language – NERC) More is possible …

Comments on AQUAINT Evaluation  Opinion Questions Well-defined tasks Clearly articulated technical problem Clear statement of issues Well executed

Finally….  AQUAINT seems like a very well-managed program Friendly PM Executive Committee Strong participation from Intelligence Community

Thank You!

What Could Be Done?  Enhance Annotations  Knowledge Resources  Modality Independent Tools

Enhance Annotations  Look for places where manual annotation effort is already being invested XML documents Intelligence report summarization Corpus construction  Use RKF tools to do annotations Annotations come from a KB Not limited to keywords Can do inference with background knowledge

Knowledge Resources  SHAKEN incorporates significant general knowledge We give our system for free for research use  We can scale it to cover sizable text  Will be fun to combine it with large text/multi-media collections for fixed- domain QA

Modality Independent Tools  QA Management Managing interaction Managing the lifecycle of a question  Answer fusion  Source validity