1. HITIQA: Scenario Based Question Answering Tomek Strzalkowski, et al The State University of New York at Albany Paul Kantor, et al Rutgers University Boris Yamrom Lehman College of City University of New York (HLT-NAACL2004 Workshop)

2. Abstract HITIQA: High Quality Interactive Question Answering HITIQA is a open-domain QA In this paper we describe some preliminary results of qualitative evaluation. It uses novel data-driven semantics to conduct a clarification dialogue.

3. Introduction(1/3) HITIQA allows user to submit exploratory, analytical Qs.( eg: What has been Russia ’ s reaction to U.S. bombing of Kosovo?) The answer is heavily conditioned by what information is in fact available on the topic, background knowledge of the user while certain type is expected (eg: diplomatic statements).

4. Introduction(2/3) HITIQA is Dialogue with Information Analytical task: topic + context (time, recipient, purpose) Detect, follow, anticipate, negotiate strategy turns and shifts

5. Introduction(3/3) Acknowledgements Based on work supported by ARDA(Advanced Research and Development Activity) ’ s AQUAINT(Advanced QA for intelligence)

6. Data Driven Semantics of Questions We use a combination of Google and InQuery to retrieve the top 50-200 documents from a large document database. 1.ranked:on target,near misses,outliers 2.extracted into topic frames.

7. Partial structuring of text data Frames are defined structures repre- senting various “ event types ” 3 broad domain frames of “ Weapon of mass destruction proliferation domain. ” WMDtransfer WMDDevelop WMD- Tready All frames contain small num of core attribute using BBN extract 24 kind.

8. Partial structuring of text data the Goal frame generated from the question, “ Where does al- Qaida have training facilities? ”

10. How it works?

11. HITIQA Dialogue capabilities System actions: offers, suggestions, clarifications Offering closely related events (time, space, etc.): Imports from Korea? Missile developments there? Suggesting similar events involving different players Sarin production in Iran? In Pakistan? Analyst ’ s actions affecting answer space Direct: accept, partially accept, or reject offers Indirect: strategy changes, diversions Support hypothesis formation Explicit: HITIQA suggestions and offers Analyst information saving actions Implicit: HITIQA: Direction of dialogue and answer formation Analyst: Deciding what question to ask next

12. Analyst: What is the history of the nuclear arms program between Russia and Iraq? HITIQA: Do you want to see material on cooperation on development of weapons between Iraq and Russia? Analyst: Yes … Who has helped finance the nuclear arms program in Iraq? HITIQA: Would you be interested in information on weapons in Iraq? Analyst: Yes … What type of nuclear weapons does Iraq possess? HITIQA: Would you be interested in information on dollars in Russia relative to your question? Analyst: Yes … What is the finance debt that Iraq owes Russia?

13. Answer generation Current Frame-based summaries Frames in the answer space rendered into headlines Passages sorted by “ threads ” : target, time, location, etc. Build more coherence into the answer Use elements Discourse Structure Theory Applied to passages from multiple documents Passages are output to form a more logical flow Maximize lucidity of the answer Use dialogue history to structure the answer → folders Compute rhetorical relations between answer elements justification, elaboration, evidence, contradiction, etc. Answer Summaries Summarize answer passages using XDoX Summarizer

14. Workshop Result It is important to determine the realism of the scenarios used during the workshop relative to the analysts current work tasks in order for any results to be meaningful. Each analyst was asked a series of five questions These 5 questions were all relative to the realism and difficulty of the scenario tasks used a scale of 1 to 5 based on their agreement with the statement.

16. Using Camtasia record the session.

19. Result “ Having HITIQA at work would help me find information faster than I can currently find it ”, our mean score was 3.83.

20. Integrated Visual interface