AQUAINT R&D Program Phase I Kickoff Workshop WELCOME

AQUAINT Kickoff – 3 December AQUAINT: User Testbed / System Integration Pull together best available system components emerging from AQUAINT Program research efforts –Couple AQUAINT components with existing GOTS and COTS software Develop end-to-end AQUAINT prototype(s) aimed at specific Operational QA environments Government-led effort: –Directly Linked into Sponsoring Agency’s Technology Insertion Organizations –Close, working relationship with working Analysts –Provide external system development support –Mitre/Bedford will lead External System Integration / Testbed efforts –Plan to also utilize additional external researchers as Consultants / Advisors

AQUAINT R&D Program Phase I Kickoff Workshop Breakout Sessions Dr. John D. Prange AQUAINT Program Director December 2001

AQUAINT Kickoff – 3 December Breakout Sessions AQUAINT Evaluation – Both Sessions –Ellen Voorhees & Donna Harman, NIST User Testbed/System Integration Issues – Both Sessions –Marc Light & John Berger, Mitre Data Requirements – 1 st Session –Richard Tong, Tarragon Consulting Sharability of Data, Components, & Resources – 2 nd Session –John Donelan, CIA NRRC Program Committee Meeting – 1 st Session –Penny Lehtola, ARDA

AQUAINT Kickoff – 3 December AQUAINT: Evaluation Issues Build upon highly successful TREC Q&A Track Evaluations -- NIST has lead and is currently developing a Phased Evaluation Plan tied to AQUAINT Program Plans Cooperate to maximum extent possible with DARPA’s RKF (Rapid Knowledge Formation) Program Evaluation Efforts First Session: Special Emphasis on Knowledge-Based Systems Second Session: Special Emphasis on Dialogue Systems

AQUAINT Kickoff – 3 December AQUAINT: User Testbed / System Integration Pull together best available system components emerging from AQUAINT Program research efforts –Couple AQUAINT components with existing GOTS and COTS software Develop end-to-end AQUAINT prototype(s) aimed at specific Operational QA environments Government-led effort: –Directly Linked into Sponsoring Agency’s Technology Insertion Organizations –Close, working relationship with working Analysts –Provide external system development support –Mitre/Bedford will lead External System Integration / Testbed efforts –Plan to also utilize additional external researchers as Consultants / Advisors

AQUAINT Kickoff – 3 December AQUAINT: Data Requirements Issues Start by Using Existing Data Collections –NIST’s TREC Text Corpora –Linguistic Data Consortium (LDC) Human Language Corpora (e.g. TDT, Switchboard, Call Home, Call Friend Corpora) –Existing Knowledge Bases and Other Structured Databases Future Data Collection & Annotation and Question/Answer Key Development will be a major effort –Current Activities to secure major new collections –Ideas for other future collections (e.g. Multiple Genre?) Will likely use combined efforts of NIST and LDC

AQUAINT Kickoff – 3 December AQUAINT: Sharability of Data, Components, & Resources Focus is on sharability across the individual projects What Data, Components and other Resources are individual projects willing to share within the AQUAINT community? Similarly what Data, Components and other Resources are individual projects looking for? Data annotations, taxonomy classifications, and/or other locally created/generated data/resources –Do we need to work toward common “structures” or “standards”? –To what extent are these sharable?

AQUAINT Kickoff – 3 December Room Assignments Session 1: 9:00-10:15am –Evaluation Main Workshop Room –User Testbed Room N3.365 –Data Requirements Room N3.555 –NRRC Program Comm. Room N3.770 Session 2: 10:45-12 noon –Evaluation Main Workshop Room –User Testbed Room N3.365 –Sharability Room N3.555

AQUAINT Kickoff – 3 December Contact Information Dr. John Prange, AQUAINT Program Director ARDA Web Pages: Phones: (Fax) Mailing:ARDA Room 12A69 NBP#1 STE Savage Road Fort Meade, MD