USNR: An Overview CAPT Mike Blair (USNR)

New Data Sets Available Maj Jeff Juhnke 12 June 2003

3 Two New Data Sets Available 1.JMITC Counter-Terrorism Training Scenario 2.WWW Corpus of Terrorism-Related Data

4 1. Joint Military Intelligence Training Center (JMITC) Scenario used to train Analysts in Counter- Terrorism Link Analysis Unclassified German Counter-Terrorism Event in Time-Released Messages in US Message Traffic Format Terrorism Database and Multimedia (Video) Request for Information / Question and Answer 2 CD’s: Entire Scenario and 42 Messages Requires Non-Disclosure Agreement (similar to CNS Data)

5 2. Terrorism -Related WWW Corpus Unclassified Search for Terrorism Topics, Human-Assembled Domain 155,000+ pages in MS Word Document format Text with Bibliographic Citation

6 Points of Contact Maj Jeff Juhnke, DIA/Joint Staff J2 Kelcy Allwein, DIA Frank Hughes, DIA JMIC Dr. Richard Tong, Tarragon Consulting Corp

7 Requirements for Data Use JMITC Data is USG/Proprietary: For Official Use Only –Requires AQUAINT-Specific Non-Disclosure Agreement –Not for use OUTSIDE of AQUAINT Project Terrorism Data from WWW –Requires User ID and Password –Taxonomy / Feedback Appreciated

Wrap-up Dr. John D. Prange AQUAINT Program Manager

9 Preparing to Explore a New Paradigm in Information Access: A Scenario Approach to Question- Answering –Jointly Explore with QA Researchers and Subject Matter Experts the nature and implications of the emerging QA Scenario Paradigm and report back on: 1. The new paradigm from the intelligence analysts’ perspective, 2. The more advanced capabilities being developed beyond simple factoid QA 3. The scenarios proposed by the Subject Matter Experts, 4. The range of QA capabilities needed to make scenario-based QA a reality. –Leader: Liz Liddy, Syracuse University FY2003 NRRC Wkshp Challenge Problems

10 Workshop Follow-up ALL: Final version of ALL Powerpoint presentations during the workshops –Technical Presentations –Breakout Sessions –Brief-Backs ALL: All notes (both recorders and anyone else) and flipcharts from all Scenario Exercise discussions –Any additional comments, feedback, observations etc. concerning any aspect of the exercise, discussion sessions ARDA: Will consolidate all presentations from Workshop and as best we can all notes from Scenario Exercises –Distribute via NIST Web Site containing all AQUAINT Workshop Materials –CD format for those who attended but are not directly associated with AQUAINT Program

11 Workshop Follow-up All: CNS DVD Distribution – Did not get CNS DVD, but want one. Please send to Heather McCallum-Bayliss with your request, mailing address and reason. ARDA will review requests and respond Project PI’s: ARDA wishes to compile a full library of all publications supported by or based upon AQUAINT sponsored projects. – copy of all papers to Heather McCallum-Bayliss –Bibliographic information on any books – Please continue to send all future publications through the end of Phase I NWRRC: Distribute CD containing all final presentations from AQUAINT Spring Symposium to all attendees + all Project PI’s who were unable to attend

12 Workshop Follow-up All - ASAP: Input for AQUAINT Fall Symposium –Identify any conflicts with proposed dates of 8-9 October 2003 (Wed-Thurs) –Provide suggestion for Symposium Focus; Volunteer to be a Presenter All: Regional Research Center Workshops –Interested in hearing your ideas on topics for workshops that would strongly support AQUAINT –Look for Call for Proposals for FY 2004 in the early Fall. Project PI’s: Does your Phase I contract end before the 5 December 2003? –Work with your Contract COTR to request a no-cost extension to insure that you will be able to attend the Phase I 24-Month Workshop on 2-5 Dec –Reason: Need to insure that you have reserved sufficient funds to prepare for and attend the Phase I 24-Month Workshop

13 Plans for AQUAINT Program Phase I 24-Month Workshop When & Where –2-5 Dec 2003 (Tues-Friday) –Opening Reception Evening of 1 Dec –MITRE, McLean, VA MITRE 1 Conference Facility (formerly Hayes Building) 7525 Colshire Drive McLean, VA Workshop Focus –Final Overview Summary of Phase I –Project Summary Presentation (30-60 minutes each) Highlight significant research activities and results Focus on Contributions to and Impact on the AQUAINT Program Final System Demonstrations MITRE 1

14 Some Final Observations Demonstrations –Very significant and very positive progress since Dec 2002 Workshop Full Systems –When we started there were only a few full Question Answering Systems –Today we have a good number, pursuing a number of different approaches Component Systems / Enabling Technologies –Expected good technical progress & we got that –But I’m especially pleased by the degree to which these are oriented towards and becoming integrated into AQUAINT

15 Some Final Observations Scenario Exercise –Nervousness was justified –Not the most ideal conditions for the exercise –More, pre-training would have helped –Clearly things that we would do differently “But overall the ‘gods’ were good!” –Membership of each team was on the high side –Another excellent opportunity for a meaningful exchange between IC Analysts and AQUAINT Research Community –Applaud the level of interaction, involvement with this exercise –But bottom line is: “Well worth the time and effort” –Next Steps: Definitely want to try again, but decisions on that will wait until post NRRC Workshop led by Liz Liddy

16 Some Final Observations Format of this Workshop –Selected, focused technical talks on project highlights vs. Project Summary / Overviews –Made for a very enjoyable, stimulating workshop –Thank you! BUT ON THE OTHER HAND –Increasingly, need to be more cognizant of the first time AQUAINT attendees or their varying backgrounds/interests –Wondering outloud --- Would it be good to offer optional tutorial (s) to bring them up to speed and/or to offer presentations at different levels of technical depth? Anything from Anyone Else?

17 Questions / Comments

18 Contact Information Dr. John Prange, AQUAINT Program Director ARDA Web Pages: Phones: (Fax) Mailing:ARDA (RA) Room 12A69 NBP#1 STE Savage Road Fort Meade, MD