1 Semantic Web for the Military User Progress Tom Martin Research Management Enterprises (571)

2 DAML Program Semantic Web Military Applications World Wide Semantic Web W3C/DAML-OIL Infrastructure, Tools, Applications, and Language Specific Projects, Military Applications Forcenet/ ESG/Doctrine/LL Web Enabled Navy Intelink Horus Center for Army Lessons Learned Thesaurus Other/TBD Ongoing Ongoing & Planned JBI Foreign Clearance Guidance

3 Semantic Web for the Military User Meetings SWMU II – Nov 12/ Agenda Attendees Approach Outcome – Working Session Results Joint SWMU/GMUG/EEE Meeting March 25-27, 2002 Background Rationale for Joint Meeting Objectives Approach/Agenda

4 SWMU II Agenda Tuesday, November 12, 2001 AM – Introduction for those who had not attended SWMU-I (June 2001) –Ontologies Introduction, Horus Overview, DAML Language, Example Demo IT Talks, & Summary of Last Meeting (SWMU-I) PM – Plenary for all attendees –Update of DAML –Several short DAML demos – BBN, DRC, and LM AeroText –CoABS Grid Military Users Group (GMUG) overview –Horus update –NWDC updates, –Updates from Breakout groups: – –Intelligence (Joe Rockmore) – –Doctrine/Lessons Learned (Alice Mulvehill) – –C2 Applications (Tom Martin) Evening Demo session Wednesday, November 13, 2001 Intel, Doctrine/LL, and C2 Breakouts and Outbriefs

5 SWMU II Attendee/Organizations Elaine Marsh/NRL Frank Muller/BBN Paul Kogut/Lockheed-Martin Joe Rockmore/Cyladian Mike Dean/BBN Rob Rasch/BCBL-L (Army) Mike Rimmer/NWDC Mark Gorniak/AFRL Ken Whitebread/LMSC/ATL Frank White/SSC Martha Kahn/Global Infotek Hal Hultgren/NWDC Paul Neves/BBN Lee Lacy/DRC Tom Martin/RME Alice Mulvehill/BBN Wayne Perras/NWDC David Rager/BBN David Aha/NRL Jake Jacobowitz/OC Inc Sally DeGozzaldi/Joint Staff/J7 Mike Pratt/OC Inc Dennis Toomey/Anteon Ted Baer/Anteon Bruce Peoples/Raytheon Mark Werner/NWDC Chuck A Menges USMC Doc Div Phillip Boos/Thomas Associates Bob Lucas/JCLL at JWFC Chuck McGrath/OC Inc/JFCOM/JWFC Kalyan Gupta/ITT Industries/NRL Mike Dietvorst/USAF Doctrine Center Marti Hall/DRC Orlando Paul Odell/NWDC

6 Intelligence Breakout Joe Rockmore - Facilitator Charter How do the ideas of the semantic web specifically apply to intelligence problems? What unique problems does the intelligence community have with respect to using semantic web technology? How can we leverage the work being done in DAML, and specifically the applications to intelligence, to other efforts?

7 Semantic Web Functional Architecture Docs DBs Markup Map { DAML } KB Analyses User interactions Browsing Visualization Q & A Etc. Ont dev

8 Intelligence Ontologies (vice C2, logistics, or others) Intelligence needs to talk about what was, is, and might be (with uncertainty), while C2 plans what to do with resources available, logistics makes resources available, etc. Ontologies need to reflect differences in data and mission Issues of interest to intelligence (primary) Money laundering, geopolitical issues, financial transactions, non-military organizations, drugs, counter- terrorism, etc. Imagery, signals, open source, & analysis of this data Generally higher levels of abstraction than C2, etc. Source info and confidence in source important Temporal and spatial reasoning important

9 Significant Issue: Geolocation & Temporal Representation Understand documents enough to know locations in a document Placename, lat/lon, BE num, UTM, etc. Disambiguation Granularity issues Understand documents enough to know temporal aspects in a document Absolute time in different granularity (date & time to milliseconds vs. season) and representations (Julian date, DTG, etc.) Disambiguation Relative time (before, after, within, overlapping, close to, etc.) Co-reference problems in geolocations and times

10 Significant Issue: Markup Tools Consumer-based and producer-based markup tools needed Combine automated and manual markup intelligently Markup as part of authoring Culture is analysts (producers) are too busy to do any additional work, such as markup, unless Its very easy to do There is clear value to producers (not just consumers) Someone measures them on the quality/quantity of markup Mid term: mixed initiative, where authoring and knowledge object creation are done in parallel and with either driving the process A long term view: author knowledge objects from the outset; form products from these objects, including English text documents Multilingual opportunities

11 Significant Issue: Access to Data Tailored push; also pull (My Intelink), including changes of sufficient magnitude Subscriptions and data descriptions for matching against subscriptions may be best done using hierarchical ontologies (vice database schemata, which are not sufficiently expressive) Crawlers of value, but may have access control issues (open source an exception) Uncertainty of data (both by source and about source) Inference-based retrieval of information Pedigree critical to maintain (but often raises the security levels) Indexing of markup important for speed of access Timelines for intelligence information. Can be long, if national Can be short, if tactical

12 Significant Issue: Collection Tie collection, processing, production together A common markup language will enhance collection, thus optimizing use of intel resources Producers and consumers have different ways of looking at the world; there is not necessarily a mapping between them Can consumers provide tasking to producers, via markup, of requirements on collection? Info data needs from UJTL tasks or other statement of data needs

13 Significant issue: Security Will DAML markup allow semantic understanding of information enough to affect releasability processes? Can we do our collection and analysis at SCI and report at lower levels (including collateral, coalition, and unclass)?

14 Recommendations Military and intelligence users that particularly should hear about semantic web: DoD elements: DIA (esp JIVA), NSA Agencies: NRO, NIMA, CIA Service intel agencies: ISCOM, AFIA, ONI, MCIA Unified commands: JICs and JACs Standards setting and interoperability orgs How do organizations understand what DAML products and approaches could help them? Focused TIEs with appropriate producers and consumers around specific value propositions Need straightforward explanation of what DAML is and its value added (over XML)

15 Doctrine/Lessons Learned: Breakout Session Out-Brief –Working Group Name: Doctrine/LL –Purpose –Significant Issues –Recommendations/Plan of Action –Working Group Name: Doctrine/LL –Purpose –Significant Issues –Recommendations/Plan of Action Outline Facilitator: Alice Mulvehill (BBN)

16 Some Common Goals Use DAML to support user groups: –Operations: Decision support –Training –Doctrine: Both producers and consumers User Processes: –Information (doctrine/lessons) access –Auto-generation of valid/reliable structured information –Just-in-time information delivery (post approval/verification) Across all levels (strategic, operational, tactical) Tailored granularity (concept, task, mission) Usable format Constraints: –Portable: Does not require net connection –Tools must be easy to use, inexpensive to acquire

17 Significant Issues/Needs Process: 1.Tools for searching and inferencing with DAML ontologies for context-sensitive search for given documents for document revision and change tracking for speeding up the markup process 2.Meta-tag generation 3.Identifying users granularity need (e.g., paragraph vs. task) 4.Approval of DAML links in doctrine generation 5.Bandwidth of existing info infrastructure systems, & search implications 6.Performance Tasks: Representations for situation collection Verification and editing Trend analysis (e.g., from lessons, commonalities among METLs) Comparative evaluations (e.g., search using DAML markup vs. not) Process: 1.Tools for searching and inferencing with DAML ontologies for context-sensitive search for given documents for document revision and change tracking for speeding up the markup process 2.Meta-tag generation 3.Identifying users granularity need (e.g., paragraph vs. task) 4.Approval of DAML links in doctrine generation 5.Bandwidth of existing info infrastructure systems, & search implications 6.Performance Tasks: Representations for situation collection Verification and editing Trend analysis (e.g., from lessons, commonalities among METLs) Comparative evaluations (e.g., search using DAML markup vs. not) Product: 1.Provide portable/local search (e.g., not connected to SIPRNET) 2.Agent-based tools for exploiting DAMLized LL/doctrine 3.Meta-tag use Product: 1.Provide portable/local search (e.g., not connected to SIPRNET) 2.Agent-based tools for exploiting DAMLized LL/doctrine 3.Meta-tag use

18 Questions 1.Can PDF files (or their indices) be DAMLized (i.e., is HTML needed)? 2. Any existing dataspace visualization tools for viewing DAML links? If so, what are their existing capabilities? 2. Any existing dataspace visualization tools for viewing DAML links? If so, what are their existing capabilities?

19 1.Build ontologies for METLs (DRC) 2.Demonstrate the utility of DAML for applying doctrine/LL in decision support systems (Several groups) 3.Provide PDF mission folder examples (with indices) for experimentation re: just-in-time index generation (C. McGrath) 4.Test ontology creation & markup tools (M. Dietvorst w/ D. Rager) 5.Provide intro descriptions & system requirements of available DAML tools (A. Mulvehill) 6.Provide feedback/assessments to Alice on DAML tools (all) 1.Build ontologies for METLs (DRC) 2.Demonstrate the utility of DAML for applying doctrine/LL in decision support systems (Several groups) 3.Provide PDF mission folder examples (with indices) for experimentation re: just-in-time index generation (C. McGrath) 4.Test ontology creation & markup tools (M. Dietvorst w/ D. Rager) 5.Provide intro descriptions & system requirements of available DAML tools (A. Mulvehill) 6.Provide feedback/assessments to Alice on DAML tools (all) Recommendations/Plan of Action 1.Make requested DAML tools available e.g., Ontology generators, auto-markup tools, DAML editors 2.Create inferencing tools for use with DAML markup 3.Advertise DAML tutorial (i.e., the walk-thru example) 4.Request DAML tool summary at next meeting 1.Make requested DAML tools available e.g., Ontology generators, auto-markup tools, DAML editors 2.Create inferencing tools for use with DAML markup 3.Advertise DAML tutorial (i.e., the walk-thru example) 4.Request DAML tool summary at next meeting

20 C2 Applications Facilitator – Tom Martin Objectives Explore Command and Control Applications for DAML developments, both long and short term –What are ways in which we can best explore the value of DAML for C2 functions

21 DAML in Expeditionary Sensor Grid (ESG) for Data and Information Fusion ESG - A sensor grid architecture that integrates sensors, networks, decision aids, warriors and supporting systems into a highly adaptive comprehensive system that operates from the surface to space and from sea to land. Widely varying heterogeneity, thousands of sensors and sources - coalition, legacy and emerging systems and sensors Fusion will require many more operators than practical and automation of fusion will depend on software agents DAML Contribution Referenced metadata regarding sensor characteristics, performance and reliability (pedigree), and decision-needs context for multiple heterogeneous sensors/sources/systems enabling high level automated (agent-based) fusion. FY 02 ESG Enabling Experiments Assess DAML markup utility for enhancement of fusion Explore bandwidth overhead and scalability issues.

22 Sensor Processing Stream using fixed structure, XML, and DAML Sensor d specificatio n Algorithm description … S b` ScSc S zz S a` Time Series Sensor Fusion Processor Event Time/Freq, Intensity, Shape Features, Sensor Data C2 Fusion Classification Confidence Localization Tracks + Event Data SdSd SeSe ……... Heterogeneous Sensors Sensor c specificatio n Algorithm description … Sensor b specificatio n Algorithm description … Semantic Resources Fixed Data Structure XMLDAML DAML in Sensor Fusion

23 DAML in Sensor Fusion Sensor Processing Stream using fixed structure, XML, and DAML Sb`Sb` ScSc S zz Time Series Event Time/Freq, Intensity, Shape Features, Sensor Data Existence of object Classification Confidence Localization Tracks + Event Data SdSd SeSe …... Heterogeneous Sensors and Sources Sensor c specificatio n Pedigree Algorithm description Unit definition Decision Context … Semantic Resources Fixed Data Structure, XML DAML DATA AND INFORMATION FUSION (Client and Services?) Level 0 Signal Data Refinement Level 1 Object Refinement Objects Level 4 Resource Refinement Object Base Situation Visualization, Management Level 2 Situation Refinement Level 3 Impact Situations Impacts High Low INFERENCE LEVEL Sensor b specificatio n Pedigree Algorithm description Unit definition Decision Context … Sensor a specification Pedigree Algorithm description Unit definition Decision Context … HUMINT Open Source … ? ?

24 By the way XML = 10x hard code DAML = 2x XML Jini/Java breakpoint

25 Recommendations/Actions

26 Communications For ESG, smart agent needed for dynamic communications management to do reasoning about the network – to support the smart agent operation, need: Comms ontology from Operational Level to Tactical Level (see next slide) Identify boundaries of sensor nets, etc.

27 Ontology Development Ontology from Operational Level (CJTF) to tactical level (weapons on target) for supporting modeling of sensor/ communications/ information management needs Benefits to ESG –Experiment design –System Concept Testing –Once refined, system design, and construction Ties to many many other needs –Intelligence, Leverage of Doctrine/TTP/Lessons Learned/Training work

28 Experimentation Experimentation (General) For both Communications and Fusion applications, refine the CoABS Grid DAML interfaces for Utility for –Dynamic Communications –Sensor Management –Sensor Fusion DAML for Fusion EEE Experimentation –Explore the Depth of where DAML markup makes the most sense –Explore Jini/Java (I.e., Grid) object translation to DAML Research –Assess DAML tools for the multiple layers from the physical to the information management – –Bandwidth tradeoffs, etc. –Modeling DAML/assessing utility in the mobile environment

29 Joint SWMU/GMUG/EEE March 25-25, 2002 SPAWAR Systems Center, San Diego, CA A new approach to the Agent Based Computing Outreach to Transition Partners

30 Joint SWMU/GMUG/EEE March 25-27, 2002 Background Vision of Future Warfare Fit to ESG/Interest of Navy in Agent Based Computing Agent-Based and DARPA Fusion Technology Roles –CoABS Grid –Semantic Web/DAML –DDB/DTT –Each Program at a turning point –Changes of the Military Environment –Changes of Management and Organization

31 Joint SWMU/GMUG/EEE Rationale for Joint Meeting Unprecedented Opportunity Fruition of DARPA Technologies Role of NWDC in ESG Experimentation FY 01 Accomplishments with CoABS Grid –Interoperability –Dynamic Reconfigurability –Security FY 02 Plans –Distributed Multiple Real vs. Virtual Sensors –Teaming with JBI and possibly AATD –Fusion efforts – DDB/AIM and DAML Joint Meeting Rationale –Many of Same Participants for all Three Meetings –Sharing with Those Greatest Stressing the Technologies –Opportunity Joint Work on Future

32 Joint SWMU/GMUG/EEE Objectives Review ESG Enabling Experiment Plans Expose non-EEE Participants to key use of ABC NWDC Explore Added Opportunities for Co-evolve Emerging Technologies with New Concepts and Doctrine Adjust/Refine EEE Plans Accordingly A Forum for Sharing Grid and Semantic Web Experiences Review of Key Lessons Learned –Challenges, Problems, Work-arounds –Feed Back to Developers and Program Managers Explore Potential for Key ABC and Fusion Technologies to Contribute to ESG/JBI in Future Identify Potential DARPA Initiatives to Address Unsolved Technology Issues Result in actionable Plan of Action and Milestones to meet those objectives above

33 Joint SWMU/GMUG/EEE Approach Joint meeting for all three groups Sequence Objectives and Sponsor Views –ONR/NWDC Objectives –DARPA – CoABS, DAML, DDB/AIM Users Reviews –EEE Plans and Progress, and Grid Lessons Learned –DAML Issues, NWDC Lessons Learned/Doctrine CALL –Horus, AATD, CECOM, AFRL/JBI Working Groups

34 Joint SWMU/GMUG/EEE Working Group Process 4 Focus Areas Connectivity, Interoperability, and Security (SSC and JBI Lead) Sensors, Fusion, and Representation Using DARPA Technologies (SSC Lead) –Highlight DARPA Projects CoABS Grid, DAML, and DDB/AIM Agents for C2 (NWDC Lead) Ontologies for Military Use – Representation, C2, Fusion, Military Lessons Learned, Doctrine, Intelligence (NWDC Lead) Objectives Focus – Issues, Problems, Lessons Learned, Opportunities Objective, POA&M with with responsibilities, milestones, and due dates assigned Interrelated, not stove-piped approaches and solutions

35 Joint SWMU/GMUG/EEE Overall Process Objectives Kickoff Sensors, Fusion, and Representation Agents In Command and Control Ontologies For Military Use Interim Briefing Objectives Kickoff Sensors, Fusion, and Representation Agents In Command and Control Ontologies For Military Use Interim Briefing Objectives Kickoff Sensors, Fusion, and Representation Agents In Command and Control Ontologies For Military Use Interim Briefing Objectives & User Review Sensors, Fusion, and Representation Agents In Command and Control Ontologies For Military Use Interim Briefing* Working Groups Final Briefs, Sponsor Closeout Connectivity… Sensors, … Agents in … Ontologies … Plenary Connectivity, Interoperability, &Security *Cross-Pollination Opportunity

36 Joint SWMU/GMUG/EEE - Working Group Process – Interim Briefing – Cross Pollination 1.Connectivity,.. 2. Sensors,.. 3. Agents in C2 4. Ontologies for.. *Cross-Pollination Opportunity

37 Joint SWMU/GMUG/EEE - Working Group Process – Final POA&Ms – Interrelated 1.Connectivity,.. 2. Sensors,.. 3. Agents in C2 4. Ontologies for..

38 Joint SWMU/GMUG/EEE Schedule AM PM Monday 3/25Tuesday 3/26 Wednesday 3/27 User Review EEE-02 Plan Review EEE-02 Progress EEE Grid Lessons Learned DAML Issues Horus Review AATD CECOM JBI/AFRL Plenary Working Group Interim Briefs ( ) then Resume Cross- Fertilization Working Groups Sponsor Intro/Joint Meeting Objectives and Process Navy ONR/NWDC – FY 02 Navy Objectives DARPA Objectives and Updates CoABS DAML DDB/AIM Workshop Process Working Groups 1. Connectivity, Interoperability, & Security 2. Sensors, Fusion, & Representation 3. Agents in C2 4. Ontologies for Military Use Working Groups Re-convene ( ) then Working Group Out- Brief and Sponsor Wrap- up/Future Planning

39 Joint SWMU/GMUG/EEE Outcome Tuesday 3/26 1.Connectivity,.. 2. Sensors,.. 3. Agents in C2 4. Ontologies for.. Wednesday 3/27