1. SBIR Topic A04-105: An Ontologically-Based Data Fusion Model Chris Matheus & Mitch Kokar Versatile Information Systems, Inc. cmatheus, mkokar@vistology.com www.vistology.com A04-105: An Ontollogically-Based Data Fusion Model

2. Outline JDL Model Motivation, objectives, requirements Meta-modeling framework IFRM – top level IFRM – first step refinements Ontologies and fusion How many ontologies can be used? Commercialization – an idea Tasks Backup slides

3. SOURCE PRE - PROCESSING LEVEL ONE PROCESSING OBJECT REFINEMENT LEVEL TWO PROCESSING SITUATION REFINEMENT LEVEL THREE PROCESSING THREAT REFINEMENT LEVEL FOUR PROCESSING PROCESS REFINEMENT DATA FUSION DOMAIN DATA BASE MANAGEMENT SYSTEM FUSION DATABASE SUPPORT DATABASE SOURCES LOCAL DISTRIBUTED NATIONAL INTEL EW SONAR RADAR... DATA BASES HUMAN COMPUTER INTERACTION USERS WA - SOT assessment for Multi - source/sensor capability: Mature / Available Lab Demos / Not Fielded Research Required * From JDL/DFG Data Fusion Model JDL Model

4. Fusion: JDL Model

5. Motivation JDL has seen a lot of attention But mainly Level 1 has been tested Recent emphasis on higher-level fusion made JDL insufficient Moreover, JDL has been misinterpreted as a data flow model Still needs a process model –Connected by “bus” – too flexible –Layered – too restrictive

6. Reference Model - Impacts Improve development efficiency Enable application portability and scalability Ease application adaptability Improve system interoperability (NCW!) Improve user productivity and portability Promote vendor independence Improve security Reduce life-cycle cost

7. Phase I Objectives Analyze reference models, identify useful features Propose adding process model to JDL Formalize proposed model Propose model evaluation approach and tools Select application for validation in Phase II Disseminate findings (papers, presentations) Prepare an RFP for the OMG

8. Model Requirements Be descriptive – serve the fusion community Capture data, functions, processes Represent fusion processes – allow comparison of systems before they are built Capture metrics of performance Ontologically based – formal, computer processable semantics Formal specification of the model itself Have a place for capturing human-in-the-loop and user models Have associated software tools Compatibility with current models (as much as possible, but not more)

9. Meta-Modeling Framework (MMF) Meta Model Objects Objects of interest in the domain (instances) Classes of objects and relations among classes (Need modeling elements to define models) Definition of modeling elements

10. MMF – Software Engineering Meta Model Objects new Employee(“John”) new Employer(“IBM”) Class, Association, … UML Class Diagrams Java Objects Employee:Class, employedBy:Association

11. MMF – Semantic Web Meta Model Objects Class, Property, … OWL Ontology Annotation (markup)

12. MMF – Information Fusion Meta Model Objects track51:Track1, tankA1:TankB, IFRM Terms (Class, Property, Track, Object, Situation) IFRM Model of Fusion System (run time) System model (Track1:Track, O1:Object, TankB:Tank)

13. Three Views

14. Refinement: Product This is just a first refinement step, not even complete …

15. OODA

16. OODA in UML

17. Refinement: Function (just an example)

18. ontology 1. the branch of metaphysics dealing with the nature of being, reality or ultimate substance (cf. phenomenology) 2. particular theory about being or reality phenomenology 1. the philosophical study of phenomena, as distinguished from ontology 2. the branch of a science that classifies and describes its phenomena without any attempt at metaphysical explanation metaphysics 1. the branch of philosophy that deals with first principles and seeks to explain the nature of being or reality (ontology); it is closely associated with the study of nature of knowledge (epistemology) Ontology (Webster)

19. An explicit specification of a conceptualization: the objects, concepts, and other entities that are assumed to exist in some area of interest and the relationships that hold among them (Genesereth & Nilsson, 1997) Definitions associate the names of entities in the universe of discourse (e.g., classes, relations, functions, or other objects) with human-readable text describing what the names mean, and formal axioms that constrain the interpretation and well-formed use of these terms. A statement of a logical theory. (Gruber) An agent commits to an ontology if its observable actions are consistent with the definitions in the ontology (knowledge level). A common ontology defines the vocabulary with which queries and assertions are exchanged among agents. Ontology (cont.)

20. Ontologies and IFRM IFRM is at the same level as UML and OWL (from the linguistic point of view) IFRM is a domain-specific (information fusion) modeling language –Ontology defines a conceptualization, a vocabulary … Thus IFRM is an ontology, provided it is specified explicitly in a formal language (formal axioms)

21. Ontologies and Fusion System Meta Model Objects Annotations IFRM = Ontology of Fusion IFRM Model of Fusion System (run time) Specific ontology

22. Commercialization Idea Achieve acceptance by the fusion community Achieve acceptance by the Government Request for Proposals (RFP) for standards - the OMG Work with OMG towards standard (IFRM) Promote the standard Build supporting tools Result: Interoperability of various fusion systems!

23. Tasks Task123456789 A. Analyze other models B. Refine the model C. Formalize top-level model D. Develop evaluation methodology E. Propose software tools F. Identify evaluation scenarios G. Prepare RFP to OMG

24. Backup Slides

25. Ontology Languages Web Ontology Language (OWL) –Lite –DL (Description Logics) –Full OWL+SWRL (Semantic Web Rule Language) RuleML First Order Predicate Calculus (FOPL) Higher Order Logics (HOL)

26. OWL & SWRL XML RDF RDFS OWL SWRL Extensible Markup Language Resource Description Framework RDF Schema Web Ontology Language Sem. Web Rule Language Note: The layering of OWL on top of RDFS is not strict.

27. How many ontologies? Net Centric Warfare (NCW) requires full interoperability and thus communication An agent commits to an ontology if its observable actions are consistent with the definitions in the ontology (knowledge level). Two agents must commit to the same ontology in order to communicate What if there are N agents? –One common ontology? Seems impossible! –N 2 ontologies? Unmanageable! –Ontology mapping? Work in progress. –Core ontology + extensions? Compromise. Somewhat similar idea to NATO’s Generic Hub (GH5) and C2 Information Exchange Data Model (C2IEDM)

28. C2IEDM Core

29. GH 5 Core CANDIDATE-TARGET-LIST CAPABILITY RULE-OF-ENGAGEMENT REPORTING-DATA LOCATION OBJECT-TYPE OBJECT-ITEM CONTEXT ACTION

30. GH5 Core A neat idea –Common core, extendable –Relatively rigorous –Substantial amount of knowledge –XML Schema available Not an ontology –Not formal (no formal semantics) –Thus cannot be processed by logical tools –Not quite compatible with MMF (mixes Object and Model levels) –But looks like a good start towards an ontology

31. VIS SAW Core Ontology SBIR Phase II, AFRL/IF Rome, Mike Hinman, John Salerno

32. Battlefield Ontology

33. What is “extension”? Add classes –E.g., Battalion is a Unit Add properties (relations) –E.g., Platoon is part-of Company Add constraints –E.g., Soldier can be part of only one Platoon But the result must be an ontology that is consistent

34. Consistency consistency agreement with what has already been done or expressed; conformity with previous practice [Webster’s] In logic: from P and not(P) can derive anything Inconsistency is a dangerous thing for autonomous agents!

35. Inconsistency: Example Battlefield Ontology happens to be consistent Suppose we have –Constraint: Unit must have at least 8 Soldiers Suppose we then extend it by adding: –Class: Group (sub-class of Unit) –Constraint: Group has 3 Soldiers Inconsistency! Easy to generate inconsistencies while developing ontologies ConsVISor to the rescue: http://www.vistology.com

36. Ontology Mapping DB 1 DB d KB 1 KB k SsSs S1S1 TtTt OoOo T1T1 O1O1 AD 1 AD d AK 1 AK k AT t AO 1 AS 1 AS s AT 1 AO o GUI Agents: commit to ontologies; negotiate mapping; use templates

37. Ontology Mapping (cont.) Need to map: –Classes to Classes –Properties to Properties –Objects to Objects –Constraint mapping implicit May result in inconsistency! ConsVISor to the rescue: http://www.vistology.com

38. Why Use Ontologies? 1. Represent theories of potential objects and relations as ontologies (OWL) 2.Represent collected data as annotations in terms of ontology (OWL) 3. Formulate any queries about situations in OQL (OWL Query Language) 4. Use a general purpose OWL reasoner to answer queries 5. Use the trace of the reasoner to give an explanation to user 6. Multiple ontologies may need to be combined into one (fusion) 7.Data Association and Fusion 1. association of objects with ontologies/annotations 2. relations among objects within ontologies/annotations 3. combining ontologies/annotations using colimit of category theory Flexibility: Can use the same reasoners on any ontologies and annotations!

39. Use Case: Ontology-Based Fusion

40. Use Case: Fusion System Development