# 1 # 1 Model-based Communication Networks, Valued Information at the Right Time (VIRT) & Rich Semantic Track (RST): Filtering Information by Value to Improve Collaborative Decision-Making Project Report on CEC Collaboration Rick Hayes-Roth June 15, 2007

# 2 # 2 Outline Overall Vision: Model-based Communication Networks, VIRT, and Rich Semantic Track Overall Vision: Model-based Communication Networks, VIRT, and Rich Semantic Track CEC/VIRT Project Results and Accomplishments CEC/VIRT Project Results and Accomplishments Rich Semantic Track Definition and Influence on Community Developments Rich Semantic Track Definition and Influence on Community Developments Semantic Modeling and Condition Monitoring: Making Information Delivery More Precise and Reducing Work Load Semantic Modeling and Condition Monitoring: Making Information Delivery More Precise and Reducing Work Load Opportunities for Further Collaboration Opportunities for Further Collaboration

# 3 # 3 Outline Overall Vision: Model-based Communication Networks, VIRT, and Rich Semantic Track Overall Vision: Model-based Communication Networks, VIRT, and Rich Semantic Track CEC/VIRT Project Results and Accomplishments CEC/VIRT Project Results and Accomplishments Rich Semantic Track Definition and Influence on Community Developments Rich Semantic Track Definition and Influence on Community Developments Semantic Modeling and Condition Monitoring: Making Information Delivery More Precise and Reducing Work Load Semantic Modeling and Condition Monitoring: Making Information Delivery More Precise and Reducing Work Load Opportunities for Further Collaboration Opportunities for Further Collaboration

# 4 # 4 COIs Overall Vision: Model-based Communication Networks, VIRT and Rich Semantic Track Past Global Information Grid Past Present Future Present Future Present Past Present Future Past Future COIs Shared World Models Common Track Semantics State-full Network

# 5 # 5 Overall Vision: Model-based Communication Networks, VIRT and Rich Semantic Track Past Present Future Present Future Present Past Present Future Past Future Common Track Semantics State-full Network Valued Bits Global Information Grid Shared World Models

# 6 # 6 Outline Overall Vision: Model-based Communication Networks, VIRT, and Rich Semantic Track Overall Vision: Model-based Communication Networks, VIRT, and Rich Semantic Track CEC/VIRT Project Results and Accomplishments CEC/VIRT Project Results and Accomplishments Rich Semantic Track Definition and Influence on Community Developments Rich Semantic Track Definition and Influence on Community Developments Semantic Modeling and Condition Monitoring: Making Information Delivery More Precise and Reducing Work Load Semantic Modeling and Condition Monitoring: Making Information Delivery More Precise and Reducing Work Load Opportunities for Further Collaboration Opportunities for Further Collaboration

# 7 # 7 CEC/VIRT Project Key Results Advanced theory and implementation of Valued Information at the Right Time (VIRT) and Rich Semantic Track (RST) Advanced theory and implementation of Valued Information at the Right Time (VIRT) and Rich Semantic Track (RST)  Demonstrated significant reductions in bandwidth from 2-5 orders of magnitude in initial studies to 45-90% in CEC-specific simulated message streams Created research opportunities for NPS faculty and students Created research opportunities for NPS faculty and students Created research and development initiatives with government and industry Created research and development initiatives with government and industry Developed simulation framework for advanced studies of VIRT and RST Developed simulation framework for advanced studies of VIRT and RST

# 8 # 8 CEC/VIRT Project Results (1) Prepared/published/presented several papers Prepared/published/presented several papers  “Model-based Communication Networks and VIRT: Filtering Information by Value to Improve Collaborative Decision- Making”  “Towards a Rich Semantic Model of Track: Essential Foundation for Information Sharing”  “Two Theories of Process Design for Information Superiority: Smart Pull vs. Smart Push”  VIRT Technical Architecture Specification (Draft)  “Hyper-Nodes for Emerging Command and Control Networks: The 8 th Layer” (with Alex Bordetsky)  “Model-Based Communication Networks and VIRT: Orders of Magnitude Better for Information Superiority”  “Decision Making in Very Large Networks” (with Peter Denning)  “Semantic Reasoning for Adaptive Management of Telecommunications Networks” (with J. C. Hoag) Established VIRT as key focus of W2COG Established VIRT as key focus of W2COG  VIRT driving various groups’ efforts

# 9 # 9 CEC/VIRT Project Results (2) Recruited & mentored three NPS thesis students Recruited & mentored three NPS thesis students Mentoring two NPS PhD students Mentoring two NPS PhD students Helped PACOM start up CMA JCTD around sharing of rich track information; leading design and development of the CMA data model Helped PACOM start up CMA JCTD around sharing of rich track information; leading design and development of the CMA data model Advising Joint Track Management (JTM) Architecture Working Group Advising Joint Track Management (JTM) Architecture Working Group Acquired workstation & CEC simulation data Acquired workstation & CEC simulation data Completed first computational model of CEC- VIRT concepts in thesis research Completed first computational model of CEC- VIRT concepts in thesis research

# 10 # 10 CEC/VIRT Project Results (3) Designed and implemented CEC-VIRT simulation framework for continuing test and experimentation Designed and implemented CEC-VIRT simulation framework for continuing test and experimentation Conducted computational analysis of CEC simulated message streams Conducted computational analysis of CEC simulated message streams Mapped CEP-to-Track-User message elements to abstract Rich Semantic Track model Mapped CEP-to-Track-User message elements to abstract Rich Semantic Track model Initiated design and development of Semantic Web representations of RST Initiated design and development of Semantic Web representations of RST Designed expressions of VIRT Conditions of Interest using Semantic Web technologies in CEC context Designed expressions of VIRT Conditions of Interest using Semantic Web technologies in CEC context Initiated design and development of formalized theory of RST Initiated design and development of formalized theory of RST Defined initial measures of performance for evaluating VIRT applications Defined initial measures of performance for evaluating VIRT applications

# 11 # 11 VIRT Implementation and Testing Described VIRT Product Line Architecture approach Described VIRT Product Line Architecture approach Built collaboration with industry to generate a draft technical architecture for VIRT Built collaboration with industry to generate a draft technical architecture for VIRT Hayes-Roth paper showed 5 orders of magnitude reduction in bandwidth using Smart Push Implementation Hayes-Roth paper showed 5 orders of magnitude reduction in bandwidth using Smart Push Implementation LCDR Acevedo thesis describing application of VIRT concepts in context of the CEC -- Under modeling assumptions, Theory 2 Smart Push model showed a decrease of nearly 200 times the bandwidth of Theory 1, Smart Pull model LCDR Acevedo thesis describing application of VIRT concepts in context of the CEC -- Under modeling assumptions, Theory 2 Smart Push model showed a decrease of nearly 200 times the bandwidth of Theory 1, Smart Pull model

# 12 # 12 Outline Overall Vision: Model-based Communication Networks, VIRT, and Rich Semantic Track Overall Vision: Model-based Communication Networks, VIRT, and Rich Semantic Track CEC/VIRT Project Results and Accomplishments CEC/VIRT Project Results and Accomplishments  Drill-down on Analysis of CEC Message Streams Rich Semantic Track Definition and Influence on Community Developments Rich Semantic Track Definition and Influence on Community Developments Semantic Modeling and Condition Monitoring: Making Information Delivery More Precise and Reducing Work Load Semantic Modeling and Condition Monitoring: Making Information Delivery More Precise and Reducing Work Load Opportunities for Further Collaboration Opportunities for Further Collaboration

# 13 # 13 CEC/VIRT Message Stream Analysis Simulated message streams provided by JHU/APL Simulated message streams provided by JHU/APL Focus on application of VIRT at an intermediary node (“VIRT Track User”) between the CEC network and the general GIG network Focus on application of VIRT at an intermediary node (“VIRT Track User”) between the CEC network and the general GIG network Value determined by “GIG User” Conditions of Interest (COIs) relating to expected position and velocity, track identification, and engagement status Value determined by “GIG User” Conditions of Interest (COIs) relating to expected position and velocity, track identification, and engagement status Computed bit traffic reduction of 45-90% in short duration, highly dynamic air tracking scenarios Computed bit traffic reduction of 45-90% in short duration, highly dynamic air tracking scenarios Demonstrated capability to “tune” bit traffic flows based on user-defined thresholds in accuracy of estimations Demonstrated capability to “tune” bit traffic flows based on user-defined thresholds in accuracy of estimations

# 14 # 14 CEC/VIRT Message Stream Scenarios Air Target Tracking scenario Air Target Tracking scenario  Unclassified message stream provided by JHU/APL  XML-formatted RTTS messages, similar in content to CEP-to- Track-User messages, but not in strict conformance to the Interface Design Document  10-minute duration Target Ring scenario Target Ring scenario  Classified message stream provided by JHU/APL  Formatted text messages conforming to the CEP-to- Track-User Interface Design Document  minute duration

# 15 # 15 Operational Assumptions (both scenarios) Detected aircraft are initially behaving in accordance with expectations Detected aircraft are initially behaving in accordance with expectations Intermediary node (Track User) between the CEC network and the general user (GIG User) shares the same expectations of target behavior as the GIG User Intermediary node (Track User) between the CEC network and the general user (GIG User) shares the same expectations of target behavior as the GIG User Track User node is given knowledge of GIG User COIs Track User node is given knowledge of GIG User COIs Only a significant deviation from expected behavior is considered “valued information” Only a significant deviation from expected behavior is considered “valued information” Under these assumptions, what portion of the message stream is passed to the user node?

# 16 # 16 CEC/VIRT Conditions of Interest (both scenarios) Significant change in expected motion of air tracks based on decision thresholds for the magnitude of change in expected position and expected velocity Significant change in expected motion of air tracks based on decision thresholds for the magnitude of change in expected position and expected velocity Significant change in Track ID information (e.g., from FRIEND to HOSTILE) Significant change in Track ID information (e.g., from FRIEND to HOSTILE) Significant change in Track IFF information (e.g., change in IFF mode responses) Significant change in Track IFF information (e.g., change in IFF mode responses) Start and end of engagements Start and end of engagements Assumption/Operational Decision: No need to send CEC network-specific Cooperating Unit, Time, and Status messages Assumption/Operational Decision: No need to send CEC network-specific Cooperating Unit, Time, and Status messages

# 17 # 17 Air Target Tracking Scenario Results CEP-to- Track User Messages* Number of Messages Number of Bits Without VIRT With VIRT** Without VIRT With VIRT** Valued Bit Ratio Track Data 1, ,483,712189, Track ID 1869,2163, Track IFF , Cooperating Unit , Engagement Status 0000N/A Status0000N/A Time0000N/A Totals2, ,712,160192, * From RTTS XML message stream ** Dead Reckoning thresholds of 100m and 10m/sec Valued bits made up 7% of the total bits transmitted – bit traffic can be reduced by 93% if only valued bits are sent!ru

# 18 # 18 Air Target Tracking Scenario Discussion Results used position and velocity information in all 3 dimensions Results used position and velocity information in all 3 dimensions If initial shared awareness is assumed, only 1 of the 18 Track ID messages needed to be sent (change from ASSUMED FRIEND to UNKNOWN) If initial shared awareness is assumed, only 1 of the 18 Track ID messages needed to be sent (change from ASSUMED FRIEND to UNKNOWN) None of the 552 Cooperating Unit messages needed to be sent (CEC domain only) None of the 552 Cooperating Unit messages needed to be sent (CEC domain only) None of the 121 Track IFF messages needed to be sent (information of significance provided in the Track ID message) None of the 121 Track IFF messages needed to be sent (information of significance provided in the Track ID message) Portion of Track Data messages sent depended on criteria applied (position or velocity) and threshold setting Portion of Track Data messages sent depended on criteria applied (position or velocity) and threshold setting  92% reduction in Track Data bit traffic at 100m position threshold and 10m/sec velocity threshold

# 19 # 19 Target Ring Scenario Results CEP-to- Track User Messages* Number of Messages Number of Bits Without VIRT With VIRT** Without VIRT With VIRT** Valued Bit Ratio Track Data 15,6258,73621,000,00011,741, Track ID ,38454, Track IFF 75526,4001, Cooperating Unit , Engagement Status 62415,8721, Status136026, Time62015, Totals16,7188,85121,356,96011,798, * From RTTS XML message stream** Dead Reckoning thresholds of 100m and 10m/sec Valued bits made up 55% of the total bits transmitted – bit traffic can be reduced by 45% if only valued bits are sent!

# 20 # 20 Target Ring Scenario Discussion Results used position and velocity information in all 3 dimensions (east/west, north/south, and altitude) Results used position and velocity information in all 3 dimensions (east/west, north/south, and altitude) Passed initial Track ID messages and changes in reported attributes Passed initial Track ID messages and changes in reported attributes  Refinements to Track ID COIs may be possible Only sent 5 of 75 Track IFF messages, representing conditions raising suspicion about the track Only sent 5 of 75 Track IFF messages, representing conditions raising suspicion about the track None of the 601 Cooperating Unit messages, 136 Status messages, and 62 Time messages needed to be sent (CEC domain only) None of the 601 Cooperating Unit messages, 136 Status messages, and 62 Time messages needed to be sent (CEC domain only) Portion of Track Data messages sent depended on criteria applied (position or velocity) and threshold setting Portion of Track Data messages sent depended on criteria applied (position or velocity) and threshold setting  44% reduction in Track Data bit traffic at 100m position threshold and 10m/sec velocity threshold

# 21 # 21 CEC Message Stream Analysis Conclusions Can achieve significant reduction in bit traffic from attention to user information needs (COIs) Can achieve significant reduction in bit traffic from attention to user information needs (COIs) Demonstrated mechanisms for enabling CEC message traffic to be filtered for non- CEC users Demonstrated mechanisms for enabling CEC message traffic to be filtered for non- CEC users Opportunity for further research into user- specified COIs and message processing using more of the message content (e.g., certainty and accuracy data) Opportunity for further research into user- specified COIs and message processing using more of the message content (e.g., certainty and accuracy data)

# 22 # 22 Outline Overall Vision: Model-based Communication Networks, VIRT, and Rich Semantic Track Overall Vision: Model-based Communication Networks, VIRT, and Rich Semantic Track CEC/VIRT Project Results and Accomplishments CEC/VIRT Project Results and Accomplishments Rich Semantic Track Definition and Influence on Community Developments Rich Semantic Track Definition and Influence on Community Developments Semantic Modeling and Condition Monitoring: Making Information Delivery More Precise and Reducing Work Load Semantic Modeling and Condition Monitoring: Making Information Delivery More Precise and Reducing Work Load Opportunities for Further Collaboration Opportunities for Further Collaboration

# 23 # 23 Rich Semantic Track Model Need practical ontologies for important domains Need practical ontologies for important domains Tracks are common information elements in numerous military and non-military domains Tracks are common information elements in numerous military and non-military domains  A common “informal” concept in military operations  Now vital in multiple domains: aircraft, ships, vehicles, people, cargo, …  Semantic hub + translator generators => information sharing Provides vocabulary for expressing COIs Provides vocabulary for expressing COIs Provides logical constructs for evaluating COIs Provides logical constructs for evaluating COIs

# 24 # 24 Rich Semantic Track (RST) Rich Semantic Track - conceptual hub for interchange and automated reasoning CEC Track Messages Joint Track Management Data Model CMA Maritime Information Exchange Model Other Track Data Models

# 25 # 25 The Rich Semantic Track Model TrackBeliefs Identity and Characteristics Dynamic State at Time T History of states (past “track”) Predicted states (future “track”) Meta-Information (applicable to each element of belief) EvidenceInferences Error and uncertainty estimates Temporal qualifications Spatial qualifications The top-level conceptual hierarchy for Track. The full hierarchy has more than 125 high level concepts.

# 26 # 26 CEC/VIRT Project Investigated mapping of CEC track data to RST conceptual model Investigated mapping of CEC track data to RST conceptual model Constructed various RST representations for software and web-based implementations Constructed various RST representations for software and web-based implementations  Formal logic representation for automating mappings across track data models  Semantic Web representations for web-based data interchange and machine reasoning Constructed various web-based expressions of COIs Constructed various web-based expressions of COIs

# 27 # 27 Rich Semantic Track: Influences on Community Developments Defines core semantic model for Track information combining Beliefs and Meta- Information on precision and certainty Defines core semantic model for Track information combining Beliefs and Meta- Information on precision and certainty  Basis for vocabulary of VIRT Conditions of Interest  Basis for interchange across multiple track data models  Basis for formal logic for track computation Advising Comprehensive Maritime Awareness (CMA) JCTD for development of the Maritime Information Exchange Model (MIEM) Advising Comprehensive Maritime Awareness (CMA) JCTD for development of the Maritime Information Exchange Model (MIEM) Advised the Joint Track Management (JTM) Enterprise Architecture Working Group on data modeling of beliefs and pedigree Advised the Joint Track Management (JTM) Enterprise Architecture Working Group on data modeling of beliefs and pedigree

# 28 # 28 MIEM Objectives Share actionable maritime intelligence in a net-centric way: Simple/raw data exchange Fusion output representation Advanced analytics support Establish unambiguous maritime lexicon that: Supports communication among data providers and consumers Embodies broad expertise covering the extent of the maritime domain Leverages existing models and knowledge bases Enable communications from/with future services and capabilities Permit extension to more sophisticated data without change to existing systems Import partners’ databases and export to them with efficient translators Dynamic nature of all quantities (potentially any belief/value can vary in time) Inexact information (“roughly 30 feet long”) Relative information (“a mile from the leader of XYZ”) Conditionals (“all facilities open on Thursdays”) Complex queries (“return last five ports of call for vessels flying Chinese of Korean flags and within 3 days of US costal waters”) Pedigree (“position was derived from AIS message A, ELINT data B, and HUMINT source C, using inference rule D”) Belief conditions (“value is considered 85% reliable”, “value was provided by source S”, etc.) Behaviors, states & histories (“the events make up an overall fishing voyage”)

# 29 # 29 Levels of Value Added Information Level Information Type Example Value added 9 (highest) Case files for key entities Histories, highlights, comprehensive details Enables in-depth predictive analysis 8 Threats & anomalies Dangerous undeclared cargo Increased pre- emptive threat reduction 7 “Of interest” conditions & watch lists Suspicious cargo on board Increased analytical efficiency 6 History, behavior & future projections Voyages & predicted courses Enables basic predictive analysis 5 Multiple alternatives & analysis Ambiguity, uncertainty Direct evidence of certainty 4 Degree of belief & pedigree Evidence, quality Direct information about quality 3 Fused data & inferred beliefs Position, crew Synergistic improvement in SA 2 Caveats & simple meta-data Sensor type, classification Indirect quality assessment 1 (lowest) Sensor system reports AIS Reduced development costs for consumers

# 30 # 30 Principal Elements of MIEM Key Domain Entities Key Domain Entities  Conveyance/Vessel  Person/Crew/Passenger  Cargo & Facilities  Measurements: Time, Position, Length, Weight, … Key Secondary Concepts Key Secondary Concepts  Life-cycle: States, Transitions, Voyages, Epochs  Of-interest  Event  Anomalies & Threats Extensive & Universally Applicable Meta-data Extensive & Universally Applicable Meta-data  Source, Confidence, Alternatives, Pedigree, Caveats, …  Past, Present & Future Universal Extensibility & Restriction Universal Extensibility & Restriction  All classes can be augmented or simplified Conceptual model in modular XML schemas Conceptual model in modular XML schemas

# 31 # 31 File & Type Hierarchy of MIEM

# 32 # 32 Basis for Further Research, Development and Experimentation Refine the RST model based on applications in CEC, CMA, and JTM domains Refine the RST model based on applications in CEC, CMA, and JTM domains Continue development of a higher order abstract formalization of RST (formal theory of Track) for extended reasoning capabilities and cross-domain mappings Continue development of a higher order abstract formalization of RST (formal theory of Track) for extended reasoning capabilities and cross-domain mappings Refine expressions of RST concepts using Semantic Web standards to promote data interchange and automated interpretation in the GIG architecture Refine expressions of RST concepts using Semantic Web standards to promote data interchange and automated interpretation in the GIG architecture Continue integration of software implementations of RST into VIRT/CEC simulation framework for test and experimentation Continue integration of software implementations of RST into VIRT/CEC simulation framework for test and experimentation

# 33 # 33 Outline Overall Vision: Model-based Communication Networks, VIRT, and Rich Semantic Track Overall Vision: Model-based Communication Networks, VIRT, and Rich Semantic Track CEC/VIRT Project Results and Accomplishments CEC/VIRT Project Results and Accomplishments Rich Semantic Track Definition and Influence on Community Developments Rich Semantic Track Definition and Influence on Community Developments Semantic Modeling and Condition Monitoring: Making Information Delivery More Precise and Reducing Work Load Semantic Modeling and Condition Monitoring: Making Information Delivery More Precise and Reducing Work Load Opportunities for Further Collaboration Opportunities for Further Collaboration

# 34 # 34 Semantic Modeling and Condition Monitoring Strong semantic representations of Track data improve automated search, information filtering and reasoning Strong semantic representations of Track data improve automated search, information filtering and reasoning Improved computer interpretation and processing of data provides better information products and reduces human processing load Improved computer interpretation and processing of data provides better information products and reduces human processing load Semantics of military orders implies critical conditions of interest – automated derivation of COIs from orders creates valued information flows Semantics of military orders implies critical conditions of interest – automated derivation of COIs from orders creates valued information flows

# 35 # 35 Condition Monitoring is Key Conditions of Interest (COIs) Conditions of Interest (COIs)  Computable expressions (“continuous queries”)  Describe critical assumptions (like CCIRs)  Depend on operator’s evolving context  Usually reflects phase of a mission & current status High-value events are detected High-value events are detected  Data describing the event match the COI  The event is “news”  The COI assures the event is still “relevant”  Bits reporting the event flow with priority Low-value data does not flow Low-value data does not flow  Generally relevant data not matching a COI  Repeated and redundant data, not newsworthy

# 36 # 36 Outline Overall Vision: Model-based Communication Networks, VIRT, and Rich Semantic Track Overall Vision: Model-based Communication Networks, VIRT, and Rich Semantic Track CEC/VIRT Project Results and Accomplishments CEC/VIRT Project Results and Accomplishments Rich Semantic Track Definition and Influence on Community Developments Rich Semantic Track Definition and Influence on Community Developments Semantic Modeling and Condition Monitoring: Making Information Delivery More Precise and Reducing Work Load Semantic Modeling and Condition Monitoring: Making Information Delivery More Precise and Reducing Work Load Opportunities for Further Collaboration Opportunities for Further Collaboration

# 37 # 37 Opportunities for Further Collaboration (1) Recommended next steps for CEC & IWS Recommended next steps for CEC & IWS  Develop and implement VIRT principles inside the CEC network and SIAP (e.g., variable length messages sending only message fields that have significantly changed value)  Develop & deliver VIRT track data using COIs to demonstrate timely sharing of valued info  Spearhead development of rich track model for SIAP  Develop enhanced toolkit for and methods for using a rich semantic track model  Semantic V&V  Interface & service code generators  Track storage & query using XML  COI specification, event computing, and AAR learning

# 38 # 38 Opportunities for Further Collaboration (2) Recommendation for JTM Enterprise Architecture Working Group Recommendation for JTM Enterprise Architecture Working Group  Examine MIEM products of the CMA JCTD  Identify common concepts and model components  Collaborate in developing interchange mechanisms for passing data between the models

# 39 # 39 Opportunities for Further Collaboration (3) Requested Funding for NPS Requested Funding for NPS  Extend formalization of RST for further experimentation  COI expressions  Computing with uncertainty  Intermediary for automated exchange of data across multiple track representations (MIEM, JTM, others)  Research automated derivation of COIs from plans and orders  Investigate Web-based techniques to create smart push of valued information from auto- generated COIs

# 40 # 40 Conclusion New problems & opportunities combine to change our concept of “collaborative communication networks” New problems & opportunities combine to change our concept of “collaborative communication networks” Humans must use computers to reduce information glut Humans must use computers to reduce information glut Collaborators need to employ common models to share model state Collaborators need to employ common models to share model state Operators and domain practitioners will use semantic technologies to create and evolve the needed models Operators and domain practitioners will use semantic technologies to create and evolve the needed models Each consumer’s plans and beliefs determine the value of information Each consumer’s plans and beliefs determine the value of information Computers can implement knowledge of decision- makers and current state to determine the flow of bits Computers can implement knowledge of decision- makers and current state to determine the flow of bits Significant increases in productivity will result Significant increases in productivity will result