International Technology Alliance in Network & Information Sciences Dave Braines, John Ibbotson, Graham White (IBM UK) SPIE Defense Security & Sensing Next Generation Analyst MIPS: A service-based aid for intelligence analysis Research was sponsored by US Army Research Laboratory and the UK Ministry of Defence and was accomplished under Agreement Number W911NF The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the US Army Research Laboratory, the U.S. Government, the UK Ministry of Defense, or the UK Government. The US and UK Governments are authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon.

AgendaAgenda What is MIPS? IT Support for the Intelligence Analysis Process MIPS and the International Technology Alliance (ITA) The MIPS Architecture and an example of its use Meeting the MIPS Objectives

Research transition project led by DSTL (Defence Science Technology Laboratory) UK Applying emerging technologies arising from the US/UK ITA research program Information Fabric, Gaian Database, Controlled English (CE) & CE Store Provide a generic service-based information processing architecture Support the information analyst in their analytic goals Notify new information relevant to their current task –In any form, from any source Automatic processing of future information to extract meaningful task- relevant domain information Management of Information Processing Services What is MIPS?

Collection IT Processing can extract, filter and transform data into information from large volumes of data Cognitive Processing is restricted to IT process configuration due to large volumes of data Processing Preparation of information for interpretation by analysts Interpretation Human activity through hypothesis testing and other cognitive techniques IT processing assists analyst through information management, rules based inferencing and visualisation MIPS is a proof of concept to investigate how analysts can configure and benefit from an improved Collect/Process/Interpret processing pipeline On the ground Increased volume and variety of information sources Cannot manually inspect, process & interpret IT is not appropriate for all phases in intelligence analysis Processing to support Intelligence Analysis IT supports extraction, storage, indexing IT supports retrieval, visualisation, presentation & communication Human configuration, algorithms, modeling Human intel analysis, hypotheses IT Support for the Intelligence Analysis Process

MIPS Builds on work from the US/UK International Technology Alliance ITA research is a US/UK collaboration between industry, academia and government Focused on: Network science, Decision-making & Coalition operations From May 2006 – May 2016 Research is fundamental (6.1), low Technical Readiness Level (TRL) Intended outputs are papers, proof of concepts Higher TRL transition contracts take the core ideas to progress further Emerging technologies arising from ITA research used by MIPS: ITA Controlled English Consumable by humans and machines ITA Information Fabric A lightweight service bus middleware Gaian Database A Dynamic, Distributed, Federated Database

What is it? Human-friendly language but machine readable Rich semantics, broad application Reasoning engine with rationale Supports agility for dynamic, evolving situations Enables hybrid human/machine collaboration Empowers non-technical users Examples: the person Dave works for the company IBM. if (the person P works for the company C) then (the company C employs the person P). the person pn0123 is a suspect in the crime c713 because the person pn0123 was sighted in… conceptualise an ~ intelligence report ~ I that ~ mentions ~ the suspect S. Why does it matter? Real fusion of machine precision and human cognition Can harness collective intelligence Facilitate human-human communication & socialisation Download the CE Store: Using human language to enable smarter human/machine processing of information ITA Controlled English

A two-way message bus and set of middleware services Connects all network assets to each other and to users Provides universal access to intelligence data, processing services and applications Implemented using a multi-hop publish/subscribe architecture Messages are efficiently propagated without duplication thereby minimising bandwidth utilisation Policies constrain how assets can be used and configured and information shared across coalition boundaries Same policy mechanism as is used in the Gaian Database Information fabric can integrate assets and services In MIPS, the Information Fabric is used to integrate information processing services A lightweight service-bus middleware designed for the edge of the network ITA Information Fabric Download the fabric:

node Gaian Database DDFDs provide distributed access to distributed heterogeneous data sources, are highly scalable & impose low management overhead Obtaining information from multiple heterogeneous data sources does not scale & requires significant management overhead DDFD is based on store locally & query from anywhere principle A dynamic distributed federated database (DDFD) The Gaian Database Download Gaian:

Objective 1: Notify analysts when new, relevant information has arrived Defining Relevant Information Maintaining and sharing analytical goals Indexing information & notifying analysts of matches with their analytical goals Operation in a mixed environment Objective 2: Automatic processing of new information Identifying processing services and data sets Standardised description of processing services to enable reasoning Visualising current capabilities for the user Efficient operation Abstract services Automatic information processing in a mixed environment Facilitating collaboration between analysts MIPS Objectives & Challenges

Composition Analyst describes a set of services and links them to form a network (to achieve a goal) Network can be deployed as a set of linked and running services Information Fabric Provides nodes for services to run on and message bus for service interconnection Services perform specialised information processing Sources & Sinks Controlled English All data produced by MIPS services is converted to CE and optionally stored in the CE Store Data is stored and queried using CE Some services also implemented in CE Analyst Tools A set of tools for users to operate within the MIPS environment MIPS Architecture

Integrates multiple SharePoint repositories Document Extractor service extracts metadata facts in CE about the documents Report Analytics text mining service extracts entities and relationships from document text Implemented as a set of CE-based services Watch List Notifier looks at extracted CE about people and queries watch lists to see if a person is present on the list Multiple types of watch list are demonstrated Notification Sink manages notifications that can be accessed by other services and to alert users Information generated is stored in CE store Example: A typical information flow

Challenge 1: Defining relevant information Analysts indicate relevant information by defining service compositions, or by querying the CE Store. Compositions and queries may be saved, shared and reused Challenge 2: Maintaining and sharing analytical goals Analysts define goals using the Fabric Service Composition Tool. Goals so defined may be reused and shared amongst teams of analysts Goals may also be encapsulated in queries (or rules) in the CE Store which may be saved, shared and reused Challenge 3: Indexing information and notifying analysts of matches with their analytical goals Indexing of information may be via a MIPS service or an external application e.g. traditional indexing within a relational database Example provided of local and external watch list inspection, notifying people matched, with a link to the original document text about the person of interest Challenge 4: Operation in a mixed environment The MIPS demonstrator is capable of interacting with and integrating different data sources and using a wide range of deployed processing services Meeting the MIPS Objectives Objective 1: Notify Analysts when new, relevant information has arrived

Challenge 1: Identifying processing services and data sets The MIPS Fabric Service Composition Tool (FSCT) allows users to identify services available to them and the data sets required and produced by the services Challenge 2: Standardised description of processing services to enable reasoning MIPS standardises the descriptions of the services available to a user together with the information exchange formats used between services Challenge 3: Visualising current capabilities for the user The MIPS processing pipeline is constructed and visualised using a graphical editor; the FSCT Complex queries to the CE Store can be constructed and visualised using a CE Query Builder; a graphical tool for creating complex queries across linked information Challenge 4: Efficient operation MIPS uses the ITA Information Fabric as its underlying messaging middleware. The fabrics publish/subscribe model provides efficient routing of messages between services Challenge 5: Abstract services Abstract services are a generalisation of a group or type of service Abstract service descriptions may be defined in MIPS before the service has been created Challenge 6: Automatic information processing in a mixed environment MIPS is capable of handling information and services from both within and external to its environment. Challenge 7: Facilitating collaboration between analysts Some rudimentary support is provided in MIPS for collaboration amongst groups of analysts. Further support is expected in future work Meeting the MIPS Objectives Objective 2: Automatic processing of new information

or Research was sponsored by US Army Research Laboratory and the UK Ministry of Defence and was accomplished under Agreement Number W911NF The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the US Army Research Laboratory, the U.S. Government, the UK Ministry of Defense, or the UK Government. The US and UK Governments are authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon. Main links: IBM developerWorks downloads: Information fabric: Gaian database: CE Store: International Technology Alliance