International Technology Alliance in Network & Information Sciences Dave Braines, David Mott, Simon Laws (IBM UK) Geeth de Mel, Tien Pham (ARL) SPIE Defense Security & Sensing Next Generation Analyst Controlled English to facilitate human/machine analytical processing Research was sponsored by US Army Research Laboratory and the UK Ministry of Defence and was accomplished under Agreement Number W911NF The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the US Army Research Laboratory, the U.S. Government, the UK Ministry of Defense, or the UK Government. The US and UK Governments are authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon.

AgendaAgenda Controlled English & the CE Store Motivations & military relevance Field Trial example Other CE research areas 2

What is Controlled English? 3 A number of research collaborations, demonstrations and transition projects are experimenting with CE related capabilities both inside and outside the ITA programme In our research activities CE has been the single core information representation format for humans and machines. No translation to underlying technical formats. The Controlled English ecosystem: More than just a language representation Agent-based execution environment Multi-modal visualisation & interaction Focus on agility & flexibility A Controlled Natural Language: Based on English, but formal and limited Directly processable by machine agents Can be used at design and/or run time Unifying information format: Model, facts, rules, queries, annotations & commands Rationale, hypotheses, assumptions & meta-data To support human/machine, machine/machine and human/human interactions Fuse formal logic with social semantics

Natural Language Why are we researching this? 4ITA Peer Review, Sept The potential value of a usable semantic processing environment in the hands of non- technical, domain-specialist users is very high, especially in dynamic situations. Bringing capabilities of machines and humans together Human brain for insight & understanding The key component is the human Harnessing collective intelligence Help make connections outside the system Java Controlled Natural Language XML Logic Prolog We would like thinking and processing to be as close as possible We need a language that is both thinkable and processable. Processing Articulation as Language Photographer: Sebastian Kaulitzki | Agency: Dreamstime.com

The CE Store (aka the IBM Controlled Natural Language Processing Environment) 5 Implemented in Java, default client runs in TomCat Core CE parsing engine: Model, fact, query, rule, command, annotation, reified sentences Rationale integration, extensible meta-model Default web-based user interface and client Customer in-memory Java based object persistence Database persistence being considered for future version Customisable alerts and triggers APIs: HTTP APIs for most common activities (JSON response) Internal Java programming APIs and extensible Agent development environment Visual query building canvas (CE Query Builder or CEQB) Available for download from: (IBM developerWorks)

Motivations and Military Relevance 6 Targeting coalition operations Dynamic and ad-hoc, but shared goals Opportunity to share information and insight Multi-agent collaborative environment For human and machine agents How can technology help? Rationale for explanation Facilitate different socio-cultural backgrounds e.g. moderate/observe information flow (to help misunderstanding) Improve operational tempo …to get inside the opponent decision cycle, or just be more effective Reduce communication overhead Dont send information that can be re-inferred at the destination The DCDP loop

Field Trial (LOSA) 7 Late 2012, UK Evaluate sensors, systems, architecture Heterogeneous environment Based around FOB activities Situation awareness & agility Hard & soft fusion This demo is driven ONLY by Controlled English + screenshots are generic – driven by CE All material is unclassified

Pre-deployment activities 8 Geo-spatial preparation ahead of deployment: Simple multi-modal integration there is a building named b1 that has as latitude and has as longitude. Dynamic model development conceptualise a ~ building ~ B that is a spatial thing. conceptualise a ~ ground feature ~ G that is a spatial thing. Use of meta-model the renderable concept 'building has '/icons/building.png' as icon file name.

Crowd-sourced Intelligence Gathering 9 Imagery Chat Custom PhotoProcessor Agent there is a photo named 'IMG_0486.JPG' that has '/photos/IMG_0486.JPG' as image url and has '/photos/thumbnail.JPG' as thumbnail url and has ' ' as latitude and has ' ' as longitude and has '52' as elevation and has '222.97' as bearing angle. Standard JSON conversion there is a field intelligence incident named 'inc_540' that has 'Droid810G' as device model and has ' ' as source id and has ' ' as latitude and has ' ' as longitude and has '48' as elevation and has 'Local trader passing through with goods for market. Inspected and passed on' as body and has 'Alpha,US' as owner and has 'normal' as priority and has ' :00:00.0' as timestamp and has 'INTEL' as type. Need highly agile and responsive system Dynamic / contextual filtering Capture new insights

Anecdote: Linking information 10 Observer: Look, that photo is of that building New idea: Photos can be of things conceptualise the photo P ~ is of ~ the thing T. Now we can state: the photo IMG_0486.JPG is of the building b1. Observer: Why cant I see the building linked to the photo? Need to infer the inverse: conceptualise the thing T has the photo P as ~ imagery ~. Write a rule to infer: [thing has photo] if (the photo P is of the thing T) then (the thing T has the photo P as imagery). the building b1 has the photo IMG_0486.JPG as imagery because the photo IMG_0486.JPG is of the building b1 [thing has photo]. Inference with rationale

Soft sources: Information extraction 11 Other ITA research is looking at Information Extraction from natural language Not covered in detail in this paper Example shows detection of attributes for fours small jeeps or cars Related to vehicles A sized group with cardinal size of 4 Used in trial for: IED mentions Statements of certainty Geo-location Associated imagery

Hard sources: Agile integration 12 Conversion of multiple on the wire formats: CSV, XML, JSON Used Information Fabric and external commercial / government sources Human led integration / mapping exercise with machine led processing of events Crude form of Situation Awareness No integration with wider models / scenarios No reasoning or inference …but lots of possibilities if time allows Good feedback from visitors and other trial participants Possibility for visit in 2013

Other CE-related research areas On-going ITA research using Controlled English: Policy A candidate light-weight, distributable, readable & executable policy language Trust & meta-data CE reification to assign subjective logic trust values and propagate using inference and rationale Collaborative coalition planning Rich semantic model for collaborative coalition planning, specialising from general concepts of space and time, through problem solving to military planning External interest: NATO, NS-CTA, TerraHarvest Other SPIE papers: Context-rich semantic framework for effective D2D in coalition networks (8742-2, Monday) MIPS: A Service-Based Aid for Intelligence Analysis ( , in this session) CE-SAM: a conversational interface for ISR mission support (8758-6, 17:20 today)

or Research was sponsored by US Army Research Laboratory and the UK Ministry of Defence and was accomplished under Agreement Number W911NF The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the US Army Research Laboratory, the U.S. Government, the UK Ministry of Defense, or the UK Government. The US and UK Governments are authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon. Main links: CE Store (from IBM developerWorks) International Technology Alliance Controlled English resources