Determining the Consistency of Information Between Multiple Subsystems used in Maritime Domain Awareness Marie-Odette St-Hilaire a, Anthony Isenor b a OODA Technologies b Defence Research and Development Canada

Objective Present the concept of information consistency using maritime domain awareness (MDA) pertinent information collected from multiple subsystems and propose a methodology to quantify the consistency.

Outlines MDA, information quality and trust Information consistency Prototype to assess consistency of information between multiple sources Cross comparison of information from multiple data sources Difficulties in comparing ship related information Consistency visualization

Maritime Domain Awareness MDA is the effective understanding of anything associated with the maritime domain that could impact the security, safety, economy, or environment.

MDA and Information Heart of MDA: Quality information gained from a combination of sources. Examples of Sources: maritime, land, air and space surveillance systems other government departments and the commercial sector public web sites

Quality and Trust Information accessibility brings issues: Information overload Lack of processing capabilities Trust Issues …  Quality of Information impacts trust

Consistency as a data quality attribute A piece of information is consistent if it does not conflict with other information. Source A IMO: Name: Albatros Flag: Canada Source B IMO: Name: Albatros Flag: Canada Source C IMO: Name: Albatros Flag: Spain Inconsistent information

Assumption Consistency in information helps build the trust we place in the information.  To develop trust in information, the similar data items from the various sources should be compared.

Compare-MDA Project Architecture Multiple information sources Consistency evaluation Consistency visualization

High Level Description A Service-Oriented Architecture (SOA) was developed to compare the information from diverse sources. It allows one to: Assess the consistency of information related to MDA contained in DRDC databases and from web sites Compare information from disparate sources Quantify a source consistency Visualize consistency results within a Google Earth environment.

Compare-MDA Framework Source_A Source_A WS Source A Source_B Source_B WS Source B Source_C Source_C WS Source C Source_D Source_D WS Source D DRDC Applications CA CA WS CA Client Web sites

Data Sources Source A and B: DRDC databases Source C: ITU web site Source D: ShipSpotting web site All exposed as web services in the framework Source_A Source_A WS Source A Source_B Source_B WS Source B Source_C Source_C WS Source C Source_D Source_D WS Source D DRDC Applications Web sites

DRDC Data Sources 2 MDA-related databases: Both contain static and positional information One contains ship photographs Both exposed as web services in the framework

ITU Web Site

ShipSpotting Web Site

Consistency Application Functionalities and information flow Consistency score Comparing similar items Consistency statistics Difficulties in assessing consistency CA CA WS

CA Functionalities Communicate with the data sources Send queries Process responses Align diverse source vocabularies Identify unique ship objects within different source responses Compare ship attributes among sources Persist comparison results and ship attributes Consistency Application CA Data Sources

Information Flow Consistency Check Compare items within a comparison group Vessel Matcher Create comparison groups (unique ship entities). Ship Descriptions CA Manager Build queries for DS services CA Queries Source A Source D … Vocabulary Solution (Alignment) Comparison groups CA

Consistency Check Compare items within a comparison group Consistency Check Identify consistent and inconsistent data among the sources: Compare a ship’s item, among sources. Provide statistics on the inconsistencies found at the various sources. These statistics reflect a general assessment of the source based on the number of inconsistencies found at the source. Persist comparison results and ship attributes. Comparison Group CA

Score quantifying a source consistency (1) The consistency check product is a score called Match(%) Used to quantify the ability of a source to provide consistent information or ability of a source to provide information that can be confirmed by other sources Match(Source, Item, Ship) = Unequal (0), Partial (0.5) or Exact (1).

Score quantifying a source consistency (2) ItemSource A (DRDC) Source B (DRDC) ITUShipSpotting Ship Name MMSI number CallSign IMO number Ship Flag Ship Type Match(Source i, Item j, Ship k ) = Propensity of Source i to provide information (Item j for Ship k ) that can be confirmed by other sources.

Ship Items Comparison (1) In order to compute consistency score (match), we need to compare ship items among sources. Source A MMSI: Name: Espoir Type: Oil Products Tanker Source B MMSI: Name: Espoire Type: Merchant Oil Tanker Source D MMSI: Name: Espoir Type: Tanker Source C MMSI: Name: Espira Type: Oil Tanker

3 Types of Comparison Hard Comparison: exact comparison (==) usually for numeric items Soft Comparison: based on string similarity (Levenshtein distance) among the different expressions to compare for some items typos are frequent (e.g. ship name) Pattern Comparison: based on the recurrence of words among the different expressions to be compared for ship type or other complex self-describing item.

Source Consistency Statistics A match is computed for each source, item and ship: Match(source, item, ship)  3 dimensions: source, item, ship Source consistency is assessed by averaging over all items and ships. Ship XXSource A Source B Source C Source D Name 1111 MMSI111 CallSign00.67 … Ship XYSource A Source B Source C Source D Name 0.67 MMSI0.670 CallSign111 …

Value for one Ship of one Item of the Source Ship Level Item Level (averaged over all Ships) Source Level (averaged over all Ships and Items) Source A 0.77 Name 0.86 Ship Ship Ship Ship N 1.0 Flag N/A MMSI 0.55 Ship 1 Ship i Ship N IMIM S1S1 S2S2 S3S3 I1I1 IiIi

Statistics for one Item of the Source Ship 1 Ship i Ship N IMIM S1S1 S2S2 S3S3 I1I1 IiIi Ship Level Item Level (averaged over all Ships) Source Level (averaged over all Ships and Items) Source A 0.77 Name 0.86 Ship Ship Ship Ship N 1.0 Flag N/A MMSI 0.55

Overall Statistics for a Source IMIM IiIi I1I1 S1S1 S2S2 S3S3 Ship 1 Ship i Ship N Ship Level Item Level (averaged over all Ships) Source Level (averaged over all Ships and Items) Source A 0.77 Name 0.86 Ship Ship Ship Ship N 1.0 Flag N/A MMSI 0.55

Difficulties in assessing information consistency Complex… and we just compared strings. Comparison is static no consistency tracking (does a source consistency evolves in time?) no comparison of dynamic information such as destination, ETA, cargo...

Consistency Visualization Google Earth information display Traffic-light color code Ship photographs Consistency Statistics (sources, ships and items decomposition) CA CA WS CA Client

Query and GE Display

Traffic Light Visualization Green: Consistency ≥ 80% Yellow: 20%<Consistency<80% Red: Consistency≤20% Gray: No consistency assessed because only one source provides information for that ship’s item.

Concluding Remarks There are many information quality attributes: uncertainty, reliability, relevance, utility, expectability... The problem comes to quantify (build metrics to assess) these quality attributes.  In this project, we showed that consistency can be concretely quantified without any a priori suppositions.  The computed consistency can then be used to identify some sources as providing more reliable information as compared to other sources for post processing (e.g. information fusion)