Accelerating the data-sharing process Copyright © 2007, Sypherlink, Inc. All Rights Reserved. Rehan Chawdry Integrated Justice Practice Leader NIEM In Practice

Launched on Feb 28, 2005 Partnership between U.S. DOJ and U.S. DHS NIEM v1.0 released 11/11/06 NIEM v2.0 released 7/31/07 Framework for a common XML data vocabulary Provides standard naming conventions, type definitions Standardizes field/element relationships Describes cardinality, domain space of fields/elements Evolved from GJXDM from justice world Uses best practices from other standards (ebXML, etc) Technical features Set of XSDs distributed with documentation XML Elements built around a common core Namespaces allow domains to evolve independently of each other, but still build on common elements Usage is growing rapidly Future DOJ State/Local funding tied to NIEM compliancy Florida, Texas, other state-wide data sharing efforts FBI N-DEx program DoD UCore support (Aug 7, 2008) Governance NIEM Program Management Office Comprised of government and industry stakeholders NIEM Overview

NIEM Usage (1 of 2) What NIEM is and is not NIEM IS not a database model NIEM does not define a set of web services NIEM IS an exchange standard NIEM is intended to be extended before it is used Intended to be a template that defines common data elements Initiatives should only pick only those elements they need to use If data elements do not exist, users are encouraged to extend using NIEM naming and design rules (NDR) There is a standard process to pick/extend NIEM for your use (IEPD process) NIEM needs to be coupled with other standards to be effective SOAP and Web Services for SOA functionality

NIEM Usage (2 of 2) Using NIEM typically involves three(3) steps 1.Requirements gathering 2.Design and documentation 3.Implementation Information Exchange Package Documentation (IEPD) Standard process that guides how to do #1 and #2 above Most current NIEM training is about IEPD development Implementation (#3) is not defined by NIEM and is rarely covered Why do this? The NIEM process is essentially used to define an API Insures that an API uses common nomenclature and structure

Example NIEM instance Bart Jojo Simpson

Post 9-11 accelerated need to share data between state, local, and federal law enforcement agencies Niche industry (over 300+ vendors) provide operational software to law enforcement agencies all over the country. CAD, RMS, JMS, Court Each vendor has a unique system with their own platforms, schemas, features, and naming standards (i.e, subject vs offender vs suspect) Agencies with the same vendor could usually data share. Agencies with different vendors often could not Jurisdictional boundaries made data sharing difficult (State cant directly connect into local police system and pull data out) Most CAD, RMS, JMS, Court systems focus on operational activities -> not great analytic capabilities, especially dealing with large volumes of data Different vendors emerged to provide analytic software and warehouses to consolidate data from different law enforcement systems (LInX, CopLink, etc) Data standards were still evolving (GJXDM), but not supported by the majority of industry initially. Public Safety Domain Background

Florida Data Sharing Initiatives Background Florida created seven (7) domestic security regions within the State to coordinate homeland security functions Each of the regions consisted of between agencies Each region independently pursued data sharing initiatives with its member agencies Four (4) regions each selected different analytic vendors. Regional implementation involved getting data out of each operational agency systems and into selected analytical system (some warehouse, some federated) Three (3) regions were looking to start data sharing initiatives Florida Department of Law Enforcement wanted to align all regions and have strategic approach to data sharing

RLEX (3 regions) RLEX (3 regions) Florida Department of Law Enforcement Data Sharing Initiatives Tallahassee Region Ft. Myers Region Miami Region Pensacola Region Jacksonville Region Tampa Region Orlando Region SmartCop LInX FINDER CopLink Florida Law Enforcement Exchange (FLEX) would link all seven Regional Project nodes together PROBLEM Each region also had their own data sharing solution ~30 agencies ~30 agencies ~90 agencies Regional Law Enforcement Exchange (RLEX) needed to integrate data from 3 regions and over 150 law enforcement agencies GOALS 1.Provide data sharing solution for three (3) regions 2.Provide data sharing solution for entire State

Requirements Provide practical framework for data sharing Neutral, vendor-agnostic approach Leverage NIEM and other national standards when possible Support warehouse (RLEX) and federated (FLEX) approaches Challenges Where and how to leverage NIEM? At an RLEX level, multiple operational systems At a FLEX level, multiple analytical systems For both projects, multiple, separate law enforcement agencies NIEM in Florida

Where to use NIEM Develop canonical data model as basis for data sharing Use NIEM to develop canonical relational database model Determine how data from all regional systems map to NIEM For RLEX, standardize on way to get data to State Publish an API that can be used to accept data into regional warehouse Work with each agency to convert their data on an ongoing basis Warehouse data from agencies in RLEX region For FLEX, standardize on way to query data between regional systems Work with analytic vendors to have them implement a common query API Each regional system continues to own their data Federated query allows access to all regional systems

Standard Canonical Model Florida NIEM Data Model Florida NIEM Data Model Common relational structure for warehoused data Data fields identified by analyzing other analytic systems in FL -> shareable fields Used NIEM naming and design rules Provided target data model that other systems could map to

Sample Tables

Next Task – Standard submission API Analytic vendor chosen by RLEX used a warehouse model Had their own database schema and submission API All agencies would need to implement it -> Once implemented, agency is locked to vendor Agency implementation is most time consuming in data sharing projects

Vendor Application Current Warehouse Models Agency 1 Law Enforcement Database Application Database Agency 2 Law Enforcement Database Agency 3 Law Enforcement Database Vendor specific Florida wanted to avoid lock-in Insure data integration could be leveraged with different vendors down the road

Enter LEXS Logical Entity eXchange Specification (LEXS) General purpose sharing API based on NIEM first developed in 2005 Developed by DOJ initially to support data sharing between its federal agencies (FBI, DEA, ATF, BOP) What problem does it address that no other current NIEM-based standard addresses? Defines a common set of NIEM elements all systems should be able to understand (who, what, where, and how things are related) Allows extensions to be written in such a way that allows an interpreting system to ignore parts it doesnt understand A single IEPD as opposed to several IEPDs that cover different areas

Enter LEXS LEXS has two parts Publish/Describe (LEXS-PD) Framework for laying out NIEM-based data Describes actions that systems should take with data (insert, update, delete) Search/Retrieval (LEXS-SR) Describes search API based on NIEM Allows wildcard searches, entity-specific searches Describes search results format Several LEXS-based APIs have been developed FBIs NDEX initiative DOJs Suspicious Activity Report (SAR) Anyone that uses a LEXS framework will be able to interoperate with other LEXS-based systems without changes Law enforcement system that describes an Arrest report can be sent and interpreted by DoD even if it doesnt understand what an Arrest report is

Understanding NIEM / LEXS NIEM LEXS SR PD SAR IEPD #2 IEPD #3 IEPD #1 breadth of applicability functionality NIEM:data definitions, element librarydesign-time compatibility LEXS:specific structure, constrained entitiesrun-time interoperability LEXS-SR:search & retrievalfederated query capability LEXS-PD:publication & discoverydata replication capability N-DEx FLEX

LEXS and Florida LEXS-PD implemented as standard API for agencies to push data to RLEX Work with vendors to get data into LEXS format Training existing law enforcement vendors on LEXS Help produce LEXS from their software using internal development team or using data integration vendors (Sypherlink) Deploy software at law enforcement agencies to provide continual data feeds Solved tactical needs but also allowed agencies to position themselves strategically Provide data sharing in ways that they couldnt before Support for emerging national systems (NDEX) without having to re-code

Vendor Application Current Warehouse Models Agency 1 Law Enforcement Database Application Database Agency 2 Law Enforcement Database Agency 3 Law Enforcement Database Vendor specific Florida wanted to avoid lock-in Insure data integration could be leveraged with different vendors down the road

Agency 2 Agency 1 Vendor Application Best Practice - Warehouse Models Application Database LEXS-PD Vendor specific LEXS-PD Law Enforcement Database NIEM Appliance Law Enforcement Database NIEM Appliance NIEM Appliance Agency 3 LEXS-PD Law Enforcement Database NIEM Appliance

Agency 2 Agency 1 Vendor Application Best Practice - Warehouse Models Application Database LEXS-PD Vendor specific LEXS-PD Agency 3 Law Enforcement Database NIEM Appliance Law Enforcement Database NIEM Appliance Law Enforcement Database NIEM Appliance NIEM Appliance Vendor Application 2 Application Database Vendor 2 specific LEXS-PD Support for multiple, simultaneous analytic vendors from same agency feed

Next Task – Standard query API RLEX (3 regions) RLEX (3 regions) Tallahassee Region Ft. Myers Region Miami Region Pensacola Region Jacksonville Region Tampa Region Orlando Region SmartCop LInX FINDER CopLink Florida Law Enforcement Exchange (FLEX) would link all seven Regional Project nodes together Each regional system had its own platform and schema Florida wanted to allow distributed approach between regions (i.e. federated system)

Region 4 Classic Federated Model Region 1 Law Enforcement Database Region 2 Law Enforcement Database Region 3 Law Enforcement Database Region specific query Each region would have to understand the search APIs of the other regions in order to perform a federated query across all regions

LEXS and Florida LEXS-SR implemented as standard API for agencies to search data between regions Each region consumes a LEXS-SR message and figures out how to map it into a local query within the regional system. Results are formatted using LEXS-SR and sent back to external originator Several vendors were already implementing LEXS-SR for other initiatives. Standardizing on LEXS-SR at a State level allows external entities to query the State at some point (i.e. NDEX and Florida)

NIEM Appliance Vendor Application Best Practice - Federated Models Agency 1 Law Enforcement Database NIEM View LEXS-SR Agency 2 Law Enforcement Database NIE M View LEXS-SR Agency 3 Law Enforcement Database NIE M View LEXS-SR Application Query FBI NDEX LEXS-SR

Conclusion NIEM can be used effectively in real-world applications, as long as it is augmented with other related standards LEXS Justice Reference Architecture (JRA) – if dealing in this domain Other examples of NIEM use NCIC project Workflow exchanges Many other Federal, State, local law enforcement sharing systems NIEM branching out into other areas April 2008 – Support for DoD UCore standard Commercial organization exposing their data as NIEM NIEM Project Management Office is soliciting other federal agencies to help expand NIEM into other functional areas.

Review Q&A Rehan Chawdry Integrated Justice Practice Leader Sypherlink, Inc