1. GJXDM User Conference Atlanta 8-10 June 2005 NLETS Lessons Learned

2. Kate Silhol NLETS Senior Software Engineer ksilhol@nlets.org (602) 224-0744 www.nlets.org

3. NLETS is a sophisticated high- speed message switching system created for and dedicated to the criminal justice community. What is NLETS?

4. Non-profit corporation chartered by the States – funded by user fees Members are all 50 states, most federal agencies & territories, and the RCMP Nearly 40 years in operation Links 30,000 agencies and over 500,000 access devices in the U.S. and Canada Topping 40 million transmissions per month NLETS is:

5. Facts and Figures 41 Million transactions per month! Only way to:  State Criminal Histories  Electronic “HIT” confirmation  Canadian Files (CPIC)  Homeland Alert Messages  Screen Officers Flying Armed  INS databases at LESC – IAQs  Driver records and vehicle registrations  Aircraft Registrations  HAZMAT, GSA Federal registrations, NICB  Push administrative messages to all CJ agencies

6. Services: Private T1 Network, NO Internet VPN – Triple DES encryption Fire-walled at every connection Hot Disaster Recovery Site - Idaho Store and Forward – Guaranteed Message Delivery Operation center 7 x 24 x 365 - Arizona 99.6% uptime

7. Requirements Represent precise data Reliability Performance Flexibility Maintainability

8. NLETS XML Operational Capabilities XML Message Router (XMR) since 2001 All NLETS transactions available in GJXDM 3.0 format plus full legacy system translation capabilities since September 750,000 XML rap sheets alone per month  All FBI, WI and KY rapsheets are in XML All NLETS states/agencies have access to XML and web services

9. XML Formats and Transport NLETS allows users to send XML via Web Services, MQ Series or TCP/IP  Allowing XML over various protocols is crucial to users in our environment NLETS transforms between XML and legacy text to allow interoperability between users All new data providers will be connecting via Web Services

10. Validates user Transforms XML to Text XML Message Text Message XMR Determines Destination & Format Validates XML XML Message Text Message Text/CDATA XML Message

11. Web Services Within a secured network, NLETS exposes a web service to which users connect and send XML as a string States wishing to receive a response via web services host own service for NLETS to connect to  NLETS specifies name and fingerprint of service, how data is handled is up to state

12. NLETS Standardized GJXDM XML Transactions All Inquiries  Standard text format is easy to convert to/from XML Criminal History Responses  Leveraging the JTF Criminal History Rapsheet Specification Hazardous Materials Responses  Standardized single-source responses from Operation Respond Drivers and Registration Responses  Standardized responses from state DMVs based on CANDLE grant products

13. So, how did we do it? Mapped elements to GJXDM Documented XML Instances Created applications to consume XML Created schemas Get users to start implementing Change management

14. Mapping NLETS Transactions First, created a generic message envelope, in NLETS namespace (prefix n) Header information remains in the payload instead of SOAP  Multiple transports  Legacy header data ….. NLETS header elements …… ….. NLETS transaction data ……

15. Mapping NLETS Transactions Next, within the generic NLETS message envelope we defined each transaction in XML  Going from a legacy system provided us a strictly defined set of information Used GJXDM elements where available Extended GJXDM elements and types when possible  Example: Agency Participated in GJXDM development by utilizing feedback mechanisms (Bugzilla, XSTF)

16. NLETS GJXDM Example RQ GA0250300 AZ TERM000000 LJB934 LJB934 PC PC 1983 1983 RQ.GA0250300.AZ.*TERM000000.TXTLIC/LJB934.LIY/1983.LIT/PC

17. NLETS GJXDM Header RQ GA0250300 AZ TERM000000 RQ.GA0250300.AZ.*TERM000000.TXT

18. NLETS GJXDM Data LJB934 LJB934 PC PC 1983 1983 </n:NLETSInquiryData> LIC/LJB934.LIY/1983.LIT/PC

19. Supporting legacy text values  Legacy text may have supported nonconforming data values (example: Expiration Date) Newly standardized transactions  Cannot find structure in older, non-standardized formats  Must support two versions of XML “3.0” Multiple ways to represent same data GJXDM not meant to support certain types of data  Messaging/Header information GJXDM Mapping Challenges

20. Schemas Each message type has its own schema “package”  Combined Document/Extension schema  Defines overall structure of message – referencing GJXDM elements and defining NLETS-specific elements  Global JXDM constraint schema  Defines GJXDM elements being used and specifies constraints (min/max occurrences) Shared supporting schemas  NCIC, NIBRS code enumerations, etc

21. Schemas <n:NLETS>…</n:NLETS> XML Instance DQ.xsd Document/Extension Schema Constraint Schema JXDM.xsd Supporting Schemas xsd.xsd ncic_2000.xsd Validates against Imports Imports

22. Schemas NLETS does not currently perform schema validation on each transaction  Table-based validation of data  Validate users’ XML in a test environment first, then trust them Schemas are provided primarily as documentation for programmers Recommend use during development and testing Schemas cannot communicate all rules for XML conformance

23. Documentation of XML Specifications XML is defined in detail in Appendix A of the NLETS User Guide For each transaction, schemas, XML instances, stylesheets and business rules are provided

24. NLETS processes, switches data but does not create, store or manipulate data Must parse text and XML in order to route, validate and convert  XPath for XML, regular expressions for text Consuming XML

25. Standardized Rap Sheets Currently Wisconsin, Kentucky, Maine and FBI sending and/or receiving rapsheets in XML – JTF 2.2 format New states will be sending JTF 3.0 XML rapsheets (compliant with GJXDM 3.0) All users receiving standard presentation format – most in legacy formats All III Inquires done - go through and are delivered by – NLETS

26. Mapping the Rapsheet Similar to the process followed in mapping NLETS transactions While we built from an existing specification, we had more flexibility  Benefited from GJXDM data dictionary  Data we had not previously thought to include  Provides more clear consistency throughout rapsheet Found a greater need to extend elements to restrict allowable values

27. CANDLE National Institute of Justice Bureau of Justice Assistance NLETS (Awardee) American Association of Motor Vehicle Administrators Advanced Technology Systems Pilot States BJA Collaboration between AAMVA and NLETS for Driver License Exchange

28. CANDLE Standardized GJXDM XML representation of DMV drivers license and registration responses Unlike NLETS Inquiries and Rapsheets, CANDLE specifications were built from scratch – not existing spec Cannot convert text driver and registration responses to XML

29. Operation Respond HAZMAT Information - live Trucking manifests and rollover data Rail Car Data Passenger Train Schematics Aerial photography via NLETS – a possibility

30. Operation Respond Not a criminal justice agency, though information is pertinent to CJ agencies GJXDM did not contain hazardous material elements Created elements in Operation Respond’s namespace, utilizing same naming conventions and GJXDM elements when available

31. Change Management Even small changes affect entire user base User concern about a “moving target” Only plan changes when necessary – or optionally adding value Plan to give states advance notice to the extent possible Work with states to minimize effects

32. User Implementation Allowed users to start slow  Specify format preference by message key for each ORI  XML can be sent via multiple protocols; not limited to web services Provide support to users Stylesheets, transformations were a necessity because users are and will continue to be at different stages As new services are offered, they may only be available via XML and web services

33. Typos, other minor mistakes  Do you fix and risk disrupting users or leave incorrect? Namespaces  Has become considerably more complicated than with initial specifications  New namespaces (example AA)  Multiple GJXDM versions/namespaces Inheritance – differing requirements for same element Support of multiple transports  Difficult to utilize SOAP header, WS-Attachments, etc Hindsight…  Multiple ways to do everything – each with own pros, cons Problems

34. Benefits Allows NLETS to share more information with users Allows users to customize data presentation via stylesheets XML is non-proprietary  Presents enormous cost savings to states

35. Benefits Implementing new services quicker and easier  If a data provider already has info in GJXDM XML, we can quickly position ourselves to receive it and provide to users Sharing data within a state is simpler – eliminates the need for XML-to-XML transformations (example: Amber Alert)

36. Lessons Learned Be sensitive to early adopters Recognize that you will need careful change management (“ripple effect”) Consider large issues carefully before beginning  message structure, namespaces, schemas Remember that the GJXDM will not provide everything

37. Future Plans Expand membership and services offered Web Services opens new doors to new providers Standardize all NLETS transactions in GJXDM-compliant XML

39. Thank you Kate Silhol NLETS Senior Software Engineer ksilhol@nlets.org (602) 224-0744 www.nlets.org