1. Delivering NIEM-aligned data services David Webber, Information Architect, Oracle Public Sector NIEM and XML for Architects and Developers

2. The following is intended to outline Oracle general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Disclaimer Notice Slide 2

3. © 2012/13 Oracle Corporation – Restricted Distribution3 Agenda Background - What are the business drivers and needs? NIEM today – Technical Introduction The Delivery Life Cycle – Lessons Learned Software Delivery - Anatomy of a NIEM “IEPD” Tooling to the Rescue! Key Pillar of Dictionaries Summary

4. BACKGROUND Business Drivers and Needs

5. © 2012/13 Oracle Corporation – Restricted Distribution5 XML is everywhere… PMIX – Prescription Drug information sharing Elections Results Reporting Pharmacy and Vaccines locating COBie Buildings Information sharing PESC – Student Post-Secondary Education City 311 Services Vehicle spare parts inventory Public Safety information sharing Healthcare services delivery coordination eDevice control templates

6. © 2012/13 Oracle Corporation – Restricted Distribution6 Public Sector Challenges Cost of building information exchanges – Reduction of development sunk costs through minimizing coding tasks – Allowing rapid prototyping and proof of concepts (agile development) – Flexibility - dynamic adaptable templates instead of rigid fixed code – Reuse – templates for common systems / solutions easily shared and adapted Government transparency and public information sharing – Structured content from data stores – e.g. election results reporting, monthly cost reports, scheduled road maintenance; obvious metadata and semantics – Simple search requests on data – e.g. part numbers, codes, licenses lookup – Not intended for unstructured and textual content document delivery Practical real world data sharing anywhere – Direct simple tools with short learning curve and plug and play deployment

7. © 2012/13 Oracle Corporation – Restricted Distribution7 Technology Capability Requirements Delivering on the Open Data Digital Strategy Vision for government Open Standards and Open Source based Plug and play with code-free templates and rapid development Supports NIEM and Open XML exchanges Leverages deployed technology today Workflow integration Secure messaging delivery and partner management Delivered across-platform and device End-to-end security & governance

8. NIEM TODAY Technical Introduction

9. The NIEM Framework NIEM connects communities of people who share a common need to exchange information in order to advance their missions, and provides a foundation for seamless information exchange between federal, state, local, and tribal agencies. Much more than a data model, NIEM offers an active user community as well as a technical and support framework. Formal Governance Processes Online Repositories Mission-Oriented Domains Self-Managing Domain Stewards Data Model XML Design Rules Development Methodology Predefined Deliverables (IEPD) Tools for Development and Discovery Established Training Program Implementation Support Help Desk & Knowledge Center

10. The NIEM Data Model NIEM’s data model is a set of common, controlled, and approved XML data structures and definitions vetted through the Federal, State, Local, Tribal and Private Sectors. Data elements are organized into core and domain-specific components Core components are used by multiple domains and can be described by structure, semantics, and definition universally Domain-specific components are continually updated by subject matter experts that are actual NIEM participants and industry experts for their particular domain NIEM Naming and Design Rules (NDR) specify how each of these components are defined and utilized

11. NIEM’s Domains View NEWEST DOMAINS NIEM Government Resource Management Domain NIEM Health Domain NIEM Human Services Domain DOMAINS IN PLANNING NIEM Agriculture NIEM Education NIEM Transportation 11

12. © 2012/13 Oracle Corporation – Restricted Distribution12 Available XML Dictionaries NIEM 2.1 dictionaries NIEM 2.1 dictionaries – CBRN dictionary – Emergency dictionary – Family dictionary – Immigration dictionary – Infrastructure dictionary – Intelligence dictionary – Justice dictionary – Maritime dictionary – Screening dictionary – Trade dictionary – Immigration blueprint – NIEM core dictionary Available from download site direct link: http://sourceforge.net/projects/camprocessor/files XML + includes spreadsheets and sample models Note: Those marked in bold are model style dictionaries with recursive components.

13. © 2012/13 Oracle Corporation – Restricted Distribution13 NIEM uses W3C Schema Collection XSD Collection *Code list xsd Namespace * Optional XML Schema xsd subset *Constraint xsd *Extension xsd *Restriction xsd Industry Domain schemas Code list xsd LEGEND: Exchange xsd NIEM Exchange Schema

14. © 2012/13 Oracle Corporation – Restricted Distribution14 NIEM Past and Present Challenges Vertical domain vocabulary for DHS and DOJ One dozen domain dictionaries Technology limited - built using XSD schema Core components highly contextual to DHS/DOJ Surprisingly – no actual logical models of information! Gap with semantic technologies integration Limited data content rules and code lists Mapping automation support missing Multi-year development life cycles

15. © 2012/13 Oracle Corporation – Restricted Distribution15 NIEM is still improving… Collections of complex XML Schema Verbose components Embedded context in names Currently facing significant scaling challenges – Inconsistencies; too much manual management; slow lifecycles Dictionary technology incubating Enhanced code lists mechanism incubating UML profile is evolving initiative with OMG – Attempt to marry modelling techniques and XSD Schema syntax Policy and Security mechanisms incubating Need to embrace enabling tools for rapid expansion of domains and community collaboration

16. DELIVERY LIFECYCLE How to build NIEM solutions Lessons learned Foundation Pillars

17. © 2012/13 Oracle Corporation – Restricted Distribution17 The 8 “D”s and NIEM Design Develop Deploy Document Dictionaries Discovery Differentiate Diagnose Repeatable, Reusable Process (Exchange Specification Lifecycle) NIEM IEPD Process *IEPD - Information Exchange Package Documentation

18. ©2012 Oracle Corporation 18 NIEM IEPD Reports DictionariesDictionaries DiscoveryDiscovery NIEM IEPD / Exchange Delivery Lifecycle DiagnoseDiagnose DifferentiateDifferentiate DesignDesign DevelopDevelop Validated Templates / Schema Validated Templates / Schema XSD Schema XML Samples XMI / UML Models Production Results Production Results Requirements Updates Requirements Updates Document ation Drag and Drop Visual Designer DocumentDocument DeployDeploy Exchange Templates Exchange Templates 1 1 2 2 3 3 4 4 5 5 Required IEPD artifacts

19. ©2012 Oracle Corporation 19 Integrating into your existing processes and control Critical to manage and steer use of NIEM Ensure business outcomes match requirements Lessons learned with NIEM The earlier in the software development process that NIEM is introduced the better the overall outcomes Each step has deliverables – business plan, delivery architecture, data model, alignment reports, schema, test results that can be monitored in your SDLC* Monitoring products reinforces reuse as best practice Tools essential in review processes Collaboration tools can coordinate artifacts across projects and participants Test bed tools and open APIs reduce integration alignment costs *SDLC – Software Development Life Cycle

20. SOFTWARE DELIVERY Anatomy of a NIEM “IEPD” It is not just all about the XSD schema

21. ©2012 Oracle Corporation 21 IEPD Components & Requirements IEPD IEM IEPD MPD NIEM Core Schema(s) Domain Schema(s) Main Document Catalog Change Log Sample XML Instance In order to be NIEM-conformant, the IEPD must adhere to: 1.NIEM Conformance Document 2.NIEM Naming and Design Rules (NDR) v1.3 3.NIEM Model Package Description (MPD) Specification v1.0

22. Formal NIEM IEPD Packaging Details IEPD ArtefactDescription Exchange Files Subset Schema (SSGT) Subset of the full NIEM schema—a compressed directory of schemas to distinguish from other schema sets. Wantlist User requirements -distinguishes user-required data components from components they depend on for conformance. Generated by and up-loaded to the Schema Subset Generation Tool (SSGT). This is an open specification and the SSGT is not required to create a wantlist, though it is easier. Exchange Schema Base document schema that defines the XML root element and is generally named after the IEPD itself. Also known as the document schema, reference schema, or root schema. Constraint Schema Constraints for separate constraint validation path—a compressed directory of schemas to distinguish from other schema sets. Extension Schema Specification for extended components—separate local name-spaces of components not contained in NIEM. Sample XML Instance Example instance(s) – may reference optional stylesheet. Stylesheet (generic) Example stylesheet for display of instance(s). Documentation Master Documentation The Master Document is the main document for which all of the context and details around the exchange are explained. This document includes, the overview, as well as details surrounding the exchange, business drivers and requirements Exchange model XMI Exchange model in standard open format (xmi, vsd, zargo) and standard open graphic (jpg, pdf, etc.) preferably a Unified Modeling Language (UML) model. Business Rules Business rules in one of the following formats: (1) plain or structured English, (2) written into master documentation, (3) Schematron or other formal business rule language, or (4) generated by a development tool. Mapping to NIEM Mapping of domain components, tagged with constraints (i.e., cardinality, etc.) to NIEM components; preferably the Component Mapping Tool (CMT). Extended Components Components created because they were not in NIEM—may be part of mapping spreadsheet and include structure and definitions of new components. Prefer CMT. Change log Record of cumulative changes from previous IEPD versions. The initial IEPD simple records its creation date. Catalog Catalog XML file A machine-readable list of artifacts in the IEPD in an open, portable and browser-displayable format (simple manual edit). Metadata XML file All metadata of owner and domain to be registered with the IEPD (simple manual edit). Generated using CAM Never Used Word Document Manual Entry

23. ©2012 Oracle Corporation 23 XML Technology Components NIEM: need to manage complete aspects of exchange delivery, not just the structure and content data model

24. ©2012 Oracle Corporation 24 Swim Lane View UML Technologies XML Technologies W3C Schema This is what NIEM exchanges are delivered with on-the-wire! UML XMI XSD Namespace Enumerations XSLT XQuery XPath XML Patterns Associations Cardinality Stereotypes Inheritance Facets Elements Attributes Content Complex Types Audience: Data Modellers / Designers Audience: Software Developers and Middleware Engineers Audience: General content delivery and applications; content engineers; rules representation systems; multimedia delivery; geospatial systems Objects WebWeb HTML JavaScript JSON REST SQL SOAP Audience: Web application developers Open-XDX 24

25. TOOLING TO THE RESCUE Accelerating and Improving the NIEM process

26. ©2012 Oracle Corporation 26 IEPD Delivery Tasks / Tools Matrix CAMCAMV Open-XDX (OPX) Exchange Schema Development X Canonical Model Development / Generation X Exchange Schema Generation X NIEM Compliance Checking / Reporting X WSDL / JAXB Binding Generation X Operation Message Test Case Generation X Production Services Message Generation X SQL Schema Mapping XX NIEM IEPD Artifact Generation X - Cross-reference spreadsheet X - UML model X - Business Rules Report X - XSD schema package X Conformance Test Suite X Run time XML instance validation X In order to be NIEM-conformant, the IEPD must adhere to: 1.NIEM Conformance Document 2.NIEM Naming and Design Rules (NDR) v1.3 3.NIEM Model Package Description (MPD) Specification v1.0

27. © 2012/13 Oracle Corporation – Restricted Distribution27 CAM Toolkit for NIEM Concepts behind the Content Assembly Mechanism approach Understanding the steps involved in top down / bottom up exchange design, testing and delivery Applicability to NIEM and IEPD / MPD delivery * NIEM – National Information Exchange Model * IEPD – Information Exchange Package Documentation *MPD – Model Package Description

28. © 2012/13 Oracle Corporation – Restricted Distribution28 CAM Templates – WYSIWYG Exchange StructureRules Documentation CAM Template http://www.oasis-open.org/committees/cam

29. © 2012/13 Oracle Corporation – Restricted Distribution29 INGEST CAM and W3C Schema Wizards EXPORT XSLT tools process entire XSD collection XSLT tools process entire XSD collection XSD Collection Structure Rules Documentation *Code list xsd Namespace * Optional XML Schema xsd subset *Constraint xsd *Extension xsd *Restriction xsd Industry Domain schemas Code list xsd LEGEND: Exchange xsd Exchange Schema CAM Template

30. http://www.cameditor.org CAM Editor – Main Screens

31. © 2012/13 Oracle Corporation – Restricted Distribution31 CAM Toolkit Features Dictionary “Drag and Drop” of components to build exchanges Visual editor and structure editing Cross-platform deploy - Windows, Ubuntu/LINUX, Mac; 32/64bit Mind map models – integration with Freemind tool UML/XMI export – import to UML tools / ArgoUML Code list table lookups – rule based and selectable This allows people to fully integrate their exchange development from dictionaries through to working deployed exchange schema. 31

32. © 2012/13 Oracle Corporation – Restricted Distribution32 XML Exchange Package Artifacts Template Evaluation Report – Naming and Design Rules assessment – Interoperability factors review – Spell checking Comparison to Dictionary – Reuse report, cross-reference spreadsheet, subset “wantlist” Generate Schema Bundle with layout styles (e.g. NIEM) Realistic test samples generation Testing with runtime validation engine (CAMV) Business information exchange documentation Models – XMI and mind maps

33. © 2012/13 Oracle Corporation – Restricted Distribution33 CAM Template Evaluation Report The purpose of the CAM template evaluation report is to provide a suite of checks against a template that are difficult, tedious or otherwise to determine by manual inspection of a template structure and its associated rules and annotation details The report covers the following areas: – alerts users to potential issues with their template and provides suggestions for improving the functional details – addresses logical and conceptual issues that simple syntax checking alone does not flag – can be used as part of a formal review process to ensure a baseline for quality control

34. © 2012/13 Oracle Corporation – Restricted Distribution34 CAM Toolkit Capabilities

35. © 2012/13 Oracle Corporation – Restricted Distribution35 Online Tutorials – www.niemtrainingvideos.org

36. KEY PILLAR OF DICTIONARIES Introduction and Concepts

37. © 2012/13 Oracle Corporation – Restricted Distribution37 The Semantic Framework and Vision XML/NIEM Components in Neutral Dictionary Representation UML Technologies XML Technologies W3C Schema Tools Semantic Representations Rules Technologies HTML 5 rendering tools Security solutions Middleware solutions

38. © 2012/13 Oracle Corporation – Restricted Distribution38 Dictionaries: Key Pillar of Information Sharing Canonical XML Components Dictionary 1 1 Component Associations and Couplings 2 2 Exchange Templates and Rules 3 3 W3C Schema and Model Representations 4 4 Delivery Control, Messaging, Security 5 5 Presentation Collaboration Services Artefact relationships Implementation Artifacts and Examples 6 6 Navigation and Query Tools Domains Reuse Library Components Automatic Rendering Tools Deployment Environments and Middleware XML User Inserts Relationship Lookups Template Catalog Testing Workbench, Rules Engine, Data Samples, Integration ETL Schema XML Models Component s Canonical Dictionary Collections

39. ©2012 Oracle Corporation 39 Dictionaries Semantics Approach Abstract dictionary representations with strong semantics, rules and code lists support Schema, models, documentation are generated from dictionary definitions automatically Contextual mechanisms allow linear number of true core components – not exponential growth Easier for practitioners to leverage with more predictable and repeatable results Enable domain use horizontally across government ETL mapping automation support Continuous collaboration environment

40. ©2012 Oracle Corporation 40 Developing domain dictionaries (EIEM) Allows domains to manage their components libraries Provides consistency for project development teams Sets of NIEM consistent XML exchange components Aligned to enterprise data stores Optimized for reuse and interoperability Save time and effort across the enterprise Perennial question for developers – when should I use NIEM components, and when our own local ones? Are there components already available for that purpose? Provide formal mechanisms and procedures to share components and collaborate across SDLC process Provide external parties consistent data views *EIEM – Enterprise Information Exchange Model

41. ©2012 Oracle Corporation 41 EIEM/BIEC Business Information Exchange Components (BIEC) NIEM-conforming XML schema data component definition that meets a particular recurring business requirement for an enterprise Enterprise Information Exchange Model (EIEM) NIEM-conforming schemas that define data components to be reused in IEPDs developed by an enterprise; collection of enterprise BIECs organized into a subset and one or more extension schemas

42. © 2012/13 Oracle Corporation – Restricted Distribution42 Managed Dictionaries – Authoritative Sources Slide 42 Centralized Distributed schema SQL SFIS Values Library Service, NIEM, Metadata Registry ALLOWED VALUES schema Managed Dictionaries Publish

43. SUMMARY Next Steps

44. © 2012/13 Oracle Corporation – Restricted Distribution44 Why CAM and NIEM? “Can you give us a brief overview of how you expect IT professionals and line of business managers to use CAM*?” – Van Ristau, CTO DLT Inc. *CAM editor toolkit – http://www.cameditor.org and OASIS Content Assembly Mechanism standard – http://docs.oasis-open.org/camhttp://www.cameditor.orghttp://docs.oasis-open.org/cam

45. © 2012/13 Oracle Corporation – Restricted Distribution45 Response Line of business managers can use CAM to drive better alignment to NIEM and faster more consistent and predictable delivery of NIEM information exchanges CAM provides tools that allow business analysts to work collaboratively with software developers to build exchanges Teams use CAM to perform the 8 “D’s” tasks when implementing NIEM exchanges CAM automates the delivery of key XML artifacts for NIEM exchange services XML validation can be deployed using the CAMV engine Open-XDX provides rapid flexible open data web services

46. © 2012/13 Oracle Corporation – Restricted Distribution46 Public Sector Open Data Services Delivery Public Sector Open Data Services Delivery Open Platform Solutions Visual Editor SQL Browser Dictionaries Drag and Drop Query/Response Generate Data Validate Data Send Data JAX-WS soapUI CAM Editor Content Models OER Integration Web Forms UI Open Data Test Bed Services Dictionaries DB Web Open XDX CAMV Open-XDX / CAM Editor Dictionary Security SQL XML SQL Repository CAM Editor Templates Policy Rules XACML Templates XACML

47. © 2012/13 Oracle Corporation – Restricted Distribution47 Technology Components and Features Providing open data sharing – Open-XDX component accelerates deployment of interfaces Aligning meaning and definitions – Collaboration using XML component dictionaries and NIEM Formal information verification and content rules – CAMV rules engine provides runtime handling – Industry domain test bed facilities – Rules documentation and sharing Business process integration – Oracle BPM and Forms combined with CAMV and Open-XDX In progress – Security profiles and policy templates – Upcoming release of CAM Editor and XACML integration

48. © 2012/13 Oracle Corporation – Restricted Distribution48 CAMV interactive testing Cloud Services Deployment Open-XDX web services XML Technologies W3C Schema Tools Semantic Representations Rules Technologies Security solutions Middleware solutions CAMV test bed for industries Registry for Dictionaries Security Profile Templates Web site and collaboration tools Policy Rules Policy Rules

49. CAMeditor.ORG Project Statistics SNAPSHOT OF PROJECT ACTIVITIES 135,000+ CAMeditor.org page visits to site 165+ countries have downloaded tools; 30% of visitors are from U.S.; 800+ downloads weekly 2000+ student views of online video training resources 8 languages now available www.cameditor.org www.niemtrainingvideos.org

50. ©2012 Oracle Corporation 50