NIEM, CAM and the 8 “D’s” David Webber - Public Sector NIEM Team, December 2011 NIEM Test Model Data Deploy Requirements Build Exchange Generate Dictionary XML Exchange Development National Information Exchange Model

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation The following is intended to outline Oracle general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Disclaimer Notice Slide 2

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Why CAM and NIEM? “Can you give us a brief overview of how you expect IT professionals and line of business managers to use CAM*?” – Van Ristau, CTO DLT Inc. *CAM editor toolkit – and OASIS Content Assembly Mechanism standard –

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Response Line of business managers can use CAM to drive better alignment to NIEM and faster more consistent delivery of NIEM information exchanges enabling projects to be performed more timely and predictably CAM provides tools that allow business analysts to work collaboratively with software developers to build exchanges Teams use CAM to perform the 8 “D’s” tasks when implementing NIEM exchanges CAM automates the delivery of key XML artifacts for NIEM exchange services CAM can be deployed using the CAMV engine

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation OASIS CAM standard and approach CAM is about content assembly for information exchanges using a visual XML paradigm with the components managed in dictionary collections NIEM provides the basis for domain dictionaries and the naming and design rules for creation of components NIEM aligns well with CAM because they share a common heritage in core component standards work CAM is designed for standards based business exchange development and simplifying that process CAM works for both top down new exchanges or bottom up customizing and tailoring of existing exchanges

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Introducing the 8 “D”s Design Develop Deploy Document Dictionaries Discovery Differentiate Diagnose Tasks in implementing NIEM

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation The 8 “D”s and NIEM Design Develop Deploy Document Dictionaries Discovery Differentiate Diagnose Repeatable, Reusable Process (Exchange Specification Lifecycle) NIEM IEPD Process *IEPD - Information Exchange Package Documentation

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation NIEM IEPD Reports DictionariesDictionaries DiscoveryDiscovery Exchange Delivery Lifecycle DiagnoseDiagnose DifferentiateDifferentiate DesignDesign DevelopDevelop Validated Templates / Schema Validated Templates / Schema XSD Schema XML Samples XMI / UML Models Production Results Production Results Requirements Updates Requirements Updates Documentation Drag and Drop Visual Designer DocumentDocument DeployDeploy Exchange Templates Exchange Templates

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation CAM: Top Down Exchange Assembly Canonical XML Components Dictionary 1 1 Component Associations and Couplings 2 2 Exchange Templates and Rules 3 3 W3C Schema and Model Representations 4 4 Delivery Control, Messaging, Security 5 5 Presentation Collaboration Services Artefact relationships Implementation Artifacts and Examples 6 6 Navigation and Query Tools Domains Reuse Library Components Automatic Rendering Tools Deployment Environments and Middleware XML User Inserts Relationship Lookups Template Catalog Testing Workbench, Rules Engine, Data Samples, Integration ETL Schema XML Models Components Canonical Dictionary Collections

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Design Create information exchange conceptual work flows including the information exchanges needed to match business process needs Determine conceptual information exchanges Understand security and information privacy needs Determine the information components for each exchange Organize the components in each exchange structure to an overall hierarchy to align with the business process needs CAM provides a visual editor for prototyping of exchange structures along with Mind map modelling to allow conceptual reviews

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Discovery CAM provides a variety of tools to assist in discovery of existing components from dictionary collections Automatic comparison tool between exchange and dictionaries – creates mapping spreadsheet Search tools within dictionary interface Visual dictionary structure viewer Mind map models of dictionary components to allow further visual inspection and analysis

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Develop Using the CAM exchange structure outlines complete the actual exchange details Select actual components either from existing NIEM dictionaries or from the enterprise dictionaries Create new components or extend existing components to meet additional design needs Add validation rules for structure and content as needed by the business process Build working exchanges using actual simulated data with realistic XML instances Test each exchange to verify validation and operation Integrate the information exchange handling with the business processes and applications

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Visual Designer with Dictionary Collection Drag n’ Drop Tool Industry dictionary Domain dictionary Component Definitions 4 4 Search Tools Search Tools 2 2 Exchange Designer Exchange Designer 1 1 Insert Dictionary Parent Components Insert Dictionary Parent Components 3 3 Completed Exchange Template 5 5 Collection

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Example - Suspicious Activity Report V2.0 dictionaries XML SAR v1.5 components NIEM core dictionary LEXS dictionary SAR v1.5 components NIEM core dictionary LEXS dictionary LEXS components referenced New structure components based on NIEM + SAR + new SAR conceptual components Definitions stored as syntax neutral canonical XML NIEM core components Dictionary Collection Namespaces of dictionary components

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Document CAM generates NIEM IEPD documentation artifacts including Business component usage report with rules and definitions Code list details and content checks UML physical models Mind map model of exchange structure and components Comparison mapping spreadsheets to NIEM dictionaries and components NIEM “want list” XML for use with NIEM subset generator tool Example XML instances

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Differentiate This step includes building in deployment specific details and rules and usage policy determinations Add additional XPath rules for local integration needs Constrain code lists to local use Limit and restrict content based on policy and role of exchange partners Contextually exclude structure components based on rules Create other integration artifacts for middleware such as policy control, partner certificates and security configuration Can configure these aspects through the CAM template and using middleware tools

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Deploy and Diagnose Once structure information exchange is complete need to test and verify it by generating realistic XML examples Validate those against the exchange template Share working examples with exchange partners Share documentation (IEPD) Deploy with CAMV and distribute test suite module CAMV is Spring aware and supports Java API Tutorial and examples available CAM Editor resources site:

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation CAM Validation Framework CAMV XML instance Template (compiled) Database Lookup (optional) Results XML Java Handler Errors and Warnings StructureRulesCode Lists SQL Query

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation CAMV / ANT Test Suites Environment XML Test Instances ANTANT CAMV XML Validation Results XSLT Reports CAM template CAMV Control Script XML Examples and instructions:

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Dictionaries Sets of NIEM domain dictionaries available Dictionaries can be grouped into collections Tools allow comparisons with existing dictionaries for reuse analysis and mapping spreadsheet generation New dictionaries can be created by Harvesting existing schema collections Export and generation from Enterprise Data Models (ERwin) Automatic Renaming tool can be used to generate NIEM consistent dictionary components for a domain Naming and Design evaluation tool provides alignment to NIEM and interoperability reporting and spell checks Dictionaries can be viewed as spreadsheets or models CAM Toolkit provides suite of tools for dictionary handling

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Developing domain dictionaries (EIEM) Allows domains to manage their components libraries Provides consistency for project development teams Sets of NIEM consistent XML exchange components Aligned to enterprise data stores Optimized for reuse and interoperability Save time and effort across the enterprise Perennial question for developers – when should I use NIEM components, and when our own local ones? Are there components already available for that purpose? Provide formal mechanisms and procedures to share components and collaborate through project lifecycle Provide external parties consistent data views *EIEM – Enterprise Information Exchange Model CAM Toolkit for EIEM generation

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation CAM Toolkit capabilities review

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation SUMMARY AND REVIEW Reviewing NIEM technologies

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Summary of NIEM and CAM Technologies  Rapid NIEM solution development – IEPDs NIEM and CAM editor toolkit Ability to use rules, code lists and SQL lookups  Dictionaries and Exchange development Simplifying and accelerating effective exchanges NIEM – canonical component management Tools for building EIEM dictionaries Drag and Drop designer using dictionary collections  Deployment CAMV validation XML framework and test suite toolkit

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Generating IEPD artifacts NDR evaluation, NIEM mapping, “want list” + subset, Exchange Schema, models of the structure components, XML instances, business rules documentation

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation IEPD Generation Steps Toolkit for IEPD artifacts generation Suite of menu options and tools in desktop CAM toolkit editor (each run custom xslt scripts on the exchange CAM template xml) CAM Template 1.Run template Evaluation Report 2.Compare to NIEM dictionary - create mapping spreadsheet - create wantlist.xml - NIEM online SSGT uploading + subset schema generation Generate business rules report 3.Generate exchange XSD schema 4.Generate XML test instance(s) EXCHANGE TEMPLATE Structure Rules Documentation

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation NDR Evaluation Report Provides scoring, alerts, warnings and potential issues including: Naming and Definition Conventions Naming / Restriction consistency checks Interoperability Enablers/Inhibitors checks Rules integrity and duplicates Statistics on exchange size

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Evaluation Report (NDR) example Part of an example report for LEXS getDataItemRequest template displayed in toolkit HTML viewer

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Compare to NIEM dictionary References NIEM dictionary of names and properties currently in NIEM 2.1 or 2.0 releases Matches on physical names Reports mapping details Compatible with Microsoft Excel Report can be used to do spell checking (in Excel) Generates wantlist.xml file Wantlist can be uploaded to SSGT and then subset schema generated ( Options.iepd ) Options.iepd

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation NIEM cross-reference spreadsheet Formatted view in Microsoft Excel of import of cross- reference report details (from generated XML file)

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation NIEM online SSGT and wantlist.xml Options menu and Load Wantlist option with resulting load of subset details

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Documentation and Schema Documentation: Create HTML report of exchange schema details and associated content and business rules Report layout and content designed to be reviewed by business analysts Schema: Generate XSD schema for exchange Customizable IEPD folder layout management by namespace for extension, subset and exchange schema components Writes XSD schema in syntax that is clear, simple and compatible with deployment tooling environments

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Business Rules Documentation Part of the example rules for LEXS getDataItemRequest template displayed in toolkit HTML viewer

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Export Exchange to XSD Schema Completed Exchange Structure Set Exchange Options Complete set of NIEM exchange schemas generated Export Template to Exchange XSD:

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Exchange Schema Generated Each namespace file is import for those specific type definitions Reviewing XSD results in a schema editor tool Set of XSD files with filename and namespace suffix

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Exchange Model Generated

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Examples Generation Support for software development testing process Designed to allow creation of concrete realistic examples not just random value based Hinting system allows insertion of actual test system values into XML examples Can create both valid and invalid examples to support unit testing of application software Exclude capability allows generator to create examples that contain only a portion of the entire exchange Control over random seed value used allows re- generation of identical test cases

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation XML example generation wizard View of CAM toolkit with LEXS getDataItemRequest and dialogue for XML test example generator tool

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Running validation rules tests Built-in validation engine allows testing of XML instances against actual exchange rules. Critical to ensure that the exchange validates actual live production example scenarios correctly Allows deployed solution to match IEPD exchange details Errors can be reviewed interactively in exchange visual interface Post-processing of validation results allows unit regression tests to be created with reporting of errors, warnings and information level notes

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Run Exchange Template Pick XML test case to validate Run validation Review results in visual editor

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Formal IEPD Packaging Details IEPD ArtefactDescription Exchange Files Subset Schema (SSGT)Subset of the full NIEM schema—a compressed directory of schemas to distinguish from other schema sets. Wantlist User requirements -distinguishes user-required data components from components they depend on for conformance. Generated by and up-loaded to the Schema Subset Generation Tool (SSGT). This is an open specification and the SSGT is not required to create a wantlist, though it is easier. Exchange Schema Base document schema that defines the XML root element and is generally named after the IEPD itself. Also known as the document schema, reference schema, or root schema. Constraint SchemaConstraints for separate constraint validation path—a compressed directory of schemas to distinguish from other schema sets. Extension SchemaSpecification for extended components—separate local name-spaces of components not contained in NIEM. Sample XML InstanceExample instance(s) – may reference optional stylesheet. StylesheetExample stylesheet for display of instance(s). Documentation Master Documentation The Master Document is the main document for which all of the context and details around the exchange are explained. This document includes, the overview, as well as details surrounding the exchange, business drivers and requirements Exchange model Exchange model in standard open format (xmi, vsd, zargo) and standard open graphic (jpg, pdf, etc.) preferably a Unified Modeling Language (UML) model. Business rules Business rules in one of the following formats: (1) plain or structured English, (2) written into master documentation, (3) Schematron or other formal business rule language, or (4) generated by a development tool. Mapping to NIEMMapping of domain components, tagged with constraints (i.e., cardinality, etc.) to NIEM components – as spreadsheet Extended componentsComponents created because they were not in NIEM— part of mapping spreadsheet and include structure and definitions of new components. Change logRecord of cumulative changes from previous IEPD versions. The initial IEPD simple records its creation date. Catalog Catalog XML fileA machine-readable list of artifacts in the IEPD in an open, portable and browser-displayable format (simple manual edit). Metadata XML fileAll metadata of owner and domain to be registered with the IEPD (simple manual edit).

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Q & A Download the CAM editor toolkit from: Resource Center link

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Reference Materials References and Links

Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation Links and References SUPPORTING NIEM MATERIALS - NIEM Naming and Design Rules (NDR) NIEM Practical Implementer’s Course (online) NIEM Course Modules (17 free PDFs) Techniques for Building and Extending NIEM XML Components M.pdfhttp:// M.pdf