Dictionaries Tutorial Overview – Public Sector NIEM Team, November 2012 CAM Test Model Data Deploy Requirements Build Exchange Generate Dictionary XML Exchange Development Content Assembly Mechanism

Copyright ©2011/12, Oracle. All Rights Reserved. The following is intended to outline Oracle general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Disclaimer Notice Slide 2

Copyright ©2011/12, Oracle. All Rights Reserved. Tutorial Overview Dictionary Concepts Dictionary Services and Facilities Examples Walkthrough Summary and Review

Copyright ©2011/12, Oracle. All Rights Reserved. DICTIONARY CONCEPTS Understanding Dictionaries Usage

Copyright ©2011/12, Oracle. All Rights Reserved. The Dictionary Role and Content Provide a set of common, controlled, and managed information data structures and component definitions for a particular purpose and community Data elements are organized into core and domain-specific components Core components are used across information exchanges and can be described by structure, semantics, and definition with global context Context-specific components are particular to a limited number of information exchanges and have a limited scope and applicability Naming and Design Rules (NDR) specify how each of these components are defined and utilized

Copyright ©2011/12, Oracle. All Rights Reserved. The 8 “D”s of software development Design Develop Document Dictionaries Discovery Differentiate Deploy Diagnose Predictable, Repeatable, Reusable Process Solution Delivery Process LIFECYCLE

Copyright ©2011/12, Oracle. All Rights Reserved. XML Samples DictionariesDictionaries DiscoveryDiscovery Exchange Delivery Lifecycle DiagnoseDiagnose DifferentiateDifferentiate DesignDesign DevelopDevelop Validated Templates / Schema Validated Templates / Schema XSD Schema Evaluation Reports XMI / UML Models Production Results Production Results Requirements Updates Requirements Updates Documentation Drag and Drop Visual Designer DocumentDocument DeployDeploy Exchange Templates Exchange Templates CAM editor Toolkit *CAM – Content Assembly Mechanism – OASIS standard

Copyright ©2011/12, Oracle. All Rights Reserved. CAM Dictionary Framework CAM Editor Drag and Drop Designer Collection Manager CAM Rules Wizards Model Generation Exchange Structure Builder Dictionary Codelist Examples and instructions: Codelist

Copyright ©2011/12, Oracle. All Rights Reserved. How to create dictionaries Top down - directly from information models Enterprise modelling tools SQL database physical table definitions Bottom up Harvesting from existing data assets Harvesting from XML exchange structures and schema Manual entry through Excel spreadsheets Using neutral abstract standards based representations

Copyright ©2011/12, Oracle. All Rights Reserved. Dictionary Content Model Child (BBIE) Item Child (BBIE) Item Parent (ABIE) Item Parent (ABIE) Item Follows Naming and Design Rule (NDR) principles and guidelines Canonical Components Dictionary XML Canonical Components Dictionary XML ebXML CCTS terms (ABIE, BBIE, ASBIE) Parent = Aggregate Business Information Entity Child = Basic Business Information Entity Attribute = Association Business Information Entity Parent (ABIE) Item Parent (ABIE) Item Parent (ABIE) Item Parent (ABIE) Item Parent (ABIE) Item Parent (ABIE) Item..... Child (BBIE) Item Child (BBIE) Item Child (BBIE) Item Child (BBIE) Item Child (BBIE) Item Child (BBIE) Item Attribute (ASBIE) Attribute (ASBIE) Attribute (ASBIE) Attribute (ASBIE) * CCTS – Core Components Technical Specification Attribute (ASBIE) Attribute (ASBIE) Attribute (ASBIE) Attribute (ASBIE) Each compound component Each atomic component Optional attributes of component

Copyright ©2011/12, Oracle. All Rights Reserved. Example – Person Name Person Name (ABIE) Verified Details? (ASBIE) First Name (BBIE) Middle Name (BBIE) Last Name (BBIE) Previous Name? (ASBIE) Language Code (BBIE) Language Code may exist independently of Person Name Verified Details and Previous Name are flags that denote additional information about the entity they are associated with There are three component items aspects: structure relationships; content rules; definitions

Copyright ©2011/12, Oracle. All Rights Reserved. Semantics Approach Neutral formatted XML dictionaries Contains core information on XML components Representation subset of UN/CEFACT Core Components Technical Specification (CCTS) Can generate XSD, OWL, XMI and more as desired (contains the vocabulary terms, definitions, purpose and content models) Renamer tool aligns with NIEM NDR

Copyright ©2011/12, Oracle. All Rights Reserved. Value Proposition Allow business data analysts to focus on information needs and build data exchanges Aligns information with industry standards and enterprise information stores Ensures consistent usage and definitions using profile of syntax-neutral terms and constructs Components can be derived from existing domain schema and data structures Supports providing sharing and collaboration services Allows development of supporting analytics tools

Copyright ©2011/12, Oracle. All Rights Reserved. Sources of Your Dictionaries Ingest existing application semantic artifacts – e.g. XML Schema, Database schema, Excel spreadsheets, Enterprise Data Models (EDM) Starting from scratch: Excel spreadsheet, or SQL table entry, or XML components entry Then generate dictionaries of components Download existing domain dictionary collection

Copyright ©2011/12, Oracle. All Rights Reserved. Renaming Tool Expands abbreviations (has built-in lists with context) Corrects common typos and spelling errors (200+) Adjusts names to camel case with no separators (_) Retains acronyms Adds name suffix according to NIEM representation terms and type of content, e.g. date, number, code, etc Optionally refactor rules by their type for interoperability; make nillable optional; relax DB length restrictions Can optionally insert namespace prefix Can be configured by editing the lists in: word-utils.xsl

Copyright ©2011/12, Oracle. All Rights Reserved. Analytics Tools Compare to Dictionary tool – provides reuse scoring statistics for information exchange structures Template Evaluation Tool – Provides Naming and Design Rule (NDR) analysis Spelling Checking Information exchange interoperability analysis Data modelling errors and warnings Naming / use inconsistencies / omissions

Copyright ©2011/12, Oracle. All Rights Reserved. CAM: Top Down Exchange Assembly Canonical XML Components Dictionary 1 1 Component Associations and Couplings 2 2 Exchange Templates and Rules 3 3 W3C Schema and Model Representations 4 4 Delivery Control, Messaging, Security 5 5 Presentation Collaboration Services Artefact relationships Implementation Artifacts and Examples 6 6 Navigation and Query Tools Domains Reuse Library Components Automatic Rendering Tools Deployment Environments and Middleware XML User Inserts Relationship Lookups Template Catalog Testing Workbench, Rules Engine, Data Samples, Integration ETL Schema XML Models Components Canonical Dictionary Collections *CAM – Content Assembly Mechanism – OASIS standard

Copyright ©2011/12, Oracle. All Rights Reserved. DICTIONARY SERVICES AND FACILITIES Tools Architecture

Copyright ©2011/12, Oracle. All Rights Reserved. Dictionary Management / Acquisition

Copyright ©2011/12, Oracle. All Rights Reserved. CAM Dictionary From Existing XML schema OASIS CAM template Source schema in XSD syntax Source schema in XSD syntax NDR Evaluation, Refactor, Renaming Tools NDR Evaluation, Refactor, Renaming Tools 3 3 Generate Standard Components Dictionary XML Generate Standard Components Dictionary XML 4 4 Canonical XML Components Dictionary XML Import Apply Build

Copyright ©2011/12, Oracle. All Rights Reserved. CAM Dictionary From ERwin models EDM Ele Def DDL 1 1 Export Components in XSD syntax Collection of objects from model From Enterprise Data Model Import XSD and refactor for use with CAM editor OASIS CAM template Model Components XSD schema NDR Evaluation, Refactor, Renaming Tool NDR Evaluation, Refactor, Renaming Tool Apply Naming and Design Rule (NDR) checks and edits 4 4 Generate Standard Components Dictionary XML Generate Standard Components Dictionary XML 5 5 Dictionary of exchange components XML Import 3 3 Analyst Review ebXML CCTS compatible (ABIE, BBIE, ASBIE) 2 2

Copyright ©2011/12, Oracle. All Rights Reserved. CAM Dictionary From SQL tables 1 1 SQL tables Drag and Drop Designer from SQL to XML Configure renamer drag and drop XML mapping 4 4 Generate Dictionary from XML components

Copyright ©2011/12, Oracle. All Rights Reserved. CAM Dictionary From Excel Spreadsheet 1 1 Dictionary Generator Spreadsheet XML data mapping Export to XML results Download Excel to Dictionary pack Enter components README has complete instructions

Copyright ©2011/12, Oracle. All Rights Reserved. Renamer Tool – Align to XML NDR -renamer added by default – otherwise edit name as desired Leave blank or pick namespace to process NIEM mode (with term representations); or CamelCase (Java style) Leave as normal Optionally insert prefix to all items without prefix Set to false to retain all rules “as is” Note: Produces renamer report XML file of old / new names. Can be reviewed as spreadsheet.

Copyright ©2011/12, Oracle. All Rights Reserved. Dictionary Collection Manager 1 1 dictionary collection control file Collection name Component dictionaries Download Excel to Dictionary pack 4 4 Editor preference

Copyright ©2011/12, Oracle. All Rights Reserved. Available XML Dictionaries NIEM 2.1 dictionariesNIEM 2.1 dictionaries CBRN dictionary Emergency dictionary Family dictionary Immigration dictionary Infrastructure dictionary Intelligence dictionary Justice dictionary Maritime dictionary Screening dictionary Trade dictionary Immigration blueprint NIEM core dictionary Available from download site direct link: XML + includes spreadsheets and sample models Note: Those marked in bold are model style dictionaries with recursive components.

Copyright ©2011/12, Oracle. All Rights Reserved. EXAMPLES WALKTHROUGH Understanding capabilities

Copyright ©2011/12, Oracle. All Rights Reserved. Visual Designer with Dictionary Collection Drag n’ Drop Tool Industry dictionary Domain dictionary Component Definitions 4 4 Search Tools Search Tools 2 2 Exchange Designer Exchange Designer 1 1 Insert Dictionary Parent Components Insert Dictionary Parent Components 3 3 Completed Exchange Template 5 5 Collection

Copyright ©2011/12, Oracle. All Rights Reserved. Developing domain dictionaries (EIEM) Allows domains to manage their components libraries Provides consistency for project development teams Sets of NIEM consistent XML exchange components Aligned to enterprise data stores Optimized for reuse and interoperability Save time and effort across the enterprise Perennial question for developers – when should I use NIEM components, and when our own local ones? Are there components already available for that purpose? Provide formal mechanisms and procedures to share components and collaborate across SDLC process Provide external parties consistent data views *EIEM – Enterprise Information Exchange Model CAM Toolkit for EIEM generation

Copyright ©2011/12, Oracle. All Rights Reserved. Example: N-DEx Dictionary and Model (Harvested from N-DEx schema – ndexia.xsd and ndexibp2.xsd) Freemind Interactive Model Dictionary viewer with N-Dex components

Copyright ©2011/12, Oracle. All Rights Reserved. Example - Suspicious Activity Report V2.0 dictionaries XML SAR v1.5 components NIEM core dictionary LEXS dictionary SAR v1.5 components NIEM core dictionary LEXS dictionary LEXS components referenced New structure components based on NIEM + SAR + new SAR components Definitions stored as syntax neutral canonical XML NIEM core components Dictionary Collection Namespaces of dictionary components

Copyright ©2011/12, Oracle. All Rights Reserved. SUMMARY AND REVIEW Reviewing Dictionary Technology

Copyright ©2011/12, Oracle. All Rights Reserved. Opportunities / Challenges Paradigm shift Teams used to manually crafting XML Schema and Java code in developer XML IDE tools Collaboration between developers and data analysts Lowering learning curve for application development teams Integration into SDLC and NIEM IEPD processes Unit test XML instance generation XML test framework and validation services

Copyright ©2011/12, Oracle. All Rights Reserved. Work in Progress NIEM ‘lite’ – easier simpler use by adding context aware mechanisms to automatically filter dictionary components based on domain use case Expose context lists to allow click and choose selections Dictionary Analytics tools – find out what is going on inside your dictionaries – consistency, omissions, content/name conflicts, orphans Store and Share Collaboration - API with SOA Suite Repository integration

Copyright ©2011/12, Oracle. All Rights Reserved. Value Proposition Contextual mechanisms allow linear number of true core components – not exponential growth Easier for practitioners to leverage with more predictable and repeatable results Enable domain use horizontally across government ETL mapping automation support Continuous collaboration environment

Copyright ©2011/12, Oracle. All Rights Reserved. CAMeditor.ORG Project Statistics ‹#› SNAPSHOT OF PROJECT ACTIVITIES 50,000 CAMeditor.org page visits in one year 140 countries have downloaded tools; 25% of visitors are from U.S.; 600+ downloads weekly 50 views of online video training resources weekly 6 languages now available