Dictionaries Tutorial Overview – Public Sector NIEM Team, November 2012 CAM Test Model Data Deploy Requirements Build Exchange Generate Dictionary XML.

Slides:



Advertisements
Similar presentations
Oracle SQL Developer Data Modeler 3.0: Technical Overview March 2011.
Advertisements

CACORE TOOLS FEATURES. caCORE SDK Features caCORE Workbench Plugin EA/ArgoUML Plug-in development Integrated support of semantic integration in the plugin.
IJIS Industry Summer Briefing – July 2011
NIEM and Content Policy briefing David Webber - Public Sector NIEM Team, April 2013 NIEM Test Model Data Deploy Requirements Build Exchange Generate Dictionary.
Future of NIEM Tools Delivery Public Sector NIEM Team, July 2012 Futureof NIEM Tools.
IEEE P1622 Meeting, Oct 2011 IEEE P1622 Meeting October 24-25, 2011 Guide to using OASIS EML v7.0 for UOCAVA Implementations David RR Webber Information.
Quick Guide to CAM Blueprints
1 1 Accelerated NIEM IEPD Development Process, Procedures and Techniques.
1 June, 2010 – CAM Draft Specification Development Related Materials Quick Guide to CAM Dictionaries Building and using canonical XML components dictionaries.
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 8 Slide 1 System modeling 2.
Understanding the Landscape and Opportunities David Webber, Information Architect, Oracle Public Sector Briefing for Japan METI Team on NIEM, Open Data.
NIEM, CAM and the 7 “D’s” David Webber - Public Sector NIEM Team, November 2011 NIEM Test Model Data Deploy Requirements Build Exchange Generate Dictionary.
Semantics and Information Exchanges Overview – Public Sector NIEM Team, June 2011 CAM Test Model Data Deploy Requirements Build Exchange Generate Dictionary.
SRDC Ltd. 1. Problem  Solutions  Various standardization efforts ◦ Document models addressing a broad range of requirements vs Industry Specific Document.
David Webber, NIEM Team, Oracle Public Sector NIEM Test Model Data Deploy Requirements Build Exchange Generate Dictionary Exchange Development Taking a.
Building Enterprise Applications Using Visual Studio ®.NET Enterprise Architect.
©Ian Sommerville 2006Software Engineering, 8th edition. Chapter 8 Slide 1 System models.
Supplement 02CASE Tools1 Supplement 02 - Case Tools And Franchise Colleges By MANSHA NAWAZ.
XML Exchange Development CAM Technology Tutorial – Public Sector NIEM Team, June 2011 CAM Test Model Data Deploy Requirements Build Exchange Generate Dictionary.
MTEI Methods & Tools for Enterprise Integration
1 1 Roadmap to an IEPD What do developers need to do?
Open Data API delivery “Open-XDX” David Webber, Information Architect, Oracle Public Sector Open Data Exchange.
NIEM Team, Oracle Public Sector Technology Introduction – September 2011 CAMV Test Model Data Deploy Requirements Build Exchange Generate Dictionary Exchange.
Open Data API delivery “Open-XDX” David Webber, Information Architect, Oracle Public Sector Open Data Exchange October, 2012.
Technical Introduction to NIEM
The Role of XML in Cloud Data Integration Presenter: David RR Webber, Oracle Corporation October 15th, 2010.
1 Dictionary Driven Exchange Content Assembly Blueprints Concepts, Procedures and Techniques (CAM – Content Assembly Mechanism Specification) Author: David.
David Webber, NIEM Team, Oracle Public Sector NIEM Test Model Data Deploy Requirements Build Exchange Generate Dictionary Exchange Development NIEM, CAM,
David Webber, NIEM Team, Oracle Public Sector NIEM Test Model Data Deploy Requirements Build Exchange Generate Dictionary Exchange Development Taking a.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 1 Quick Tutorial – Part 1 Using Oracle BPM with Open Data Web Services David Webber.
NIEM and Information Exchanges Overview – Public Sector NIEM Team, December 2011 NIEM Test Model Data Deploy Requirements Build Exchange Generate Dictionary.
1Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8 Reporting from Contract.
James Cabral, David Webber, Farrukh Najmi, July 2012.
NIEM and Rules David Webber – Public Sector NIEM Team, November 2011 NIEM Test Model Data Deploy Requirements Build Exchange Generate Dictionary XML Exchange.
1 1 Roadmap to an IEPD What do developers need to do?
NIEM, CAM and the 8 “D’s” David Webber - Public Sector NIEM Team, December 2011 NIEM Test Model Data Deploy Requirements Build Exchange Generate Dictionary.
Chapter 4 System Models A description of the various models that can be used to specify software systems.
Delivering NIEM-aligned data services David Webber, Information Architect, Oracle Public Sector NIEM and XML for Architects and Developers.
UN/CEFACT Forum Wednesday, 16 March 2005 Lunch & Learn ATG XML NDR Mark Crawford ATG2 Chair U NITED N ATIONS C ENTRE F OR T RADE F ACILITATION A ND E LECTRONIC.
Introduction to MDA (Model Driven Architecture) CYT.
NIEM Blue Team Presentation April 20, 2010 Phil Letowt, Mini Kanwal, Ken Sall, David Webber ICE OCIO / Task ASAS ICE Information Exchange Reuse with NIEM.
9/11 - Tens Years On Overview – Public Sector NIEM Team, October 2011 NIEM Test Model Data Deploy Requirements Build Exchange Generate Dictionary XML Exchange.
1 Quick Guide to CAM Dictionaries Building and using Canonical XML dictionaries for CAM (CAM – Content Assembly Mechanism Specification) Author: David.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 1 Quick Tutorial – Part 2 Open Data Web Services for Oracle BPM August, 2013 Forms.
David Webber, NIEM Team, Oracle Public Sector Rapid NIEM XML Exchange Design, Semantics and UML Models NIEM Test Model Data Deploy Requirements Build Exchange.
1Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8 Contract Management.
1 Twitter #NIEMNTE3 Lightning-Fast IEPD Development Techniques Presenter David RR Webber Information Architect Oracle Public Sector.
Interfacing Registry Systems December 2000.
Public Sector NIEM Team, December 2011 NIEM Test Model Data Deploy Requirements Build Exchange Generate Dictionary XML Exchange Development National Information.
Development Process and Testing Tools for Content Standards OASIS Symposium: The Meaning of Interoperability May 9, 2006 Simon Frechette, NIST.
Chapter 7 System models.
DEV337 Modeling Distributed Enterprise Applications Using UML in Visual Studio.NET David Keogh Program Manager Visual Studio Enterprise Tools.
1 Quick Guide to CAM Blueprints Using blueprints to develop XML exchange templates and schema (CAM – Content Assembly Mechanism Specification) Author:
1 1 CAM Toolkit for NIEM IEPD Development Phil Letowt – DHS/ICE David Webber – ICE Data Architect.
NIEM Information Exchange Package Documentation (IEPD) Mini Kanwal NIEM Technical Advisor Department of Homeland Security September, 7 th 2006.
NIEM WHAT IS IT REALLY?. XML ORB: VIEWS & INSIGHTS David Webber XML savant blog: XMLOrb CAMeditor.org.
“This presentation is for informational purposes only and may not be incorporated into a contract or agreement.”
U NITED N ATIONS C ENTRE F OR T RADE F ACILITATION A ND E LECTRONIC B USINESS Under the auspices of United Nations Economic Commission for Europe UN/CEFACT.
Dictionary based interchanges for iSURF -An Interoperability Service Utility for Collaborative Supply Chain Planning across Multiple Domains David Webber.
Leveraging UBL for Developing Justice XML (GJXDM) Reference Documents John Ruegg County of Los Angeles Information Systems Advisory Body GJXDM User Conference.
® IBM Software Group © 2007 IBM Corporation Module 1: Getting Started with Rational Software Architect Essentials of Modeling with IBM Rational Software.
Manufacturing Systems Integration Division Development Process and Testing Tools for Content Standards Simon Frechette National Institute of Standards.
Viewpoint Modeling and Model-Based Media Generation for Systems Engineers Automatic View and Document Generation for Scalable Model- Based Engineering.
1 Copyright © 2008, Oracle. All rights reserved. Repository Basics.
Computer Aided Software Engineering (CASE)
The Re3gistry software and the INSPIRE Registry
GJXDM Tools Team Georgia Tech Research Institute
Using JDeveloper.
NIEM Tool Strategy Next Steps for Movement
SDMX IT Tools SDMX Registry
Presentation transcript:

Dictionaries Tutorial Overview – Public Sector NIEM Team, November 2012 CAM Test Model Data Deploy Requirements Build Exchange Generate Dictionary XML Exchange Development Content Assembly Mechanism

Copyright ©2011/12, Oracle. All Rights Reserved. The following is intended to outline Oracle general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Disclaimer Notice Slide 2

Copyright ©2011/12, Oracle. All Rights Reserved. Tutorial Overview Dictionary Concepts Dictionary Services and Facilities Examples Walkthrough Summary and Review

Copyright ©2011/12, Oracle. All Rights Reserved. DICTIONARY CONCEPTS Understanding Dictionaries Usage

Copyright ©2011/12, Oracle. All Rights Reserved. The Dictionary Role and Content Provide a set of common, controlled, and managed information data structures and component definitions for a particular purpose and community Data elements are organized into core and domain-specific components Core components are used across information exchanges and can be described by structure, semantics, and definition with global context Context-specific components are particular to a limited number of information exchanges and have a limited scope and applicability Naming and Design Rules (NDR) specify how each of these components are defined and utilized

Copyright ©2011/12, Oracle. All Rights Reserved. The 8 “D”s of software development Design Develop Document Dictionaries Discovery Differentiate Deploy Diagnose Predictable, Repeatable, Reusable Process Solution Delivery Process LIFECYCLE

Copyright ©2011/12, Oracle. All Rights Reserved. XML Samples DictionariesDictionaries DiscoveryDiscovery Exchange Delivery Lifecycle DiagnoseDiagnose DifferentiateDifferentiate DesignDesign DevelopDevelop Validated Templates / Schema Validated Templates / Schema XSD Schema Evaluation Reports XMI / UML Models Production Results Production Results Requirements Updates Requirements Updates Documentation Drag and Drop Visual Designer DocumentDocument DeployDeploy Exchange Templates Exchange Templates CAM editor Toolkit *CAM – Content Assembly Mechanism – OASIS standard

Copyright ©2011/12, Oracle. All Rights Reserved. CAM Dictionary Framework CAM Editor Drag and Drop Designer Collection Manager CAM Rules Wizards Model Generation Exchange Structure Builder Dictionary Codelist Examples and instructions: Codelist

Copyright ©2011/12, Oracle. All Rights Reserved. How to create dictionaries Top down - directly from information models Enterprise modelling tools SQL database physical table definitions Bottom up Harvesting from existing data assets Harvesting from XML exchange structures and schema Manual entry through Excel spreadsheets Using neutral abstract standards based representations

Copyright ©2011/12, Oracle. All Rights Reserved. Dictionary Content Model Child (BBIE) Item Child (BBIE) Item Parent (ABIE) Item Parent (ABIE) Item Follows Naming and Design Rule (NDR) principles and guidelines Canonical Components Dictionary XML Canonical Components Dictionary XML ebXML CCTS terms (ABIE, BBIE, ASBIE) Parent = Aggregate Business Information Entity Child = Basic Business Information Entity Attribute = Association Business Information Entity Parent (ABIE) Item Parent (ABIE) Item Parent (ABIE) Item Parent (ABIE) Item Parent (ABIE) Item Parent (ABIE) Item..... Child (BBIE) Item Child (BBIE) Item Child (BBIE) Item Child (BBIE) Item Child (BBIE) Item Child (BBIE) Item Attribute (ASBIE) Attribute (ASBIE) Attribute (ASBIE) Attribute (ASBIE) * CCTS – Core Components Technical Specification Attribute (ASBIE) Attribute (ASBIE) Attribute (ASBIE) Attribute (ASBIE) Each compound component Each atomic component Optional attributes of component

Copyright ©2011/12, Oracle. All Rights Reserved. Example – Person Name Person Name (ABIE) Verified Details? (ASBIE) First Name (BBIE) Middle Name (BBIE) Last Name (BBIE) Previous Name? (ASBIE) Language Code (BBIE) Language Code may exist independently of Person Name Verified Details and Previous Name are flags that denote additional information about the entity they are associated with There are three component items aspects: structure relationships; content rules; definitions

Copyright ©2011/12, Oracle. All Rights Reserved. Semantics Approach Neutral formatted XML dictionaries Contains core information on XML components Representation subset of UN/CEFACT Core Components Technical Specification (CCTS) Can generate XSD, OWL, XMI and more as desired (contains the vocabulary terms, definitions, purpose and content models) Renamer tool aligns with NIEM NDR

Copyright ©2011/12, Oracle. All Rights Reserved. Value Proposition Allow business data analysts to focus on information needs and build data exchanges Aligns information with industry standards and enterprise information stores Ensures consistent usage and definitions using profile of syntax-neutral terms and constructs Components can be derived from existing domain schema and data structures Supports providing sharing and collaboration services Allows development of supporting analytics tools

Copyright ©2011/12, Oracle. All Rights Reserved. Sources of Your Dictionaries Ingest existing application semantic artifacts – e.g. XML Schema, Database schema, Excel spreadsheets, Enterprise Data Models (EDM) Starting from scratch: Excel spreadsheet, or SQL table entry, or XML components entry Then generate dictionaries of components Download existing domain dictionary collection

Copyright ©2011/12, Oracle. All Rights Reserved. Renaming Tool Expands abbreviations (has built-in lists with context) Corrects common typos and spelling errors (200+) Adjusts names to camel case with no separators (_) Retains acronyms Adds name suffix according to NIEM representation terms and type of content, e.g. date, number, code, etc Optionally refactor rules by their type for interoperability; make nillable optional; relax DB length restrictions Can optionally insert namespace prefix Can be configured by editing the lists in: word-utils.xsl

Copyright ©2011/12, Oracle. All Rights Reserved. Analytics Tools Compare to Dictionary tool – provides reuse scoring statistics for information exchange structures Template Evaluation Tool – Provides Naming and Design Rule (NDR) analysis Spelling Checking Information exchange interoperability analysis Data modelling errors and warnings Naming / use inconsistencies / omissions

Copyright ©2011/12, Oracle. All Rights Reserved. CAM: Top Down Exchange Assembly Canonical XML Components Dictionary 1 1 Component Associations and Couplings 2 2 Exchange Templates and Rules 3 3 W3C Schema and Model Representations 4 4 Delivery Control, Messaging, Security 5 5 Presentation Collaboration Services Artefact relationships Implementation Artifacts and Examples 6 6 Navigation and Query Tools Domains Reuse Library Components Automatic Rendering Tools Deployment Environments and Middleware XML User Inserts Relationship Lookups Template Catalog Testing Workbench, Rules Engine, Data Samples, Integration ETL Schema XML Models Components Canonical Dictionary Collections *CAM – Content Assembly Mechanism – OASIS standard

Copyright ©2011/12, Oracle. All Rights Reserved. DICTIONARY SERVICES AND FACILITIES Tools Architecture

Copyright ©2011/12, Oracle. All Rights Reserved. Dictionary Management / Acquisition

Copyright ©2011/12, Oracle. All Rights Reserved. CAM Dictionary From Existing XML schema OASIS CAM template Source schema in XSD syntax Source schema in XSD syntax NDR Evaluation, Refactor, Renaming Tools NDR Evaluation, Refactor, Renaming Tools 3 3 Generate Standard Components Dictionary XML Generate Standard Components Dictionary XML 4 4 Canonical XML Components Dictionary XML Import Apply Build

Copyright ©2011/12, Oracle. All Rights Reserved. CAM Dictionary From ERwin models EDM Ele Def DDL 1 1 Export Components in XSD syntax Collection of objects from model From Enterprise Data Model Import XSD and refactor for use with CAM editor OASIS CAM template Model Components XSD schema NDR Evaluation, Refactor, Renaming Tool NDR Evaluation, Refactor, Renaming Tool Apply Naming and Design Rule (NDR) checks and edits 4 4 Generate Standard Components Dictionary XML Generate Standard Components Dictionary XML 5 5 Dictionary of exchange components XML Import 3 3 Analyst Review ebXML CCTS compatible (ABIE, BBIE, ASBIE) 2 2

Copyright ©2011/12, Oracle. All Rights Reserved. CAM Dictionary From SQL tables 1 1 SQL tables Drag and Drop Designer from SQL to XML Configure renamer drag and drop XML mapping 4 4 Generate Dictionary from XML components

Copyright ©2011/12, Oracle. All Rights Reserved. CAM Dictionary From Excel Spreadsheet 1 1 Dictionary Generator Spreadsheet XML data mapping Export to XML results Download Excel to Dictionary pack Enter components README has complete instructions

Copyright ©2011/12, Oracle. All Rights Reserved. Renamer Tool – Align to XML NDR -renamer added by default – otherwise edit name as desired Leave blank or pick namespace to process NIEM mode (with term representations); or CamelCase (Java style) Leave as normal Optionally insert prefix to all items without prefix Set to false to retain all rules “as is” Note: Produces renamer report XML file of old / new names. Can be reviewed as spreadsheet.

Copyright ©2011/12, Oracle. All Rights Reserved. Dictionary Collection Manager 1 1 dictionary collection control file Collection name Component dictionaries Download Excel to Dictionary pack 4 4 Editor preference

Copyright ©2011/12, Oracle. All Rights Reserved. Available XML Dictionaries NIEM 2.1 dictionariesNIEM 2.1 dictionaries CBRN dictionary Emergency dictionary Family dictionary Immigration dictionary Infrastructure dictionary Intelligence dictionary Justice dictionary Maritime dictionary Screening dictionary Trade dictionary Immigration blueprint NIEM core dictionary Available from download site direct link: XML + includes spreadsheets and sample models Note: Those marked in bold are model style dictionaries with recursive components.

Copyright ©2011/12, Oracle. All Rights Reserved. EXAMPLES WALKTHROUGH Understanding capabilities

Copyright ©2011/12, Oracle. All Rights Reserved. Visual Designer with Dictionary Collection Drag n’ Drop Tool Industry dictionary Domain dictionary Component Definitions 4 4 Search Tools Search Tools 2 2 Exchange Designer Exchange Designer 1 1 Insert Dictionary Parent Components Insert Dictionary Parent Components 3 3 Completed Exchange Template 5 5 Collection

Copyright ©2011/12, Oracle. All Rights Reserved. Developing domain dictionaries (EIEM) Allows domains to manage their components libraries Provides consistency for project development teams Sets of NIEM consistent XML exchange components Aligned to enterprise data stores Optimized for reuse and interoperability Save time and effort across the enterprise Perennial question for developers – when should I use NIEM components, and when our own local ones? Are there components already available for that purpose? Provide formal mechanisms and procedures to share components and collaborate across SDLC process Provide external parties consistent data views *EIEM – Enterprise Information Exchange Model CAM Toolkit for EIEM generation

Copyright ©2011/12, Oracle. All Rights Reserved. Example: N-DEx Dictionary and Model (Harvested from N-DEx schema – ndexia.xsd and ndexibp2.xsd) Freemind Interactive Model Dictionary viewer with N-Dex components

Copyright ©2011/12, Oracle. All Rights Reserved. Example - Suspicious Activity Report V2.0 dictionaries XML SAR v1.5 components NIEM core dictionary LEXS dictionary SAR v1.5 components NIEM core dictionary LEXS dictionary LEXS components referenced New structure components based on NIEM + SAR + new SAR components Definitions stored as syntax neutral canonical XML NIEM core components Dictionary Collection Namespaces of dictionary components

Copyright ©2011/12, Oracle. All Rights Reserved. SUMMARY AND REVIEW Reviewing Dictionary Technology

Copyright ©2011/12, Oracle. All Rights Reserved. Opportunities / Challenges Paradigm shift Teams used to manually crafting XML Schema and Java code in developer XML IDE tools Collaboration between developers and data analysts Lowering learning curve for application development teams Integration into SDLC and NIEM IEPD processes Unit test XML instance generation XML test framework and validation services

Copyright ©2011/12, Oracle. All Rights Reserved. Work in Progress NIEM ‘lite’ – easier simpler use by adding context aware mechanisms to automatically filter dictionary components based on domain use case Expose context lists to allow click and choose selections Dictionary Analytics tools – find out what is going on inside your dictionaries – consistency, omissions, content/name conflicts, orphans Store and Share Collaboration - API with SOA Suite Repository integration

Copyright ©2011/12, Oracle. All Rights Reserved. Value Proposition Contextual mechanisms allow linear number of true core components – not exponential growth Easier for practitioners to leverage with more predictable and repeatable results Enable domain use horizontally across government ETL mapping automation support Continuous collaboration environment

Copyright ©2011/12, Oracle. All Rights Reserved. CAMeditor.ORG Project Statistics ‹#› SNAPSHOT OF PROJECT ACTIVITIES 50,000 CAMeditor.org page visits in one year 140 countries have downloaded tools; 25% of visitors are from U.S.; 600+ downloads weekly 50 views of online video training resources weekly 6 languages now available