1 1 Roadmap to an IEPD What do developers need to do?

2 2 Agenda IEPD Deliverables NIEM IEPD – Base Components Needed Top Down or Bottom Up? Tools and XML Technology How do these tools all play together? Dictionaries – NIEM / Domain Reference Materials Illustrative Examples Resources and Contacts

3 3 IEPD Base Components XML Schema Documentation (Project business details ( MS Word ), NIEM cross-reference Spreadsheet ) XML artifacts ( wantlist.xml, subset schema. Optional - extension, restriction and reference schemas ) Examples & Test Cases (Realistic test XML instance(s) for exchange interoperability testing )     Required items Exchange Schema XSD (aligns to information model, NIEM core plus domain dictionary and ICE NDR principles)

4 4 Schemas for data exchange To produce “exchange schemas” which offer a high degree of clarity, interoperability and reusability while being agnostic to the actual deployment software tools. Documentation for understanding business context and usage Supporting XML Artifacts Examples and test cases for verification Cross-reference to NIEM dictionary to ensure consistency, interoperability and reuse within and across domains Alignment to domain data model components dictionary – interoperability across domain applications Consistency: conforms to domain naming and design rules principles Technical Goals

5 5 XML, XSD, WSDL XSD Wantlist IEPD Development Overview CAM toolkit ConTesA SSGT IEPD Repository Conformance Tool NIEM dictionary Microsoft Excel Microsoft Word Exchange XSD IEPD Package Top Down Bottom Up ICE EDM dictionary CAM = Content Assembly Mechanism Subset Schema Generation Tool Testing Tool

6 6 IEPD – Task Breakdown Component TasksTimingsConstraints Collect exchange needs Perform XSD schema development with NIEM alignment Check NIEM NDR conformance and ICE principles and rules alignment Documentation of each element Document NIEM element alignment (want list) – compare to dictionary report Create test cases and examples Perform interoperability testing Create IEPD document Model information needs XSD syntax writing Desk check conformance Produce evaluation report HTML report formats Excel spreadsheet + Wantlist.xml Sets of XML instances Build test environments Word documentation Days Hours Minutes Hours Days Assisted analysis Reduce need to know detailed schema syntax. Automatic generation + manual desk check Automatic generation + manual desk check Automatic + content hints Test harnesses consistent Manual preparation When requirements change – Regenerate Automatically!

7 7 Tools and XML Technologies How do these pieces all play together?

8 8 NIEM Exchange Schema Exchange xml *Code list xsd Namespace * Optional XML Schema xsd subset *Constraint xsd *Extension xsd *Restriction xsd *Reference xsd NIEM domain schemas Code list xsd Wantlist xml LEGEND: Exchange XSD

9 9 Conceptual Process Overview Def Ele NIEM / ICE dictionaries formatted as XML CAM toolkit Automated Manual LEGEND Definitions Repository (XML) Domain dictionary formatted as XML Build Exchange Designer Tool User Interface Review Structure Expand Assembly Pick Components Structure Outline DHS/ICE applications EDM Ele Def DDL IEPD Package IEPD Package Exchange Components Exchange Components

10 Top Down Approach - Steps Overview Expander Tool Expander Tool NIEM dictionary Domain dictionary Component Definitions 4 4 SSGT Excel Searc h Tools Searc h Tools 2 2 Exchange Designer Exchange Designer 1 1 Pick Top Level Nodes Outline Pick Top Level Nodes Outline 3 3 Completed Exchange steps in the process; leverages existing NIEM search tools + dictionaries

11 Vehicle Collision Report Example NIEM Core Your node names outline Structure Details Inserted Rules Inserted ICE EDM Lookups Dictionaries Expander Tool Expander Tool COMPLETED EXCHANGE TOP LEVEL DESIGN

12 Top Down – Desktop Editor Exchange Structure items Rules for each item Domain Components Namespaces Can add new custom domain Items Build New Exchange from Dictionary pieces

13 Export Template to Exchange XSD Completed Exchange Structure Set Exchange Options Complete set of NIEM exchange schemas generated Export Template to Exchange XSD:

14 Exchange Schema Generated Each namespace file is import for those specific type definitions Reviewing XSD results in a schema editor tool Set of XSD files with filename and namespace suffix

15 IEPD Development Summary PROCESS Toolkit for IEPD base components generation Suite of menu options and tools in CAM toolkit editor (run custom xslt scripts on CAM template xml) Structure Rules Documentation CAM Template 1.Run template Evaluation Report 2.Compare to NIEM dictionary - create spreadsheet - create wantlist.xml - NIEM online SSGT uploading + subset schema generation Generate business rules report 3.Generate XML instance(s)

16 NIEM Considerations Naming and Design Rules (NDR) ICE Principles and Rules IEPD Packaging

17 NDR evaluation example Part of an example report for LEXS getDataItemRequest template displayed in toolkit HTML viewer

18 ICE Principles and Rules Schema Conformance (subset of NIEM rules) Naming and Definition Conventions Element and Attribute Prohibitions Namespace Usage Versioning Reuse Techniques Interoperability Enablers/Inhibitors Documentation available in ELMS

19 Formal IEPD Packaging Details IEPD ArtifactDescription Exchange Files Subset Schema (SSGT)Subset of the full NIEM schema—a compressed directory of schemas to distinguish from other schema sets. Wantlist User requirements -distinguishes user-required data components from components they depend on for conformance. Generated by and up-loaded to the Schema Subset Generation Tool (SSGT). This is an open specification and the SSGT is not required to create a wantlist, though it is easier. Exchange Schema Base document schema that defines the XML root element and is generally named after the IEPD itself. Also known as the document schema, reference schema, or root schema. Constraint SchemaConstraints for separate constraint validation path—a compressed directory of schemas to distinguish from other schema sets. Extension SchemaSpecification for extended components—separate local name-spaces of components not contained in NIEM. Sample XML InstanceExample instance(s) – may reference optional stylesheet. Style sheetExample stylesheet for display of instance(s). Documentation Master Documentation The Master Document is the main document for which all of the context and details around the exchange are explained. This document includes, the overview, as well as details surrounding the exchange, business drivers and requirements Exchange model (import to ArgosUML) Exchange model in standard open format (xmi, vsd, zargo) and standard open graphic (jpg, pdf, etc.) preferably a Unified Modeling Language (UML) model. Business rules Business rules in one of the following formats: (1) plain or structured English, (2) written into master documentation, (3) Schematron or other formal business rule language, or (4) generated by a development tool. Mapping to NIEM Mapping of domain components, tagged with constraints (i.e., cardinality, etc.) to NIEM components; preferably the Component Mapping Tool (CMT). Extended components Components created because they were not in NIEM—may be part of mapping spreadsheet and include structure and definitions of new components. Prefer CMT. Change logRecord of cumulative changes from previous IEPD versions. The initial IEPD simple records its creation date. Catalog Catalog XML fileA machine-readable list of artifacts in the IEPD in an open, portable and browser-displayable format (simple manual edit). Metadata XML fileAll metadata of owner and domain to be registered with the IEPD (simple manual edit).

20 Bottom Up Development Leveraging existing schema:

21 Approach in a Nutshell INGEST Structure Rules Documentation XSLT tools process entire XSD collection XSLT tools process entire XSD collection CAM = Content Assembly Mechanism XSD Collection Re-factor to CAM abstraction format CAM Template *Code list xsd Namespace * Optional XML Schema xsd subset *Constraint xsd *Extension xsd *Restriction xsd NIEM domain schemas Code list xsd LEGEND: Exchange xsd

22 Localization Approach Steps LEXS XML XSD XML XSD Model Exchange Manage develop 1 1 ingest 2 2 Base CAM Template Base CAM Template localize 3 3 Revised CAM Template Revised CAM Template Generate 4 4 Local XML XSD Local XML XSD Compare 5 5 NIEM/LEXS Dictionary NIEM/LEXS Dictionary Steps: 1. LEXS develops search query and publish process XSD schema 2. Ingest the standard XSD into base CAM template 3. Apply localizations to CAM template  Exclude unwanted optional components  Extend for local needs, add new components were permitted 4. Generate new localized XSD schema 5. Build dictionary cross-reference of localization usage 6. Publish documentation and XML examples as IEPD Publish 6 6 Documentation + Examples Documentation + Examples

23 Bottom Up – Refine Model Add / Refine Structure, Rules, Documentation Structure Rules Documentation Existing LEXS exchange

24 Run Template Compress tool Compress: globally remove unnecessary NIEM model items

25 LEXS Exchange Model (UCORE) Request Response LEXS and UCORE model of information exchange components Toolkit can provide templates of these for rapid development

26 LEXS Exchange Template Outline Template Outline matches LEXS model Expander Builds LEXS Exchange Console Log detail

27 Summary Top Down development Bottom Up support Dictionary driven reuse Automated IEPD generation Alignment to ICE Principles and Rules

28 Review Top Down development Use of NIEM, ICE EDM and LEXS dictionary components Create exchange outline Run Expander tool Refine desired structure in visual editor Bottom Up development Ingest existing XSD schema Refine structure model Produce IEPD components Dictionary driven reuse Enterprise Data Model and NIEM/LEXS components ICE Principles and Rules Best practices for interoperability and reuse

29 Reference Materials Illustrative Examples References and Contacts

30 NIEM cross-reference spreadsheet Formatted view in Microsoft Excel of import of cross- reference report details (from generated XML file)

31 NIEM online SSGT and wantlist.xml Options menu and Load Wantlist option with resulting load of subset details

32 Business Rules Documentation Part of the example rules for LEXS getDataItemRequest template displayed in toolkit HTML viewer

33 XML example generation wizard View of CAM toolkit with LEXS getDataItemRequest and dialogue for XML test example generator tool

34 Phil Letowt Mini Kanwal David Webber Ken Sall Forthcoming: ICE Data Management Working Group mailbox Points of Contact

35 Links and References XML Schema Part 1: Structures and XML Schema Part 2: Datatypes (and Part 0: Primer) – We are using XML Schema version NIEM Naming and Design Rules (NDR) NIEM Practical Implementer’s Course (online) NIEM Course Modules (17 free PDFs) Techniques for Building and Extending NIEM XML Components NIEM 2.0 Conformant LEXS 3.1: Logical Entity eXchange Specifications ndard=486 ndard=486 IEPD Preparation Guide – DRAFT (To be renamed CAM Toolkit User Guide for IEPD Development) CAM Toolkit download