NIEM Tool Strategy Next Steps for Movement NIEM Technical Architecture Committee Sept 18 2018

Long term vision What do we want from a tool architecture for NIEM, in the long term? Open source & user-installable. accreditation in DOD and other environments. Modular to support incremental improvments. Support a self-service model. Provide capabilities that satisfy the needs of the community and that aren't available from other tools.

Tools SSGT Movement Contesa / NIEM Conformance tool Code list generation tool

Movement: Goals Should make the NIEM model more comprehensible Should be accessible to a non-technical user (e.g., an easy mode tool) Should do what it says it does If it claims to satisfy JSON or subset requirements, then it should do so, or the claim should be redacted

Movement: Capabilities Search Top-down browsing Open source & user-installable

Movement: Issues Can only select simple content Generated subset is not suitable for a purpose The JSON Schema it generates doesn't provide a JSON capability for defining an exchange Does not read or write in a way compatible with the SSGT Does not generate what the SSGT generates Limitations on browsing Can't browse across types Can't browse across base types Can't browse across uses of an element Only supports a single version of the model Users assume using Movement will satisfy JSON requirements for an exchange, which is not true

Suggested Course of Action Leave the current Movement capability Deployed on beta.movement.niem.gov. Stand up a separate IOC Movement capability that omits JSON claims and subset selection Deploy on movement.niem.gov Improve Movement to provide additional capability

COA Step 1: Leave current Movement Movement is currently deployed on beta.movement.niem.gov Keep this deployment location for the current development version of Movement e.g., a nightly build of Movement Don't direct new users to the beta The beta is for experienced users who want new and experimental features Add text on beta.movement.niem.gov that clarifies it is a development version, and directs users to movement.niem.gov

COA Step 2: Create an IOC Movement Deploy a new Movement instance on movement.niem.gov Will contain Movement without: Add to subset buttons Generate subset button Reduce the claims it makes about satisfying JSON requirements This limits IOC Movement to a search tool. Will enable us to direct users to Movement for search, without explaining or apologizing for misleading JSON and subset claims Let it do what it does best: search NIEM Update NIEM webpages to clarify what Movement does includes niem.github.io/movement, niem.gov/about-niem/news/introducing- movement-easier-simpler-way-explore-and-use-niem

COA Step 3: Improve Movement to provide additional capability UI improvements: Improve component display & navigation so you can find context & uses of properties & see and navigate through types. Allow addition of types and complex elements to the subset Enable Movement to generate subsets using a web service common to it and the SSGT. Would allow building NIEM subset schemas directly from the the Movement tool. Would allow Movement and SSGT (and other tools that use the web service) to build XML Schema & JSON Schema, when that functionality is developed.

Current State: SSGT & Movement “Easy-mode” Tool (Movement) User Interface JSON Schema NIEM v4.1 JSON Schema Generator “Expert-Mode” Tool (SSGT) Want List XML User Interface This is the current state Restricts innovation Hard to maintain consistency across tools Can't use definitions across the tools; they're isolated / siloed Can only do JSON Schema on one tool, only XML Schema on the other Broken workflow Have to use "expert mode" tool for XML Schema Forces you to use both tools to do XML Schema & JSON Schema Required redo of work to use both tools Architecture outdated: Not scalable Only 1 web service exposed (XSD generation), and it's SOAP Bottom line: existing tools have good capability, BUT... NIEM Releases XML Schema Generator ZIP file Generator XML Schema ZIP

Phase 1: Modularize Core Services “Easy-mode” UI (Movement) Model Description JSON Schema Generator Model Source NIEM Releases ZIP file Generator XML Schema Generator NIEM Data Model Software Development Develop “Model Description” format Deploy NIEM Data Model Service Develop & Deploy Subset XML Schema Generation Service Develop & Deploy Subset JSON Schema Generation Service Refactor Movement Tool Refactor SSGT MVP = minimal viable product This is the 1st stage of updating the tools. Base-line, near-term. Opens / provides core capabilities to both tools and external tools functionality is consistent across all tools Provides interoperability across all tools Extensible: provides for an ecosystem of capabilities Enables new providers to innovate and provide additional functionality It's a platform for new capabilities Readmap / intuitive. Separates complex tools into separate components that are simpler and serve a single purpose Simplifies users experience with NIEM: multiple paths: easy or expert. REST Query Model Description “Expert-mode” UI (SSGT)

Phase 2: Improved Function via Additional Services “Easy-mode” UI (Movement) NIEM Releases Model Description JSON Schema Generator Git User's EIEM / IEPD / Extensions ZIP file Generator XML Schema Generator Model Source Documentation Generator Long-term: vision / roadmap Could be other tools that use these same services Ecosystem Marketplace of providers 2 kinds of tools: 1. self-contained / locally-installable 2. Using services in a build process 3. web app using web services Show how to connect in new things: Generator services Search services Applications calling APIs Locally installed components Can add ingestion / model source of other kinds of artifacts: UML STIX EDXL Spreadsheets Less training required – support production of artifacts the developers/users are familiar with. could add search module inputs data definitions XML Schemas Databases selections extensions rules outputs schemas diagrams spreadsheets zip files with stuff in it REST Query Domain Updates Model Description “Expert-mode” UI (SSGT)