IPDA PDS4 Report PDS Team July 2015

PDS4: The Next Generation PDS PDS4 is a PDS-wide project to upgrade from PDS version 3 to version 4 to address many of these challenges An explicit information architecture All PDS data tied to a common model to improve validation and discovery Use of XML, a well-supported international standard, for data product labeling, validation, and searching. A hierarchy of data dictionaries built to the ISO 11179 standard, designed to increase flexibility, enable complex searches, and make it easier to share data internationally. An explicit software/technical architecture Distributed services both within PDS and at international partners Consistent protocols for access to the data and services Deployment of an open source registry infrastructure to track and manage every product in PDS A distributed search infrastructure

Challenge: End-to-End System and Data Integration Core PDS Data Providers Transform Ingest PDS Data Management Distribution Transform Users Improve efficiency and support to deliver high quality science products to PDS Preserve and ensure the stability and integrity of PDS data Improve user support and usability of the data in the archive

Information Architecture Concepts Product Information Model Tagged Data Object (Information Object) Used to Create <Array_2D_Image>     <local_identifier>MPFL-M-IMP_IMG_GRAYSCALE…     <offset unit="byte">0</offset>     <axes>2</axes>     <axis_index_order>Last Index Fastest…     <Element_Array>         <data_type>UnsignedMSB2</data_type>         <unit>data number</unit>     </Element_Array>                <Axis_Array>         <axis_name>Line</axis_name>         <elements>248</elements>         <unit>not applicable</unit>         <sequence_number>1</sequence_number>     </Axis_Array>     <Axis_Array>         <axis_name>Sample</axis_name>         <elements>256</elements>         <sequence_number>2</sequence_number> </Array_2D_Image> Label Schema Validates Expressed As Extracted/Specialized Data Element Class has Planetary Science Data Dictionary Describes Data Object

System Design Approach Based on a distributed information services architecture (aka SOA-style) Allow for common and node specific network-based (e.g., REST) services. Allow for integrating with other systems through IPDA standards. System includes services, tools and applications Use of online registries across the PDS to track and share information about PDS holdings Implement distributed services that bring PDS forward into the online era of running a national data system With good data standards, they become critical to ultimately improving the usability of PDS Support on-demand transformation to/from PDS Client B Client A Service C Service Interface

Summary of Progress to Date Requirements in place (approved by MC 3/2010) PDS-wide Architecture defined Major reviews conducted PDS4 products generated for LADEE and MAVEN missions System builds grouped by purpose: build 1,2,3, 4, 5 Used to establish a rigorous release process Includes deployment of software, data standards, documents and system integration testing As stability has increased, we have moved to 6 month builds Operational capabilities deployed Registry and Harvest infrastructure in place at EN and many PDS nodes Central catalog migrated to registry; High level search migrated Information model formally released with each build PDS3 catalog data now all in PDS4 to link to PDS3 LADEE and MAVEN ingested for search/access from pds.nasa.gov

PDS4 Software Development Lifecycle NASA Software Management Guidebook – NASA-GB-001 Evaluation by users as early as possible!

Project Lifecycle – mapped to NPR 7120.8 per SDR1 Pre- Formulation Formulation Implementation KDP: Study KDP: Project Plan & Arch KDP: Prelim Design KDP: Release LADEE/ MAVEN Build 1: Prototype build Project Lifecycle Gates & Major Events 1a 1b 1c 1d Build 2: Release to LADEE/MAVEN Begin Study Project Project Plan PDS4 Prelim Architecture PDS4 Design Study/ Concepts Build 3: V1.0 of PDS4 Stsandards PDSMC Preliminary Design (August 2009) Build 1b Internal Stds Assessment (Dec 2010) PDS System Review II (June 2011) Operational Readiness Review (LADEE/MAVEN) (November 2011) Project Reviews PDSMC Concept Review (Dec 2007) PDSMC Impl Review (July 2008) PDSMC Arch Review (Nov 2008) PDS System Review (Mar 2010) Build 1c IPDA Stds Assessment (April 2011) Build 1d External Stds Assessment (Aug 2011) Phoenix Beta Test (Dec 2012)

PDS4 Planned Mission Support LADEE (NASA) InSight (NASA) BepiColumbo (ESA/JAXA) MAVEN (NASA) Osiris-REx (NASA) ExoMars (ESA/Russia) JUICE (ESA) …also Hyabussa-2, Chandryaan-2 Endorsed by the International Planetary Data Alliance in July 2012 – https://planetarydata.org/documents/steering-committee/ipda-endorsements-recommendations-and-actions

PDS3 Implementation PDS3 Pipeline PDS3 Ingest PDS3 Central Catalog Current Missions PDS3 Archive @ DNs Datasets + Products PDS3 Pipeline PDS3 Ingest PDS3 Central Catalog PDS3 Services PDS3 Metadata Index PDS Catalog Info Users PDS Portal

Transition Current Missions PDS3 Archive @ DNs Datasets + Products PDS3 Pipeline PDS3 Ingest PDS3 Central Catalog PDS3 Services PDS3 Metadata Index PDS Catalog Info PDS Catalog Info Users toPDS4 Transform Updated PDS Portal PDS4 XML Label PDS4 Registry PDS4 Metadata Index PDS4 infrastructure deployed at EN; Central catalog migrated.

Support for LADEE/MAVEN Current Missions PDS3 Archive @ DNs Datasets + Products PDS3 Pipeline PDS3 Ingest PDS3 Central Catalog PDS3 Services NOTE: PDS3 Services phased out overtime PDS3 Metadata Index PDS Catalog Info PDS Catalog Info Users toPDS4 Transform Updated PDS Portal PDS4 XML Label PDS4 Registry PDS4 Metadata Index PDS4 Pipeline PDS4 Ingest PDS4 Archive @ DNs PDS4 Services Build 4 New Missions (LADEE, MAVEN, O-Rex, Insight) PDS3 Data Migration PDS4 infrastructure deployed at EN; Central catalog migrated. Working towards acceptance of new PDS4 mission data

The Information Model Driven Process & Artifacts Protégé Ontology Modeling Tool Requirements & Domain Knowledge The Information Model Driven Process & Artifacts PDS4 Information Model PDS4 Data Dictionary (ISO/IEC 11179) Filter and Translator XML Schema (pds) Query Models PDS4 Data Dictionary (Doc and DB) Registry Configuration Parameters XML Document (Label Template) Information Model Specification XMI/UML This is given to data providers

Capabilities/Operational Deployment Design and Generate PDS4 Products Information Model 0310b frozen (build 3b) COTS tools tested and in use to support PDS4 Schema development (internal to PDS) (build 2) DN tools for generating PDS4 products developed at multiple nodes (build 2) Validation Validate tool developed for PDS4 label validation (build 2) Harvest/Registration PDS3 central catalog migrated to registry (build 3a) CI tool in place to register PDS3 catalog data (done, e.g., MSL) in a PDS4 registry (build 3a) Harvest in place to register PDS4 bundles and resources in PDS4 registry (build 3a) Registration of PDS4 web resources (build 3a) Registration of PDS4 bundles (build 3b) Search/Access Deployment of the PDS4 search service at EN (build 3a) Generation of a PDS4 search index for PDS3 data (build 3a) High level search of PDS3 data sets, PDS4 bundles, web resources, IPDA (build 3b) Distribution/user tools (build 4) This the need area for future builds, but is dependent on having good bundles … We need to take advantage of the flexibility of XML in presenting PDS4 to end users

Build Deliverables* Software System Data Standards* Registry Service Harvest Tool Validate Tool Security Service Report Service Search Service Transform Service CI Tool Upgraded portal search and page views to support PDS4 Information Model XML Schemas Data Dictionary Concepts Document Standards Reference Data Providers Handbook PDS4 Example Products * Posted to http://pds.nasa.gov/pds4

PDS4 Home Page https://pds.jpl.nasa.gov/pds4/about/

PDS4 CCB Process

PDS4 Archive Support Pages Cassini Archive Support page has been received well and used as a model LADEE and MAVEN pages now operational Those pages show up at the top of a search now with the new search service

IPDA Search Architecture Search Service supports the PDS and PDAP protocols enabling development of other portals and applications on this infrastructure. Registered Objects: Websites Data Sets Investigations, Instruments, etc. Tools and Services Search results include mission support pages and other more specific search interfaces. July 13, 2012 Cross-Agency Search and Access

IPDA Search Results Data sets and other results returned include products from NASA and PSA. PSA data sets currently link directly to a PSA web-based interface.

More information To be presented by Steve on the Information Model To be presented by Santa on the assessment project To be presented on interoperable access and search

Backup