PDS4 Project Report PDS MC F2F Columbia, MD Dan Crichton April 2-3,

PDS4: The Next Generation PDS PDS4 is a PDS-wide project to upgrade from PDS version 3 to version 4 to address many of these challenges An explicit information architecture All PDS data tied to a common model to improve validation and discovery Use of XML, a well-supported international standard, for data product labeling, validation, and searching. A hierarchy of data dictionaries built to the ISO standard, designed to increase flexibility, enable complex searches, and make it easier to share data internationally. An explicit software/technical architecture Distributed services both within PDS and at international partners Consistent protocols for access to the data and services Deployment of an open source registry infrastructure to track and manage every product in PDS A distributed search infrastructure 2

Information Architecture Concepts 3 Tagged Data Object (Information Object) Label Schema Used to Create Describes Extracted/Specialized Information Model Data Object Data Element Class has Planetary Science Data Dictionary Expressed As Product Validates MPFL-M-IMP_IMG_GRAYSCALE… 0 2 Last Index Fastest… UnsignedMSB2 data number Line 248 not applicable 1 Sample 256 not applicable 2

System Design Approach Based on a distributed information services architecture (aka SOA-style) Allow for common and node specific network-based (e.g., REST) services. Allow for integrating with other systems through IPDA standards. System includes services, tools and applications Use of online registries across the PDS to track and share information about PDS holdings Implement distributed services that bring PDS forward into the online era of running a national data system With good data standards, they become critical to ultimately improving the usability of PDS Support on-demand transformation to/from PDS 4 Client BClient A Service CService Interface

Summary of Progress to Date Requirements in place (approved by MC 3/2010) PDS-wide Architecture defined Major reviews conducted Initial PDS products defined using maturing PDS4 specification LADEE, MAVEN teams developing labels System builds grouped by purpose: build 1(a,b,c,d), 2 (a,b,c), 3(a) Used to establish a rigorous release process Includes deployment of software, data standards, documents and system integration testing As stability has increased, we have moved to 6 month builds Operational capabilities deployed Registry and Harvest infrastructure in place at EN Central catalog migrated to registry; High level search migrated Information model released for limited use to select data providers PDS3 data now being ingested into PDS4 system at EN Beta test conducted by Atmospheres to get user input on PDS4 concepts (XML, Bundles, etc) LADEE PDS4 Peer Reviews conducted IPDA endorsement and plans to move to PDS4 International implementation by PSA for Bepi Colombo underway International access prototype demonstrated at IPDA 2012 SC 5

PDS4 Software Development Lifecycle 6 NASA Software Management Guidebook – NASA-GB-001 Evaluation by users as early as possible!

Project Lifecycle – mapped to NPR per SDR1 7 Project Lifecycle Pre- Formulation Implementation Project Lifecycle Gates & Major Events Project Reviews PDSMC Concept Review (Dec 2007) Study/ Concepts Project Plan PDS4 Prelim Architecture PDSMC Impl Review (July 2008) PDSMC Preliminary Design (August 2009) PDSMC Arch Review (Nov 2008) Build 1b Internal Stds Assessment (Dec 2010) PDS System Review (Mar 2010) Build 1c IPDA Stds Assessment (April 2011) PDS System Review II (June 2011) Build 1d External Stds Assessment (Aug 2011) Operational Readiness Review (LADEE/MA VEN) (November 2011) PDS4 Design Begin Study Project Build 1: Prototype build Build 2: Release to LADEE/MAVEN Build 3: V1.0 of PDS4 Stsandards 1a 1b 1c 1d KDP: Study KDP: Prelim Design KDP: Project Plan & Arch KDP: Release LADEE/ MAVEN Phoenix Beta Test (Dec 2012)

PDS4: Build Structure 88 PhasePurposeReleaseDate Build 1 Prototype Early formalization of the data standards Early formalization of software Integration between software and data standards Stakeholder input PDS4 beta info model, standards reference, data dictionary, schemas baseline Early releases of Harvest, Registry and Security services First set of process, documentation and tutorial October 2010 Build 2 Early Release to Select Data Providers Support LADEE, MAVEN and Early Migration Planning Begin deployment of PDS4 at Engineering Node and transition Stakeholder input, particularly from LADEE and MAVEN PDS4 beta info model, standards reference, data dictionary, schemas baseline Production release of Harvest, Registry, Report and Security services Validation and catalog ingest tools Updated documentation March 2012 (build 2b) Build 3 V1.0 of PDS Data Standards Transition entire PDS3 catalog to PDS4 Baseline PDS4 standards as version 1.0 Support validation of PDS4 bundles Support ingestion and PDS4 data into the PDS4 registry Support search and access to PDS3 data sets and PDS4 bundles Ensure LADEE/MAVEN can move on Stable release of data standards Incremental releases of validation and PD3 catalog ingest tools Incremental releases of harvest, registry, report and security services Deployment of the PDS4 search service at EN October (build 3a) March 2013 (build 3b) Build 4 User Services Support PDS4 data distribution services for LADEE, MAVAEN Support PDS4 data transformation PDS4 user tools Incremental release of PDS4 Standards Distribution services for initial PDS4 bundles Transformation services for initial PDS4 bundles September 2013/ March 2014

Build 2 ORR Begin PDS4 label design for LADEE and MAVEN; Begin planning/testing migration Deploy a PDS4 standard that supports the Policy on “Acceptable PDS4 Data Formats” Transition the central catalog to the registry infrastructure Deploy early PDS4 software tools and services 9

PDS4 Planned Mission Support 10 BepiColumbo (ESA/JAXA) Osiris-REx (NASA) MAVEN (NASA) LADEE (NASA) InSight (NASA) ISRO, JAXA planning PDS4 missions

PDS3 Implementation 11 PDS3 Pipeline PDS3 Ingest Current Missions PDS3 DNs PDS3 Services PDS3 Central Catalog Users PDS Portal Datasets + Products PDS Catalog Info PDS3 Metadata Index

Today 12 PDS3 Pipeline PDS3 Ingest Current Missions PDS3 DNs PDS3 Services PDS3 Central Catalog Users PDS4 Registry Updated PDS Portal toPDS4 Transform Datasets + Products PDS4 XML Label PDS Catalog Info PDS4 Metadata Index (1)PDS4 infrastructure deployed at EN; Central catalog migrated. PDS Catalog Info PDS3 Metadata Index

Support for LADEE/MAVEN 13 PDS3 Pipeline PDS3 Ingest Current Missions PDS3 DNs PDS3 Services PDS3 Central Catalog Users PDS4 Ingest PDS4 Registry PDS4 DNs PDS4 Pipeline PDS4 Services New Missions (LADEE, MAVEN, O-Rex, Insight) Updated PDS Portal toPDS4 Transform Datasets + Products PDS4 XML Label PDS Catalog Info PDS4 Metadata Index NOTE: PDS3 Services phased out overtime (1)PDS4 infrastructure deployed at EN; Central catalog migrated. (2)Working towards acceptance of new PDS4 mission data PDS Catalog Info PDS3 Metadata Index Build 4 PDS3 Data Migration

Requirements & Domain Knowledge PDS4 Information Model Query Models Information Model Specification XML Schema (pds) Filter and Translator Protégé Ontology Modeling Tool PDS4 Data Dictionary (Doc and DB) PDS4 Data Dictionary (Doc and DB) XML Document (Label Template) XMI/UML Registry Configuration Parameters PDS4 Data Dictionary (ISO/IEC 11179) The Information Model Driven Process & Artifacts This is given to data providers

Capabilities/Operational Deployment Design and Generate PDS4 Products Information Model 0310b frozen (build 3b) COTS tools tested and in use to support PDS4 Schema development (internal to PDS) (build 2) DN tools for generating PDS4 products developed at multiple nodes (build 2) Validation Validate tool developed for PDS4 label validation (build 2) Harvest/Registration PDS3 central catalog migrated to registry (build 3a) CI tool in place to register PDS3 catalog data (done, e.g., MSL) in a PDS4 registry (build 3a) Harvest in place to register PDS4 bundles and resources in PDS4 registry (build 3a) Registration of PDS4 web resources (build 3a) Registration of PDS4 bundles (build 3b) Search/Access Deployment of the PDS4 search service at EN (build 3a) Generation of a PDS4 search index for PDS3 data (build 3a) High level search of PDS3 data sets, PDS4 bundles, web resources, IPDA (build 3b) Distribution/user tools (build 4) This the need area for future builds, but is dependent on having good bundles … We need to take advantage of the flexibility of XML in presenting PDS4 to end users 15

Build 3b Deliverables* 16 Software System Registry Service Harvest Tool Validate Tool Security Service Report Service Search Service CI Tool Upgraded portal search and page views to support PDS4 Data Standards* Information Model XML Schemas Data Dictionary Concepts Document Standards Reference Data Providers Handbook PDS4 Example Products * Posted to

Build 3b Screenshot 17

Build 3b Finalization March 2013 Freeze the model Synchronize the model and software 29 March 2013 Capture in CM; Deploy IM, documentation, software, test plans, etc to PDS4 build 3b site April 2013 EN regression testing to verify software against PDS4 examples Compile test report Pending no major issue, deploy Standards as V1.0 to support LADEE, MAVEN SIS finalization and IPDA SC 18

Build 3b Testing Process Engineering has developed a test plan (Emily Law will lead) Test cases Data product matrix Engineering performs regression tests using example bundles and data products Validate functional requirements L3 – L5 Engineering will write a test report 19

Beta Testing PDS4 development has performed increasing levels of beta testing with users since build 1 Latest beta test by Atmospheres included review of PDS4 labels, bundles and documentation 20 Search Service Deployed for Build 3a Includes links to Archive Resources

PDS4 Archive Support Pages Cassini Archive Support page has been received well In PDS4, we are developing archive pages (e.g. Phoenix) for our migrated data We can add tools/services over time - this gets the data online Those pages show up at the top of a search now with the new search service from build 3a 21

IPDA Interactions IPDA endorsement of PDS4 (July 2012) PSA moving forward with PDS4 Dave Heather leading a IPDA PDS4 implementation project Will be at JPL in May 2013 for coordination meetings for Bepi Columbo Search/access to international archives Demonstrated as a prototype at IPDA SC and COSPAR in July 2012 Leverages the PDS4 REST-based architecture Will be operational with build 3b deployment 22 7 th IPDA Steering Committee (July 2012) (ASI, CNES, ESA, ISRO, JAXA, NASA)

Architecture July 13, 2012Cross-Agency Search and Access23 Registered Objects: Websites Data Sets Investigations, Instruments, etc. Tools and Services Search results include mission support pages and other more specific search interfaces. Search Service supports the PDS and PDAP protocols enabling development of other portals and applications on this infrastructure.

IPDA Search Results 24 Data sets and other results returned include products from NASA and PSA. PSA data sets currently link directly to a PSA web-based interface.

Summary Significant progress has been made and operational capabilities have been deployed since build 2b We need stable data standards (v1.0) so we can move forward with LADEE, MAVEN and others, and focus on data distribution services. We need to be producing bundles That will push us forward We have international alignment! We should good progress at the July 2013 meeting 25

Backup 26