PDS 2010 Project Update Dan Crichton MC Face-to-Face Washington DC March 25-26, 2010

Purpose Today Provide an update on the PDS 2010 project The Data Design and System Design will be reported separately A report on the system review will also be provided 2

What is PDS 2010 A PDS-wide project to upgrade PDS from PDS3 to PDS4 A transition from a 20-year-old collection of standards and tools to a modern system constructed using best practices for data system development. Fewer, simpler, and more rigorously defined formats for science data products. Use of XML, a well-supported international standard, for data product labeling, validation, and searching. A hierarchy of data dictionaries built to the ISO standard, designed to increase flexibility, enable complex searches, and make it easier to share data internationally. 3

Key Design Decisions & Recommendation from Aug Replace PDS3 ad hoc information model with a PDS4 information model that is now managed in modern tools (DDWG) Replace ad hoc PDS3 product definitions with PDS4 products that are defined in the model (DDWG) Require data product formats to be derivations from a core set; Support transformation from the core set (DDWG) Replace “homegrown” PDS data dictionary structure with an international standard (ISO RIM) (DDWG) Adopt a modern data language/grammar (XML) where possible for all tool implementations (SDWG) Adopt system of registries to support improved tracking and access (SDWG) Support remote access to data and services to bring the federation together both for ingestion and distribution (SDWG) 4

Summary of Progress to Date Great progress by the teams! Project plan defined and being executed Overguide Funding In Place PDS-wide Architecture defined Iterative development releases of PDS4 standards coupled with nodes exercising and validating it against products Data provider engagement Software prototypes for tradeoffs of key services Software specifications for key infrastructure services generated Reviews at multiple levels (including a system review) Multiple IPDA projects working towards a common PDS 2010 goal Transition plan developed for transitioning both software and data over time

Progress from Design Teams Data Design Regular telecons F2F meeting in January Iterative Releases of the Data Model Simplified Product Model & Product Schemas Prototyping Preliminary Data Dictionary System/Software Regular telecons Software Architecture Specification Software Design Specifications for each of the proposed services Pilots for security and registry Regular interfaces with the DDWG 6

Key Project Documentation in Place Project Plan Contains management approach and implementation plan Multiple Concept Papers Architecture, user services and data design System Architecture Defines data and software architecture Operations Concept Interactions of PDS across the mission phases and from ingestion thru to distribution System Design Specifications for services and tools

Document Tree

PDS 1/2/3 Requirements Simpson/Crichton provided minor updates to address gaps for PDS 2010 Service-oriented Architecture (2.8,3.3) Security (2.9) Need MC input on whether they want to close out the requirements or provide additional comments More on this Friday morning 9

Baseline Plan Deliver an initial PDS4 Data Standard by October 2010 to begin coordinating PDS4 product definitions and systems development Deliver a prototype infrastructure build by October 2010 to test ingestion of PDS4 sample data products Deliver an initial operational capability for PDS4 by October 2011 to support ingestion and distribution of PDS4 data products Deliver additional tools and a framework to support PDS4 transformation and science services/distribution capabilities by June 2012

Resources for PDS 2010 Working to resource plan presented in December DN request is to predominately focus on PDS4 data standards in FY10, then begin shift to system development effort EN staff in place 11March 22, 2010PDS 2010 Resources

Data Standards Development March 22, 2010PDS 2010 Resources12 DDWG Support thru September 2009 Anne Raugh/SBN (.3) Mitch Gordon/Rings (.3) Lyle Huber/Atmos (.3) Ed Guinness/Geosciences (.3) Steve Joy/Joe Mafi/PPI (.3) Boris Semenov/NAIF (.1) Dick Simpson/Radio Science (.2) Elizabeth Rye (.8) (.4 EN;.4 IMG) Steve Hughes/EN (.5) Ron Joyner/EN (.5) EN: 1.4 FTE; DN: 2.2 FTE *.2 support/node for burst activities, etc

System Development System Design and Development Support (FY10) Sean Hardman/Engineering (.5) Todd King/PPI (.05) Tom Stein/Geosciences (.05) Alice Stanboli/Sciences (.05) Mike Martin (.05) EN Development Support (2.5) FY11/FY12 Planned Support 3.0 FTE (Engineering) 0.3 FTE/node for Development; 0.2 FTE for Standards 1.0 FTE (ARC) GUI/tool development March 22, 2010PDS 2010 Resources13

On-going PDS3 Maintenance PDS3 development and standards kept to a minimum as discussed in December F2F. March 22, PDS 2010 Resources

Reviews Preliminary MC design discussion – August 2009 Complete System Review (Architecture, Ingestion) – March 2010 Complete System Review (Distribution) – March 2011 Sub-system/component level reviews take place throughout PDS 2010 development Prototyping of both software and data products is used as risk mitigation approach March 22, 2010PDS 2010 Resources15

System Review 6 Review Board Members All felt that a tremendous amount of good work has been done. They’ve made some recommendations that Dave Heather will share. In particular, they felt the PDS 2010 architecture and design was very solid This was the focus of the review No major technical issues related to the design Some recommendations Map PDS development lifecycle to NPR Requirements traceability (but we didn’t present level 1/2/3/4 other than to post) An absolute date by which new missions will start to be PDS4 compliant Amount of centralization vs. de-centralization in terms of system elements and governance The board will release a report in mid April that will be distributed 16

PDS4 Data Standards Schedule March 22, 2010PDS 2010 Resources17

PDS 2010 Software System Schedule March 22, 2010PDS 2010 Resources18

Project Next Steps Meetings DDWG F2F (April) Proposed Tech Session F2F (September/October 2010) Review PDS4 Tutorials, transition, and software development plans Resources/Workforce Continue in current working group structure to get a v4.0 data standard Note: we expect efforts to continue on data standards, but this gives us a baseline for system development and mission planning Tech F2F will be a natural time to begin increasing emphasis on the software/systems-side across PDS Baseline plan on track 19

Backup 20

Project Schedule March 22, 2010PDS 2010 Resources21 Study Phase (August March 2008) Project Definition (January July 2008) High Level Architecture (July 2008 – January 2009) Include trade studies and transition planning Development (2009 –2012) Build (October 2009 – September 2010) Build (July 2010 – September 2011) Build (April 2011 – May 2012) Deployment* (2010 –2012) Build (October 2010) Build (October 2011) Build (June 2012) * Incremental releases occur between deployments….

Data Standards Breakdown for FY10 Data Model (25% effort) thru Sept 2010 Common Model (DDWG members) Discipline Model (20% EN, 80% DN); effort needs to ramp up in conjunction with DD Data Dictionary (25% effort) thru Sept 2010 (40% EN, 60% DN) Continuing to ramp up XML Product Definition and Best Practices (10% effort) thru Sept 2010 (30% EN, 70% DN) This needs be a larger focus in Jan/Feb Tutorial Material (15% effort) Jan 2010 – Aug 2010 (70%, 30% EN) Initial discussions at the January F2F Standards Reference (25% effort) Thru Sept 2010 (60% EN, 40% DN) Ramp up will occur in spring timeframe March 22, 2010PDS 2010 Resources22

Operations* FY10 Planned I&T, Deployment Support 0.4 FTE (Engineering) - develop Build I test plan; support Build I I&T, deploy Build I H/W, S/W, support CM & build management 0.1 FTE (one or 2 selected node) - support Build I I&T and deployment at the node(s) FY11 Planned I&T, Deployment & Prototype Support 0.75 FTE (Engineering) - support Build I prototype activities (catalog migration, data prototype validation etc); develop Build II test plan, support Build II I&T, deploy Build II H/W, S/W, support CM & build management 0.1 FTE (all nodes) - support Build II I&T and deployment at the nodes FY12 Planned I&T, Deployment & Operations Support.75 FTE (Engineering) - support Build II operations; develop Build III test plan & Mirror Site, support Build III I&T, deploy Build III H/W, S/W, support CM & build management 0.1 FTE (all nodes) - support Build III test, deployment and integration at the node(s); development of Build III science services test plan March 22, 2010PDS 2010 Resources23 * EN Operations, will migrate personnel to support PDS 2010

Current PDS3 Support March 23, 2010Transition & Migration Plan24 PDS3 Pipeline PDS3 Ingest Current Missions (e.g MER,CAS) PDS3 DNs PDS3 Services Central Catalog Homepage Users Datasets + Products Datasets + Products Dataset Metadata (PDS3) Dataset Metadata

Transition to PDS4 Support March 23, 2010Transition & Migration Plan25 PDS3 Pipeline PDS3 Ingest Current Missions (e.g MER,CAS) PDS3 DNs PDS3 Services Central Catalog Users PDS4 Ingest PDS2010 Registry PDS4 DNs PDS4 Pipeline PDS2010 Services New Missions (e.g LADEE, MAVEN) PDS Portal Harvest Product Metadata (PDS3) toPDS4 Transform Datasets + Products Dataset Metadata (PDS3) Dataset Metadata (PDS3) Metadata Index NOTE: PDS3 Services phased out overtime

Migration Concept March 23, 2010Transition & Migration Plan26 PDS3 DNs PDS2010 Registry PDS4 DNs toPDS4 Transform NSSDC/Deep Archive PDS3 Products (Metadata + Data) PDS4 Products (Metadata + Data) PDS4 Products (Metadata) PDS4 Products (Metadata + Data) NOTE: Deliveries to the Deep archive will be performed if actual data has been transformed.

PDS 2010 Overall Transition Approach Analyze tradeoffs and impacts Develop translation software from PDS3 to PDS4 to support existing PDS3 pipelines Allow for phased/decoupled transition to PDS4 over time All PDS do not need to transition at exactly the same time Ensure PDS 2010 will serve data from PDS3 and PDS4 repositories Migrate data sets to PDS4 as needed March 23, 2010Transition & Migration Plan27

System Transition Concept PDS Central Catalog will be replaced with a registry system that will support BOTH PDS3 and PDS4 data collection and product registration Tool support for PDS3 and PDS4 Central PDS homepage will link to both PDS3 and PDS4 resources as they are available Expectation is that PDS3 resources will decrease and PDS4 will increase overtime Each node will execute their own transition timeline and plan to upgrade to PDS4 Overall plan is that existing PDS3 services will remain while new PDS4 services will be added March 23, 2010Transition & Migration Plan28