1. IPDA July 2013
CDF and PDS
Todd King, Joseph Mafi, Steven Joy

2. PDS (Planetary Data System)
NASA's project charged with archiving planetary data.
An active archive that makes available well documented, peer reviewed data to the research community.
Establishes the requirements, specifications and guidelines for proper archiving.
9 April 2019
Information and Overview

3. PDS Archive Requirements
PDS is required to preserve archived data for 200 years.
Adopted standards are defined by recognized organizations.
Data storage formats must be durable.
Useful as-is.
Data values are stored, no software based calculations
Data structures must be described in a way that allows users to access the data in the absence of any format-specific readers.
9 April 2019
Information and Overview

4. CDF (Common Data Format)
Self-describing format for the storage of scalar and multidimensional data in a platform and discipline independent way.
Libraries and toolkits are available for the most commonly used platforms and programming languages.
CDF specification is self-published and publicly available (currently version 3.4).
Blends metadata (attributes) and data (variables) into one format.
Additional specifications for metadata (CDF Tools, ISTP/IACG)
9 April 2019
Information and Overview

5. Information and Overview
CDF Internals
Magic Numbers
CDF Descriptor Record (CDR)
Global Descriptor Record (GDR)
Attribute Descriptor Record (ADR) *
Attribute zEntry Descriptor Record (AzVDR)
zVariable Descriptor Record (zVDR) *
Variable indeX Record (VXR)
Binary Data
Variable Values Record (VVR)
9 April 2019
Information and Overview

6. Information and Overview
The Need to Support CDF
Various NASA (Heliophysics Division) and international projects have chosen CDF as their official science data format.
MAVEN
BepiColombo
and so have space agencies
JAXA
PDS prefers to archive projects’ official science products.
Instrument/Science team use of archive data products best method of insuring data quality.
9 April 2019
Information and Overview

7. Archiving CDF in PDS4 … is possible
Restrictions for CDF files:
Single file CDF
MSB byte order (Network Encoding)
Contiguous variable storage (no frgamentation)
All records and values are written (physical)
Adhere to CDF Tools and ISTP/IACG metadata guidelines.
No sparse records.
No compression.
CDF data will be described using PDS4 data structures.
CDF attributes (metadata): Header objects.
CDF variables (data): Array objects.
9 April 2019
Information and Overview

8. Information and Overview
Archiving Support
Copies of the CDF Specification will be included in the archive.
Tools:
CDF tools (Goddard)
PDS4 metadata generation tool
igpp.docgen with pds.cdf parser
Format conversion tool
CDF to ASCII table (in development)
9 April 2019
Information and Overview

9. Additional Information
White Paper - Supporting CDF in PDS4, Todd King
Java tools:
igpp.docgen – Process Apache Velocity Templates
pds.cdf – Pure Java CDF parser. Library, command-line tool, and embedded in igpp.docgen.
9 April 2019
Information and Overview

10. Information and Overview
Recommendation
Allow CDF
which meet archive standards.
Call it “CDF-A”?
9 April 2019
Information and Overview