Mike Folk, Peter Cao, Kent Yang Ruth Duerr Christopher Lynnes Improving long-term preservation EOS data by independently mapping HDF data objects Mike Folk, Peter Cao, Kent Yang Ruth Duerr Christopher Lynnes

Annual HDF Briefing to ESDIS Two projects Productizing HDF4 Mapping schema and tools for deployment Implement HDF5 mapping prototype March 31, 2009 Annual HDF Briefing to ESDIS

I. Productizing HDF4 Mapping schema and tools for deployment March 31, 2009 Annual HDF Briefing to ESDIS

Productizing HDF4 mapping Investigate integration of mapping schema with existing standards Determine HDF-EOS 2 requirements Redesign the XML schema Implement production quality HDF4 map writer Develop demo HDF4 map reader and optional general reader Develop test suite and validation utility for layouts and datatypes Deploy tools at other NASA data centers. March 31, 2009 Annual HDF Briefing to ESDIS

HDF4/HDF-EOS2 mapping workflow March 31, 2009 Annual HDF Briefing to ESDIS

Annual HDF Briefing to ESDIS Estimated cost in fte The HDF Group: 29.8 fte months GES: 2.9 fte months + time for technology infusion NSIDC: 3.3 fte months + 3 months summer student + time for technology infusion Project duration: approximately 32 months March 31, 2009 Annual HDF Briefing to ESDIS

Annual HDF Briefing to ESDIS HDF-EOS 2 analysis Some crucial HDF-EOS2 information is not directly available in a prototype-mapping file Example: Latitude and longitude in an HDF-EOS2 grid are provided implicitly by projection parameters in an EOS2-specific HDF4 attribute “StructMetadata.” When apps use the HDF-EOS2 library to access a grid, the library computes the lat/lon at each data point based on the StructMetadata value. It would be burdensome for apps to implement on their own these computations for all projections. Currently, the StructMetadata attribute is in the mapping file, but no interpretation of lat and lon is provided. March 31, 2009 Annual HDF Briefing to ESDIS

Annual HDF Briefing to ESDIS HDF-EOS 2 analysis Hence the project includes a study of the mapping requirements for HDF-EOS2 grid, swath, and point objects. Will do a comprehensive analysis of issues related to the handling of NASA HDF-EOS2 files, aimed at making crucial EOS-specific data easily accessible to apps through the same tools used to access HDF4 files. March 31, 2009 Annual HDF Briefing to ESDIS

II. Implement HDF5 mapping prototype March 31, 2009 Annual HDF Briefing to ESDIS

Implement HDF5 mapping prototype Assess and categorize EOS data products in HDF5. Investigate methods to map objects in HDF5 files, and evaluate effort needed to implement methods. Identify tools to facilitate creating and using maps; investigate requirements to develop such tools. Carry out a proof of concept with selected files Create maps, read back data using the maps Create prototype tools to create map and read back map Present results and options to the community Assess usefulness of approach, desired modifications. Evaluate the effort required for a full solution. Submit a proposal for a full solution. March 31, 2009 Annual HDF Briefing to ESDIS

Annual HDF Briefing to ESDIS Estimated costs The HDF Group: 7 months fte GES: 1.3 months fte NSIDC: 4.4 months fte March 31, 2009 Annual HDF Briefing to ESDIS