1. Mike Folk, Peter Cao, Kent Yang Ruth Duerr Christopher Lynnes
Improving long-term preservation EOS data by independently mapping HDF data objects
Mike Folk, Peter Cao, Kent Yang Ruth Duerr Christopher Lynnes

2. Annual HDF Briefing to ESDIS
Two projects
Productizing HDF4 Mapping schema and tools for deployment
Implement HDF5 mapping prototype
March 31, 2009
Annual HDF Briefing to ESDIS

3. I. Productizing HDF4 Mapping schema and tools for deployment
March 31, 2009
Annual HDF Briefing to ESDIS

4. Productizing HDF4 mapping
Investigate integration of mapping schema with existing standards
Determine HDF-EOS 2 requirements
Redesign the XML schema
Implement production quality HDF4 map writer
Develop demo HDF4 map reader and optional general reader
Develop test suite and validation utility for layouts and datatypes
Deploy tools at other NASA data centers.
March 31, 2009
Annual HDF Briefing to ESDIS

5. HDF4/HDF-EOS2 mapping workflow
March 31, 2009
Annual HDF Briefing to ESDIS

6. Annual HDF Briefing to ESDIS
Estimated cost in fte
The HDF Group: fte months
GES: 2.9 fte months + time for technology infusion
NSIDC: 3.3 fte months + 3 months summer student + time for technology infusion
Project duration: approximately 32 months
March 31, 2009
Annual HDF Briefing to ESDIS

7. Annual HDF Briefing to ESDIS
HDF-EOS 2 analysis
Some crucial HDF-EOS2 information is not directly available in a prototype-mapping file
Example:
Latitude and longitude in an HDF-EOS2 grid are provided implicitly by projection parameters in an EOS2-specific HDF4 attribute “StructMetadata.”
When apps use the HDF-EOS2 library to access a grid, the library computes the lat/lon at each data point based on the StructMetadata value.
It would be burdensome for apps to implement on their own these computations for all projections.
Currently, the StructMetadata attribute is in the mapping file, but no interpretation of lat and lon is provided.
March 31, 2009
Annual HDF Briefing to ESDIS

8. Annual HDF Briefing to ESDIS
HDF-EOS 2 analysis
Hence the project includes a study of the mapping requirements for HDF-EOS2 grid, swath, and point objects.
Will do a comprehensive analysis of issues related to the handling of NASA HDF-EOS2 files, aimed at making crucial EOS-specific data easily accessible to apps through the same tools used to access HDF4 files.
March 31, 2009
Annual HDF Briefing to ESDIS

9. II. Implement HDF5 mapping prototype
March 31, 2009
Annual HDF Briefing to ESDIS

10. Implement HDF5 mapping prototype
Assess and categorize EOS data products in HDF5.
Investigate methods to map objects in HDF5 files, and evaluate effort needed to implement methods.
Identify tools to facilitate creating and using maps; investigate requirements to develop such tools.
Carry out a proof of concept with selected files
Create maps, read back data using the maps
Create prototype tools to create map and read back map
Present results and options to the community
Assess usefulness of approach, desired modifications.
Evaluate the effort required for a full solution.
Submit a proposal for a full solution.
March 31, 2009
Annual HDF Briefing to ESDIS

11. Annual HDF Briefing to ESDIS
Estimated costs
The HDF Group: 7 months fte
GES: 1.3 months fte
NSIDC: 4.4 months fte
March 31, 2009
Annual HDF Briefing to ESDIS