- 1 - HDF5, HDF-EOS and Geospatial Data Archives HDF and HDF-EOS Workshop VII September 24, 2003

Don Keefer Illinois State Geological Survey Mike Folk U. of Illinois & NCSA

- 3 - NARA’s data challenge Includes electronic records Unprecedented volume, diversity, complexity Must be able to organize, mine, query and access to satisfy public and government needs Includes geospatial data The National Archives and Records Administration (NARA) has the duty to preserve the nation’s history through archival storage and management of federal records.

- 4 - Project goals Evaluate HDF5 and HDF-EOS as formats for archival storage of geospatial data –Need to represent many geosptatial data models –Need to accommodate large volumes of data –Need high performance data access –Need visualization and other applications

- 5 - Why HDF5 and HDF-EOS? Established open standard –Earth science focus –Manages huge data files Scalablility –Efficient storage –High performance access Geospatial adaptability –HDF5 storage structures –HDF-EOS data models Format features –Self-describing –Alternate storage structures –Extendable Software features –Flexible API –Portable library –High quality –Many applications/tools

- 6 - Mapping Data Structures to Information Types 1D, 2D, 3D array Raster image Vector features 2D Curtain Table Attribute HDF-EOS Grid HDF-EOS Swath HDF-EOS Point Digital orthophoto quad Digital elevation model Geologic well log Map Contoured surface Volume model Cross-section Color map Geospatial metadata Geospatial index Data structuresInformation types

- 7 - What we are doing Exploring use of HDF5 and HDF-EOS datatypes for non remote-sensed, geospatial data types –HDF-EOS grids for geospatial surfaces and volumes –HDF5 for raster (DOQ, DEM), vector (GIS), tabular data Addressing format conversion –Map other formats to HDF5 and HDF-EOS –Convert from other formats to HDF5 and HDF-EOS –Examine performance and storage efficiency Exploring visualization issues –Identifying technical requirements for visualization tools –Evaluating I/O efficiency

- 8 -

- 9 -

- 10 -

- 11 -Acknowledgements This report is based upon work supported in part by the National Archives and Records Administration (NARA) through the grant NARA NSF 0202 GPG, by a Cooperative Agreement with the National Aeronautics and Space Agency (NASA) under NASA grant NAG and NAG NCCS-599, and by the Illinois State Geological Survey (ISGS). Other support provided by NCSA and other sponsors and agencies ( Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NARA, NASA or the ISGS.