1. Transition from HDF4 to HDF5: Issues
Robert E. McGrath
NCSA
University of Illinois at Urbana-Champaign
9/19/2000
McGrath 9/19/2000

2. For more details see the white paper at:
Overall Goal
A discussion of compatibility, conversion, and interoperability issues for HDF4 and HDF5, and HDFEOS.
For more details see the white paper at:
9/19/2000
McGrath 9/19/2000

3. Conclusions (Preview) Transition from HDF4 to HDF5 is not necessarily transparent or effortless.
9/19/2000
McGrath 9/19/2000

4. Conclusions: HDF
Transition from HDF4 to HDF5 is not transparent: in most cases source code will have to be written or rewritten.
Many tools will support both HDF4 and HDF5.
Default translation from HDF4 to HDF5 will be supported as a utility and library.
9/19/2000
McGrath 9/19/2000

5. Conclusions: HDF-EOS
Transition from HDFEOS2 to HDFEOS3 is not transparent: in most cases source code will have to be written or rewritten
Translation of ‘pure’ HDF-EOS will be supported
Translation of ‘hybrid’ HDF-EOS files is very difficult.
Not possible to support V.2 and V.3 in a single program
9/19/2000
McGrath 9/19/2000

6. What Does Compatibility Mean?
9/19/2000
McGrath 9/19/2000

7. What does “compatibility” mean?
Ability to “do the same things” with new software
I.e., “compatibility” is relative to some purpose
Many dimensions and shades of compatibility
No golden solution sufficient for every use
Assume HDF4 users want to continue doing what they are already doing, but move to HDF5
9/19/2000
McGrath 9/19/2000

8. Some Types of Compatibility
9/19/2000
McGrath 9/19/2000

9. Compatibility of HDF and HDF-EOS
9/19/2000
McGrath 9/19/2000

10. Interoperation
My definition of “interoperation”: one program or tool may use HDF4 or HDF5 or both
9/19/2000
McGrath 9/19/2000

11. NCSA HDF4 and HDF5
9/19/2000
McGrath 9/19/2000

12. Interoperability: Any program may use either HDF4 or 5 or both:
Not Supported
image2
image1
dataset1
dataset2
Single Combined HDF Library
HDF 4 Library
HDF 5 Library
Stored as
HDF4 Objects
Stored as
HDF5 Objects
HDF4 file
HDF5 file
9/19/2000
McGrath 9/19/2000

13. Application specific Library
Interoperability: Specific Application or tool may use either HDF4 or 5 or both:
Up to Application and Tool Developers
image2
image1
dataset1
dataset2
Application specific Library
Application Specific Library
HDF 4 Library
HDF 5 Library
Stored as
HDF4 Objects
Stored as
HDF5 Objects
HDF4 file
HDF5 file
9/19/2000
McGrath 9/19/2000

14. HDF-EOS Version 2 and Version 3
9/19/2000
McGrath 9/19/2000

15. Interoperability: Any program may using HDF-EOS Objects with either HDF4 or HDF5 or both:
Not supported
HDFEOS Library
Grid1
Grid2
Grid3
Grid4
HDF 4 Library
HDF 5 Library
Stored as
HDF4 Objects
Stored as
HDF5 Objects
HDF4 file
HDF5 file
9/19/2000
McGrath 9/19/2000

16. Interoperability: One application may use HDF-EOS Objects with either HDF4 or HDF5 or both:
Not supported
Grid1
Grid2
Grid3
Grid4
HDFEOS V2 Library
HDFEOS V3 Library
HDF 4 Library
HDF 5 Library
Stored as
HDF4 Objects
Stored as
HDF5 Objects
HDF4 file
HDF5 file
9/19/2000
McGrath 9/19/2000

17. Translation Convert HDF4 objects (file) to HDF5 objects (file)
9/19/2000
McGrath 9/19/2000

18. Translation of HDF4 to HDF5
Supported: NCSA mapping and h4toh5 tool and library
HDF4 Library
HDF5 Library /
Stored as
HDF4 Objects
Stored as Analogous
HDF5 Objects
HDF5 File
HDF4 File
9/19/2000
McGrath 9/19/2000

19. Translation of HDF-EOS V2 to V3
Supported
HDF EOS V. 2.x Library
HDF EOS V. 3 Library
Grid1
Grid2
Grid1
Grid2
SAME
HDFEOS
Objects
HDFEOS
Objects
HDF4 Library
HDF5 Library
Stored as
HDF4 Objects
Stored as
HDF5 Objects /
HDF5 file
HDF4 file
9/19/2000
McGrath 9/19/2000

20. “Hybrid” HDF-EOS files
HDF-EOS files may contain native HDF objects (images, arrays, etc.) as well as HDF-EOS objects (Grid, etc.)
Almost all HDF-EOS files are ‘hybrids’.
Very difficult to automatically translate hybrid files.
9/19/2000
McGrath 9/19/2000

21. Science Application Code
“Hybrid” HDF-EOS File
Science Application Code
HDFEOS
Other
NCSA HDF
Components of
HDFEOS Objects
stored in HDF
Objects
Other HDF
Objects Stored
Directly Through
HDF
Very difficult
to tell which
objects are part
of HDF-EOS
One HDF file, objects
mixed together.
9/19/2000
McGrath 9/19/2000

22. Conclusions Transition from HDF4 to HDF5 is not necessarily transparent or effortless.
9/19/2000
McGrath 9/19/2000

23. Conclusions: HDF
Transition from HDF4 to HDF5 is not transparent: in most cases source code will have to be rewritten.
Many tools will support both HDF4 and HDF5.
Default translation from HDF4 to HDF5 will be supported as a utility and library.
9/19/2000
McGrath 9/19/2000

24. Conclusions: HDF-EOS
Transition from HDFEOS2 to HDFEOS3 is not transparent: in most cases source code will have to be rewritten
Translation of ‘pure’ HDF-EOS will be supported
Translation of ‘hybrid’ HDF-EOS files is very difficult.
Not possible to support V.2 and V.3 in a single program
9/19/2000
McGrath 9/19/2000

25. For More Information
NCSA White paper: “Transition From HDF4 to HDF5”:
“Guidelines for Mapping HDF4 to HDF5 Objects”,
h4toh5 tool: Soon
9/19/2000
McGrath 9/19/2000