1. www.hdfgroup.org The HDF Group Support for NPP/NPOESS by The HDF Group Mike Folk, Elena Pourmal, Peter Cao The HDF Group November 5, 2009 November 3-5, 2009HDF/HDF-EOS Workshop XIII1

2. www.hdfgroup.org Goal 2 To develop and adapt HDF5 software and provide HDF5 support for NPP/NPOESS. HDF/HDF-EOS Workshop XIIINovember 3-5, 2009

3. www.hdfgroup.org Areas of immediate need Ensure easy and intuitive data access and use by diverse communities Enable producers and consumers to view content, manage metadata, and convert data to other formats. Provide high-quality, rapid-response support of HDF5 for NPOESS users at all levels. November 3-5, 20093HDF/HDF-EOS Workshop XIII

4. www.hdfgroup.org 2009-2009 Priorities Data accessibility and usability Developed library of high level APIs to support NPP/NPOESS data management Tool development Modify h5dump to enable viewing of region references and quality flags Modify HDFView to enhance viewing of references and quality flags System maintenance User support November 3-5, 20094HDF/HDF-EOS Workshop XIII

5. www.hdfgroup.org NPP/NPOESS Project Information Project Web site http://www.hdfgroup.org/projects/npoess/ Project description Released software Documentation November 3-5, 20095HDF/HDF-EOS Workshop XIII

6. www.hdfgroup.org NPOESS Web site November 3-5, 20096HDF/HDF-EOS Workshop XIII

7. www.hdfgroup.org NPOESS Software FTP Site November 3-5, 20097HDF/HDF-EOS Workshop XIII

8. www.hdfgroup.org NPOESS Project Information Project Wiki site http://wiki.hdfgroup.org/NPOESS-Project Monthly progress reports, links to software under development, presentations, etc. Requires a password Contact epourmal@hdfgroup.orgepourmal@hdfgroup.org November 3-5, 20098HDF/HDF-EOS Workshop XIII

9. www.hdfgroup.org HDF NPOESS Wiki November 3-5, 20099HDF/HDF-EOS Workshop XIII

10. www.hdfgroup.org The HDF Group Library of High-Level APIs to support NPOESS data November 3-5, 2009HDF/HDF-EOS Workshop XIII10

11. www.hdfgroup.org NPOESS High-Level Library Library to support HDF5 region references and packed bits in support of NPOESS quality flags C APIs Documentation Tests Examples Source available from ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/NPOESS/s ource/ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/NPOESS/s ource/ Documentation is available from http://www.hdfgroup.org/projects/npoess/ Still a prototype! November 3-5, 2009HDF/HDF-EOS Workshop XIII11

12. www.hdfgroup.org Documentation November 3-5, 2009HDF/HDF-EOS Workshop XIII12

13. www.hdfgroup.org The HDF Group h5dump New Features to Support Region References and Quality Flags November 3-5, 2009HDF/HDF-EOS Workshop XIII13

14. www.hdfgroup.org h5dump and Region Reference Data New flag –R to display data pointed to by region references Improved display of subsets An element of a dataset is specified by its indices Available in HDF5 1.8.4, November 2009 November 3-5, 2009HDF/HDF-EOS Workshop XIII14

15. www.hdfgroup.org h5dump and Quality Flags New flag –M to display packed bit-field data Available only in the HDF5 NPOESS branch Will NOT be in HDF5 1.8.4 release Source and pre-built binaries can be found at ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/NPOESS/ November 3-5, 2009HDF/HDF-EOS Workshop XIII15

16. www.hdfgroup.org The HDF Group HDFView New Features to Support References and Quality Flags November 3-5, 2009HDF/HDF-EOS Workshop XIII16

17. www.hdfgroup.org HDFView and references Aggregated granules: show the whole dataset pointed to by an object reference Granules: show data pointed to by a dataset region reference November 3-5, 200917HDF/HDF-EOS Workshop XIII

18. www.hdfgroup.org Objects Pointed to by Object References November 3-5, 2009HDF/HDF-EOS Workshop XIII18

19. www.hdfgroup.org Data Pointed to by Region References November 3-5, 2009HDF/HDF-EOS Workshop XIII19

20. www.hdfgroup.org HDFView and quality flags By default, integers with quality flags are displayed as 8-bit decimal values Needed: a way to show values of specific quality flag bit fields November 3-5, 200920HDF/HDF-EOS Workshop XIII

21. www.hdfgroup.org Default display: 8-bit decimal integers November 3-5, 2009HDF/HDF-EOS Workshop XIII21

22. www.hdfgroup.org Displaying quality flags in 8-bit integers Use “Open As” on dataset with quality flags. Apply bitmask to choose individual quality flags November 3-5, 2009HDF/HDF-EOS Workshop XIII22

23. www.hdfgroup.org Examples of bit masks Applied mask in red at top-right, e.g., “By bitmask {0, 1}” November 3-5, 2009HDF/HDF-EOS Workshop XIII23

24. www.hdfgroup.org Quality Flags Data can be displayed in decimal, binary, or hexadecimal November 3-5, 2009HDF/HDF-EOS Workshop XIII24 binary decimal hexadecimal

25. www.hdfgroup.org HDFView plug-in to Show Quality Flags Implemented using “TreeView” plug-in Open HDFView Select “User Options” Select NPOESS plug-in Restart HDFView Available with HDFView 2.5 patch3 November 3-5, 2009HDF/HDF-EOS Workshop XIII25

26. www.hdfgroup.org The HDF Group Possible Future Work November 3-5, 2009HDF/HDF-EOS Workshop XIII26

27. www.hdfgroup.org Possible future work Add new features Enhance current features System support User support November 3-5, 2009HDF/HDF-EOS Workshop XIII27

28. www.hdfgroup.org Maintenance Maintenance NPOESS/NPP features High-Level Library H5dump, other tools? HDFView Testing, documentation, releases November 3-5, 2009HDF/HDF-EOS Workshop XIII28

29. www.hdfgroup.org New features Support Variable-Length datasets Ease access to information about quality flags Use definition of quality flags, which are in: The Common Data Format Control Book XML files as part of the CDFCB-X delivery Apply the structure of the XML to determine whether a field is a packed quality flag byte Tell us what you need! November 3-5, 2009HDF/HDF-EOS Workshop XIII29

30. www.hdfgroup.org Enhance current features Extend reference display options E.g., show properties of the referenced data Revise default TreeView and TableView to improve NPOESS plug-in support Improve quality of testing, documentation, and binary and source distribution Tell us what you need! November 3-5, 2009HDF/HDF-EOS Workshop XIII30

31. www.hdfgroup.org NPOESS/NPP System Support Daily testing on systems critical to NPOESS/NPP Linux 32 and 64-bit Windows AIX 5.3 Solaris 32 and 64-bit Regular testing of NPOESS high-Level library Linux 32 and 64-bit Windows Solaris 32 and 64-bit Porting to requested compilers Performance tuning for applications Improved support for C++ Support for netCDF-4 access to NPOESS SDR and EDR products November 3-5, 200931HDF/HDF-EOS Workshop XIII

32. www.hdfgroup.org NPOESS/NPP User Support Priority support to NPOESS/NPP users Send email to help@hdfgroup.orghelp@hdfgroup.org Add NPOESS to the subject field Example: Subject: [NPOESS] Where is the plug-in? Response in less than 4 hours Continue gather requirements and enhance HDF5 software as needed We want to hear from you! November 3-5, 200932HDF/HDF-EOS Workshop XIII

33. www.hdfgroup.org Special Research Projects Improvements to HDF5 or HDF5 NPOESS products with the data management portion of the Raytheon Algorithm Development Library (ADL) Implement OpeNDAP server for HDF5 NPOESS SDR and EDR products Develop NPOESS specialized version of the HDF5 tool h5check Performance tuning of HDF-NPOESS product files November 3-5, 200933HDF/HDF-EOS Workshop XIII

34. www.hdfgroup.org Questions? November 3-5, 2009HDF/HDF-EOS Workshop XIII34

35. www.hdfgroup.org The HDF Group Questions/comments? November 3-5, 2009HDF/HDF-EOS Workshop XIII35

36. www.hdfgroup.org Acknowledgements This work was supported by cooperative agreement number NA08AANEG0128from the National Aeronautics and Space Administration (NASA). Any opinions, findings, conclusions, or recommendations expressed in this material are those of the author[s] and do not necessarily reflect the views of the National Aeronautics and Space Administration. November 3-5, 2009HDF/HDF-EOS Workshop XIII36