1. The HDF Group www.hdfgroup.org July 8, 20142014 Summer ESIP Federation Meeting How to Meet the CF Conventions with NcML for NASA HDF/HDF-EOS Hyo-Kyung Joe Lee and Ted Habermann The HDF Group 1

2. www.hdfgroup.org 2014 Summer ESIP Federation Meeting The CF Conventions 2 Big and complex! Just focus on a few key conventions: coordinate variables & attributes (bald) valid_range / _FillValue (fat) scale_factor / add_offset (short) units ($ vs. ₵)

3. www.hdfgroup.org 2014 Summer ESIP Federation Meeting Why the key CF conventions matter 3 If a data product doesn’t follow them, your NetCDF visualization tool like IDV is useless.

4. www.hdfgroup.org 2014 Summer ESIP Federation Meeting But if you follow them carefully… 4 you can visualize data instantly & correctly.

5. www.hdfgroup.org 2014 Summer ESIP Federation Meeting OBPG L3 NetCDF-4 Example 5 The previous screenshots are created from a NASA OBPG beta product. The product doesn’t work with IDV as is. But we made the product work by augmenting data with NcML.

6. www.hdfgroup.org 2014 Summer ESIP Federation Meeting NcML 6 NetCDF Markup Language XML representation of netCDF metadata It can be an input file for editing metadata.

7. www.hdfgroup.org 2014 Summer ESIP Federation Meeting Why NcML? 7 Isn’t CDL enough? No. NcML has more features and flexibility. Plus, NcML works with THREDDS Data Server.

8. www.hdfgroup.org 2014 Summer ESIP Federation Meeting What OBPG NC4 missed - #1 8 Coordinate variables are not provided. Dataset { String palette[rgb = 3]; Int16 /geophysical_data/sst[rows = 4320][columns = 8640]; Byte /geophysical_data/sst_qual[rows = 4320][columns = 8640]; Float32 /geophysical_data/Lat[rows= 4320]; Float32 /geophysical_data/Lon[columns = 8640]; } testAll/A20030602003090.L3m_MO_AT109_SST_4.nc4; The CF convention expects lat/lon variables.

9. www.hdfgroup.org 2014 Summer ESIP Federation Meeting Add new coordinate variables in NcML. 9 <netcdf location="c:/tomcat/webapps/thredds/share/testdata/cdmUnitTest/it/content/thr edds/public/testdata/A20030602003090.L3m_MO_AT109_SST_4.nc4" xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"> …

10. www.hdfgroup.org 2014 Summer ESIP Federation Meeting Now IDV can visualize data but… 10 No data on Sea w/ Range = [-1, -1]? Examine units.

11. www.hdfgroup.org 2014 Summer ESIP Federation Meeting What OBPG NC4 missed - #2 11 “units” attribute has “degrees-C” value. Accepted values in the CF Convention: Celsius / celsius / degree_centigrade / degC / degreeC / degree_C / degree_c / deg_C / deg_c Remove “s-” from “degrees-C”.

12. www.hdfgroup.org 2014 Summer ESIP Federation Meeting Modify attribute value with NcML. 12 … … Before: After:

13. www.hdfgroup.org 2014 Summer ESIP Federation Meeting What OBPG NC4 missed - #3 13 The actual fill value is -1 according to HDFView. Attributes { /geophysical_data/sst { Int16 _FillValue -32767; Float64 valid_min -5.0; Float64 valid_max 50.0; } HDFView

14. www.hdfgroup.org 2014 Summer ESIP Federation Meeting Modify _FillValue attribute with NcML. 14 … … Now every value is a fill value – “missing” in IDV.

15. www.hdfgroup.org 2014 Summer ESIP Federation Meeting What OBPG NC4 missed - #4 15 valid_min / valid_max doesn’t match the range of actual data. Attributes { /geophysical_data/sst { Int16 _FillValue -32767; Float64 valid_min -5.0; Float64 valid_max 50.0; } No scale/offset attributes are defined either.

16. www.hdfgroup.org 2014 Summer ESIP Federation Meeting Add scale/offset attributes with NcML. 16 Please note that all types match as float. _FillValue’s type matches the data type.

17. www.hdfgroup.org 2014 Summer ESIP Federation Meeting Finally, OBPG met the CF conventions. 17

18. www.hdfgroup.org 2014 Summer ESIP Federation Meeting But tools behave differently. 18 The same NcML doesn’t work for GoDiva2. NC4 “Group” is the reason. GoDiva2: lat/lon outside group IDV/Panopy: lat/lon inside group Is CF/NcML ready for “Group”?

19. www.hdfgroup.org 2014 Summer ESIP Federation Meeting GoDiva2 19

20. www.hdfgroup.org 2014 Summer ESIP Federation Meeting Panoply 20

21. www.hdfgroup.org 2014 Summer ESIP Federation Meeting Acknowledgement 21 This work was supported by Subcontract number 114820 under Raytheon Contract number NNG10HP02C, funded by the National Aeronautics and Space Administration (NASA). Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of Raytheon or the National Aeronautics and Space Administration.