The HDF Group July 8, Summer ESIP Federation Meeting How to Meet the CF Conventions with NcML for NASA HDF/HDF-EOS Hyo-Kyung Joe Lee and Ted Habermann The HDF Group 1

Summer ESIP Federation Meeting The CF Conventions 2 Big and complex! Just focus on a few key conventions: coordinate variables & attributes (bald) valid_range / _FillValue (fat) scale_factor / add_offset (short) units ($ vs. ₵)

Summer ESIP Federation Meeting Why the key CF conventions matter 3 If a data product doesn’t follow them, your NetCDF visualization tool like IDV is useless.

Summer ESIP Federation Meeting But if you follow them carefully… 4 you can visualize data instantly & correctly.

Summer ESIP Federation Meeting OBPG L3 NetCDF-4 Example 5 The previous screenshots are created from a NASA OBPG beta product. The product doesn’t work with IDV as is. But we made the product work by augmenting data with NcML.

Summer ESIP Federation Meeting NcML 6 NetCDF Markup Language XML representation of netCDF metadata It can be an input file for editing metadata.

Summer ESIP Federation Meeting Why NcML? 7 Isn’t CDL enough? No. NcML has more features and flexibility. Plus, NcML works with THREDDS Data Server.

Summer ESIP Federation Meeting What OBPG NC4 missed - #1 8 Coordinate variables are not provided. Dataset { String palette[rgb = 3]; Int16 /geophysical_data/sst[rows = 4320][columns = 8640]; Byte /geophysical_data/sst_qual[rows = 4320][columns = 8640]; Float32 /geophysical_data/Lat[rows= 4320]; Float32 /geophysical_data/Lon[columns = 8640]; } testAll/A L3m_MO_AT109_SST_4.nc4; The CF convention expects lat/lon variables.

Summer ESIP Federation Meeting Add new coordinate variables in NcML. 9 <netcdf location="c:/tomcat/webapps/thredds/share/testdata/cdmUnitTest/it/content/thr edds/public/testdata/A L3m_MO_AT109_SST_4.nc4" xmlns=" …

Summer ESIP Federation Meeting Now IDV can visualize data but… 10 No data on Sea w/ Range = [-1, -1]? Examine units.

Summer ESIP Federation Meeting What OBPG NC4 missed - #2 11 “units” attribute has “degrees-C” value. Accepted values in the CF Convention: Celsius / celsius / degree_centigrade / degC / degreeC / degree_C / degree_c / deg_C / deg_c Remove “s-” from “degrees-C”.

Summer ESIP Federation Meeting Modify attribute value with NcML. 12 … … Before: After:

Summer ESIP Federation Meeting What OBPG NC4 missed - #3 13 The actual fill value is -1 according to HDFView. Attributes { /geophysical_data/sst { Int16 _FillValue ; Float64 valid_min -5.0; Float64 valid_max 50.0; } HDFView

Summer ESIP Federation Meeting Modify _FillValue attribute with NcML. 14 … … Now every value is a fill value – “missing” in IDV.

Summer ESIP Federation Meeting What OBPG NC4 missed - #4 15 valid_min / valid_max doesn’t match the range of actual data. Attributes { /geophysical_data/sst { Int16 _FillValue ; Float64 valid_min -5.0; Float64 valid_max 50.0; } No scale/offset attributes are defined either.

Summer ESIP Federation Meeting Add scale/offset attributes with NcML. 16 Please note that all types match as float. _FillValue’s type matches the data type.

Summer ESIP Federation Meeting Finally, OBPG met the CF conventions. 17

Summer ESIP Federation Meeting But tools behave differently. 18 The same NcML doesn’t work for GoDiva2. NC4 “Group” is the reason. GoDiva2: lat/lon outside group IDV/Panopy: lat/lon inside group Is CF/NcML ready for “Group”?

Summer ESIP Federation Meeting GoDiva2 19

Summer ESIP Federation Meeting Panoply 20

Summer ESIP Federation Meeting Acknowledgement 21 This work was supported by Subcontract number under Raytheon Contract number NNG10HP02C, funded by the National Aeronautics and Space Administration (NASA). Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of Raytheon or the National Aeronautics and Space Administration.