GHRSST Users Symposium 2009 Working with the GHRSST Data Format Experiences of the GCOS SST/SI Intercomparison Working Group Tess Brandon, NOAA National Oceanographic Data Center 28 May 2009

GHRSST Users Symposium 2009 Outline Background: the GCOS SST/SI Intercomparison Framework Methodology: GCOS SST Intercomparisons and the GHRSST Data Specification User reports –Statistics –Anecdotes Conclusions and Future work

GHRSST Users Symposium 2009 GCOS SST Intercomparison Framework Current Status 1.Six historical products (1850 on), as well as standard diagnostics and browse graphics, added to website 2.Standard diagnostics added for all data sets w.r.t. their own climatologies 3.Pilot study underway to determine the origins of observed differences (from QC, analysis methodology) using a common set of input data Project Goals 1.To understand differences between global and near-global SST analyses with the aim of producing better, long- term SST climate data records 2.To use this understanding to link the modern satellite-based records with the historical, primarily in situ-based records Looking Forward 1.Add SST analyses from pilot study to intercomparison framework 2.Complete standard diagnostics and update web interface 3.Summarize differences as a set of strengths and weaknesses for each product to provide guidance to users of SST analysis products

GHRSST Users Symposium 2009 GCOS SST Intercomparison Data Sets Data Set Name Satellite Era ( ) Historical Era ( ) Original Format AVHRR Pathfinder V5XHDF Operational AVHRRXText NOAA Optimum Interpolation V2XnetCDF NOAA Daily ¼-degree OI V2XText Hadley Centre SST V2XXnetCDF Hadley Centre Sea Ice and SST V1XXnetCDF NOAA Extended Reconstruction V3XXText Kaplan ReconstructedXXnetCDF International COADS V2.4XnetCDF COBE AnalysisXText

GHRSST Users Symposium 2009 Using the GHRSST Data Specification Structure –“Data cubes” at weekly, one degree and monthly, five degree resolution  time dimension > 1 –GHRSST-compliant netCDF –Matlab mimicry Content –Default to Level 4 content specification –Some Level 3-like products –Missing required ancillary fields for Level 2P –Inconsistent error fields

GHRSST Users Symposium 2009 SST Intercomparison File Format: netCDF dimensions: time = 1366 ; lat = 180 ; lon = 360 ; variables: double time(time), shape = [1366] time:long_name = "reference time of sst field" time:standard_name = "time" time:units = "seconds since :00:00" time:axis = "T" time:calendar = "Gregorian" time:add_offset = e+07 double lat(lat), shape = [180] lat:long_name = "latitude" lat:standard_name = "latitude" lat:units = "degrees_north" lat:axis = "Y" double lon(lon), shape = [360] lon:long_name = "longitude" lon:standard_name = "longitude" lon:units = "degrees_east" lon:axis = "X" short analysed_sst(time,lat,lon), shape = [ ] analysed_sst:long_name = "analysed sea surface temperature" analysed_sst:standard_name = "sea_surface_temperature" analysed_sst:type = "SST_blend" analysed_sst:units = "kelvin" analysed_sst:scale_factor = 0.01 analysed_sst:add_offset = analysed_sst:valid_min = -300 s analysed_sst:valid_max = 4500 s analysed_sst:_FillValue = s

GHRSST Users Symposium 2009 SST Intercomparison File Format: Matlab NameSizeBytesClass CoordinateVariables1x13270struct Dimensions1x1552struct GlobalAttributes1x15958struct Variables1x17501struct analysed_sst1366x180x int16 analysis_error1366x180x struct lat1x double lon1x double mask1366x180x int8 sea_ice_fraction1366x180x int16 time1366x110928double >> Variables.analysed_sst ans = long_name:‘analysed sea surface temperature’ standard_name:‘sea_surface_temperature’ type:‘SST_blend’ units:‘kelvin’ scale_factor: add_offset: valid_min:-300 valid_max:4500 missing_value:-32768

GHRSST Users Symposium 2009 Intercomparison Facility Files File typeData and information Zipped netCDF (.nc.gz)- weekly, one degree data cubes (satellite era only) netCDF (.nc)- all monthly, five degree data cubes - all standard diagnostics data - all climatology data Matlab (.mat)- all data cubes - all climatology data Image (.jpg)- all browse images (from standard diagnostics) Matlab figure (.fig)- browse images from historical era only Other (.m, ascii)- processing code, ancillary files used in processing code

GHRSST Users Symposium 2009 Intercomparison Facility: Access Statistics User statistics for all GCOS intercomparison facility files downloaded between February 2008 and May 2009

GHRSST Users Symposium 2009 Intercomparison Facility: Access Statistics File typeNumber of requests % all bytes requested % Available files of that type netCDF, zipped (.nc.gz) Graphic (.jpg) Matlab (.mat) netCDF (.nc) Matlab figure (.fig) User statistics for all GCOS intercomparison facility files downloaded between February 2008 and May 2009

GHRSST Users Symposium 2009 Intercomparison Facility: Access Statistics Number of requests by domain Number of requests by organization User statistics for all GCOS intercomparison facility files downloaded between February 2008 and May 2009

GHRSST Users Symposium 2009 User Anecdotes Working group members’ experiences and opinions: –One member: Lack of simple documentation for Climate Forecast convention: best practices, explanation of rationale for the standard –Another member: Preference for formatted text accompanied by a format standard, due to steep learning curve and initial set-up time required for netCDF –The argument for simple text: “…anyone with any computer could most easily use the data.” Responses varied according to the format of choice already being employed by each member for his or her own data set.

GHRSST Users Symposium 2009 Conclusions and Future Work Overall, the GHRSST format specification was found to be flexible enough to accommodate data sets for the GCOS SST intercomparison framework. In general, users will choose formats that allow them to work with software and scientific analysis tools they already employ. NetCDF can be read by a wide array of tools, and in the case of GCOS SST products, is preferred over the Matlab format. In order to better compare formats by user preference (and perhaps, better address the needs of the user community), the intercomparison facility should add simple text with format statements as one of the standard formats for GCOS data sets. The LTSRF will explore capturing current GHRSST data sets in “data cubes” similar to those used in this project (netCDF4?) to facilitate access to time series of GHRSST data.

GHRSST Users Symposium 2009 References 1.4 km AVHRR Pathfinder Version 5 SST Project, 2.The GHRSST-PP Science Team and GDS-TAG working group, 2008: The Recommended GHRSST- PP Data Processing Specification GDS (Version 2.0 revision 0.1) L4 product specification. 3.Ishii, M. et al., 2005: Objective Analyses of SST and Marine Meteorological Variables for the 20 th Century using ICOADS and the KOBE Collection. Int. J. Climatol., 25, Kaplan, A. et al,. 1998: Analyses of global sea surface temperature J. Geophys. Res., 103, Rayner, N.A. et al., 2003: Global analyses of sea surface temperature, sea ice, and night marine air temperature since the late nineteenth century. J. Geophys. Res., 108 (D14), /2002JD Rayner, N.A. et al., 2006: Improved analyses of changes and uncertainties in sea surface temperature measured in situ since the mid-nineteenth century: the HadSST2 data set. J. Climate, 19(3) Reynolds, R.W., 1988: A real-time global sea surface temperature analysis. J. Climate, 1, Reynolds, R.W. et al., 2002: An improved in situ and satellite SST analysis for climate. J. Climate, 15, Smith, T. M. et al., 2008: Improvements to NOAA’s historical merged land-ocean surface temperature analysis ( ). J. Climate, 21, Woodruff, S.D. et al, 2005: ICOADS Release 2.1 data and products. Int. J. Climatol. (CLIMAR-II Special Issue), 25, (DOI: /joc.1166). Thank you! Questions?