GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 Fabrice Hernandez (Mercator Océan, Fr) and Matt Martin (Met Office, UK) GODAE OceanView Inter-comparison and Validation Task Team: Work Plan Main objectives of the task team: Provide a demonstration of inter-comparison and validation in an operational framework Improve inter-comparison and validation methodology for operational oceanography Link with other groups and activities

GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 IV-TT worplan Pursues activities developed during GODAE Develop a framework for comparing outputs of the various operational ocean forecasting systems (OOFS) –leads to improvements to the systems and to the quality of products from those systems –provides a framework for scientific discussions on forecasting system performance assessment, on ocean analysis and validation from numerical simulations –offers a demonstration of the work in GODAE OceanView and has the potential to increase the visibility to the external community

GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 IV-TT worplan Coordinates and promotes the development of scientific validation and intercomparison of operational oceanography systems –For short term forecast assessment, but benefit for/from reanalysis assessment; and for medium range predictions (eg seasonal) in collaboration with CLIVAR/GSOP –include the definition of metrics to assess the quality of analyses and forecasts (e.g. forecast skills) both for physical and biogeochemical parameters and the setting up of specific global and regional intercomparison experiments –metrics related to specific applications also considered (eg, lagrangian statistics) –New approach for ocean estimates, like multi-model ensemble, should be promoted –links with the OSE-TT –links with the JCOMM ET-OOFS team for operational implementation

GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 Contribution to IV task team Australia: BlueLink Development of validation packages and validation methodologies for both reanalysis and ocean forecast systems. Development of automated inter-comparison with other GODAE OceanView partner forecast systems. Brazil: Start participation in 2011 by preparing routines to validate ocean forecasts and assess the short-term predictability of the systems with focus on the METAREA V and regions along the Brazilian shore. As soon as the results become available, they will be presented on the REMO website. From 2012 on more active participation will occur, including participation in the Coastal Ocean and Shelf Seas TT. Canada: DFO/CONCEPT Development of validation packages and validation methodologies for both reanalysis and ocean forecast systems. This includes presentation methodologies over the web for demonstrating value of forecast systems. Development of automated inter-comparison with other GODAE OceanView partner forecast systems. China: SOA/NMEFC

GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 Contribution to IV task team Europe: ECMWF Development of validation packages and validation methodologies for both reanalysis and ocean forecast systems. This includes presentation methodologies over the web for demonstrating value of forecast systems, as well as published reports France: Mercator Ocean Co-chair this TT Implementation of new diagnostics / metrics specifically designed for each OOFS or application. This should take into account new parameters / processes like meso-scale dynamics, biological parameters, areas where nesting is done, downstream applications, data assimilation efficiency. Inter comparisons with other OOFSs when possible India: INDOFOS Italy: INGV/MFS Development of validation packages and validation methodologies for MFS by including a large number of independent observations collected by different instates in the Mediterranean. The validation is presented on a web page that is linked with the web page for the analyses and forecast

GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 Contribution to IV task team Japan: MRI/JMA We have a plan for validation in JMA groups. It is not clear that JMA will participate in intercomparison exercises with other GODAE OceanView partner forecast systems. It depends on the experimental conditions of the work plan under the task team. Norway: NERSC/TOPAZ Update of MERSEA metrics for the Arctic. Validation methodology for vector variables (sea-ice drift) provided by NIERSC, Russia. Comparison and validation of systems producing Arctic metrics. Routines provided on demand. Automatic identification of eddies through circlelets (see Chauris et al., 2010). UK: Met Office/FOAM Co-chair this TT. Develop software to perform routine Class 4 inter-comparisons, and share the relevant code with the other GODAE OceanView groups. Develop a system for plotting and summarizing the routine inter-comparison results. Contribute to proposed inter-comparison aimed at monitoring climate indices. Contribute to and help organize working group meetings.

GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 Contribution to IV task team USA: NOAA Development of validation packages and validation methodologies for ocean forecast systems. This includes presentation methodologies over the web for demonstrating value of forecast systems. Development of automated inter-comparison with other GODAE OceanView partner forecast systems. USA: US Navy GOFS Validation testing will examine accuracy in hindcasts, nowcasts and forecasts of sea surface height, the 3-dimensional temperature, salinity and current structure, mixed layer depth, sonic layer depth and the deep sound channel. GOFS V3.0 will provide boundary conditions to regional and nested coastal models and this will be examined along with longer-range (14-day) forecast skill of the quantities listed above. Evaluations will be global, and additional emphasis will be placed on the western Pacific (South China Sea, Yellow Sea, and Japan/East Sea) and Indian (Arabian Sea) Oceans. USA ECCO ECCO will participate intercomparison effort that focuses on climate time scales

GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 Implementing new intercomparison activity Assessing Performance of Ocean Forecasting Systems, using “Class 4” metrics

GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 Class 4 intercomparison implementation Up to now there has been little inter-comparison of the class 4 metrics (forecast performance in the observation space). Request from OSE-TT to provide metrics dedicated to performance assessment Would like to start performing some routine and ongoing inter-comparisons between the GODAE OceanView systems. A technical proposal, based on Class 4 metrics has been defined for routine inter-comparison: in common formats; for a core sub-set of the various data types which the systems assimilate (or not); to enable common assessment of forecasts.

GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 Class 4 intercomparison implementation Common set of reference data: Temperature and salinity profiles from Argo GDAC (USGODAE or Coriolis) Sea level data from Aviso and tides gauges (Coriolis) Sea surface temperature data from in situ surface drifters (USGODAE) and from L3 AATSR (MyOcean) The MDT used by each group will be provided in a separate netCDF file OOFS outputs to be evaluated: [Best estimates] & [1-5 d. forecast] are compared to reference data, and stored in standard files, whose format is specified and will contain: Observation time, location (lat/long), depth, value, type and ID; Best estimate model field interpolated to observation locations; Forecast values interpolated to observation location (from previous forecasts); Persistence values (from previous nowcast); Climatological values; QC information.

GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 Class 4 intercomparison implementation Sharing data and calculating results: Files made available via ftp to the USGODAE server within 10 days of the observation time. The files will be archived on the USGODAE server for future use. Any of the participants should be able to perform the inter-comparison as they like. But a standard set of statistics calculated on a monthly basis is already planned: –RMS differences, bias and anomaly correlation against forecast time for various regions (ie using the existing GODAE region definitions). The results of the monthly inter-comparison could be made available on the GODAE OceanView or JCOMM ETOOFS web-sites Results produced by any of the participants will not be publicised outside of the group of participants, without agreement of all participants. Existing software to be shared (M. Martin)

GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 Class 4 intercomparison implementation Operational Class 4 OND 2010

GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 Class 4 intercomparison implementation Operational Class 4 JFM 2011

GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 Link with OSE-OSSE TT activities: Use the metrics already defined for operational systems, and assess data impact in a comparable manner

GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 Link with other intercomparison activities: Intercomparison performed by the “operational reanalysis” community Y. Xue (NOAA), M. Balmaseda (ECMWF), A. Kumar (NOAA), N. Ferry (M-O), M. Rienecker (NASA), T. Rosati (NOAA), S. Good (UK-Met), I. Iishikawa (JMA), T. Boyer (NOAA), O. Alves (BoM)

GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 Link with “operational reanalysis” community Objectives of ocean operational analysis intercomparison –from OceanObs’09 White Paper: OCEAN STATE ESTIMATION FOR GLOBAL OCEAN MONITORING: ENSO AND BEYOND ENSO by Yan Xue et al. –Then discussion with Yan Xue (NCEP) and M. Balmaseda (ECMWF) –Raising interest in the CLIVAR-WGSIP (Working Group on Seasonal Interannual Prediction) action item "...[explore] who is interested in participating in developing a strategy to assess real-time ocean analyses..." –Routine comparison of operational analysis, eg, in a 3-month basis, in a collaboration/or as part of the IV-TT is welcomed –Possible benefit from GODAE ocean forecasting group: Share expertise on “climate signal” detection with ”ORA groups” Assess the reanalysis performed by GODAE ocean forecasting groups Measure the robustness of climate assessment by including high resolution analysis in the intercomparison

GOV Technical Workshop OSE-IV - Santa Cruz, USA, June 2011 Link with “operational reanalysis” community Possible objectives of ocean operational analysis intercomparison: –« how to derive ocean climate indices from multiple operational ocean analyses » –Produce ensemble ORA heat content indices, which will provide monitoring at quasi real time of climate signals with uncertainty estimations. –Verification of the different ORA products by comparison with observational estimates (Altimeter, EN3, NODC, …) when possible. This raises awareness on the weakness/strengths of the different data assimilation products, and identifies the areas where consensus on the current state of the ocean is possible and those where more efforts are needed. –Impact of data distribution on the quality of the ocean analysis, and seasonal forecast errors –Ongoing work: Describe the heat content indices in the context of understanding global energy budget, monitoring and forecasting climate on seasonal to decadal time scales. Paper in preparation, which goal is to assess the quality of upper 300m heat content analysis derived from multi-model operational ocean reanalyses, and their capabilities in describing climate signals associated with interannual variability (ENSO, IOD), decadal variability (PDO, TAV and AMOC) and trends.