S2S sub-project on verification (and products) Objectives: Recommend verification metrics and datasets for assessing forecast quality of S2S forecasts Provide guidance for a potential centralized effort for comparing forecast quality of different S2S forecast systems, including the comparison of multi-model and individual forecast systems and consider linkages with users and applications Issues to be addressed: Identification of current practises in sub-seasonal to seasonal forecasts Identification of user-relevant variables and quantities to be verified Provision of guidance on minimum hindcast standards (hindcast length and ensemble size) Promotion of subseasonal forecasting intercomparison efforts and evaluation of benefit of multi-model approach Links with WMO Joint Working Group on Forecast Verification Research (JWGFVR), WMO CBS/CCl ET-OPSLS, WMO LC-LRFMME and other S2S sub-projects Sub-project members: Caio Coelho, Andy Robertson, Yuhei Takaya, Richard Graham, Debra Hudson, Joanne Robbins, Angel Muñoz

S2S sub-project on verification (and products) Science plan developed with inputs from JWGFVR, SERA and S2S: http://www.s2sprediction.net/resources/documents/sub-projects/Verification.pdf Wiki page: Chanel for sub-project activities dissemination http://s2sprediction.net/xwiki/bin/view/Main/Verification Guidance document to stimulate scientific community to address S2S verification problems

Sub-project activities List of published literature on verification methods of relevance to S2S verification posted on the sub-project wiki page including: Books and technical reports Scientific papers

Sub-project activities Literature survey on S2S verification posted on the sub-project wiki page List of published literature on S2S verification addressing the following topics: Assessment of S2S systems forecast skill Assessment of MJO/ISO forecast skill Assessment of monsoon systems forecast skill and associated characteristics Applications Seamless verification

Sub-project activities Collaboration bwt S2S and WMO: Questionnaire on subseasonal verification practices in operational centers (GPC) prepared, with the purpose of sharing current practices used to verify subseasonal forecasts (both for operations and research) and also help identify gaps and guide novel developments. Summary of responses discussed with WMO CBS/CCl ET-OPSLS and posted on the sub-project wiki page Development of pilot real-time sub-seasonal MME predictions: WMO LCLRFMME developed a pilot system for real-time multi-model subseasonal forecasts using real-time forecasts (and hindcasts) from a subset of models contributing to the S2S project accessible via ECMWF data archive. The S2S wiki page provides a link for the S2S research community to see the initial developments and provide feedback for future developments and improvements in this pilot under development system. Coordination of input for WMO ET-OPSLS on S2S application areas under development and operational needs and posted on wiki page

Sub-project activities Following suggestion of S2S SG-LG meeting (Jeju 2015) a comprehensive collection of links to available datasets for S2S verification posted on the sub-project wiki page Links to reference datasets (Reanalysis and gridded products) for assessing S2S forecast quality Atmospheric parameters (e.g. geopotential height, temperature, SLP, wind, etc) Oceanic parameters Surface parameters Datasets accessible via the KNMI Climate Explorer including links for data access via the IRI data library

Sub-project activities Links and instructions on how to access S2S project model datasets posted on wiki Summary table of S2S project models Links for accessing S2S models data at ECMWF and CMA data portals Links to instructions on how to extract S2S models data from ECMWF S2S portal Link to IRI Data Library for accessing subset of these data (ECMWF, NCEP and CMA models): various file formats including OpenDAP access, including link to an introduction to the IRI Data Library Links to S2S forecast products webpages developed by ECMWF and S2S Museum (University of Tsukuba/Oxford) posted on wiki

Sub-project activities S2S included in the JWGFVR user-oriented verification challenge https://www.wmo.int/pages/prog/arep/wwrp/new/FcstVerChallenge.html Organizing S2S verification session at the 7th International Verification Methods Workshop (IVMW) in coordination with JWGFVR, Berlin,8-11 May 2017, preceded by verification tutorial, 3-6 May 2017. First announcement for abstract submission already circulated to S2S SG-LG mailing list. Workshop website will soon be available. Key-note talk by S2S co-chair (Frederic Vitart) on verification methods and procedures used in sub-seasonal verification

Sub-project activities Preparing forecast verification chapter for S2S book co-edited by S2S co-chairs Proposed chapter outline 1) Introduction Definition of what is meant by forecast verification Explanation of why forecasts need to be verified Definition of forecast goodness Attributes of forecast quality Touch on what makes S2S forecast verification different from verification at other time scales 2) Identification of verification goals Designing verification questions to be addressed Choosing verification measures/graphics 3) Observational references Recognizing uncertainties in observational datasets Matching forecasts and observations 4) Review of most common verification measures Deterministic measures Probabilistic measures 5) Types of S2S forecasts and current verification practices Literature review of what has been done so far in S2S verification S2S verification examples 6) Summary, recommendations and challenges in S2S verification Chapter authors: Caio Coelho, Barbara Brown, Laurie Wilson, Marion Mittermaier, Barbara Casati (JWGFVR members)