1. Data services: Accessing EURO4M and UERRA data in MARS 3-4.2.2016UERRA Workshop Toulouse1 Richard Mládek, Manuel Fuentes (ECMWF)

2. ECMWF MARS MARS - (Meteorological Archive and Retrieval System) a managed archive. GRIB, BUFR and ODB (in future NetCDF). Managed archive means the data has to follow a certain structure, based on archiving and retrieval patterns (needs to know how the data is going to be produced/used before deciding how to store it) Efficient data retrieval – it is important to follow recommendations how to retrieve data especially when requesting large amounts. 3-4.2.2016UERRA Workshop Toulouse2

3. EURO4M testbed  Archive some data from EURO4M before UERRA samples are available  Solution chosen:  Original data which is available in GRIB1 (no common definitions of the parameters) for period 2008-2009  If possible provide 15 selected surface parameters only  By the end of 2015 two year sample data was archived from 4 centres  Data is very varying among centres (different parameters, starting times, levels.. ) => difficult to use for inter-comparison  Data is available only to authorized users via standard MARS retrieval tools available 3-4.2.2016UERRA Workshop Toulouse3

4. EURO4M testbed 3-4.2.2016UERRA Workshop Toulouse4 https://software.ecmwf.int/wiki/display/UER/Progress+status EURO4M domain

5. EURO4M testbed  10 meter u-velocity  10 meter v-velocity  Convective precipitation  Large-scale precipitation  Orography  Snow depth water equivalent  Snow fall water equivalent  Surface air maximum temperature  Surface air minimum temperature 3-4.2.2016UERRA Workshop Toulouse5  Surface air relative humidity  Surface air temperature  Time-integrated surface net solar radiation  Time-integrated surface solar radiation downwards  Total cloud cover  Total precipitation 15 selected surface parameters:

6. EURO4M testbed CentreUM/4DVAR (MO)HIRLAM (SMHI) MESCAN (MF)COSMO (DWD) Domain size (projection) 480*384 (rot.latlon) 326*341 (rot.latlon) 1080*1000 (Lamb.conf) 848*824 (rot.latlon) Typesfcan / fc an Times3, 9, 15, 21h0, 6, 12, 18h 0, 1, 2...23h Steps(fc only)0, 1, 2...27h6, 12…36h6h Levelsml(71) / pl(20) / sfcsfc Parameters7(ml)/5(pl)/51(sfc)5(an) / 15(fc) 3(an) / 4(fc)16 (fluxes are not accumulated!) 3-4.2.2016UERRA Workshop Toulouse6 ml..model levels pl..pressure levels sfc..surface level an..analysis fc..forecast rot.latlon..Rotated Latitude/Longitude grid Lamb.conf..Lambert Conformal

7. EURO4M testbed CentreUM/4DVAR (MO)HIRLAM (SMHI)MESCAN (MF) Parameters type=fc level=sfc o 1.5m specific humidity o 1.5m temperature over land o 10 metre wind gust in the last 24 hours o 10m U wind over land o 10m V wind over land o Clear-sky (II) down surface sw flux o Clear-sky (II) up surface sw flux o Convective precipitation (water) o Convective rainfall rate o Convective snow o Convective snowfall rate o Fraction of sea-ice in sea o High cloud cover o Land-sea mask o Large scale rainfall rate o Large scale precipitation o Large scale snow o Large scale snowfall rate o Latent heat flux o Long wave radiation flux o Low cloud cover o Maximum temperature at 1.5m since previous post- processing o Mean sea level pressure o Medium cloud cover o Minimum temperature at 1.5m since previous post- processing o Net long-wave radiation flux (surface) o Net short-wave radiation flux (surface) o Orography o Precipitation rate o Relative humidity at 1.5m o Sensible heat flux o Short wave radiation flux at surface o Short wave radiation flux at top of atmosphere o Soil temperature layer 1 o Soil temperature layer 2 o Soil temperature layer 3 o Soil temperature layer 4 o Surface pressure o Temperature o Total Cloud Cover o Total Precipitation o Total cloud amount - random overlap o Total cloud amount in lw radiation o Total column water vapour o Very low cloud amount o Visibility at 1.5m o Volumetric soil water layer 1 o Volumetric soil water layer 2 o Volumetric soil water layer 3 o Volumetric soil water layer 4 o 10 metre U wind component o 10 metre V wind component o 2 metre temperature o Convective precipitation (water) o Convective snow o Global radiation flux o Large scale precipitation o Large scale snow o Long wave radiation flux o Net long-wave radiation flux (surface) o Net short-wave radiation flux (surface) o Relative humidity o Snow depth, cold snow o Total Cloud Cover o Total precipitation o 10 metre U wind component o 10 metre V wind component o Surface solar radiation downwards o Surface thermal radiation downwards 3-4.2.2016UERRA Workshop Toulouse7

8. EURO4M testbed CentreHIRLAM (SMHI)MESCAN (MF)COSMO (DWD) Parameters type=an level=sfc o 10 metre U wind component o 10 metre V wind component o 2 metre temperature o Relative humidity o Total Cloud Cover o 2 metre temperature o Relative humidity o Total precipitation o 10 metre U wind component o 10 metre V wind component o 2 metre temperature o 2m Relative Humidity o Convective rain rate o Convective snowfall rate water equivalent o Downward direct short wave radiation flux at surface (mean over forecast time) Initialisation o Geometric Height of the earth surface above sea level o Large scale rain rate o Large scale snowfall rate water equivalent o Max 2m Temperature (i) Initialisation o Min 2m Temperature (i) Initialisation o Net short wave radiation flux (at the surface) o Snow depth o Total Cloud Cover o Total Precipitation 3-4.2.2016UERRA Workshop Toulouse8

9. EURO4M testbed EURO4M data in MARS 2 main MARS key words identifying EURO4M datasets:  class = rm  expver = 1, 2, 3, 4 1 => UM/4DVAR (MO) 2 => HIRLAM (SMHI) 3 => MESCAN (MF) 4 => COSMO (DWD) Data volume (13.4 GB): UM/4DVAR (MO): 7.8 GB HIRLAM (SMHI): 4.1 GB MESCAN (MF): 952 MB COSMO (DWD): 525 MB 3-4.2.2016UERRA Workshop Toulouse9

10. MARS access 3-4.2.2016UERRA Workshop Toulouse10 1.via MARS batch requests run on unix command line 2.via ECMWF WebAPI ( an alternative way for downloading MARS data in a programmatic way via Web ) ECMWF WebAPI is a set of services developed by ECMWF to allow users from the outside to access some internal features and data of the centre. So far 2 services: Access MARS Access ECMWF public datasets (TIGGE, S2S, ERA40, UERRA…) EURO4M

11. Accessing EURO4M 3-4.2.2016UERRA Workshop Toulouse11 (via Access MARS service) Follow 3 easy steps: 1. Install ECMWF WebAPI follow instructions at https://software.ecmwf.int/wiki/display/WEBAPI/Access+MARS https://software.ecmwf.int/wiki/display/WEBAPI/Access+MARS & download the “mars” sample script 2. Check EURO4M data availability browse MARS web catalogue at http://apps.ecmwf.int/mars-catalogue/?class=rmhttp://apps.ecmwf.int/mars-catalogue/?class=rm or write MARS list request (mars_list.batch) and run it:mars mars_list.batch 3. Get the data Write MARS retrieval request (mars.batch) and run it:mars mars.batch

12. Check for availability View MARS request Estimate download size Retrieve.. Retrieve now Active request Finished! Download the data

13. Retrieve.. 3-4.2.2016UERRA Workshop Toulouse13

14. Links  ECMWF Web API tutorial for accessing MARS https://software.ecmwf.int/wiki/display/WEBAPI https://software.ecmwf.int/wiki/display/WEBAPI 3-4.2.2016UERRA Workshop Toulouse14 Retrieve now

15. Summary Done: New GRIB2 class=ur (uerra) approved by WMO UERRA website at ECMWF Preliminary work on 4 EURO4M datasets archiving in MARS In progress: Finish archiving of EURO4M Gathering first UERRA data sample GRIB2 common definitions and convertion tools to achieve identical UERRA/WMO compliant format for all datasets Data checking tool amendment (tigge_check) TOP PRIORITY: Inventory of all UERRA parameters & datasets' metadata to be able to start MARS and GRIB_API related developments 3-4.2.2016UERRA Workshop Toulouse15