1. © Crown copyright Met Office The EN dataset Simon Good and Claire Bartholomew

2. © Crown copyright Met Office What is the EN dataset? Climate dataset of temperature and salinity profiles Data quality controlled using a suite of automatic checks Monthly objective analyses are created from the data (and used in quality control) EN name has origins in European projects that funded initial versions Current publicly available version of the EN dataset is EN3 (v2a) (see www.metoffice.gov.uk/hadobs/en3)www.metoffice.gov.uk/hadobs/en3 A new version is being prepared (EN4; Good et al. 2013 submitted) In this presentation I am mostly information the new version (EN4)

3. © Crown copyright Met Office The EN dataset Table of Contents Data sources Quality control procedures Data format and dissemination Data users Performance of quality control

4. © Crown copyright Met Office Data sources

5. © Crown copyright Met Office Data sources We use data from World Ocean Database 2009 (WOD09) Global Temperature-Salinity Profile Program (GTSPP) Argo Arctic Synoptic Basinwide Observations project Main data source is WOD09 Monthly updates performed using data from GTSPP and Argo Data from any profiling instrument are used

6. © Crown copyright Met Office Data sources

7. © Crown copyright Met Office Data processing and quality control

8. © Crown copyright Met Office How the dataset is made Quality control Analysis Persistence forecast Output as NetCDF file One month of observations processed per cycle Output as NetCDF file Available from Met Office website Observations

9. © Crown copyright Met Office Quality control Incorporates a duplicate check adapted from the Gronell and Wijffels (2008) Profiles with >400 levels are subsampled Quality control is of both temperature and salinity and is mostly automatic Exceptions are exclusion lists obtained externally and a list of manual rejections developed for a previous version of data EN system shares a code base with ocean forecasting system requiring real time quality control, hence good automatic methods are required Currently do not have any effort given to manual quality control of subsurface data

10. © Crown copyright Met Office Quality control Manual exclusions Track check Profile check (spikes etc.) Thinning (informational) Stability check Background checks Buddy check Multi level check Argo delayed mode flags Argo grey list Argo altimetry quality control External quality information Automatic quality checks Bathymetry check Measurement depths check Waterfall check Near surface and deep BTs

11. © Crown copyright Met Office Track check Spike check Background check

12. © Crown copyright Met Office Quality control code Written in Fortran Owing to way the data are processed and stored, can require a lot of memory if there are a lot of profiles and/or levels Number of levels are thinned to 400 if profile has more A month of data requires between a few seconds to ~25 minutes to run (depending on data quantity) on a desktop machine

13. © Crown copyright Met Office Data availability and use Each month of data is added around the middle of the following month Data are provided on Met Office website (current version at www.metoffice.gov.uk/hadobs/en3) www.metoffice.gov.uk/hadobs/en3 Uses are varied Gridded products Monitoring ocean conditions Time series of ocean heat content Initialising seasonal/decadal predictions Ocean reanalysis Comparisons to climate model data

14. © Crown copyright Met Office Comparison between results from the EN system and the QuOTA dataset

15. © Crown copyright Met Office Comparison between QuOTA and EN processing For profiles with between 2 and 400 temperature levels Results are preliminary – we are still working on understanding the quality control flags and the impact that the differences have We count the number of profiles with any levels rejected Level by level comparison is difficult EN system = automated system CSIRO system = semi-automated system

16. © Crown copyright Met Office Method Comparison For profiles with AutoQC >0 (i.e. flagged up in semi- automated system and so have gone through the Mquest QC – 268550 of these such profiles): All profiles across Indian Ocean and Tasman Sea (582486 in total) 1942-2005 Majority of ‘good profiles failed’ are due to bad fill values in profiles – so are actually correct rejects by the automated system

17. © Crown copyright Met Office Proportion of failed levels For all profiles that have one or more temperature level rejected, the average proportion of levels rejected per profile is: for automated (MO) system: 0.025 for semi-automated (CSIRO) system: 0.169 However, when looking over all profiles, the difference between the two systems is less significant: for automated (MO) system: 0.0056 for semi-automated (CSIRO) system: 0.0065 (as semi-automated system has more profiles with all levels passed, and so help to lower the average when looking at all profiles.) This analysis is done over 4 months of all profiles (not just ones with AutoQC > 0).

18. © Crown copyright Met Office ‘Typical’ profiles missed by EN system but rejected by CSIRO system: Act codes: TO(2) TV(2) Act codes: TO(3) Act codes: TO(2) TZ(2)

19. © Crown copyright Met Office 15 0, 1, 2 ‘Typical’ profiles rejected by EN system but not by CSIRO system: 15

20. © Crown copyright Met Office Typical profiles caught and rejected by both systems EN - Rejected by Bayesian and buddy checks CSIRO – Temperature offset flag – erroneous temp compared to climatology/neighbouring profiles – full profile rejected EN - Rejected by Bayesian and buddy checks and temperature being out of a reasonable range CSIRO – Wire break flag CSIRO – Gradient spike Temperature offset flag EN - Rejected by Bayesian and buddy checks

21. © Crown copyright Met Office Impact on applications (only had a quick look so far) Localised differences can be fairly large Area average differences tend to be small

22. © Crown copyright Met Office Questions and answers

23. © Crown copyright Met Office Bit codes Profiles: 0 – Temperature reject 2 – Thinning flag (not a reject) 3 – Track check 4 – Vertical stability 6 – Bathymetry reject 11 – Vertical check 12 – No background 13 – 50% levels rejected Levels: 0 – Temperature reject 2 – Vertical stability 3 – Increasing depth 8 – BT shallow or deep obs reject 10 – Temperature out of range 12 – Vertical check 13 – No background 14 – Background check on observed levels 15 – Background/buddy 16 – Level reinstated after background reject

24. © Crown copyright Met Office

25. QC decisions for profiles rejected by the CSIRO system but missed by EN system: BO – Bowing problem or bowed mixed layer CL – Contact Lost (probe records before entering water) CS – Surface spike CT – Constant temperature, hit bottom DD – Statistical screening flag DO – DP - Duplicate drop or depth corrected DT - DU – Duplicate drop GL – Gradient long (inversion) GS – Gradient short (spike) HB – Hit Bottom IP – Input record LE – Leakage M1/2 – Statistical screening flag NG – No good trace NT – No trace PE – Position error SO – Surface offset (stats screening) SP – Spike TO – Temperature offset UR – Under resolved WB – Wire break Just selecting QC flags with associated flag severity of 3 or 4.

26. © Crown copyright Met Office QC decisions for profiles rejected by the EN system but passed by CSIRO system: Bit codes: 2 – Thinning flag (not a reject flag) 3 – Track check reject 4 – Profile vertical stability reject 6 – Bathymetry reject 11 – Temperature level rejected because on EN3 reject list 12 - Temperature level rejected due to vertical check (spikes etc.) 13 – Temperature level rejected because no background value available 14 - Temperature level rejected due to Bayesian background check 15 - Temperature level rejected due to Bayesian and buddy checks 16 – Temperature level reinstated after rejection by the Bayesian and buddy checks (Removed 0, 8 and 10 from plots)

27. © Crown copyright Met Office QC decisions for ‘bad’ profiles caught in both systems All flag severities Flag severity 3 or 4