Guidance for managing international precipitation data Markus Ziese (GPCC) Agenda item number 5.2

Background and Agenda of GPCC Global collection and analysis of in situ data of land-surface precipitation Established in early 1989 at Deutscher Wetterdienst (DWD) on request of WMO WCRP. More than 25 years of service indicates GPCC’s long term experience on the job Current staffing, 10 individuals: 1 head, 3 permanent scientists , 1 project scientist, 1 programmer, 4 support staff Contributing to GEWEX (Global Energy and Water Exchanges Project) and GCOS (Global Climate Observing System) Many users world wide, analyses used in IPCC-AR5, WMO Climate Statements, Copernicus Services and BAMS SoC Data sources: SYNOP, CLIMAT, SYNOP from CPC, ECA&D, CRU, FAO, GHCN, national meteorological services, regional data collections

GPCC Data Base Number of stations in data bank: more than 115,000 Monthly totals, collected since 1989 Daily totals, collected since 2012 2017 Number of stations in data bank: more than 115,000

GPCC Data Flow Data delivered in different formats All data in same format Data stored in data bank QC QC Extracted data for analyses QC

Data Bank Model Provider based Parameter based Parameter based Station based Provider Station Parameter Data Parameter Provider Station Data Station Parameter Provider Data

Data Bank Model Provider based Parameter based Parameter based Station based Pro: Easy import as data feed in as delivered Contra: Station synchronizing at data post-processing needed Difficult comparison between providers Provider specific metadata Pro: Easy import as data feed in as delivered Contra: Synchronizing metadata between parameters Difficult cross-validation between parameters Pro: Stations synchronizing at data import Easy comparison between providers No multiple station for post-processing One station history for all providers Contra: Large effort to import data due to metadata synchronizing

GPCC Data Bank Station based data bank scheme One station can get data from different data providers Allows comparison and quality control between different sources Station Provider A Provider B Provider C Provider D Date 1 value value value Date 2 value value value Date 3 value value value Date 4 value value value

Quality Control Delivered original data needs to be checked Many causes for errors possible Not every apparently wrong value is erroneous, but could be caused by an extreme event QC mustn’t delete correct data Track changes during QC in data bank Metadata needs to be checked

Quality Control Automatic QC at aggregation of SYNOP data Manual QC of offline checked statistically conspicuous aggregated SYNOP and CLIMAT data Manual QC of metadata and temporal shift at reformatting of non-real-time data Automatic QC of data and additional manual QC of statistically peculiar data during import in data bank

Quality Control Metadata: During import: Station within borders of country? Station coordinates and elevation consistent with other sources (OSCAR, Google Maps, GEONAMES) ? Station over land? Spelling of station name During import: Data belongs to existing station in data bank? Metadata needs to be updated? New station?

Quality Control One station: Huddur Huduur Hudur Hodur Oddur Xuddur Xudur 1982 1980 1953 1952 1951 1979

Station History Metadata can be changed due to relocations, automatisation etc. over time Store metadata as station history with validity date (valid from … till …) Coordinates & elevation Station number (national number, WMO-number) e.g., Erfurt-Bindersleben: 09554 from 1946/04/01 to 1990/10/02 10554 since 1990/10/03

Station History Use these metadata, which are valid at the date of the value/observation Location of station can change over time in the time series

Tracking Changes in Data Reproduce changes at QC Use “best values” for further processing Keep a copy of original values in data bank

Tracking Changes in Data Quality control leads to: Confirmed data: flagged values are caused by extreme events Corrected data: original value was wrong and replaced by better one, e.g., due to factor 10 error Deleted data: original value was obviously wrong, but no information available to correct it

Tracking Changes in Data For one station - one provider - one date: Maximum two values can exist Original and confirmed original value Original and corrected value Original and deleted value Wrong data are flagged to be ignored in analyses, but not physically deleted from data bank -> option to still confirm original value if later arriving additional and more reliable information supports this

Experience from GPCC Store data in a station specific data bank Bundle station history with observations Store data separated according to provider to allow for cross-comparison Apply advanced quality control procedures Track changes during QC in data bank, do not delete original data physically Use best available data at each time step Respect and document copyrights and intellectual property rights (IPR) of data providers

Thank you Merci