NOAA National Climatic Data Center Dr. Karsten Shein Climatologist NOAA/NESDIS/NCDC 151 Patton Ave. Asheville, NC Interactive Quality assurance practices A look at some methods for evaluating COOP data at the NOAA National Climatic Data Center IIPS 6A.9

NOAA National Climatic Data Center 2 of 19 COOP Climate Data ~ 8500 stations reporting ~ 1,000,000 observations per month Manual observations and reporting –Most stations observe PRCP (SNOW) –Many also observe TMAX, TMIN, TOBS, SNWD –Few: DYSW, EVAP, WTEQ, WDMV, Soil T Daily data / Monthly processing Arrive at NCDC in electronic (daily) and paper (monthly) formats.

NOAA National Climatic Data Center 3 of 19 Sources of bias in COOP data Observations –Rounding, Instrument error, incorrect obs technique Recording –Transposition, Wrong column, Wrong units, Wrong sign, illegibility, non- entries, wrong resolution, wrong date, wrong time. Transcription (keying) –Most keying errors are due to bias already introduced by the recording step Transmission (file corruption) Validation / Quality Control –Failure to review forms prior to NCDC –Incorrect metadata –Inappropriate validation, flagging, estimation

NOAA National Climatic Data Center 4 of 19 Not all COOP observations are created equally

NOAA National Climatic Data Center 5 of 19

NOAA National Climatic Data Center 6 of 19 10° F or 12° F ? Error corrections or error creations? ° F ° F TMAX of 19° F with TOBS of 21 ° F ? 20 inches of snow or 2.0?Keyers must “key what they see.” TMIN and TOBS in wrong columns.

NOAA National Climatic Data Center 7 of 19 Automated COOP QC Primary checks on Temperature and Precipitation elements –Around 10 million values per year Internal consistency –Logical (e.g., TMAX ≥ TMIN) –Spikes, flatliners, outliers, excessive range, change points –Date shifting Spatial consistency Values not automatically invalidated unless logically or meteorologically impossible. –Suspect data are reviewed by operators Original values are NEVER changed or edited –unless they were keyed incorrectly

NOAA National Climatic Data Center 8 of 19 First stage Interactive QC “Quasi-Interactive” Applied to values deemed suspect and where a logical operation will not provide resolution (e.g., date shifting) Value compared to climatological neighbors and to computed grids (GEA, TempVAL, PrecipVAL) –See extended abstract or for references and details. Decision is made to accept/reject valid status assigned by automated QC Operator intervention is part of decision process

NOAA National Climatic Data Center 9 of 19 Fully-Interactive QC Health of the Network Datzilla

NOAA National Climatic Data Center 10 of 19 Health of the Network –Available once final QC has been completed –Output for: TMAX, TMIN, TOBS, PRCP, SNOW, SNWD –Accessible by anyone via the Internet –Track QC by station, state, WFO, NWS region, or RCC region. –Graphical and tabular reports to highlight quality issues. Web-based tool for viewing the results of NCDC QC processing.

NOAA National Climatic Data Center 11 of 19 Health of the Network Data Completeness (number of obs) Quality Assurance (invalid and missing) Missing Data / Non- reporting stations Data validity (% unflagged, non-missing) Watch list (change points detected w/o corresponding metadata)

NOAA National Climatic Data Center 12 of 19

NOAA National Climatic Data Center 13 of 19 Summary from HoN All TMAX, TMIN, TOBS, PRCP, SNOW, SNWD data subjected to automated QC % of checked 2006 COOP data declared valid (no further checks). 3.28% declared invalid (331,777) –14.65% no estimate (48,602) –85.36% estimated (283,175) ◦31% of estimates supplied by TempVal (87,820) Thanks to Helen Frederick, HoN administrator, for supplying the numbers.

NOAA National Climatic Data Center 14 of 19 Datzilla Fully manual, interactive quality assurance Web-based tool to report and track errors in NOAA-held data, metadata or associated delivery systems. Developed and maintained by Kevin Robins at the SRCC

NOAA National Climatic Data Center 15 of 19

NOAA National Climatic Data Center 16 of 19 Some possible reasons for a Datzilla ticket to NCDC Data issue (usually) –That TMAX of 74 should be a 47! –Your QC clobbered my data! System issue –CDO inventory doesn’t match the data I got! Metadata issue –This station’s COOP number is wrong!

NOAA National Climatic Data Center 17 of 19 When NCDC receives a Datzilla ticket … Datzilla gatekeeper Initial determinations Reassignment Investigation Course of action Resolution Closure Your friendly Datzilla Gatekeeper

NOAA National Climatic Data Center 18 of 19 Datzilla Summary Began operation early 2005 As of 1/15/08: –865 Datzilla entries (564 NCDC) –226 open (109 NCDC) ◦Receive about 15 new tickets per month –455 of the 564 resolved –317 verified errors with archive fix ◦Most are single values or metadata ◦Few involve many values or larger station issues Has resulted in improvement to the historical climate record. Data quality Data errors

NOAA National Climatic Data Center Dr. Karsten Shein Climatologist NOAA/NESDIS/NCDC 151 Patton Ave. Asheville, NC Thank You