NCAR’s Reanalysis Effort : Past, Present, Future by Joey Comeaux Steve Worley Bob Dattore 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS INTRODUCTION Recap of NCAR’s Efforts Observations Provided Upper Air Surface Aircraft Satellite Data Processing QC Station Histories/Libraries Report Header 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS INTRODUCTION What’s New In house at DSS / already provided Potential sources Experience/Suggestions 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS DATA PREPARED by DSS Upper Air Raobs, Pibals, Dropsondes Over 70,000,000 observations More than 3,000 stations About 50 different sources 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS Upper Air Data New Data Updated Unknown Not DSS Description Period NNR ERA-40 C-CARDS Raobs 1949-1965 1949-1956 ???? CARDS 542 1946-1947 China R & P 1954-1962 1957-1962 Countries R & P 1946-1993 1957-1978 French R & P 1948-1979 1958-1978 GATE R & P 1974 Line Island 1967 MIT Raobs 1958-1963 Navy Kunia 1966-1969 Navy Spot Soundings 1966-1973 NCEP ADP 1973-Cont 1973-2000? 1980-1994 NCEP B3 1962-1972 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS New Data Updated Unknown Not DSS Description Period NNR ERA-40 Permanent Ship 1973-1980 ???? Ptarmigan Drops 1950-1961 Russian NP 1950-1991 1954-1957 Russian Ship 1947-1990 1948-1967 Russian Upper Air 1960-1990 1960-1978 Scherhag Raobs 1954-1962 South African 1961-1967 TD390 Raobs 1943-1962 TD52 Pibals 1922-1971 1948-1971? 1957-1971 TD53 Pibals 1919-1969 1948-1957 1957-1969 TD54 Raobs 1943-1974 1957-1973 TWERLE 1975-1976 US Control 1946-Cont 1958-1978 USAF Upper Air 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS DATA PROVIDED by DSS Surface Data Land surface stations ~ 1 Billion observations >10,000 stations 1901-2002 Ships ICOADS 1784-2002 Update thru 2004 to be completed in 2005 185,000,000 observations 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS SURFACE DATA New Data Updated Unknown Not DSS Description Period NNR ERA-40 TD13 1901-1973 1948-1967 1957-1973 TD14 1938-Cont 1957-1978 NCEP GTS 1975-Cont ???? 1976-1979 USAF-Dick Davis 1967-1980 1967-1976 Russian Sfc Hrly 1936-1986 1948-1955 UW Antarctic 1980-Cont 1980-1998 ICOADS 1784-2000 1960-1979? 1957-1997 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS DATA PROVIDED by DSS Aircraft Provided data for 1947-2002 ~190,000,000 observations 11 different sources 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS AIRCRAFT DATA New Data Updated Unknown Not DSS Description Start NNR ERA-40 U.S.A.F. TD57 1947-1958 Sadler 1960-1973 NCEP B3 1962-1972 1963-1972? NCEP ADP ACFT 1973-Cont 1973-1997? 1973-1978 NCEP ACARS 1991-Cont 1991-2000? 1991-1994 USAF GTS 1976-1985 ???? Navy 1971-1996 1971-1978 New Zealand 1979-1988 1978-1988 Australia 1972-1989 GASP 1975-1979 1975-1978 NAVY SPOT GATE 1974 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS SATELLITE DATA New Data Updated Unknown Not DSS Description Start NNR ERA-40 VTPR 1972-1979 1975-1978 TOVS 1978-1992 1960-1992? SIRS 1969-1972 1969-1972? GOES Cloud Winds 1974-1984 1979-1979 B3 Cloud Winds 1967-1972 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS Data Processing QC Station History Report Headers 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS QUALITY CONTROL QC/QA Checks Duplicate Checks Duplicate Elimination - some Date/Time check Range checks on Data Location checks - some Unit checks Track checks Cross-dataset compares Corrected Winds, Heights, Temps, Dew Points Hydrostatic checks 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Original Failure Rate Final Failure Rate Correction Rate

Comeaux/Worley/Dattore - SCD/DSS Station Histories Process for Creating Station Histories Create History Locations/elevations on observations Calculate elevations from observations Interpolate elevations from terrain data Digitize Station Dictionaries Use Atlas Internal consistency checks Compare to many other histories/libraries Quite a few datasets with no locations/elevations 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Sources for WMO Libraies Station Histories Sources for WMO Libraies SOURCE START END NCEP OBS 1962 2002 C-CARD 1948 1965 CARDS 1900 1999 GAFFEN 1995 WMO Cont NAVY 1964 TD54 1941 1967 USAF 1977 1997 (1999) WBAN DSS Elevations Upper Air WMO 1948-2000 WBAN (1900-Cont) TD52 Library Surface WMO 1975-2002 WBAN 1900-Cont TD13 1900-1973 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

No Elevations in B3 B3/ADP Change Over

Bad GTS Locs Aug 20-21, 2002 GTS Elv = 2m

Comeaux/Worley/Dattore - SCD/DSS DSS Report Header Report Header Added to all conventional obs Contains info on Source Format Type of obs Type of ID Station History - most recent lat,lon New Version (8/2005) Processing date Station history name and version Data provider 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS NEW DATA Reside at DSS / already provided Potential sources 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS NEW DATA at NCAR UPPER AIR Galapagos Isl 1991-1998 SHEBA 10/1997-10/1998 Other Field Projects ???? 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS NEW DATA at NCAR SURFACE DATA Canadian Sfc Hrly 1947-1973 MDL U.S. & Can 1977-Cont UW Antarctic Ship ????-Cont Australian Sfc 1939-1983 Russian NP Ice Isl 1951-1991 NCDC ISH 1901-Cont U.S. SAO 1893-1948 Brazil 1951-1980 Riverflow ???? Precipitation 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Improvement in 1-Degree Box Coverage in ERA-40 for Jan, 1960

POTENTIAL SOURCES for NEW DATA New Canadian data SFC for 1974 - later U/A pre-1950’s Brazillian u/a and sfc data Japan u/a South Africa aircraft for 1950’s CDMP Restoration for about 21 African u/a stations for 1950’s - 2000 DSS Restoration project w/ CDMP Working 7 and 9 track tape drives Univ of Alaska Canada ASLI 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

LESSONS LEARNED/SUGGESTIONS QC & station history works takes 2x as long as expected Merge process should allow for update of individual datasets Insure updated versions of data overide older data GTS Datasets which have been corrected 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

LESSONS LEARNED/SUGGESTIONS Data provided should contain Original source of data Data provider Type of ID Processing date Station history identifier and version This info should be carried into BUFR and Feedback Improved communication Better documentation More info on data downloaded and merged Improved feedback from model 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Archive Data Refresh Cycle detail labels on all records (e.g. data and library version, ..) inventories first order data checks and systematic error fixes Archive Center Collections Collection Merging create merged record format convert units QA/QC evaluations Assimilation Analysis verify data receipt with archive center feedback problems discovered in QA/QC possible request to fix and re-supply the data carry forward all detail record identification insert metadata relative to observed data quality, etc communicate back to archive center Feedback output 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS