NCAR’s Reanalysis Effort : Past, Present, Future

Slides:



Advertisements
Similar presentations
José Meitín NOAA Office of Global Programs Steve Williams UCAR Joint Office for Science Support José Meitín NOAA Office of.
Advertisements

8D Corrective Action. 2 8D Problem Solving & Corrective Action: Initiate 8D Corrective Action D1 - Create Problem Solving Team D2 - Define the Problem.
Matthew Hendrickson, and Pascal Storck
Introduction to GCOS Richard K. Thigpen Implementation Project Manager GCOS Secretariat Geneva.
ICOADS Archive Practices at NCAR JCOMM ETMC-III 9-12 February 2010 Steven Worley.
The Integrated Surface Hourly (ISH) Global Database ESDIM/OGP funded, FCC effort 2 QC phases thus far Full POR online via FTP, partial POR via CDO ISH.
Surface Marine Data International Comprehensive Ocean-Atmosphere Data Set (ICOADS) Steven Worley, NCAR Scott Woodruff, NOAA/ERSL Eric Freeman, NOAA/NCDC.
NOAA Joint OSSE System’s Applications for WISDOM project Y. Zhang, Y. Xie, N. Prive, and B. Mock Jan 18, 2012 GSD/FAB.
Report from the GCOS Archive/Analysis Center Matthew Menne NOAA/National Centers for Environmental Information Center for Weather and Climate (NCEI-Asheville)
The International Surface Pressure Databank (ISPD) and Twentieth Century Reanalysis at NCAR Thomas Cram - NCAR, Boulder, CO Gilbert Compo & Chesley McColl.
A Focus on International Activities NOAA’s Climate Database Modernization Program (CDMP)
Scientific Investigations; Support from Research Data Archives for Computing in Atmospheric Sciences October, 2001 Steven Worley National Center.
U.S. Surface Archives Sent to China ( ) 7 th PRC-U.S. Joint Coordination Panel for Data and Information Cooperation 29 Nov. – 1 Dec, 2000 Steven.
Dr Mark Cresswell Model Assimilation 69EG6517 – Impacts & Models of Climate Change.
Meteorological Observatory Lindenberg – Richard Assmann Observatory The GCOS Reference Upper Air Network.
Details for Today: DATE:18 th November 2004 BY:Mark Cresswell FOLLOWED BY:Literature exercise Model Assimilation 69EG3137 – Impacts & Models of Climate.
MADIS to LITTLE_R Converter MADIS observation types MADIS to LITTE_R converter Future work.
Growing and Future Datasets in the SCD Research Data Archives for NSF SCD Review Panel 16 October 2001 Steven Worley Scientific Computing Division Data.
Workshop on Evaluation of Reanalysis Baltimore, MD, November 1-3, Atmospheric Data Recovery and Biases in Observations Russell S. Vose Product Development.
April nd IBTrACS Workshop 1 Operational Procedures How can we build consistent, homogeneous, well- documented climate quality data?
Scientific Investigations; Support from Research Data Archives for Joint Office for Science Support 26 February, 2002 Steven Worley SCD/DSS.
Slide 1 Sakari Uppala and Dick Dee European Centre for Medium-Range Weather Forecasts ECMWF reanalysis: present and future.
Technical Working Group, II Teruko Manabe Steven Worley Miroslaw Mietus Shawn Smith Simon Tett Volker Wagner Scott Woodruff David Berry Liz Kent.
A Comparison of the Northern American Regional Reanalysis (NARR) to an Ensemble of Analyses Including CFSR Wesley Ebisuzaki 1, Fedor Mesinger 2, Li Zhang.
Rod SET QA/QC Protocols Leigh Anne Sharp, LDNR/CRD LFO Data Recording Data Entry Reporting Deliverables.
Data Access to Marine Surface Observations and Products from COADS 29 January, 2002 Steven Worley National Center for Atmospheric Research.
Page 1© Crown copyright Report of the Global Collecting Centres Elanor Gowland, GCC UK SOT-IV, 16 th - 21 st April 2007.
ARCSS Data Management Support Overview and Update James Moore Steve Williams NCAR Earth Observing Laboratory 3-5 October 2007.
Operational Issues from NCDC Perspective Steve Del Greco, Brian Nelson, Dongsoo Kim NOAA/NESDIS/NCDC Dongjun Seo – NOAA/NWS/OHD 1 st Q2 Workshop Archive,
Data Discovery and Access to The International Surface Pressure Databank (ISPD) 1 Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley.
ACRE Workshop Zurich, Switzerland, June 23-25, GCOS/WCRP Working Group on Observational Data Sets For Reanalysis Russell Vose Chief, Climate Analysis.
CPC Unified Precipitation Project Pingping Xie, Wei Shi, Mingyue Chen and Sid Katz NOAA’s Climate Prediction Center
IHOP_2002 DATA MANAGEMENT UPDATE Steve Williams UCAR/Joint Office for Science Support (JOSS) Boulder, Colorado 2 nd International IHOP_2002 Science Workshop.
IHOP-2002 Data Archive and Development of Composite Data Sets Steven F. Williams, Scot M. Loehrer, Linda E. Cully, Darren R. Gallant, Janine Goldstein,
Legacy Data. From the 2011 EOL Science Review Team Report “The EOL effort to rescue and digitally archive historical field data that are presently stored.
DATA MANAGEMENT UPDATE Steve Williams, Linda Cully, and Scot Loehrer NCAR/Earth Observing Laboratory (EOL) NAME SWG (7.5) Meeting 28 October 2005.
Marine Surface and Climate Data Gaps in the archives at the National Center for Atmospheric Research for 15 th U.S. – China Marine and Fishery Science.
NCEP Data Reanalysis 陳漢卿. Data analyses Use the data available for the original operational NCEP analyses. (available from 1962) Add other datasets to.
NCAR April 1 st 2003 Mesoscale and Microscale Meteorology Data Assimilation in AMPS Dale Barker S. Rizvi, and M. Duda MMM Division, NCAR
Data Discovery and Access to The International Surface Pressure Databank (ISPD) 1 Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley.
5-7 May 2003 SCD Exec_Retr 1 Research Data, May Archive Content New Archive Developments Archive Access and Provision.
The TIGGE Model Validation Portal: An Improvement in Data Interoperability 1 Thomas Cram Doug Schuster Hannah Wilcox Michael Burek Eric Nienhouse Steven.
Station lists and bias corrections Jemma Davie, Colin Parrett, Richard Renshaw, Peter Jermey © Crown Copyright 2012 Source: Met Office© Crown copyright.
88 th Annual American Meteorological Society Meeting New Orleans, LA January 20-25, The Integrated Surface Database: Partnerships and Progress Neal.
June 20, 2005Workshop on Chemical data assimilation and data needs Data Assimilation Methods Experience from operational meteorological assimilation John.
Atmospheric Sciences 370 Observing Systems January 2010.
The National Center for Atmospheric Research is operated by the University Corporation for Atmospheric Research under sponsorship of the National Science.
Quality Control of OLYMPEX sounding data
Atmospheric Sciences 370 Observing Systems January 2009
Downloading Weather Observations
Introduction to WP1 objectives and initial activities planning
Rapid Update Cycle-RUC
Update on the Northwest Regional Modeling System 2013
Chair: Kevin Trenberth Panel: Data issues:
Atmospheric Sciences 370 Observing Systems Winter 2018
Rapid Update Cycle-RUC Rapid Refresh-RR High Resolution Rapid Refresh-HRRR RTMA.
Update on the READER project
Development and Futures of Research Data Archives
Technical Coordinator - Data Buoy Cooperation Panel
Research Data Archives at NCAR
Key new features in the research data archive

Steven Worley, Douglas Schuster,
International Environmental Data Rescue Organization
Group: Data analysis.
Comeaux and Worley, NSF/NCAR/SCD
Robert Dattore and Steven Worley
Data Curation in Climate and Weather
Status of the Regional OSSE for Space-Based LIDAR Winds – Feb01
Comeaux and Worley, NSF/NCAR/SCD
Presentation transcript:

NCAR’s Reanalysis Effort : Past, Present, Future by Joey Comeaux Steve Worley Bob Dattore 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS INTRODUCTION Recap of NCAR’s Efforts Observations Provided Upper Air Surface Aircraft Satellite Data Processing QC Station Histories/Libraries Report Header 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS INTRODUCTION What’s New In house at DSS / already provided Potential sources Experience/Suggestions 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS DATA PREPARED by DSS Upper Air Raobs, Pibals, Dropsondes Over 70,000,000 observations More than 3,000 stations About 50 different sources 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS Upper Air Data New Data Updated Unknown Not DSS Description Period NNR ERA-40 C-CARDS Raobs 1949-1965 1949-1956 ???? CARDS 542 1946-1947 China R & P 1954-1962 1957-1962 Countries R & P 1946-1993 1957-1978 French R & P 1948-1979 1958-1978 GATE R & P 1974 Line Island 1967 MIT Raobs 1958-1963 Navy Kunia 1966-1969 Navy Spot Soundings 1966-1973 NCEP ADP 1973-Cont 1973-2000? 1980-1994 NCEP B3 1962-1972 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS New Data Updated Unknown Not DSS Description Period NNR ERA-40 Permanent Ship 1973-1980 ???? Ptarmigan Drops 1950-1961 Russian NP 1950-1991 1954-1957 Russian Ship 1947-1990 1948-1967 Russian Upper Air 1960-1990 1960-1978 Scherhag Raobs 1954-1962 South African 1961-1967 TD390 Raobs 1943-1962 TD52 Pibals 1922-1971 1948-1971? 1957-1971 TD53 Pibals 1919-1969 1948-1957 1957-1969 TD54 Raobs 1943-1974 1957-1973 TWERLE 1975-1976 US Control 1946-Cont 1958-1978 USAF Upper Air 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS DATA PROVIDED by DSS Surface Data Land surface stations ~ 1 Billion observations >10,000 stations 1901-2002 Ships ICOADS 1784-2002 Update thru 2004 to be completed in 2005 185,000,000 observations 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS SURFACE DATA New Data Updated Unknown Not DSS Description Period NNR ERA-40 TD13 1901-1973 1948-1967 1957-1973 TD14 1938-Cont 1957-1978 NCEP GTS 1975-Cont ???? 1976-1979 USAF-Dick Davis 1967-1980 1967-1976 Russian Sfc Hrly 1936-1986 1948-1955 UW Antarctic 1980-Cont 1980-1998 ICOADS 1784-2000 1960-1979? 1957-1997 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS DATA PROVIDED by DSS Aircraft Provided data for 1947-2002 ~190,000,000 observations 11 different sources 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS AIRCRAFT DATA New Data Updated Unknown Not DSS Description Start NNR ERA-40 U.S.A.F. TD57 1947-1958 Sadler 1960-1973 NCEP B3 1962-1972 1963-1972? NCEP ADP ACFT 1973-Cont 1973-1997? 1973-1978 NCEP ACARS 1991-Cont 1991-2000? 1991-1994 USAF GTS 1976-1985 ???? Navy 1971-1996 1971-1978 New Zealand 1979-1988 1978-1988 Australia 1972-1989 GASP 1975-1979 1975-1978 NAVY SPOT GATE 1974 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS SATELLITE DATA New Data Updated Unknown Not DSS Description Start NNR ERA-40 VTPR 1972-1979 1975-1978 TOVS 1978-1992 1960-1992? SIRS 1969-1972 1969-1972? GOES Cloud Winds 1974-1984 1979-1979 B3 Cloud Winds 1967-1972 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS Data Processing QC Station History Report Headers 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS QUALITY CONTROL QC/QA Checks Duplicate Checks Duplicate Elimination - some Date/Time check Range checks on Data Location checks - some Unit checks Track checks Cross-dataset compares Corrected Winds, Heights, Temps, Dew Points Hydrostatic checks 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Original Failure Rate Final Failure Rate Correction Rate

Comeaux/Worley/Dattore - SCD/DSS Station Histories Process for Creating Station Histories Create History Locations/elevations on observations Calculate elevations from observations Interpolate elevations from terrain data Digitize Station Dictionaries Use Atlas Internal consistency checks Compare to many other histories/libraries Quite a few datasets with no locations/elevations 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Sources for WMO Libraies Station Histories Sources for WMO Libraies SOURCE START END NCEP OBS 1962 2002 C-CARD 1948 1965 CARDS 1900 1999 GAFFEN 1995 WMO Cont NAVY 1964 TD54 1941 1967 USAF 1977 1997 (1999) WBAN DSS Elevations Upper Air WMO 1948-2000 WBAN (1900-Cont) TD52 Library Surface WMO 1975-2002 WBAN 1900-Cont TD13 1900-1973 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

No Elevations in B3 B3/ADP Change Over

Bad GTS Locs Aug 20-21, 2002 GTS Elv = 2m

Comeaux/Worley/Dattore - SCD/DSS DSS Report Header Report Header Added to all conventional obs Contains info on Source Format Type of obs Type of ID Station History - most recent lat,lon New Version (8/2005) Processing date Station history name and version Data provider 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS NEW DATA Reside at DSS / already provided Potential sources 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS NEW DATA at NCAR UPPER AIR Galapagos Isl 1991-1998 SHEBA 10/1997-10/1998 Other Field Projects ???? 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Comeaux/Worley/Dattore - SCD/DSS NEW DATA at NCAR SURFACE DATA Canadian Sfc Hrly 1947-1973 MDL U.S. & Can 1977-Cont UW Antarctic Ship ????-Cont Australian Sfc 1939-1983 Russian NP Ice Isl 1951-1991 NCDC ISH 1901-Cont U.S. SAO 1893-1948 Brazil 1951-1980 Riverflow ???? Precipitation 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Improvement in 1-Degree Box Coverage in ERA-40 for Jan, 1960

POTENTIAL SOURCES for NEW DATA New Canadian data SFC for 1974 - later U/A pre-1950’s Brazillian u/a and sfc data Japan u/a South Africa aircraft for 1950’s CDMP Restoration for about 21 African u/a stations for 1950’s - 2000 DSS Restoration project w/ CDMP Working 7 and 9 track tape drives Univ of Alaska Canada ASLI 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

LESSONS LEARNED/SUGGESTIONS QC & station history works takes 2x as long as expected Merge process should allow for update of individual datasets Insure updated versions of data overide older data GTS Datasets which have been corrected 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

LESSONS LEARNED/SUGGESTIONS Data provided should contain Original source of data Data provider Type of ID Processing date Station history identifier and version This info should be carried into BUFR and Feedback Improved communication Better documentation More info on data downloaded and merged Improved feedback from model 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS

Archive Data Refresh Cycle detail labels on all records (e.g. data and library version, ..) inventories first order data checks and systematic error fixes Archive Center Collections Collection Merging create merged record format convert units QA/QC evaluations Assimilation Analysis verify data receipt with archive center feedback problems discovered in QA/QC possible request to fix and re-supply the data carry forward all detail record identification insert metadata relative to observed data quality, etc communicate back to archive center Feedback output 4/30/2019 Comeaux/Worley/Dattore - SCD/DSS