Research Data Archives at NCAR

Slides:

Advertisements

Similar presentations

Use of VOS data in Climate Products Elizabeth Kent and Scott Woodruff National Oceanography Centre, Southampton NOAA Earth System Research Laboratory.

Advertisements

Data management in SCD Steven Worley General Categories –The Mass Storage System –NCAR user file services (home directories) –Computer attached storage.

SCD Research Data For UCAR Data Management Working Group January 10, 2001 Steven Worley Scientific Computing Division Data Support Section.

ICOADS Archive Practices at NCAR JCOMM ETMC-III 9-12 February 2010 Steven Worley.

The International Surface Pressure Databank (ISPD) and Twentieth Century Reanalysis at NCAR Thomas Cram - NCAR, Boulder, CO Gilbert Compo & Chesley McColl.

TPAC Digital Library Talk Overview Presenter:Glenn Hyland Tasmanian Partnership for Advanced Computing & Australian Antarctic Division Outline: TPAC Overview.

Scientific Investigations; Support from Research Data Archives for Computing in Atmospheric Sciences October, 2001 Steven Worley National Center.

U.S. Surface Archives Sent to China ( ) 7 th PRC-U.S. Joint Coordination Panel for Data and Information Cooperation 29 Nov. – 1 Dec, 2000 Steven.

Dr Mark Cresswell Model Assimilation 69EG6517 – Impacts & Models of Climate Change.

October 16-18, Research Data Set Archives Steven Worley Scientific Computing Division Data Support Section.

Details for Today: DATE:18 th November 2004 BY:Mark Cresswell FOLLOWED BY:Literature exercise Model Assimilation 69EG3137 – Impacts & Models of Climate.

Research Data at NCAR 1 August, 2002 Steven Worley Scientific Computing Division Data Support Section.

Growing and Future Datasets in the SCD Research Data Archives for NSF SCD Review Panel 16 October 2001 Steven Worley Scientific Computing Division Data.

Data for Climate and Energy Studies Steven Worley Computational and Information Systems Laboratory NCAR.

Collaborative Research: Toward reanalysis of the Arctic Climate System—sea ice and ocean reconstruction with data assimilation Synthesis of Arctic System.

Scientific Investigations; Support from Research Data Archives for Joint Office for Science Support 26 February, 2002 Steven Worley SCD/DSS.

NCAR Computation and Information Systems Laboratory Facilities and Support Overview UCAR Postdoctoral and Graduate Fellows March 16, 2007 Mike Page Consulting.

A Comparison of the Northern American Regional Reanalysis (NARR) to an Ensemble of Analyses Including CFSR Wesley Ebisuzaki 1, Fedor Mesinger 2, Li Zhang.

Data to Support Ocean-Atmosphere Research NCAR Research Data Archive (RDA), Zaihua Ji, NCAR Steven Worley, NCAR Scott Woodruff,

Archive and Access Practices that Support Data Reuse and Transparency Steven Worley Doug Schuster Bob Dattore National Center for Atmospheric Research.

CISL/DSS & MMM Data Discussion 19 March Who CISL/DSS - maintain NCEP operational analyses and observation datasets – Gregg Walters, Doug Schuster,

Global, Basin and Shelf Ocean Applications of OPA An Inter-Agency Canadian Initiative EC-DFO-DND + Universities + Mercator-Ocean  CONCEPTS -- Canadian.

Operational sub-regional Long-Range Forecasting Unit at RA VI Regional Climate Center – South-East European Virtual Climate Change Center Vladimir Djurdjevic.

Analyzed Data Products Available from NCAR that Support Marine Climate Research JCOMM ETMC-III 9-12 February 2010 Steven Worley Doug Schuster.

TPAC Tasmanian Partnership for Advanced Computing Partner in APAC (Australian Partnership for Advanced Computing) Expertise centre for Earth Systems Science.

Observing System Monitoring Center (OSMC) Status Update April 2005 Steve Hankin – PMEL (co-PI) Kevin Kern – NDBC (co-PI)

Data Discovery and Access to The International Surface Pressure Databank (ISPD) 1 Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley.

Content, Discovery, and Accessibility Enhancements to the NCAR Research Data Archive Doug Schuster and Steve Worley NCAR.

RDA Data Support Section. Topics 1.What is it? 2.Who cares? 3.Why does the RDA need CISL? 4.What is on the horizon?

NQuery: A Network-enabled Data-based Query Tool for Multi-disciplinary Earth-science Datasets John R. Osborne.

Review of Meteorological Data Sharing Project in China Anyuan XIONG (National Meteorological Information Center, CMA, CHINA)

International Workshop on Antarctic Clouds Columbus, OH Polar Meteorology Group, Byrd Polar Research Center, The Ohio State University, Columbus, Ohio.

The TIGGE Model Validation Portal: An Improvement in Data Interoperability 1 Thomas Cram Doug Schuster Hannah Wilcox Steven Worley National Center for.

29 March 2004 Steven Worley, NSF/NCAR/SCD 1 Research Data Stewardship and Access Steven Worley, CISL/SCD Cyberinfrastructure meeting with Priscilla Nelson.

1 Accomplishments. 2 Overview of Accomplishments  Sustaining the Production Earth System Grid Serving the current needs of the climate modeling community.

SCD Research Data Archives; Availability Through the CDP About 500 distinct datasets, 12 TB Diverse in type, size, and format Serving 900 different investigators.

Marine Surface and Climate Data Gaps in the archives at the National Center for Atmospheric Research for 15 th U.S. – China Marine and Fishery Science.

The Research Data Archive at NCAR: A System Designed to Handle Diverse Datasets Bob Dattore and Steven Worley National Center for Atmospheric Research.

NCEP Data Reanalysis 陳漢卿. Data analyses Use the data available for the original operational NCEP analyses. (available from 1962) Add other datasets to.

Data Discovery and Access to The International Surface Pressure Databank (ISPD) 1 Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley.

RDA Data Support Section. Topics 1.What is it? 2.Who cares? 3.Why does the RDA need CISL? 4.What is on the horizon?

SCD Research Data for Ocean Observatories Steering Committee June 18, 2001 Steven Worley Scientific Computing Division Data Support Section.

5-7 May 2003 SCD Exec_Retr 1 Research Data, May Archive Content New Archive Developments Archive Access and Provision.

The TIGGE Model Validation Portal: An Improvement in Data Interoperability 1 Thomas Cram Doug Schuster Hannah Wilcox Michael Burek Eric Nienhouse Steven.

IASC Workshop Potsdamr, Germany Polar Meteorology Group, Byrd Polar Research Center, The Ohio State University, Columbus, Ohio, USA The Arctic System Reanalysis.

2nd Workshop on WIGOS Data Quality Monitoring and Incident Management (Geneva, December 2015) JCOMM Quality Monitoring and the role of JCOMMOPS Etienne.

Russian Research Institute of Hydrometeorological Information

Report by Japan Meteorological Agency (JMA)

INTAROS – Integrated Arctic Observation System

Ice sheets and their relation to sea level

12th – 14th January 2017 Tim Oakley, GCOS Network Manager

Integrating Data and Information Across Observing System

TIGGE Data Archive and Access System at NCAR

Candyce Clark JCOMM Observations Programme Area Coordinator

CHALLENGES IN PROVIDING MARINE DATA IN FORMATS DESIGNED FOR LAND-BASED DATA Karen Douglas|

User Requirements for Climate Monitoring

Contributions to WIGOS David Meldrum, vice chair, JCOMM OCG

Development and Futures of Research Data Archives

(WCRP Seasonal Prediction Workshop) Applied Meteorology Group

Key new features in the research data archive

Steven Worley, NSF/NCAR/SCD

Status of Existing Observing Networks

CISL’s Research Data Archive (RDA) : Description and Methods

Comeaux and Worley, NSF/NCAR/SCD

Long-Lived Data Collections

Data Management Components for a Research Data Archive

Robert Dattore and Steven Worley

Can the increase of Polar Stratospheric Clouds explain the Antarctic Winter Tropospheric warming? Tom Lachlan-Cope (W. M. Connolley, J. Turner, H. Roscoe,

Data Curation in Climate and Weather

Comeaux and Worley, NSF/NCAR/SCD

Presentation transcript:

Research Data Archives at NCAR RDA, a continually evolving data resource Content is improved and extended Metadata is developed to improve discovery Access methods are many and expanding Now; Web, FTP, NCAR MSS, writable media Future; via CDP and network servers (e.g. GDS, OPeNDAP, etc) 5 February 2019 Steven Worley, NSF/NCAR/SCD

Steven Worley, NSF/NCAR/SCD RDA definition Collection of reference datasets used in atmospheric and related sciences Over 600 datasets currently archived 10-20 new datasets added annually The RDA is maintained by 9 staff First established about 40 years All data on the MSS 548K files 100.5 TB 5 February 2019 Steven Worley, NSF/NCAR/SCD

Steven Worley, NSF/NCAR/SCD RDA Content Many categories of data Observations Model Output NWP Reanalyses Climatology Data Long-term SST analyses Monthly summary of the day station data Satellite derived datasets Global topography and ocean depth Station Libraries Much more …. 5 February 2019 Steven Worley, NSF/NCAR/SCD

Steven Worley, NSF/NCAR/SCD RDA Observations Observations Conventional meteorology and oceanography Surface Global ocean from ships and buoys (ICOADS) Global meteorology stations Gridded ocean and station obs Profiles Global ocean hydrography Global upper air measurements 5 February 2019 Steven Worley, NSF/NCAR/SCD

Steven Worley, NSF/NCAR/SCD RDA Model Output Model Output Operational Model Output NCEP Global NCEP Regional North American ECMWF Global Reanalysis Projects NCEP Global Reanalysis I & II ECMWF Global ERA-40 5 February 2019 Steven Worley, NSF/NCAR/SCD

Steven Worley, NSF/NCAR/SCD Eta and NARR domains 5 February 2019 Steven Worley, NSF/NCAR/SCD

Steven Worley, NSF/NCAR/SCD Arctic Specific data Univ Wash. PSC ARGOS Buoy Observations, 1979-1990 Polar Science Center Hydrographic Climatology (PHC) U. Wisconsin Greenland Surface Observations, 1987-1991 AARI Russian North Polar Drifting Station Data, 1937- Arctic and Antarctic Sea Ice Concentration, Walsh 1901-1990 Arctic Sea Ice Thickness Climatology 1960-1982 Western Arctic Ice Drift Stations, + ships Maud and Fram 1893-1976 Hadley Centre Global Sea Ice and Sea Surface Temperature Analyses 1870-2005 USSR. Arctic Ice Island Radiosonde Observations, 1950-1991 NMC Nimbus-5 Sea Ice (ESMR) Data for February 1975 Climatic Boundary Conditions for 18,000 years before present Ptarmigan Dropsondes (Arctic reconnaissance) for 1950-1961 Norwegian Marine Surface Data, 1867-1912 5 February 2019 Steven Worley, NSF/NCAR/SCD

Steven Worley, NSF/NCAR/SCD RDA Metadata All datasets Described with individual web pages Assigned data consultant Catalogued in UCAR Community Data Portal (CDP) 5 February 2019 Steven Worley, NSF/NCAR/SCD

Steven Worley, NSF/NCAR/SCD 5 February 2019 Steven Worley, NSF/NCAR/SCD

MSS All RDA data RDA Server CDP NCAR User Internet User Data User’s Functional Diagram for RDA Access MSS All RDA data RDA Server Some data All Metadata CDP Standardized Metadata NCAR User Internet User 5 February 2019 Steven Worley, NSF/NCAR/SCD

MSS All RDA data RDA Server CDP NCAR User Internet User Data User’s Functional Diagram for RDA Access MSS All RDA data RDA Server Some data All Metadata CDP Standardized Metadata NCAR User Internet User metadata 5 February 2019 Steven Worley, NSF/NCAR/SCD

MSS All RDA data RDA Server CDP NCAR User Internet User Data User’s Functional Diagram for RDA Access MSS All RDA data RDA Server Some data All Metadata CDP Standardized Metadata NCAR User Internet User metadata data 5 February 2019 Steven Worley, NSF/NCAR/SCD

MSS All RDA data RDA Server CDP NCAR User Internet User Data User’s Functional Diagram for RDA Access MSS All RDA data RDA Server Some data All Metadata CDP Standardized Metadata NCAR User Internet User metadata data future data 5 February 2019 Steven Worley, NSF/NCAR/SCD

RDA MSS Server All RDA data CDP Operational Function Diagram for RDA RDA Database RDA Server Some data All Metadata MSS All RDA data SAN CDP Standardized Metadata Data Ingest 5 February 2019 Steven Worley, NSF/NCAR/SCD

RDA MSS Server All RDA data CDP Operational Function Diagram for RDA RDA Database Integrity Monitor RDA Server Some data All Metadata MSS All RDA data SAN Top Collections CDP Standardized Metadata Data Ingest Network Media 5 February 2019 Metadata Steven Worley, NSF/NCAR/SCD

RDA MSS Server All RDA data CDP Operational Function Diagram for RDA RDA Database Integrity Monitor RDA Server Some data All Metadata MSS All RDA data SAN Top Collections CDP Standardized Metadata Data Ingest Network Media 5 February 2019 Metadata Data Steven Worley, NSF/NCAR/SCD

RDA MSS Server All RDA data CDP Operational Function Diagram for RDA RDA Database Integrity Monitor RDA Server Some data Metadata MSS All RDA data SAN Top Collections CDP Metadata Data Data Ingest Network Media 5 February 2019 Metadata Data Steven Worley, NSF/NCAR/SCD Future Data

Steven Worley, NSF/NCAR/SCD Key Case – ERA-40 35 TB collection, 30 distinct product lines Added about 10 products (computed in SCD) Support Climate Modeling – T85 Guassian grids to match NCAR CCSM Metrics for 2004 Web & FTP = MSS in Data Amount Over 20 TB delivered 28K versus 13K files, RDA Server - MSS 5 February 2019 Steven Worley, NSF/NCAR/SCD

Questions / Discussion 5 February 2019 Steven Worley, NSF/NCAR/SCD