The International Surface Pressure Databank (ISPD) and Twentieth Century Reanalysis at NCAR Thomas Cram - NCAR, Boulder, CO Gilbert Compo & Chesley McColl.

Slides:

Advertisements

Similar presentations

1 NASA CEOP Status & Demo CEOS WGISS-25 Sanya, China February 27, 2008 Yonsook Enloe.

Advertisements

ECMWF June 2006Slide 1 Access to ECMWF data for Research Manuel Fuentes Data and Services Section, ECMWF ECMWF Forecast Products User Meeting.

Managing Dataset DOIs and Versions in a Changing Archive Steven Worley Bob Dattore Zaihua Ji National Center for Atmospheric Research Boulder, Colorado,

Data management in SCD Steven Worley General Categories –The Mass Storage System –NCAR user file services (home directories) –Computer attached storage.

New Resources in the Research Data Archive Doug Schuster.

SCD Research Data For UCAR Data Management Working Group January 10, 2001 Steven Worley Scientific Computing Division Data Support Section.

ICOADS Archive Practices at NCAR JCOMM ETMC-III 9-12 February 2010 Steven Worley.

ERA-Interim and ASR Data Management at NCAR

The Research Data Archive at NCAR Doug Schuster and Steve Worley NCAR.

Operational Dataset Update Functionality Included in the NCAR Research Data Archive Management System 1 Zaihua Ji Doug Schuster Steven Worley Computational.

NOAA National Climate Model Portal D. Michael Grogan Lead Systems Engineer National Climatic Data Center.

TPAC Digital Library Talk Overview Presenter:Glenn Hyland Tasmanian Partnership for Advanced Computing & Australian Antarctic Division Outline: TPAC Overview.

Introduction Downloading and sifting through large volumes of data stored in differing formats can be a time-consuming and sometimes frustrating process.

Obtaining MISR Data and Information Jeff Walter Atmospheric Science Data Center April 17, 2009.

Scientific Investigations; Support from Research Data Archives for Computing in Atmospheric Sciences October, 2001 Steven Worley National Center.

U.S. Surface Archives Sent to China ( ) 7 th PRC-U.S. Joint Coordination Panel for Data and Information Cooperation 29 Nov. – 1 Dec, 2000 Steven.

October 16-18, Research Data Set Archives Steven Worley Scientific Computing Division Data Support Section.

EGU 2011 TIGGE, TIGGE LAM and the GIFS T. Paccagnella (1), D. Richardson (2), D. Schuster(3), R. Swinbank (4), Z. Toth (3), S.

Coordinated Energy and water-cycle Observations Peroject A Well Organized Data Archive System Data Integrating/Archiving Center at University of Tokyo.

TIGGE Archive Highlights. First Service Date ECMWF – October 2006 NCAR – October 2006 CMA – June 2007.

Research Data at NCAR 1 August, 2002 Steven Worley Scientific Computing Division Data Support Section.

Bringing it All Together: NODC’s Geoportal Server as an Integration Tool for Interoperable Data Services Kenneth S. Casey, Ph.D. YuanJie Li NOAA National.

Data for Climate and Energy Studies Steven Worley Computational and Information Systems Laboratory NCAR.

Unidata’s TDS Workshop TDS Overview – Part II October 2012.

Unidata TDS Workshop TDS Overview – Part I XX-XX October 2014.

THREDDS Data Server Ethan Davis GEOSS Climate Workshop 23 September 2011.

Scientific Investigations; Support from Research Data Archives for Joint Office for Science Support 26 February, 2002 Steven Worley SCD/DSS.

Data to Support Ocean-Atmosphere Research NCAR Research Data Archive (RDA), Zaihua Ji, NCAR Steven Worley, NCAR Scott Woodruff,

Archive and Access Practices that Support Data Reuse and Transparency Steven Worley Doug Schuster Bob Dattore National Center for Atmospheric Research.

Data Access to Marine Surface Observations and Products from COADS 29 January, 2002 Steven Worley National Center for Atmospheric Research.

Integrated Grid workflow for mesoscale weather modeling and visualization Zhizhin, M., A. Polyakov, D. Medvedev, A. Poyda, S. Berezin Space Research Institute.

CISL/DSS & MMM Data Discussion 19 March Who CISL/DSS - maintain NCEP operational analyses and observation datasets – Gregg Walters, Doug Schuster,

ICOADS: Update Status and Data Distribution Steven J. Worley Scott D. Woodruff Sandra J. Lubker Ziahua Ji J. Eric Freeman NCAR, NOAA/ESRL, NOAA/NCDC CLIMAR-III,

Analyzed Data Products Available from NCAR that Support Marine Climate Research JCOMM ETMC-III 9-12 February 2010 Steven Worley Doug Schuster.

ESIP Federation 2004 : L.B.Pham S. Berrick, L. Pham, G. Leptoukh, Z. Liu, H. Rui, S. Shen, W. Teng, T. Zhu NASA Goddard Earth Sciences (GES) Data & Information.

Data Discovery and Access to The International Surface Pressure Databank (ISPD) 1 Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley.

Content, Discovery, and Accessibility Enhancements to the NCAR Research Data Archive Doug Schuster and Steve Worley NCAR.

APEC Climate Center Data Service System Chi-Yung Francis Tam APCC.

TIGGE Data Archive and Access at NCAR November 2008 November 2008 Steven Worley National Center for Atmospheric Research Boulder, Colorado, U.S.A.

TIGGE Data Archive at NCAR 8th GIFS-TIGGE Working Group World Meteorological Organization Geneva February, 2010 Doug Schuster Steven Worley Dave.

The TIGGE Model Validation Portal: An Improvement in Data Interoperability 1 Thomas Cram Doug Schuster Hannah Wilcox Steven Worley National Center for.

29 March 2004 Steven Worley, NSF/NCAR/SCD 1 Research Data Stewardship and Access Steven Worley, CISL/SCD Cyberinfrastructure meeting with Priscilla Nelson.

1 Adventures in Web Services for Large Geophysical Datasets Joe Sirott PMEL/NOAA.

TIGGE Archive Status at NCAR THORPEX Workshop and 6th GIFS-TIGGE Working Group Meetings WMO Headquarters Geneva September 2008 Steven Worley Doug.

SCD Research Data Archives; Availability Through the CDP About 500 distinct datasets, 12 TB Diverse in type, size, and format Serving 900 different investigators.

The Research Data Archive at NCAR: A System Designed to Handle Diverse Datasets Bob Dattore and Steven Worley National Center for Atmospheric Research.

TIGGE Archive Access at NCAR Steven Worley Doug Schuster Dave Stepaniak Hannah Wilcox.

Research Data Archive (RDA) Access and Services from Yellowstone Grace Peng and Doug Schuster 1.

Data Discovery and Access to The International Surface Pressure Databank (ISPD) 1 Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley.

SCD Research Data for Ocean Observatories Steering Committee June 18, 2001 Steven Worley Scientific Computing Division Data Support Section.

Support to scientific research on seasonal-to-decadal climate and air quality modelling Pierre-Antoine Bretonnière Francesco Benincasa IC3-BSC - Spain.

5-7 May 2003 SCD Exec_Retr 1 Research Data, May Archive Content New Archive Developments Archive Access and Provision.

AOLI 2015 The NMME Experience: A Research Community Archive Lessons learned from Climate Model data archive and use AOLI Meeting 2015 Eric Nienhouse NCAR.

The TIGGE Model Validation Portal: An Improvement in Data Interoperability 1 Thomas Cram Doug Schuster Hannah Wilcox Michael Burek Eric Nienhouse Steven.

1. Gridded Data Sub-setting Services through the RDA at NCAR Doug Schuster, Steve Worley, Bob Dattore, Dave Stepaniak.

A41I-0105 Supporting Decadal and Regional Climate Prediction through NCAR’s EaSM Data Portal Doug Schuster and Steve Worley National Center for Atmospheric.

Introduction What purpose does a data archive center serve if users can’t find or access the holdings they might need to facilitate their research discoveries?

Standardization Promotes Biogeochemical Data Management and Use in Multidisciplinary Environmental Research Yaxing Wei, Suresh Vannan, Robert B. Cook,

TIGGE Archives and Access

TIGGE Data Archive and Access System at NCAR

gWRF Workflow and Input Data Requirements

Operational Dataset Update Functionality Included in the NCAR Research Data Archive Management System Zaihua Ji Doug Schuster Steven Worley Computational.

Development and Futures of Research Data Archives

TIGGE Data Archive at NCAR

Research Data Archives at NCAR

Access and visualization of theTwentieth Century Reanalysis dataset

Steven Worley, Douglas Schuster,

Comeaux and Worley, NSF/NCAR/SCD

Robert Dattore and Steven Worley

Comeaux and Worley, NSF/NCAR/SCD

Presentation transcript:

The International Surface Pressure Databank (ISPD) and Twentieth Century Reanalysis at NCAR Thomas Cram - NCAR, Boulder, CO Gilbert Compo & Chesley McColl – Univ. of Colorado-Boulder/NOAA-CIRES Doug Schuster & Steven Worley - NCAR, Boulder, CO 7th ACRE Workshop, Toronto, Aug 2014

Topics ● RDA overview ● ISPD and 20CR overview ● RDA services o Data discovery & access o Custom services o Data citation

Research Data Archive (RDA) at NCAR rda.ucar.edu ●600+ distinct datasets for climate and weather research ●Free and open access ●10,000+ unique users in FY2013 ●Collections: ocean & atmosphere observations, analyses, reanalyses, operational NWP outputs ●Each collection managed by a RDA data specialist

RDA data collections ● Historical observations (ISPD, ICOADS, NCDC ISD, Upper Air Database) ● Reanalyses (20CR, ECMWF ERA, JRA-25, JRA-55, NCEP CSFR, BPRC ASR, … ) ● Operational (NCEP observations, analyses, & forecasts) ● Climate/regional models (CMIP5, NRCM, … ) ● Remote sensing ● Ancillary datasets--topography/bathymetry, vegetation, land use

ISPD (version 2) ● World’s largest collection of surface & sea level pressure observations o Land station o Marine observations (ICOADS) o Tropical cyclone best track (IBTrACS) ● Period: 1768 – 2012 ● Volume: 562 GB ● 60+ contributors ● doi: /D6SQ8XDW ● Data access: rda.ucar.edu/datasets/ds132.0

Twentieth Century Reanalysis (20CR) ● Global reanalysis ● Observational inputs o Surface pressure (ISPD) o Sea surface temperature o Sea ice distribution ● Period: near present (currently through 2012) ● Volume: 13.6 TB ● Ensemble mean & spread analysis: 2 x 2 deg grid ● Ensemble mean & spread forecast fields: T62 Gaussian grid ● doi: /D6QR4V37 ● Data access: rda.ucar.edu/datasets/ds131.1

Data discovery ● Detailed metadata collection o Dataset-, group-, and file-level metadata o Drive interfaces for file grouping and sub-setting tools o Support efficient back-end processing o Improve scalability o Provide “quick look” at data samples

ISPD Metadata: Dataset level Comprehensive view of dataset-level metadata ●Observing platform ●Data types ●Data frequency ●Temporal & geographical coverage

ISPD Metadata discovery: File level ● Hourly files grouped into yearly tarballs ● Looking glass icon loads discovery tool

ISPD Metadata discovery: File level ● Interactive map ● Browse and inspect observations

20CR Metadata discovery ●Gridded data ●Metadata classified by parameter

ISPD data access page

Data access features ● Direct file download ● Customized subsetting ● Data format conversion o ISPD: HDF5 → ASCII, NetCDF o 20CR: GRIB → NetCDF ● THREDDS services (OPeNDAP, WMS, WCS, etc.) -- currently 20CR only ● RDA API web service (Python) ● Supporting software

20CR data subsetting Temporal Selection Parameter selection Format conversion Spatial selection

ISPD data subsetting Temporal selection Spatial Selection (Lat/Lon region or Station ID) Observation type Output format (ASCII or NetCDF)

2014 online data downloads ● Direct file downloads o ISPD: 869 GB o 20CR: 3.23 TB ● Subset requests o ISPD: 3.14 TB accessed -> 694 GB served o 20CR: 2.65 TB accessed -> 1.27 TB served

Dataset citation service ● Citation design – ESIP Guidelines Compo, G. P., et al International Surface Pressure Databank (ISPDv2). Research Data Archive at the National Center for Atmospheric Research, Computational and Information Systems Laboratory. Accessed dd mmm yyyy. ● Citation styles o ESIP o AMS o AGU o DataCite

Upcoming ISPD ● ISPD version 3 ( ) ● Geoscience Data Journal article (in review) ● THREDDS services? General ● Globus data transfer service (GridFTP protocol) CR Version 2c ( ) Version 3 (~1 deg resolution)

ISPD rda.ucar.edu/datasets/ds CR rda.ucar.edu/datasets/ds131.1