Presentation is loading. Please wait.

Presentation is loading. Please wait.

Data Discovery and Access to The International Surface Pressure Databank (ISPD) 1 Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley.

Published byRosalyn Robertson Modified over 8 years ago

Similar presentations

Presentation on theme: "Data Discovery and Access to The International Surface Pressure Databank (ISPD) 1 Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley."— Presentation transcript:

1 Data Discovery and Access to The International Surface Pressure Databank (ISPD) 1 Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley National Center for Atmospheric Research, Boulder, CO *NOAA/CIRES, Boulder, CO AGU 2012 Fall Meeting: IN44A-05

2 2 Research Data Archive (RDA) at NCAR rda.ucar.edu 1.600+ distinct datasets for climate and weather research 2.Collections: ocean & atmosphere observations, analyses, reanalyses, operational NWP outputs 3.Free and open access http://rda.ucar.edu AGU 2012 Fall Meeting: IN44A-05

3 ISPD Overview 3 World’s largest collection of surface & sea level pressure observations  Land station  Marine observations  Tropical cyclone best track Period (version 2): 1768 – 2010 Volume: 465 Gbyte Available since Aug 2010 AGU 2012 Fall Meeting: IN44A-05

4 ISPD Overview 4 60+ Contributors o Atmospheric Circulation Reconstructions over the Earth (ACRE) o Australian Bureau of Meteorology o British Antarctic Survey o Cook Islands Meteorological Service o Danish Meteorological Institute o Deutscher Wetterdienst (DWD; German Weather Service) o European and North Atlantic Daily to Multidecadal Climate Variability (EMULATE) o ETH Zurich, Switzerland o GCOS/WCRP Working Group on Observational Data Sets for Reanalysis o MANY MANY MORE…. Assembled by NOAA/ESRL, CIRES (Univ. of Colorado), & NOAA/NCDC AGU 2012 Fall Meeting: IN44A-05

5 20 th Century Reanalysis 5 Global reanalysis of atmospheric circulation Period: 1869 – 2010 Assimilates ISPDv2 as input Compo et al. (2011) QJRMS AGU 2012 Fall Meeting: IN44A-05

6 20 th Century Reanalysis: Oct 1950 mean 1000 hPa temperature 6  ISPD stations  ISPD marine obs AGU 2012 Fall Meeting: IN44A-05

7 7  ISPD stations  ISPD marine obs AGU 2012 Fall Meeting: IN44A-05 20 th Century Reanalysis: Nov 1960 mean 1000 hPa temperature

8 8 ISPD obs assimilated into 20CR 20CR data quality control feedback contained in ISPD ­ Provides estimated uncertainty in obs ­ Helps improve underlying observational database AGU 2012 Fall Meeting: IN44A-05 ISPD & the 20 th Century Reanalysis

9 9 ISPD Sample Annual Station Distribution 1850 * Land stations only * No marine stations AGU 2012 Fall Meeting: IN44A-05

10 1850 10 1900 * Land stations only * No marine stations AGU 2012 Fall Meeting: IN44A-05 ISPD Sample Annual Station Distribution

11 1850 11 1900 1950 * Land stations only * No marine stations AGU 2012 Fall Meeting: IN44A-05 ISPD Sample Annual Station Distribution

12 1850 12 1900 1950 2000 * Land stations only * No marine stations AGU 2012 Fall Meeting: IN44A-05 ISPD Sample Annual Station Distribution

13 13 ISPD Observations/Year Figure courtesy Chesley McColl, NOAA/ESRL 2010: 53 Million ~ 1.5 Billion total observations AGU 2012 Fall Meeting: IN44A-05

14 Data Access: Problem Background Large computational/storage resources needed –Store data –Extract desired data from large grids/files –Convert data to desirable format(s) 14 Scientific data centers have these resources Individual researchers generally don’t AGU 2012 Fall Meeting: IN44A-05

15 Goals –Make data more accessible and easier to use for individual researchers Reasonable access volumes Desired data formats User defined parameters/grids 15 Researchers stay focused on research AGU 2012 Fall Meeting: IN44A-05 Data Access: Problem Background

16 ISPD Data Access Services 16 Powerful computing resources @ NCAR Large disk storage (~ 0.5 PB) Rich and detailed metadata Direct file download via web Customized data sub-setting HDF-5 to ASCII software tools AGU 2012 Fall Meeting: IN44A-05

17 17 ISPD Metadata Features Both group- and file-level metadata Drive interfaces for file grouping and sub- setting tools Support efficient back-end processing Improve scalability Provide “quick look” at data samples AGU 2012 Fall Meeting: IN44A-05

18 ISPD Metadata Interface Example 18 AGU 2012 Fall Meeting: IN44A-05

19 19 AGU 2012 Fall Meeting: IN44A-05

20 20 AGU 2012 Fall Meeting: IN44A-05

21 21 AGU 2012 Fall Meeting: IN44A-05

22 Data Access: ISPD Subset Interface 22 AGU 2012 Fall Meeting: IN44A-05

23 ISPD Data Access Services 23 Temporal range sub- setting (daily) Spatial sub-setting Lat/Lon region Individual station ID

24 ISPD Data Access Services 24 Data sub-setting options (cont.) Observation type Land station Marine obs Radiosonde Dropsonde TC best track

25 ISPD Data Access Services 25 Subsetting processed in delayed mode E-mail notification Download via server-provided scripts (wget)

26 26 ISPD 2012 Subset metrics Data accessed: ~ 6.5 TB Data served: ~ 46 GB

27 Summary & Future Directions 27 RDA – Supply “User Friendly” Data  Parameter & spatial sub-setting  Metadata discovery  Format conversion  Improved and additional services NWSC-Cheyenne opening – more computing power AGU 2012 Fall Meeting: IN44A-05

28 28 DOI assignment Geoscience Data Journal article ISPD v3 (1755-2010)  Spring 2013 AGU 2012 Fall Meeting: IN44A-05 ISPD Forthcoming

29 29 rda.ucar.edu/ds132.0 tcram@ucar.edu AGU 2012 Fall Meeting: IN44A-05

Download ppt "Data Discovery and Access to The International Surface Pressure Databank (ISPD) 1 Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley."

Similar presentations

WP1.2: Developing gridded datasets based on long-term remote sensing data (DWD) MOHC (results to be used in WP3): max and min surface air temperature products.

WP1.2: Developing gridded datasets based on long-term remote sensing data (DWD) MOHC (results to be used in WP3): max and min surface air temperature products.
Managing Dataset DOIs and Versions in a Changing Archive Steven Worley Bob Dattore Zaihua Ji National Center for Atmospheric Research Boulder, Colorado,

Managing Dataset DOIs and Versions in a Changing Archive Steven Worley Bob Dattore Zaihua Ji National Center for Atmospheric Research Boulder, Colorado,
Ryan Truchelut and Robert E. Hart Florida State University, Department of Earth, Ocean, and Atmospheric Science Florida State.

Ryan Truchelut and Robert E. Hart Florida State University, Department of Earth, Ocean, and Atmospheric Science Florida State.
Data management in SCD Steven Worley General Categories –The Mass Storage System –NCAR user file services (home directories) –Computer attached storage.

Data management in SCD Steven Worley General Categories –The Mass Storage System –NCAR user file services (home directories) –Computer attached storage.
New Resources in the Research Data Archive Doug Schuster.

New Resources in the Research Data Archive Doug Schuster.
Data rescue of historical typhoon tracks over the western north Pacific back to late 19 th century Hisayuki Kubota Research Institute for Global Change,

Data rescue of historical typhoon tracks over the western north Pacific back to late 19 th century Hisayuki Kubota Research Institute for Global Change,
SCD Research Data For UCAR Data Management Working Group January 10, 2001 Steven Worley Scientific Computing Division Data Support Section.

SCD Research Data For UCAR Data Management Working Group January 10, 2001 Steven Worley Scientific Computing Division Data Support Section.
ICOADS Archive Practices at NCAR JCOMM ETMC-III 9-12 February 2010 Steven Worley.

ICOADS Archive Practices at NCAR JCOMM ETMC-III 9-12 February 2010 Steven Worley.
Report from the GCOS Archive/Analysis Center Matthew Menne NOAA/National Centers for Environmental Information Center for Weather and Climate (NCEI-Asheville)

Report from the GCOS Archive/Analysis Center Matthew Menne NOAA/National Centers for Environmental Information Center for Weather and Climate (NCEI-Asheville)
The International Surface Pressure Databank (ISPD) and Twentieth Century Reanalysis at NCAR Thomas Cram - NCAR, Boulder, CO Gilbert Compo & Chesley McColl.

The International Surface Pressure Databank (ISPD) and Twentieth Century Reanalysis at NCAR Thomas Cram - NCAR, Boulder, CO Gilbert Compo & Chesley McColl.
Operational Dataset Update Functionality Included in the NCAR Research Data Archive Management System 1 Zaihua Ji Doug Schuster Steven Worley Computational.

Operational Dataset Update Functionality Included in the NCAR Research Data Archive Management System 1 Zaihua Ji Doug Schuster Steven Worley Computational.
Dr Rob Allan, ACRE Project Manager, Climate Monitoring and Attribution Group, Met Office Hadley Centre, Exeter, Devon, United Kingdom The Atmospheric Circulation.

Dr Rob Allan, ACRE Project Manager, Climate Monitoring and Attribution Group, Met Office Hadley Centre, Exeter, Devon, United Kingdom The Atmospheric Circulation.
TPAC Digital Library Talk Overview Presenter:Glenn Hyland Tasmanian Partnership for Advanced Computing & Australian Antarctic Division Outline: TPAC Overview.

TPAC Digital Library Talk Overview Presenter:Glenn Hyland Tasmanian Partnership for Advanced Computing & Australian Antarctic Division Outline: TPAC Overview.
Introduction Downloading and sifting through large volumes of data stored in differing formats can be a time-consuming and sometimes frustrating process.

Introduction Downloading and sifting through large volumes of data stored in differing formats can be a time-consuming and sometimes frustrating process.
Obtaining MISR Data and Information Jeff Walter Atmospheric Science Data Center April 17, 2009.

Obtaining MISR Data and Information Jeff Walter Atmospheric Science Data Center April 17, 2009.
Reanalysis of the 20th Century Gilbert P. Compo, Jeffrey S. Whitaker, and Prashant D. Sardeshmukh Univ. of Colorado/CIRES Climate Diagnostics Center and.

Reanalysis of the 20th Century Gilbert P. Compo, Jeffrey S. Whitaker, and Prashant D. Sardeshmukh Univ. of Colorado/CIRES Climate Diagnostics Center and.
U.S. Surface Archives Sent to China (1997-2000) 7 th PRC-U.S. Joint Coordination Panel for Data and Information Cooperation 29 Nov. – 1 Dec, 2000 Steven.

U.S. Surface Archives Sent to China ( ) 7 th PRC-U.S. Joint Coordination Panel for Data and Information Cooperation 29 Nov. – 1 Dec, 2000 Steven.
October 16-18, 20011 Research Data Set Archives Steven Worley Scientific Computing Division Data Support Section.

October 16-18, Research Data Set Archives Steven Worley Scientific Computing Division Data Support Section.
Coordinated Energy and water-cycle Observations Peroject A Well Organized Data Archive System Data Integrating/Archiving Center at University of Tokyo.

Coordinated Energy and water-cycle Observations Peroject A Well Organized Data Archive System Data Integrating/Archiving Center at University of Tokyo.
TIGGE Archive Highlights. First Service Date ECMWF – October 2006 NCAR – October 2006 CMA – June 2007.

TIGGE Archive Highlights. First Service Date ECMWF – October 2006 NCAR – October 2006 CMA – June 2007.

Similar presentations

Ads by Google