The TIGGE Model Validation Portal: An Improvement in Data Interoperability 1 Thomas Cram Doug Schuster Hannah Wilcox Michael Burek Eric Nienhouse Steven.

Slides:

Advertisements

Similar presentations

ECMWF June 2006Slide 1 Access to ECMWF data for Research Manuel Fuentes Data and Services Section, ECMWF ECMWF Forecast Products User Meeting.

Advertisements

Data management in SCD Steven Worley General Categories –The Mass Storage System –NCAR user file services (home directories) –Computer attached storage.

New Resources in the Research Data Archive Doug Schuster.

WRF Modeling System V2.0 Overview

ICOADS Archive Practices at NCAR JCOMM ETMC-III 9-12 February 2010 Steven Worley.

The Research Data Archive at NCAR Doug Schuster and Steve Worley NCAR.

NCAR GIS Program : Bridging Gaps

CEOP Centralized Data System and integrated analysis tools Kenji Taniguchi 1), Toshihiro Nemoto 2), Eiji Ikoma 3), Masaki Yasukawa 2), Toshio Koike 1)

The International Surface Pressure Databank (ISPD) and Twentieth Century Reanalysis at NCAR Thomas Cram - NCAR, Boulder, CO Gilbert Compo & Chesley McColl.

Operational Dataset Update Functionality Included in the NCAR Research Data Archive Management System 1 Zaihua Ji Doug Schuster Steven Worley Computational.

Introduction Downloading and sifting through large volumes of data stored in differing formats can be a time-consuming and sometimes frustrating process.

Office of Research and Development National Exposure Research Laboratory, Atmospheric Modeling Division, Applied Modeling Research Branch October 8, 2008.

October 16-18, Research Data Set Archives Steven Worley Scientific Computing Division Data Support Section.

EGU 2011 TIGGE, TIGGE LAM and the GIFS T. Paccagnella (1), D. Richardson (2), D. Schuster(3), R. Swinbank (4), Z. Toth (3), S.

Coordinated Energy and water-cycle Observations Peroject A Well Organized Data Archive System Data Integrating/Archiving Center at University of Tokyo.

TIGGE Archive Highlights. First Service Date ECMWF – October 2006 NCAR – October 2006 CMA – June 2007.

Research Data at NCAR 1 August, 2002 Steven Worley Scientific Computing Division Data Support Section.

Data for Climate and Energy Studies Steven Worley Computational and Information Systems Laboratory NCAR.

The National Center for Atmospheric Research is operated by the University Corporation for Atmospheric Research under sponsorship of the National Science.

TIGGE Data Archive and Access System at NCAR 5th GIFS-TIGGE Working Group South African Weather Service Pretoria March 2008 Steven Worley Doug Schuster.

Scientific Investigations; Support from Research Data Archives for Joint Office for Science Support 26 February, 2002 Steven Worley SCD/DSS.

Describe workflows used to maintain and provide the RDA to users – Both are 24x7 operations Transition to the NWSC with zero downtime NWSC is new environment.

Data Access to Marine Surface Observations and Products from COADS 29 January, 2002 Steven Worley National Center for Atmospheric Research.

Reducing Canada's vulnerability to climate change - ESS J28 Earth Science for National Action on Climate Change Canada Water Accounts AET estimates for.

THORPEX Interactive Grand Global Ensemble (TIGGE) China Meteorological Administration TIGGE-WG meeting, Boulder, June Progress on TIGGE Archive Center.

All about DATASETS Description and Algorithms Description and Algorithms Source Source Spatial and temporal Resolutions Spatial and temporal Resolutions.

Improved Access to RDA from the MSS OSD Executive Meeting April 28, 2009.

ICOADS: Update Status and Data Distribution Steven J. Worley Scott D. Woodruff Sandra J. Lubker Ziahua Ji J. Eric Freeman NCAR, NOAA/ESRL, NOAA/NCDC CLIMAR-III,

Analyzed Data Products Available from NCAR that Support Marine Climate Research JCOMM ETMC-III 9-12 February 2010 Steven Worley Doug Schuster.

Modern Era Retrospective-analysis for Research and Applications: Introduction to NASA’s Modern Era Retrospective-analysis for Research and Applications:

Aristeidis K. Georgoulias Konstantinos Kourtidis Konstantinos Konstantinidis AMFIC Web Data Base AMFIC Annual Meeting - Beijing October 2008 Democritus.

1 Motivation Motivation SST analysis products at NCDC SST analysis products at NCDC  Extended Reconstruction SST (ERSST) v.3b  Daily Optimum Interpolation.

Data Discovery and Access to The International Surface Pressure Databank (ISPD) 1 Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley.

TIGGE, an International Data Archive and Access System Steven Worley Doug Schuster Dave Stepaniak Nate Wilhelmi (NCAR) Baudouin Raoult (ECMWF) Peiliang.

Content, Discovery, and Accessibility Enhancements to the NCAR Research Data Archive Doug Schuster and Steve Worley NCAR.

TIGGE Data Archive and Access at NCAR November 2008 November 2008 Steven Worley National Center for Atmospheric Research Boulder, Colorado, U.S.A.

Slide 1 GO-ESSP Paris. June 2007 Slide 1 (TIGGE and) the EU Funded BRIDGE project Baudouin Raoult Head of Data and Services Section ECMWF.

MMET Team Michelle Harrold Tracy Hertneky Jamie Wolff Demonstrating the utility of the Mesoscale Model Evaluation Testbed (MMET)

TIGGE Data Archive at NCAR 8th GIFS-TIGGE Working Group World Meteorological Organization Geneva February, 2010 Doug Schuster Steven Worley Dave.

The TIGGE Model Validation Portal: An Improvement in Data Interoperability 1 Thomas Cram Doug Schuster Hannah Wilcox Steven Worley National Center for.

TIGGE Archive Status at NCAR THORPEX Workshop and 6th GIFS-TIGGE Working Group Meetings WMO Headquarters Geneva September 2008 Steven Worley Doug.

SCD Research Data Archives; Availability Through the CDP About 500 distinct datasets, 12 TB Diverse in type, size, and format Serving 900 different investigators.

The Research Data Archive at NCAR: A System Designed to Handle Diverse Datasets Bob Dattore and Steven Worley National Center for Atmospheric Research.

TIGGE Archive Access at NCAR Steven Worley Doug Schuster Dave Stepaniak Hannah Wilcox.

Proposed THORPEX/HEPEX Hydrologic Ensemble Project (THEPS) Presentation for 3 rd THORPEX Science Symposium September 14-18, 2009 Prepared by John Schaake,

Data Discovery and Access to The International Surface Pressure Databank (ISPD) 1 Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley.

Enabling the Transition of CPC Products to GIS Format Brian Doty Jennifer Adams Michael Halpert Viviane Silva.

5-7 May 2003 SCD Exec_Retr 1 Research Data, May Archive Content New Archive Developments Archive Access and Provision.

1. Gridded Data Sub-setting Services through the RDA at NCAR Doug Schuster, Steve Worley, Bob Dattore, Dave Stepaniak.

A41I-0105 Supporting Decadal and Regional Climate Prediction through NCAR’s EaSM Data Portal Doug Schuster and Steve Worley National Center for Atmospheric.

Introduction What purpose does a data archive center serve if users can’t find or access the holdings they might need to facilitate their research discoveries?

Verify WRF Forecast using MET

The Use of AMET and Automated Scripts for Model Evaluation

MERRA Data Access and Services

TIGGE Archives and Access

TIGGE Data Archive and Access System at NCAR

Jennifer Boehnert Emily Riddle Tom Hopson

gWRF Workflow and Input Data Requirements

National Water Model (Provided by NOAA)

Operational Dataset Update Functionality Included in the NCAR Research Data Archive Management System Zaihua Ji Doug Schuster Steven Worley Computational.

Development and Futures of Research Data Archives

TIGGE Data Archive at NCAR

Research Data Archives at NCAR

Steven Worley, Douglas Schuster,

Implementation and Plans for TIGGE at NCAR and ECMWF

Comeaux and Worley, NSF/NCAR/SCD

Robert Dattore and Steven Worley

Data Curation in Climate and Weather

Status of the Regional OSSE for Space-Based LIDAR Winds – Feb01

Comeaux and Worley, NSF/NCAR/SCD

Presentation transcript:

The TIGGE Model Validation Portal: An Improvement in Data Interoperability 1 Thomas Cram Doug Schuster Hannah Wilcox Michael Burek Eric Nienhouse Steven Worley National Center for Atmospheric Research Boulder, CO dss.ucar.edu SEA Software Engineering Conference, Boulder, CO 24 Feb 2012

NCAR Research Data Archive Total archive volume over 1.3 PB unique users annually 2 Meteorological and Oceanographic Observations Operational and Reanalysis model outputs Remote Sensing Observations Topography/Bathym etry, Vegetation, Land Use

THORPEX Interactive Grand Global Ensemble (TIGGE) 3 Mission: Foster and accerlerate 1-day to 2-week high- impact weather forecasts.

NCAR TIGGE Data Portal 4

5 tigge.ucar.edu Most recent 3 months of ensemble forecast data Filter data request by NWP center, date range, regional & variable subsets Delayed mode request with notification

TIGGE Model Validation Portal at NCAR 6 Ensemble Forecast Observations

TIGGE Model Validation Portal at NCAR 7 Ensemble Forecast Observations Match observations with ensemble forecasts Streamline access to observational data to support model validation Add value to ensemble forecast research One-stop shop approach

User-side resources Data Access - Common Approach 8 Data User Data Storage Format Conversion Re-gridding Subset Extraction Post Processing Data Storage Format Conversion Re-gridding Subset Extraction Post Processing Forecast Validation

TIGGE MVP Data Flow 9 NCAR RDA system processing Data Storage Format Conversion Re-gridding Subset Extraction Data Storage Format Conversion Re-gridding Subset Extraction Validation Data User

How It Works 10

How It Works 11

Parameter selection 12

Regrid & Spatial Subsetting 13

Data request review 14

Data request review 15 Click on checkbox – get matching obs data notification when data are ready

Download page 16

Under the hood 17 dates= : :00&lats=30.0 N 50.0 N&lons=120.0 W 90.0 W &params=APCP TMP HGT&qualmark=2&tigge=true&fh ours= DSRQST daemon receives data request MySQL db request record created Perl script parses request information

Data Description 18 GeopotentialRel. Humidity TemperatureSpecific Humidity Wind vectors & speedDew Point Mean SLPVapor Mixing Ratio NCEP PREPBUFR Observations (1997 – present) Surface and upper air reports – used for NCEP GDAS

CMORPH Precipitation (coming soon) Precipitation derived from multiple satellites Dec – present 0.25 x 0.25 grid 3-hourly Data (cont.) 19 Validation

20 Output Format – NetCDF PREPBUFR obs Model Evaluation Tools software NetCDF format o MET software ingests: o forecast o observations CMORPH CF-1.5 compliant

21 Access to full archives of data

22 Access to full archives of data TIGGE forecasts (2006-present): dss.ucar.edu/datasets/ds330.0 PREPBUFR (1997-present): dss.ucar.edu/datasets/ds337.0 CMORPH (2002-present): dss.ucar.edu/datasets/ds502.0

23 Data file downloads

24 Data subset interface

25 Data subset interface

Summary 26 Integrated access for ensemble forecasts and observations that can be used for validation Output compatible with MET software Service brings together diverse datasets (burden taken off researcher) Modular approach – Integrated & stand- alone access More info: tigge.ucar.edu