PRP Weather Data Transfer PRPv2 Workshop February 21, 2017

Slides:

Advertisements

Similar presentations

DRC 2006.Presenters Name.March 7-9.p 1 Californias enormous computing resources allow climate simulations at unprecedented resolution capturing Californias.

Advertisements

Quantification of Spatially Distributed Errors of Precipitation Rates and Types from the TRMM Precipitation Radar 2A25 (the latest successive V6 and V7)

AR Science Gaps/Objectives Duane Waliser on behalf of Calwater 2 SSG Jet Propulsion Laboratory/Caltech Pasadena, CA CalWater 2015 – ACAPEX Campaign Planning.

Scaling Laws, Scale Invariance, and Climate Prediction

A Tutorial on MODIS and VIIRS Aerosol Products from Direct Broadcast Data on IDEA Hai Zhang 1, Shobha Kondragunta 2, Hongqing Liu 1 1.IMSG at NOAA 2.NOAA.

Atmospheric Reanalyses Update Mike Bosilovich. ReanalysisHoriz.ResDatesVintageStatus NCEP/NCAR R1T present1995ongoing NCEP-DOE R2T present2001ongoing.

0 Future NWS Activities in Support of Renewable Energy* Dr. David Green NOAA, NWS Office of Climate, Water & Weather Services AMS Summer Community Meeting.

CO 2 in the middle troposphere Chang-Yu Ting 1, Mao-Chang Liang 1, Xun Jiang 2, and Yuk L. Yung 3 ¤ Abstract Measurements of CO 2 in the middle troposphere.

The Earth System Analyzer: Using all of the Data to Improve NOAA’s Mission Capabilities Alexander E. MacDonald NOAA Earth System Research Laboratory.

1 G EOSS A nd M AHASRI E xperiment in T ropics (GaME-T) Taikan Oki and Shinjiro.

Environmental Modeling Center ______________________________________________ Climate Change and Air Quality Workshop MCNC On the intercontinental transport.

THORPEX-Pacific Workshop Kauai, Hawaii Polar Meteorology Group, Byrd Polar Research Center, The Ohio State University, Columbus, Ohio David H. Bromwich.

The International Surface Pressure Databank (ISPD) and Twentieth Century Reanalysis at NCAR Thomas Cram - NCAR, Boulder, CO Gilbert Compo & Chesley McColl.

Grid resources for NWP models at national level in Korea Korean Meteorological Administration Super Computer Center Korea Meteorological Administration.

Dr Mark Cresswell Model Assimilation 69EG6517 – Impacts & Models of Climate Change.

DISTRIBUTED DATA FLOW WEB-SERVICES FOR ACCESSING AND PROCESSING OF BIG DATA SETS IN EARTH SCIENCES A.A. Poyda 1, M.N. Zhizhin 1, D.P. Medvedev 2, D.Y.

South Eastern Latin America LA26: Impact of GC on coastal areas of the Rio de la Plata: Sea level rise and meteorological effects LA27: Building capacity.

ESSE Environmental Scenario Search Engine for the Data Services Grid Mikhail Zhizhin, Geophysical Center Russian Academy of Sciences Eric Kihn,

“An Integrated Science Cyberinfrastructure for Data-Intensive Research” Panel CISCO Executive Symposium San Diego, CA June 9, 2015 Dr. Larry Smarr Director,

UMAC data callpage 1 of 11NLDAS EMC Operational Models North American Land Data Assimilation System (NLDAS) Michael Ek Land-Hydrology Team Leader Environmental.

April nd IBTrACS Workshop 1 Operational Procedures How can we build consistent, homogeneous, well- documented climate quality data?

Birmingham Urban Heat Islands during 2003 Heatwave Period  Preliminary WRF/BEP Simulation Xiaoming Cai, Richard Bassett and John E. Thornes School of.

Scientific Investigations; Support from Research Data Archives for Joint Office for Science Support 26 February, 2002 Steven Worley SCD/DSS.

“Comparative Human Microbiome Analysis” Remote Video Talk to CICESE Big Data, Big Network Workshop Ensenada, Mexico October 10, 2013 Dr. Larry Smarr Director,

A Comparison of the Northern American Regional Reanalysis (NARR) to an Ensemble of Analyses Including CFSR Wesley Ebisuzaki 1, Fedor Mesinger 2, Li Zhang.

Integrated Grid workflow for mesoscale weather modeling and visualization Zhizhin, M., A. Polyakov, D. Medvedev, A. Poyda, S. Berezin Space Research Institute.

The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

JCSDA Summer Colloquium Erica Dolinar 4 August 2015.

Comparison of Atmospheric Rivers depicted from satellite and NWP reanalysis Wenze Yang 1 and Ralph Ferraro 2 1. UMD/ESSIC/CICS, College Park, MD .

Modern Era Retrospective-analysis for Research and Applications: Introduction to NASA’s Modern Era Retrospective-analysis for Research and Applications:

A Portable Regional Weather and Climate Downscaling System Using GEOS-5, LIS-6, WRF, and the NASA Workflow Tool Eric M. Kemp 1,2 and W. M. Putman 1, J.

NASA Snow and Ice Products NASA Remote Sensing Training Geo Latin America and Caribbean Water Cycle capacity Building Workshop Colombia, November 28-December.

Aristeidis K. Georgoulias Contribution of Democritus University of Thrace-DUTH in AMFIC-Project Democritus University of Thrace Laboratory of Atmospheric.

Thomas R. Karl Director, National Climatic Data Center, NOAA Editor, Journal of Climate, Climatic Change & IPCC Climate Monitoring Panel Paul D. Try, Moderator.

CE-QUAL-W2 Shasta Modeling

Simulated and Observed Atmospheric Circulation Patterns Associated with Extreme Temperature Days over North America Paul C. Loikith California Institute.

Cyberinfrastructure to promote Model - Data Integration Robert Cook, Yaxing Wei, and Suresh S. Vannan Oak Ridge National Laboratory Presented at the Model-Data.

1 National HIC/RH/HQ Meeting ● January 27, 2006 version: FOCUSFOCUS FOCUSFOCUS FOCUS FOCUSFOCUS FOCUSFOCUS FOCUSFOCUS FOCUSFOCUS FOCUSFOCUS FOCUSFOCUS.

Evapotranspiration Estimates over Canada based on Observed, GR2 and NARR forcings Korolevich, V., Fernandes, R., Wang, S., Simic, A., Gong, F. Natural.

The PRPv1 Architecture Model Panel Presentation Building the Pacific Research Platform Qualcomm Institute, Calit2 UC San Diego October 16, 2015.

“CAMERA Goes Live!" Presentation with Craig Venter National Press Club Washington, DC March 13, 2007 Dr. Larry Smarr Director, California Institute for.

Earth System Curator and Model Metadata Discovery and Display for CMIP5 Sylvia Murphy and Cecelia Deluca (NOAA/CIRES) Hannah Wilcox (NCAR/CISL) Metafor.

“The UCSD Big Data Freeway System” Invited Short Talk Workshop on “Enriching Human Life and Society” UC San Diego February 6, 2014 Dr. Larry Smarr Director,

This study compares the Climate System Forecast Reanalysis (CFSR) tropospheric analyses to two ensembles of analyses. The first ensemble consists of 12.

GEOGRAPHY 120: EARTH SYSTEM II. Atmospheric Sciences at a Glance.

One-year re-forecast ensembles with CCSM3.0 using initial states for 1 January and 1 July in Model: CCSM3 is a coupled climate model with state-of-the-art.

Overview ABSTRACT As part of its Polar Satellite Program, NOAA operates the Jason-2 altimetry satellite and shares production of near real-time (NRT) products.

Support to scientific research on seasonal-to-decadal climate and air quality modelling Pierre-Antoine Bretonnière Francesco Benincasa IC3-BSC - Spain.

GPM-GLOBE Precipitation Student Field Campaign Global Precipitation Measurement Mission Developed by the GPM Education and Communications Team NASA Goddard.

EVALUATION OF A GLOBAL PREDICTION SYSTEM: THE MISSISSIPPI RIVER BASIN AS A TEST CASE Nathalie Voisin, Andy W. Wood and Dennis P. Lettenmaier Civil and.

Importance of the atmospheric boundary layer (2).

NASA, CGMS-44, 7 June 2016 Coordination Group for Meteorological Satellites - CGMS SURFACE PRESSURE MEASUREMENTS FROM THE ORBITING CARBON OBSERVATORY-2.

OceanWatch Central Pacific Satellite Oceanography Products & Applications PIFSC Melanie Abecassis, PhD Ecosystems and Oceanography Program Ecosystems Sciences.

NASA, CGMS-43, May 2015 Coordination Group for Meteorological Satellites - CGMS Use of Satellite Observations in NASA Reanalyses: MERRA-2 and Future Plans.

WMO Global Atmosphere Watch (GAW) – oppertunities for WGNE 31 st WGNE meeting, April 2016, CSIR, Pretoria, South Africa.

High Performance Cyberinfrastructure Discovery Tools for Data Intensive Research Larry Smarr Prof. Computer Science and Engineering Director, Calit2 (UC.

Center for Satellite Applications and Research (STAR) Review 09 – 11 March 2010 Image: MODIS Land Group, NASA GSFC March 2000 STAR Enterprise Synthesis.

A New Climatology of Surface Energy Budget for the Detection and Modeling of Water and Energy Cycle Change across Sub-seasonal to Decadal Timescales Jingfeng.

“The Pacific Research Platform Two Years In”

Data Browsing/Mining/Metadata

Database management system Data analytics system:

National Aeronautics and Space Administration

MERRA Data Access and Services

TOWARDS HIGH-RESOLUTION GLOBAL SATELLITE PRECIPITATION ESTIMATION

South Eastern Latin America

Optical SIG, SD Telecom Council

Global Monsoon Onset and Retreat based on Precipitable Water and Wind

VIFI : Virtual Information Fabric for Data-Driven Discovery from Distributed Fragmented Repositories PI: Dr. Ashit Talukder Bank of America Endowed Chair.

Data Curation in Climate and Weather

Presentation transcript:

PRP Weather Data Transfer PRPv2 Workshop February 21, 2017 Scott L. Sellars1, Phu Nguyen2, John Graham3, Joulien Tatar3, Larry Smarr3, Tom DeFanti3, F. Martin Ralph1 and Soroosh Sorooshian2 1Center for Western Weather and Water Extremes, UCSD, La Jolla, CA 2Center for Hydrometeorology and Remote Sensing, UCI, Irvine, CA 3California Institute for Telecommunications and Information Technology (Calit2), UCSD, La Jolla, CA

Weather Data Transfer Real time observations and Models Satellites, Ground based observations, Numerical Weather Prediction “Reanalysis” data NASA (M2I3NPASM): Assimilated Meteorological Fields (~15TB) (M2T1NXADG): Aerosol Diagnostics (extended) (~15TB) (M2I3NVAER): Aerosol Mixing Ratio (~15TB) (M2I3NVASM): Assimilated Meteorological Fields (M2I3NVCHM): Carbon Monoxide and Ozone Mixing Ratio (~8TB) (M2I6NVANA): Analyzed Meteorological Fields (~15TB) (M2T1NXLND): Land Surface Diagnostics (~5TB) NCEP/NCAR ECMWF (ERA-40) NOAA Climate Models CMIP5 (2,200TB) CMIP6 (?) 1127 variables…

NASA MERRA2 IVT (kg m-1 s-1 ) Latitude Longitude

Big Data Transformed Into Insight Novel approach in Computational Earth Sciences (Sellars et al. 2013,2015) Dr. Scott L. Sellars and Dr. Phu Nguyen (among others) CONNECT: Object Segmentation Object Storage (PostgreSQL) Data Hypercube: 5mm/hr Rainfall Longitude Latitude Time t=5 t=4 t=3 t=2 t=1 Latitude Longitude Database Indexes: Set Object Criteria: Data Object ID Number Latitude (of each voxel in objects) Longitude (of each voxel in objects) Time (hour) Precipitation Intensity (mm/hour) Each voxel must have 1mm/hr Each object must exist for 24 hours 6 voxel connections (i.e. voxel face connections) 60N-60S, 0-360 lat and long Hourly time step March 1st, 2000 to January 1st, 2011

Recent Big Data Analysis Pipeline: One Variable CHRS 2.4T Time: 7d 10h 49min 2.4T to 100GB Time: 10d 5h 05min 100GB to 50GB Time: ~1d 14h 00m 50GB to 100MB Time: ~1d 5h 00m NASA CW3E CHRS C3WE Data Visualization And Search CW3E Download Data organization Variable format CONNECT Segmentation CONNECT Characteristic Calc. Total time: ~20d 11h 0m Not including data visualization Not including data mining/machine learning jobs Assumes we know what we are doing Data mining and Discovery, Machine Learning

Pacific Research Platform (10-100 Gb/s) SDSC’s COMET Calit2’s FIONA Calit2’s FIONA

Opportunities Data Download via PRP from NASA (among others) to FIONA Using PRP network Download speed increased 4x using PRP network (40MB/s which is 4x faster than the 10MB/s standard connection previously being used by researchers at SIO). Live Thematic Real-time Environmental Distributed Data Services server (THREDDS) Set up CONNECT algorithm and workflow (and other research scripts) to harness FIONA tech, Super Computing Resources, and PRP