NASA Langley Research Center ASAP Science Meeting, Boulder, CO 13-14 Apr 2005 Louis Nguyen, Patrick Minnis, John Murray, and Rabindra Palikonda NASA Langley.

Slides:

Advertisements

Similar presentations

1 GOES-R Hurricane Intensity Estimation (HIE) Validation Tool Development Winds Application Team Tim Olander (CIMSS) Jaime Daniels (STAR)

Advertisements

Center for Satellite Applications and Research (STAR) Review 09 – 11 March 2010 Image: MODIS Land Group, NASA GSFC March 2000 A Cloud Object Based Volcanic.

Numerical Weather Prediction Readiness for NPP And JPSS Data Assimilation Experiments for CrIS and ATMS Kevin Garrett 1, Sid Boukabara 2, James Jung 3,

Retrieving Cloud Properties for Multilayered Clouds Using Simulated GOES-R Data Fu-Lung Chang 1, Patrick Minnis 2, Bing Lin 2, Rabindra Palikonda 3, Mandana.

LAPS Analysis & Software by Steve Albers. 2 Basic Solution LAPS coupled with MM5 NWP model Use diabatic initialization (“hot start”) Utilize parallel.

IBM 1350 Cluster Expansion Doug Johnson Senior Systems Developer.

McIDAS-Lite Dave Santek Program Manager Space Science & Engineering Center University of Wisconsin-Madison 5 December 2002.

McIDAS-V McIDAS-V The 5 th Generation of McIDAS by Tom Whittaker Space Science and Engineering Center University of Wisconsin-Madison USA with contributions.

2. Point Cloud x, y, z, … Complete LiDAR Workflow 1. Survey 4. Analyze / “Do Science” 3. Interpolate / Grid USGS Coastal & Marine.

February 28, 2005 NOAA / NOS / CO-OPS An Oceanographic Processing Environment: From System Design to Data Quality Chris Paternostro, Oceanographer NOAA,

02/07/2001 EOSDIS Core System (ECS) COTS Lessons Learned Steve Fox

Chapter 2 Introduction to Systems Architecture. Chapter goals Discuss the development of automated computing Describe the general capabilities of a computer.

November 2011 At A Glance GREAT is a flexible & highly portable set of mission operations analysis tools that increases the operational value of ground.

SDI (Satellite Data Ingestor) 2015 McIDAS Users Group Meeting June 8, 2015.

Information Technology for Ocean Observations and Climate Research TYKKI Workshop, December 9-11, 1998, Tokyo, Japan Nancy N. Soreide NOAA Pacific Marine.

Plans for AWIPS Next Generation 1April 12, 2005 AWIPS Technology Infusion Darien Davis NOAA/OAR Forecast Systems Laboratory Systems Development Division.

Storm Tracker – a Web-Based Tool for Working with the Automated Tracking of Thunderstorm Clusters Bob Rabin NOAA / NSSL / CIMSS Tom Whittaker Space Science.

McIDAS: The First True GIS Mashup Tommy Jasmin UW / CIMSS / SSEC McIDAS Users’ Group Meeting June 9, 2015.

EGU 2011 TIGGE, TIGGE LAM and the GIFS T. Paccagnella (1), D. Richardson (2), D. Schuster(3), R. Swinbank (4), Z. Toth (3), S.

Environmental Satellite Product and Distribution System (ESPDS) GOES-R Access Subsystem (GAS) John Linn ESPDS System Engineering 7 October 2009.

McIDAS Advisory Committee (MAC) Sponsored: Group Discussion Joleen Feltz McIDAS Advisory Committee Chair from Cooperative Institute for Meteorological.

National Weather Service National Weather Service Central Computer System Backup System Brig. Gen. David L. Johnson, USAF (Ret.) National Oceanic and Atmospheric.

, Implementing GIS for Expanded Data Accessibility and Discoverability ASDC Introduction The Atmospheric Science Data Center (ASDC) at NASA Langley Research.

Status of RSA LAPS/MM5 System Sustainment John McGinley, Steve Albers*, Chris Anderson*, Linda Wharton NOAA Earth System Research Laboratory Global Systems.

Unidata TDS Workshop TDS Overview – Part I XX-XX October 2014.

Tools Developed For AVHRR Data Production and Analysis The AVHRR modular framework can produce three output datasets in NetCDF format when AVHRR NetCDF.

ATMOSPHERIC SCIENCE DATA CENTER ‘Best’ Practices for Aggregating Subset Results from Archived Datasets Walter E. Baskin 1, Jennifer Perez 2 (1) Science.

MURI Hardware Resources Ray Garcia Erik Olson Space Science and Engineering Center at the University of WI - Madison.

Mathematics and Computer Science & Environmental Research Divisions ARGONNE NATIONAL LABORATORY Regional Climate Simulation Analysis & Vizualization John.

NOAA Satellite Proving Ground/User Readiness Meeting

The Namibia Flood Dashboard Satellite Acquisition and Data Availability through the Namibia Flood Dashboard Matt Handy NASA Goddard Space Flight Center.

CMAQ Runtime Performance as Affected by Number of Processors and NFS Writes Patricia A. Bresnahan, a * Ahmed Ibrahim b, Jesse Bash a and David Miller a.

Robert Hartman Acting Director NWS Office of Hydrologic Development GPM in the NOAA Integrated Water Forecasting Program.

AGU 2002 Fall Meeting NASA Langley Research Center / Atmospheric Sciences Validation of GOES-8 Derived Cloud Properties Over the Southeastern Pacific J.

LAPS __________________________________________ Analysis and nowcasting system for Finland/Scandinavia Finnish Meteorological Institute Erik Gregow.

Using Meteosat-8 SEVIRI as a Surrogate for Developing GOES-R Cloud Algorithms P. Minnis, W. L. Smith Jr., L. Nguyen, J. K. Ayers, D. R. Doelling, R. Palikonda,

Center for Satellite Applications and Research (STAR) Review 09 – 11 March 2010 Image: MODIS Land Group, NASA GSFC March 2000 POES Microwave Products Presented.

EOS Terra MODIS Land Processing and Distribution Overview Joseph M Glassy, Director, MODIS Software Development at NTSG School of Forestry, Numerical Terradynamics.

The Next Generation McIDAS A Look Towards the Future 21 st IIPS 8.2 San Diego, CA - 11 January 2005 Dave Santek Tom Whittaker Thomas Achtor Space Science.

2015 GLM Annual Science Team Meeting: Cal/Val Tools Developers Forum 9-11 September, 2015 DATA MANAGEMENT For GLM Cal/Val Activities Helen Conover Information.

View_hdf Kam-Pui Lee Science Applications International Corporation CERES Data Management Team Linda Hunt Computer Sciences Corporation Atmospheric Sciences.

A Portable Regional Weather and Climate Downscaling System Using GEOS-5, LIS-6, WRF, and the NASA Workflow Tool Eric M. Kemp 1,2 and W. M. Putman 1, J.

NetCDF file generated from ASDC CERES SSF Subsetter ATMOSPHERIC SCIENCE DATA CENTER Conversion of Archived HDF Satellite Level 2 Swath Data Products to.

Cooperative Institute for Meteorological Satellite Studies University of Wisconsin - Madison ABI and AIRS Retrievals in McIDAS-V Kaba Bah.

MODIS Direct Broadcast Products at UW-Madison Recent Highlights Terra MODIS passes acquired IMAPP Level 2 Atmosphere Products released in May 2002.

January 7, 2015 Walter Wolf, Jaime Daniels, and Lihang Zhou NOAA/NESDIS, Center for Satellite Applications and Research (STAR) Shanna Sampson, Tom King,

Access to Real-time and Historical Satellite Products from a Mobile Application Thad Chee 1, Matthew Kelly 1, Louis Nguyen 2, Patrick Minnis 2, Rabindra.

Near Real-Time Verification At The Forecast Systems Laboratory: An Operational Perspective Michael P. Kay (CIRES/FSL/NOAA) Jennifer L. Mahoney (FSL/NOAA)

MADIS Airlines for America Briefing Meteorological Assimilated Data Ingest System (MADIS) FPAW Briefing Steve Pritchett NWS Aircraft Based Observations.

August 2003 At A Glance The IRC is a platform independent, extensible, and adaptive framework that provides robust, interactive, and distributed control.

Overview of Satellite-Derived Cirrus Properties During SPARTICUS and MACPEX P. Minnis, L. Nguyen NASA Langley Research Center, Hampton, VA R. Palikonda,

ASAP In-Flight Icing Research at NCAR J. Haggerty, F. McDonough, J. Black, S. Landolt, C. Wolff, and S. Mueller In collaboration with: P. Minnis and W.

MACPEX Planning Meeting, Houston, TX, 2-3 February 2011 NASA LaRC Satellite Cloud Remote Sensing & Field Mission/Satellite Support Patrick Minnis, Louis.

AQUA AMSR-E MODIS POES AVHRR TRMM TMI ENVISAT AATSR GOES Imager Multi-sensor Improved SST (MISST) for GODAE Part I: Chelle Gentemann, Gary Wick Part II:

Earth Observing Satellites Update John Murray, NASA Langley Research Center NASA Aviation Weather Satellites Last Year NASA’s AURA satellite, the chemistry.

NASA Program Review 11/18/08 © 2008 UCAR Space Weather Displays for the Aviation Digital Data Service (ADDS)‏ Arnaud Dumont Nelson Hillyer Greg Thompson.

MODIS Data at NSIDC MODIS Science Team Meeting - Nov. 2, 2006.

McIDAS-X Software Development and Demonstration Dave Santek and Jay Heinzelman 2 June 2009 PDA Animated Weather (PAW) Status by Russ Dengel.

NASA Earth Exchange (NEX) Earth Science Division/NASA Advanced Supercomputing (NAS) Ames Research Center.

Central Operations Ben Kyger Acting Director / NCEP CIO.

1 Application of MET for the Verification of the NWP Cloud and Precipitation Products using A-Train Satellite Observations Paul A. Kucera, Courtney Weeks,

NOAA Near Real-Time AIRS Processing and Distribution System Walter Wolf Mitch Goldberg Lihang Zhou NESDIS/ORA/CRAD.

INTRODUCTION TO XSEDE. INTRODUCTION  Extreme Science and Engineering Discovery Environment (XSEDE)  “most advanced, powerful, and robust collection.

Band 14 (11um) Winds Low-Level >700 mb Mid-Level mb High-Level mb   NPP VIIRS Polar Winds Products The GOES-R AWG Derived Motion Winds.

GOES-R AIT: Updating the Data Processing System with data from the Himawari-8 Geostationary Satellite Jonathan Wrotny1, A. Li1, A. Ken1, H. Xie1, M. Fan1,

SDI (Satellite Data Ingestor)

The Six Parts of a GIS.

Tool for Storm Analysis Using Multiple Data Sets

Generation of Cloud Products from NOAA’s Operational Satellite Imagers

Start Hot-Start Section

Presentation transcript:

NASA Langley Research Center ASAP Science Meeting, Boulder, CO Apr 2005 Louis Nguyen, Patrick Minnis, John Murray, and Rabindra Palikonda NASA Langley Research Center Hampton, Virginia, USA Migration of LaRC Icing Products to Columbia Supercomputer

NASA Langley Research Center SPIE 49th, Denver, CO 2-6 Aug 2004 Objectives Improve processing system to produce reliable icing products Increase temporal and spatial resolution of these products Improve latency time Port and migrate code to Project Columbia Supercomputer Transition research code to operational code Transition and Integrate the Langley’s Satellite Products to Operations in the operational National Weather Service Aviation Weather Center CIP product

NASA Langley Research Center SPIE 49th, Denver, CO 2-6 Aug 2004 Outline Overview of LaRC Icing processing environment Project Columbia Supercomputer Migration plan Utilize parallel processing Benefits from improved computer capability Summary and future plans

NASA Langley Research Center SPIE 49th, Denver, CO 2-6 Aug 2004 Overview of LaRC Processing Environment Real-time Data Acquisition System GOES-East & West data from via McIDAS ADDE server (SSEC)GOES-East & West data from via McIDAS ADDE server (SSEC) RUC data obtained from SSEC and NOAA Forecast Systems Lab RUC data obtained from SSEC and NOAA Forecast Systems Lab Snow maps from NOAA via FTP site Snow maps from NOAA via FTP site Transition to Operations, utilize the NOAA NESDIS operational 24/7 ADDE servers to ingest satellite and model data Transition to Operations, utilize the NOAA NESDIS operational 24/7 ADDE servers to ingest satellite and model data Processing Environment Uses the Man computer Interactive Data Access System (McIDAS, developed by SSEC Univ. Wisc)Uses the Man computer Interactive Data Access System (McIDAS, developed by SSEC Univ. Wisc) McIDAS is a programmable suite of software and interactive data visualization tool used for displaying, analyzing, and acquiring datasetsMcIDAS is a programmable suite of software and interactive data visualization tool used for displaying, analyzing, and acquiring datasets Runs on SGI Unix platform Runs on SGI Unix platform

NASA Langley Research Center SPIE 49th, Denver, CO 2-6 Aug 2004 Overview of LaRC Icing Processing (con’t) Source Code Code Modularization (R. Palikonda)Code Modularization (R. Palikonda) -satellite modules for GOES (8-12), MSG, AVHRR, MODIS - sounding module for RUC, ECMWF, GFS/AVN - helps isolates sections of the algorithm to improve cloud mask, corr-k, cloud parameterization, etc. - most modules are written in C Main driver program is in McIDAS environmentMain driver program is in McIDAS environment Cloud algorithms are written in FortranCloud algorithms are written in Fortran Data inputs and outputs utilizes McIDAS API Data inputs and outputs utilizes McIDAS API Approximately 75,000 lines of code Approximately 75,000 lines of code

NASA Langley Research Center SPIE 49th, Denver, CO 2-6 Aug 2004 Overview of LaRC Icing Processing (con’t) Current Computer Resources Used for ASAP SGI Orgin 3800 with 40 CPUsSGI Orgin 3800 with 40 CPUs Only 8 CPU is available for Icing (other CPUs are used by CERES & CALIPSO project)Only 8 CPU is available for Icing (other CPUs are used by CERES & CALIPSO project) Limitation of Current Computer Resources Run code every half hour during the day and hourly at nightRun code every half hour during the day and hourly at night Process GOES-West (10) at 8km res: ~30 minsProcess GOES-West (10) at 8km res: ~30 mins Process GOES-East (12) at 8km res: ~35 minsProcess GOES-East (12) at 8km res: ~35 mins Need Additional Resources Run at full 4km resolution every 15 minsRun at full 4km resolution every 15 mins Would take ~70 min to process GOES-West and ~80 min for GOES-East at full resolutionWould take ~70 min to process GOES-West and ~80 min for GOES-East at full resolution Project Columbia Supercomputer offers needed resourceProject Columbia Supercomputer offers needed resource

NASA Langley Research Center SPIE 49th, Denver, CO 2-6 Aug 2004 NASA’s newest supercomputer is one of the world’s fastestNASA’s newest supercomputer is one of the world’s fastest Supports the agency’s 4 mission directorates (Aeronautics, Exploration, Science, & Space), NASA Engineering and Safety Center (NESC), and Vision for Space ExplorationSupports the agency’s 4 mission directorates (Aeronautics, Exploration, Science, & Space), NASA Engineering and Safety Center (NESC), and Vision for Space Exploration Support general science community: GSFC runs a General Circulation Model to generate real-time numerical weather prediction (NWP) to improve hurricane tracking and intensity forecast for Hurricane CenterSupport general science community: GSFC runs a General Circulation Model to generate real-time numerical weather prediction (NWP) to improve hurricane tracking and intensity forecast for Hurricane Center Funds for Columbia pre-paid forFunds for Columbia pre-paid for thru FY05-07 from the 4 mission directorates and NESC Requested 35,000 cpu hrs FY05-06Requested 35,000 cpu hrs FY05-06 from Aviation Safety Program allotment Project Columbia Supercomputer

NASA Langley Research Center SPIE 49th, Denver, CO 2-6 Aug SGI Altix 3700 superclusters, each with 512 processors20 SGI Altix 3700 superclusters, each with 512 processors 10,240 Intel Itanium-2 processors running at 1.5 GHz10,240 Intel Itanium-2 processors running at 1.5 GHz Capable of 51.9 teraflops, ranked second behind DOE Blue GeneCapable of 51.9 teraflops, ranked second behind DOE Blue Gene 20 Terabyte of Memory20 Terabyte of Memory Linix based operation systemLinix based operation system 40 TB of fiber channel RAID storage40 TB of fiber channel RAID storage Capable of storing 10 petabytes of archive storageCapable of storing 10 petabytes of archive storage Project Columbia Supercomputer Technical Hardware Description

NASA Langley Research Center SPIE 49th, Denver, CO 2-6 Aug 2004 Migration to Project Columbia NASA Advance Supercomputing (NAS) accounts requestedNASA Advance Supercomputing (NAS) accounts requested McIDAS software installationMcIDAS software installation Over 1000 programs and subroutinesOver 1000 programs and subroutines Testing the software componentsTesting the software components Port LaRC Icing code to Columbia platformPort LaRC Icing code to Columbia platform Data ingest system (satellite, model, input maps)Data ingest system (satellite, model, input maps) Processing script and aux program packageProcessing script and aux program package LaRC cloud algorithmsLaRC cloud algorithms Test and check for consistency of Icing products produced from LaRC and ColumbiaTest and check for consistency of Icing products produced from LaRC and Columbia Possible security/firewall issues related to ingest and dissemination of data and products via ADDE and FTP ports

NASA Langley Research Center SPIE 49th, Denver, CO 2-6 Aug 2004 Parallel Processing GOES-10 Phase GOES-12 Phase GOES-12 & GOES-10 are processed using single CPU - at 8km takes 30 mins - at 4km takes 80 mins Time includes 5-8 mins for Input and output Two parallel processing approach: - run sub-domains using multiple CPUs (use same binary) - multi-threaded process for looping through each grid box Parallel processing will use 16 CPUs (8 for ea GOES) - Columbia CPU are ~2.5x faster

NASA Langley Research Center SPIE 49th, Denver, CO 2-6 Aug 2004 Parallel Processing (con’t) Full Resolution 80 min process should be reduced to 9-12 mins

NASA Langley Research Center SPIE 49th, Denver, CO 2-6 Aug 2004 Summary and Future Plans LaRC Satellite products available near real-time in several formats (McIDAS ADDE server, RUC Gridded netCDF, and LaRC Packed Binary) LaRC Icing code will be adapted to run in parallel processing mode Parallel processing will allow GOES Icing products to run at full resolution every 15 min Porting to Project Columbia Supercomputer to be completed by end of FY05 (ASAP project milestone) Work with the NASA Applied Sciences Program, the FAA and NOAA to integrate and transition the LaRC Icing products to the operational NWS Aviation Weather Center CIP product by end of FY06 This research was support by the NASA Aviation Safety Program.