PRELIMINARY DESIGN REVIEW NPP MICROWAVE SOUNDER-BASED TROPICAL CYCLONE PRODUCTS (NTCP) OCTOBER 16, 2012 PRESENTED BY: ANDREA SCHUMACHER1, MARK DEMARIA2, LIMIN ZHAO3 1. CIRA / COLORADO STATE UNIVERSITY 2. NOAA/NESIDS/STAR/RAMMB 3. NOAA/NESDIS/OSPO

Preliminary Design Review: NPP Microwave-based TC Products Outline Introduction Demaria Project Requirements Schumacher Operations Concept Zhao Algorithm Theoretical Basis Schumacher Software Architecture & Interfaces Schumacher Quality Assurance Zhao Risks and Actions Demaria Summary and Conclusions Demaria Preliminary Design Review: NPP Microwave-based TC Products

Preliminary Design Review: NPP Microwave-based TC Products Introduction Presented by Mark Demaria (STAR/CIRA) Preliminary Design Review: NPP Microwave-based TC Products

Preliminary Design Review: NPP Microwave-based TC Products Introduction Project Objective Project Stakeholders Project Plan Preliminary Design Review: NPP Microwave-based TC Products

Preliminary Design Review: NPP Microwave-based TC Products PROJECT OBJECTIVES Provide TC intensity and structure estimates and mean layer winds from ATMS during the NPP and JPSS era to support the continuity of operation. The products include: Tropical cyclone maximum wind Minimum sea level pressure Radii of 34, 50, 64 kt winds in 4 quadrants relative to the storm center Balanced horizontal winds from 1000 to 200 mb within 600km of the storm center Preliminary Design Review: NPP Microwave-based TC Products

Project Stakeholders – Development Team STAR and CIRA Mark DeMaria Andrea Schumacher John Knaff OSPO Limin Zhao Hazari Syed Preliminary Design Review: NPP Microwave-based TC Products

Project Stakeholders – O & M OSPO Products Area Lead – Liqun Mao OSPO QA Lead – Zhaohui Cheng Operational Implementation and Maintenance Team Hazari Syed Jiande Wang Science Maintenance Team Mark DeMaria Andrea schumacher Preliminary Design Review: NPP Microwave-based TC Products

Project Stakeholders – Users National Hurricane Center Michael Brennan (michael.j.brennan@noaa.gov) Jack Beven (john.l.beven@noaa.gov) Joint Typhoon Warning Center Edward Fukada (edward.fukada@navy.mil) Central Pacific Hurricane Center Satellite Analysis Branch Mike Turk (michael.turk@noaa.gov) Preliminary Design Review: NPP Microwave-based TC Products

Project Plan - Ipt Members IPT Lead: Mark DeMaria (STAR) IPT Backup Lead: Limin Zhao (OSPO) NESDIS Team: STAR: Mark DeMaria, John Knaff OSPO: Limin Zhao, Liqun Ma (TC Products PAL) OSD: Tom Schott Data Center: Phil Jones Others: Andrea Schumacher (CIRA), Jiande Wang (SSAI), Hazari Syed (SSAI) User Team Lead: Michael Brennan and Jack Beven (NWS/NHC) Oversight Panel (OP) lead: ICAPOP Other Ops Involved: Soundings, Precipitation Preliminary Design Review: NPP Microwave-based TC Products

PROJECT PLAN - STAKEHOLDER INVOLVEMENT Development Team Coding and documentation Quality assurance Data verification and validation Configuration Management Operation and Maintenance Code Acceptance Running system Interface with operations Monitoring Distribution Users Supply customer request Review project development Archive data Communicate with development team Preliminary Design Review: NPP Microwave-based TC Products

Project Plan - Task and Schedules Tasks: Defined in “FY13_NPP_Tropical_Cyclone_Products_v2.ppt” (SPSRB PSDI) Schedule (key milestones): Preliminary Design Review – July, 2012  Oct 16, 2012 Critical Design Review – Nov, 2012 Software Review – Feb, 2013 System Readiness Review – July 2013 SPSRB Briefing – Aug 2013 Operations Commence – Aug 2013 Preliminary Design Review: NPP Microwave-based TC Products

Project Requirements Presented by Andrea Schumacher (STAR/CIRA) Preliminary Design Review: NPP Microwave-based TC Products

Project requirements: user request SPSRB User Request #1009-0017 submitted by NHC Requesting the current AMSU-based TC intensity and structure estimates and mean layer winds be generalized from ATMS during the NPP and JPSS era to support the continuity of operation. User community National Hurricane Center and Central Pacific Hurricane Center (CPHC) Joint Typhoon Warning Center (JTWC) & Dept. of Defense NWS Offices Benefit to user Continuity and potential improvement of TC products that provide guidance to NHC forecasters NOAA Mission Goal supported Weather and Water, Commerce and Transportation Mission priority Optimal Preliminary Design Review: NPP Microwave-based TC Products

Project requirements: Ingest NCEP GFS analyses and forecasts MIRS ATMS temperature profile retrievals Current TC positions and intensities NHC a-decks and b-decks, available at ftp://ftp.nhc.noaa.gov/atcf/ and currently being ingested by OSPO JTWC a-decks and b-decks, currently being ingested by OSPO Preliminary Design Review: NPP Microwave-based TC Products

Project requirements: Processing Product name: ATMS-based TC Products Accuracy (same or better than current AMSU-based products) Intensity estimate – MAE of 13.5 kt / 9 hPa Wind radii estimates – MAE of 32, 18, and 10 nmi for 34-, 50-, and 64-kt radii, respectively Latency Available ~ 3 hours after satellite pass Timeliness Run frequency: New product generated every 6 hours at synoptic times (0, 6, 12, 18 UTC) Updated every 2 hours Run duration: Less than 5 minutes (~30 sec per active TC) Preliminary Design Review: NPP Microwave-based TC Products

Project requirements: Dissemination Product Dissemination NDE/ESPC DDS Text outputs will be integrated into Automated Tropical Cyclone Forecast (ATCF) system. The data include: TC intensity estimates Surface wind radii estimates (R34, R50 and R64)* 2-d balance winds *Will need to be hosted in real time on ftp site so they can be retrieved and integrated into ATCF Preliminary Design Review: NPP Microwave-based TC Products

Project requirements: System Code development CIRA/Colorado State and NOAA/NESDIS/STAR/RAMMB, Fort Collins, CO A. Schumacher and M. DeMaria Operational implementation NTCP runs on the ESPC operational environment with only extra requirements for disk storage and computing power tcfprod/vmtcfpdev (Linux VMWare) Memory ~ 100 mb Job Run Time: ~ 30 sec per active TC ~50 GB disk storage gp5/gp50 (Linux Web server) ~ 1 GB disk storage DDS (IBM Data Distribution Server) Preliminary Design Review: NPP Microwave-based TC Products

Preliminary Design Review: NPP Microwave-based TC Products Operations Concept Presented by Limin Zhao NOAA/NESDIS/OSPO Preliminary Design Review: NPP Microwave-based TC Products

Operations Concept - Overview Review the concept of operations that the customers/users has documented in the SPSRB user request Interact with customers/users to produce an initial algorithm/system design that is consistent with their concept of operations Review the answers to the following questions based on customer/user needs and expectations and production constraints What is the product? Why is this product being produced? How will this product be used? How should this product be produced (operational scenario)? The operations concept will be refined by the NTCP IPT, in consultation with customers/users, as the product solution and design are matured through the development phase. Preliminary Design Review: NPP Microwave-based TC Products

Preliminary Design Review: NPP Microwave-based TC Products NPP TC Products Users The product requirements are originated from NWS through the SPSRB user request, and its users include: National Hurricane Center and Central Pacific Hurricane Center (CPHC) Joint Typhoon Warning Center (JTWC) & Dept. of Defense Preliminary Design Review: NPP Microwave-based TC Products

Preliminary Design Review: NPP Microwave-based TC Products What is the Product? Tropical Cyclone intensity and structure estimates and mean layer winds from NPP ATMS Specific Requirements from users Parameters Specifications Environmental parameter Tropical cyclone maximum wind, minimum sea level pressure, radii of 34, 50, 64 kt winds in 4 quadrants relative to the storm center, balanced horizontal winds from 1000 to 200 mb within 600km of the storm center Geographical coverage North Atlatic and Northeast Pacific Tropical cyclone basins Vertical Resolution Point values for intensity/radii; mandatory levels for balanced wind Horizontal resolution Point values for intensity/radii; 0.2 lat/lon grids for balanced winds Measurement range Similar to current AMSU-based products Measurement precision Measurement accuracy Latency 3 hours Refresh 6 hours Preliminary Design Review: NPP Microwave-based TC Products

Why Are The Products Being Produced? Current operational TC products are available from NOAA-15/16/18 Users need the operational capability for TC products from the NPP/JPSS satellites in supporting the continuity of operation The products will be produced using the MiRS ATMS temperature retrievals Preliminary Design Review: NPP Microwave-based TC Products

How Will The Products Be Used? The NPP TC products will be used at NHC, JTWC, CPHC and WMO RMSCs for the diagnosis of tropical cyclone including intensity, size and warm/cold core structures The NPP TC products will be integrated into ATCF for users to access. Preliminary Design Review: NPP Microwave-based TC Products

How Should The Products Be Produced? (1/6) Satellite Data for Product Generation ATMS-based temperature and moisture profiles from MiRS (netCDF4) Ancillary Data for Product Generation ATCF TC positions provided by HRC GFS Analysis and Forecasts from NCEP Ancillary Data for Product Validation Hurricane flight reconnaissance intensities and wind radii (ground truth) AMSU-derived products Preliminary Design Review: NPP Microwave-based TC Products

How Should The Products Be Produced? (2/6) There will be three distinct environments Development Environment (CIRA, Linux) Development and testing of pre-operational codes on Redhat Linux OS Test environment (ESPC, vmtcfpdev) Pre-operational code (DAP) received from CIRA will be tested on the designated VMWare Linux machine at ESPC and modified as needed before it is promoted to operation Operation Environment (ESPC, tcfprod) Operational DAP will be run to generate the products on the designated VMWare machine at ESPC and the products will be distributed to user through ESPC distribution system (DDS) Preliminary Design Review: NPP Microwave-based TC Products

How Should The Products Be Produced? (3/6) Coverage Global Ocean Formats ASCII, netCDF4 Latency 3 hours after satellite pass Run time ~ 30 seconds per active TC Resolution Point values for intensity and radii where storms exist 0.2 lat/lon grid for balanced winds Products Tropical cyclone maximum wind Minimum sea level pressure Radii of 34, 50, 64 kt winds in 4 quadrants relative to the storm center Balanced horizontal winds from 1000 to 200 mb within 600km of the storm center Preliminary Design Review: NPP Microwave-based TC Products

How Should The Products Be Produced? (4/6) Production and Delivery Scenarios ESPC will pull the MiRS SND products from NDE following the existing data access protocol NTCP will pull the required ancillary data that currently available on the ESPC internal servers NTCP will collect the satellite inputs and required ancillary data to run the TC algorithm The product will be generated in ASCII and netCDF formats The product users will be granted access to the ESPC distribution system through the data access request submission process. ESPC will handle the distribution of the NPP TC products Preliminary Design Review: NPP Microwave-based TC Products

How Should The Products Be Produced? (5/6) Production Monitoring and Maintenance Scenarios The product process will be monitored by ESPC and email alerts will be generated when an error occurs The NPP TC algorithm will provide overall quality control flags and quality summary level information ESPC will cover the monitoring of input data latency, product latency, and product distribution Offline Statistic comparison with NHC best track intensity and radii estimates for cases within 3 hrs of aircraft fix The TC PAL and maintenance personnel at OSPO will monitor the product quality and will coordinate with the STAR/CIRA scientists for any product quality issues Preliminary Design Review: NPP Microwave-based TC Products

How Should The Products Be Produced? (6/6) Production Archival Scenarios The TC intensity/structure products will be provided to NHC as a text bulletin in Automated Tropical Cyclone Format (ATCF) and will be archived there (same as current AMSU products). TC balanced winds products will be archived at NCDC, and a Data Submission Agreement will be developed and submitted to NCDC/CLASS for it. Local archive requires to keep three day worth of data for real-time use Preliminary Design Review: NPP Microwave-based TC Products

Algorithm Theoretical Basis Presented by Andrea Schumacher (STAR/CIRA) Preliminary Design Review: NPP Microwave-based TC Products

algorithm overview: Science description ATMS-based TC products will be based on AMSU-based algorithm already in operations at NCEP Intensity estimates (operational since 2003) Uses combination of AMSU-based soundings, the hydrostatic relationship, and statistics Described in Demuth, J., M. DeMaria, and J.A. Knaff, 2006: Improvement of Advanced Microwave Sounding Unit Tropical Cyclone Intensity and Size Estimation Algorithms, Journal of Applied Meteorology and Climatology, 45, 1573-1581. Demuth, J., M. DeMaria, J.A. Knaff and T. H. Vonder Haar, 2004: Validation of an advanced microwave sounder unit (AMSU) tropical cyclone intensity and size estimation algorithm, J. App. Met., 43, 282-296. Wind radii estimates (operational since 2005) Estimated from 2-D winds created by AMSU-derived height field and solving non-linear balance equations Bessho, K., M. DeMaria, and J.A. Knaff, 2006: Tropical cyclone wind retrievals from the Advanced Microwave Sounder Unit: Application to Surface Wind Analysis. J. Appl. Meteor. and Clim., 45,399-415. Preliminary Design Review: NPP Microwave-based TC Products

algorithm overview: Science maturity NOAA-15/16/18 AMSU-based balanced wind algorithms have been running operationally since 2005. Algorithm and validation details are provided in: Bessho, K., M. DeMaria, and J.A. Knaff, 2006: Tropical cyclone wind retrievals from the Advanced Microwave Sounder Unit: Application to Surface Wind Analysis. J. Appl. Meteor. and Clim., 45,399-415. Preliminary Design Review: NPP Microwave-based TC Products

Algorithm overview - benefits Algorithm updates Use of operational MIRS sounding products Replaces Goldberg/Barnett statistical retrievals Will eliminate duplication of efforts in generating retrievals Current Cal/Val project at CIRA has being evaluated impacts of this change Goal is that increased resolution of ATMS (vs. AMSU) will provide more accurate estimates of TC intensity and structure Adding another source of polar orbiting satellite microwave-based TC intensity and structure estimates will complement the existing AMSU-based products to improve the overall latency and availability of the NESDIS microwave-based TC products suite. Preliminary Design Review: NPP Microwave-based TC Products

algorithm overview: preliminary validation plan Compare ATMS-derived intensity estimates and wind radii to AMSU-derived products Hurricane flight reconnaissance intensities and wind radii (ground truth) ATMS-based products expected to exceed current AMSU-based products Improved horizontal resolution Preliminary Design Review: NPP Microwave-based TC Products

Software Architecture & Interfaces Presented by Andrea Schumacher (STAR/CIRA) Preliminary Design Review: NPP Microwave-based TC Products

Preliminary Design Review: NPP Microwave-based TC Products System design Languages and tools Fortran 90 (pgf90, ifort or g95 compilers) Bash, Perl, and Python (version 2.7 or higher) shell scripts Gnuplot GrADS HTML Java /JavaScripts Source of codes CIRA/CSU (most written in-house following the SPSRB F90 guidelines) Preliminary Design Review: NPP Microwave-based TC Products

Preliminary Design Review: NPP Microwave-based TC Products SYSTEM DESIGN (CONT…) Directory Structure Use standard OPS directory structure /bin Contains compiled executables /data Contains input and output data files /logs Contains all log files /scripts Contains all script files /setup Contains configuration and setup files /src Contains all source code Preliminary Design Review: NPP Microwave-based TC Products

Preliminary Design Review: NPP Microwave-based TC Products SYSTEM DESIGN (CONT…) Input data and formats NCEP GFS analysis (packed ASCII, 1-degree grid) MIRS ATMS vertical temperature and moisture profiles (netCDF4) TC positions and intensities (text a-deck and b-deck files) Ancillary data and auxiliary files Statistical algorithm coefficient data files (text) Output files and format TC intensity and radii (text) Balanced wind (netCDf4) Run schedules New runs every 6 hours at synoptic times Preliminary Design Review: NPP Microwave-based TC Products

IT System Architecture, Data Flow, Security DDS Dissemination NHC Input Data Includes: MIRS ATMS temperature profile retrievals, ATCF TC positions, GFS analyses Internet OSPO Online QC Monitoring ESPC IT Environment CLASS/NCDC Archive TC “CIRA Linux Server” NTCP “Production ” “System Test” NCEP GFS NPP ATMS TC Products NTCP “Development” ATMS TC products MiRS ATMS SND Products DAP delivery NDE “DDS” Preliminary Design Review: NPP Microwave-based TC Products 39 39

NTCP Software Flow Diagram MIRS ATMS sounding (netCDF) GFS Analysis (grib) Python script Perl script + Fortran Sounding Retrieval (ASCII) ATCF TC Positions (text) GFS Analysis (PACK ASCII) Vertical and Horizontal Integrations (Fortran) Statistical Analysis (Fortran) Quality Control (Fortran) Bias Correction (Fortran) Outputs (ASCII) 2D azimuthal mean radial/hgt cross sections (.RZA) Balanced 2-D wind field (.XYA) Outputs (ASCII) TC Intensity and wind radii estimates (.FIX) ATCF fix data (.AFX) Preliminary Design Review: NPP Microwave-based TC Products 40 40

Preliminary Design Review: NPP Microwave-based TC Products Input / output files File name Format Description Size (max) G000{YY}_X/Y{MMDD}_PACK.DAT Packed ASCII, input 1-degree GFS analysis data file 9 MB {swath/satellite ID}.nc NetCDF4, input MIRS retrieval data file 5 MB aal{storm number}{YYYY}.dat ASCII text, input Official positions and intensities of current active global TCs 10 MB per active TC {stormid}_{date}_{satellite}.FIX ASCII text, main output Swath used, intensity and wind radii estimates 1.4 kB {stormid}_{date}_{satellite}.AFX ATCF fix file (i.e., intensity and wind radii estimate in ATCF format) < 1 kB {stormid}_{date}_{satellite}.RZA ASCII text, intermediate/output Azimuthal mean radial/height cross sections (p, ρ, T, gradient wind) 20 MB {stormid}_{date}_{satellite}.XYA 2-D balance U, V, T, and Z data 3.0 MB {stormid}_{date}_{satellite}.STA ASCII text, intermediate Analysis statistics 1.1 kB {stormid}_{date}_{satellite}.ret MIRS retrieval data rewritten in simple ASCII format {stormid}_{date}_{satellite}.LOC Swath lat/lon data 200 kB Preliminary Design Review: NPP Microwave-based TC Products Total < 1 Gb

Preliminary Design Review: NPP Microwave-based TC Products Quality Assurance Presented by Limin Zhao NOAA/NESDIS/OSPO Preliminary Design Review: NPP Microwave-based TC Products

Preliminary Design Review: NPP Microwave-based TC Products QA Review QA Purpose Problem Prevention Early Error Detection Risk Identification and Mitigation QA Activities SPSRB Processes Configuration Management Project Monitoring System Monitoring Product Performance Monitoring Preliminary Design Review: NPP Microwave-based TC Products

Preliminary Design Review: NPP Microwave-based TC Products SPSRB Process Review Processes PDR CDR Software Code Review Operational System Readiness Review Documentations Algorithm Theoretical Basis End User Manual Systems Maintenance Manual Archive Data Submission Agreement ESPC Monitoring Procedures Documentation/Standards OSPO Code Standards SPSRB Documentation Templates Preliminary Design Review: NPP Microwave-based TC Products

Configuration Management CM Tool (CVS) CM Stakeholders Dorothy Brown CM Lead Zhaohui Cheng QA Lead Joseph McIlvenny CM Administrator Ed Ladd, Russ Lancaster CM installation NTCP Team Developers CM Users The ESPC CM Procedures will be enforced Preliminary Design Review: NPP Microwave-based TC Products

Preliminary Design Review: NPP Microwave-based TC Products Project Monitoring Regular Team meeting to discuss the project status Identify and track the issues and risks throughout the lifecycle Provides managements with objective feedback regarding process compliance to approved plans, procedures, standards, and analyses Preliminary Design Review: NPP Microwave-based TC Products

QA – PRODUCT MONITORING Error tracking Intensity & wind radii estimates will be added to ATCF system Can be routinely compared with similar estimates from other platforms Can be done by STAR/RAMMB, in coordination with SAB Real-time monitoring/feedback obtained from NHC Nonlinear balance winds are input to the Multi-Platform Tropical Cyclone Surface Wind Analysis system Operational at NESDIS Includes automated input quality control that can be leveraged to routinely monitor the nonlinear balance winds Near-real-time monitoring 24 x monitoring by ESPC help desk Monitor product log files each run cycle Preliminary Design Review: NPP Microwave-based TC Products

QA - science monitoring Annual verification will be performed by STAR / CIRA CIRA will work with NHC to perform seasonal verifications based on best track wind intensities and aircraft measured pressures Verification statistics computed as compared to ATCF intensity and wind radii diagnoses Annual updates will be performed by STAR / CIRA Bug fixes, updates to algorithm coefficients as needed STAR / CIRA to provide near-real-time science support E.g., answering algorithm-related questions, trouble-shooting As needed during normal business hours (M-F, 8-5pm MST) Preliminary Design Review: NPP Microwave-based TC Products

Preliminary Design Review: NPP Microwave-based TC Products QA Summary QA activities will be performed throughout the software development life cycle. Code development and documentation will follow the guidelines/standards of SPSRB. The QA process assures faster and more efficient research to operation transitions. Preliminary Design Review: NPP Microwave-based TC Products

Risks and Actions NOAA/NESDIS/STAR Presented by Mark Demaria Preliminary Design Review: NPP Microwave-based TC Products

Preliminary Design Review: NPP Microwave-based TC Products Risks and Actions PDR Risk 1: MiRS retrieval limitations may cause product degradations Risk Assessment: Preliminary validation of AMSU MiRS data show temperature/moisture retrievals don’t always converge near storm center Impact: If too many soundings are not available near the storm center, product accuracy will be degraded Likelihood: It is likely that some ATMS MiRS retrievals will be missing near the storm center. The impact on the product is unknown. Based on AMSU results, the most likely outcome is that the MiRS retrievals will provide tropical cyclone products of comparable accuracy to the current version with statistical retrievals. Mitigation: Work with MIRS development team to improve MiRS retrieval in cloudy regions if necessary Status: Open Preliminary Design Review: NPP Microwave-based TC Products

Summary and Conclusions Presented by Mark Demaria NOAA/NESDIS/STAR Preliminary Design Review: NPP Microwave-based TC Products

Review Objectives Have Been Addressed Project Plan has been reviewed Operations concept has been reviewed Project requirements have been reviewed Algorithm theoretical basis has been reviewed Software system architecture and interfaces have been reviewed Plans for quality assurance have been reviewed Open Risks and Actions have been reviewed Preliminary Design Review: NPP Microwave-based TC Products

Preliminary Design Review: NPP Microwave-based TC Products Next Steps Code Development Phase Code development Test data generation Unit tests Prepare documentation Critical Design Review (Dec, 2012) Preliminary Design Review: NPP Microwave-based TC Products

Preliminary Design Review: NPP Microwave-based TC Products Open Discussion The review is now open for free discussion Preliminary Design Review: NPP Microwave-based TC Products