Lessons Learned from the Deployment and Integration of a Microwave Sounder Based Tropical Cyclone Intensity and Surface Wind Estimation Algorithm into.

Slides:

Advertisements

Similar presentations

Improvements to Statistical Intensity Forecasts John A. Knaff, NOAA/NESDIS/STAR, Fort Collins, Colorado, Mark DeMaria, NOAA/NESDIS/STAR, Fort Collins,

Advertisements

A Blended, Multi-Platform Tropical Cyclone Rapid Intensification Index

Status of Operational Implementation of MIRS V2.0 at OSDPD Limin Zhao, Aiwu Li and Jiang Zhao April 18, 2008.

Andrea Schumacher, CIRA/CSU Mark DeMaria, NOAA/NESDIS/StAR Dan Brown and Ed Rappaport, NHC.

Future Plans  Refine Machine Learning:  Investigate optimal pressure level to use as input  Investigate use of neural network  Add additional input.

Future Plans  Refine Machine Learning:  Investigate optimal pressure level to use as input  Investigate use of neural network  Add additional input.

Data Assimilation Cycling with GSI and AIRS Sixth Meeting of the Science Advisory Committee 28 February – 1 March, 2012 National Space Science and Technology.

Further Development of a Statistical Ensemble for Tropical Cyclone Intensity Prediction Kate D. Musgrave 1 Mark DeMaria 2 Brian D. McNoldy 3 Yi Jin 4 Michael.

Microwave Imagery and Tropical Cyclones Satellite remote sensing important resource for monitoring TCs, especially in data sparse regions Passive microwave.

Himawari-8 Project Plans and Status. Background 2 NESDIS is implementing a capability to ingest Advanced Himawari Imager data from JMA, process and generate.

Diagnosing Tropical Cyclone Structure Presented by John Knaff with input and efforts from A. Schumacher, R. DeMaria, G. Chirokova, C. Slocum.

1 Short term plan to assist 2011 HWRF implementation HFIP Stream 1 Regional Hurricane Model Diagnostics Planning Meeting, 11/08/2010.

Suomi National Polar-orbiting Partnership (SNPP) Data Access NOAA Satellite Conference April 8-12, 2013 Kevin Berberich NESDIS/OSD NDE Project Photographs.

Formation of a tropical cyclone eye is often associated with intensification [1]. Currently, determination of eye formation from satellite imagery is generally.

V14.1 Jun SPSRB Project Plan on “MIRS-Based Tropical Cyclone Intensity and Structure Estimation” Project Lead: John Knaff Backup Project Lead: Liqun.

Abstract The automated multi-platform software nightly build system is a major component in the ATLAS collaborative software organization, validation and.

1. Outline RAMMB/CIRA Overview RAMMB/CIRA real-time product development & deployment environment GOES-R Proving Ground AWIPS II Overview AWIPS II Configuration.

PRELIMINARY DESIGN REVIEW NPP MICROWAVE SOUNDER-BASED TROPICAL CYCLONE PRODUCTS (NTCP) OCTOBER 16, 2012 PRESENTED BY: ANDREA SCHUMACHER1, MARK DEMARIA2,

Lessons Learned from the Deployment and Integration of Microwave Sounder Based Tropical Cyclone Intensity and Surface Wind Estimation Algorithms into NOAA.

Advanced Applications of the Monte Carlo Wind Probability Model: A Year 1 Joint Hurricane Testbed Project Update Mark DeMaria 1, Stan Kidder 2, Robert.

Advanced Applications of the Monte Carlo Wind Probability Model: A Year 2 Joint Hurricane Testbed Project Update Mark DeMaria 1, Robert DeMaria 2, Andrea.

Development of a Probabilistic Tropical Cyclone Genesis Prediction Scheme Jason Dunion1, John Kaplan2, Andrea Schumacher3, Joshua Cossuth4, & Mark DeMaria5.

A. Schumacher, CIRA/Colorado State University NHC Points of Contact: M. DeMaria, D. Brown, M. Brennan, R. Berg, C. Ogden, C. Mattocks, and C. Landsea Joint.

Applications of ATMS/CrIS to Tropical Cyclone Analysis and Forecasting Mark DeMaria and John A. Knaff NOAA/NESDIS/STAR Fort Collins, CO Andrea Schumacher,

Team Lead: Mark DeMaria NOAA/NESDIS/STAR Fort Collins, CO

Improvements in Deterministic and Probabilistic Tropical Cyclone Wind Predictions: A Joint Hurricane Testbed Project Update Mark DeMaria and Ray Zehr NOAA/NESDIS/ORA,

V Jan SPSRB Project Plan “NPP Microwave Sounder-based Tropical Cyclone Products (NTCP)” IPT Lead: Mark DeMaria, STAR/NESDIS IPT Backup.

Lessons Learned from the Deployment and Integration of a Microwave Sounder Based Tropical Cyclone Intensity and Surface Wind Estimation Algorithm into.

Center for Satellite Applications and Research (STAR) Review 09 – 11 March 2010 Image: MODIS Land Group, NASA GSFC March 2000 Center for Satellite Applications.

New and Updated Operational Tropical Cyclone Wind Products John A. Knaff – NESDIS/StAR - RAMMB, Fort Collins, CO Alison Krautkramer – NCEP/TPC - NHC, Miami,

Overview: RAMMB Tropical Cyclone (TC) Research and Applications Presented by John Knaff.

Development of a Real-Time Automated Tropical Cyclone Surface Wind Analysis: Development of a Real-Time Automated Tropical Cyclone Surface Wind Analysis:

Improvements in Deterministic and Probabilistic Tropical Cyclone Surface Wind Predictions Joint Hurricane Testbed Project Status Report Mark DeMaria NOAA/NESDIS/ORA,

Version 1.2, Feb. 2013Modular structure 1/22WW Winter School 2013 Modular structure of WAVEWATCH III and general features Arun Chawla The WAVEWATCH III.

April nd IBTrACS Workshop 1 Operational Procedures How can we build consistent, homogeneous, well- documented climate quality data?

An Improved Wind Probability Program: A Year 2 Joint Hurricane Testbed Project Update Mark DeMaria and John Knaff, NOAA/NESDIS, Fort Collins, CO Stan Kidder,

A Comparison of Two Microwave Retrieval Schemes in the Vicinity of Tropical Storms Jack Dostalek Cooperative Institute for Research in the Atmosphere,

Hurricane Intensity Estimation from GOES-R Hyperspectral Environmental Suite Eye Sounding Fourth GOES-R Users’ Conference Mark DeMaria NESDIS/ORA-STAR,

Center for Satellite Applications and Research (STAR) Review 09 – 11 March 2010 Image: MODIS Land Group, NASA GSFC March 2000 Improving Hurricane Intensity.

Ocean Surface Winds Research Summary: Meteorological Applications Mark DeMaria, NOAA/NESDIS, Fort Collins, CO Ocean Surface Winds Workshop NCEP/Tropical.

Introduction More delays for Flavio. Visa appointment in Mexico for Jan 9 th Likely start date: Second half of Jan. MIRS 3 rd Release delivered on.

COPC Meeting, May 27 – 28 th 2015 NDE Operational Status and PDA Update Prepared by: Chris Sisko, NDE Operational Status.

NHC/JHT Products in ATCF Buck Sampson (NRL, Monterey) and Ann Schrader (SAIC, Monterey) IHC 2007 Other Contributors: Chris Sisko, James Franklin, James.

A. FY12-13 GIMPAP Project Proposal Title Page Title: Improvements to the Advanced Dvorak Technique Status: New – but continuing work from GIMPAP FY07-09.

The Joint Polar Satellite System (JPSS) is the next generation polar-orbiting operational environmental satellite system. The first satellite in the JPSS.

Development of Probabilistic Forecast Guidance at CIRA Andrea Schumacher (CIRA) Mark DeMaria and John Knaff (NOAA/NESDIS/ORA) Workshop on AWIPS Tools for.

March 2004 At A Glance autoProducts is an automated flight dynamics product generation system. It provides a mission flight operations team with the capability.

NCEP ESMF GFS Global Spectral Forecast Model Weiyu Yang, Mike Young and Joe Sela ESMF Community Meeting MIT, Cambridge, MA July 21, 2005.

Improvements to Statistical Forecast Models and the Satellite Proving Ground for 2013 Mark DeMaria, John Knaff, NOAA/NESDIS/STAR John Kaplan, Jason Dunion,

Can Dvorak Intensity Estimates be Calibrated? John A. Knaff NOAA/NESDIS Fort Collins, CO.

Tropical Cyclone Rapid Intensity Change Forecasting Using Lightning Data during the 2010 GOES-R Proving Ground at the National Hurricane Center Mark DeMaria.

G.Govi CERN/IT-DB 1 September 26, 2003 POOL Integration, Testing and Release Procedure Integration  Packages structure  External dependencies  Configuration.

Center for Satellite Applications and Research (STAR) Review 09 – 11 March 2010 Applications of AMSU-Based Hydrological Products for Climate Studies Ralph.

Improved Statistical Intensity Forecast Models: A Joint Hurricane Testbed Year 2 Project Update Mark DeMaria, NOAA/NESDIS, Fort Collins, CO John A. Knaff,

1 Recommendations from the 2 nd GOES-R Users’ Conference: Jim Gurka Tim Schmit NOAA/ NESDIS Dick Reynolds Short and Associates.

Evolution of the JPSS Ground Project Calibration and Validation System Patrick Purcell, Gyanesh Chander and Peyush Jain JPSS Ground Project NASA, GSFC.

TC Projects Joint Hurricane Testbed, Surface winds GOES-R, TC structure – TC Size TPW & TC size (Jack Dostalek) IR climatology – RMW/wind profile Proving.

Development and Implementation of NHC/JHT Products in ATCF Charles R. Sampson NRL (PI) Contributors: Ann Schrader, Mark DeMaria, John Knaff, Chris Sisko,

Overview of CIRA and NESDIS Global TC Services Presented by John Knaff NOAA/NESDIS Regional and Mesoscale Meteorology Branch Fort Collins, CO USA For The.

The National Weather Service Goes Geospatial – Serving Weather Data on the Web Ken Waters Regional Scientist National Weather Service Pacific Region HQ.

Satellite + Aircraft Tropical Cyclone Surface Wind Analysis Joint Hurricane Testbed.

Transition of Science Algorithms into Enterprise Product Generation Operations 2016 AMS Annual Meeting Dylan Powell, Ph.D. Lockheed Martin ESPDS Science.

Space Science and Engineering Center University of Wisconsin-Madison Space Science and Engineering Center University of Wisconsin-Madison 1 NPP Atmosphere.

Preparing for GOES-R and JPSS: New Tropical Cyclone Tools Based on 30 Years of Continuous GOES IR Imagery John A. Knaff1, Robert T. DeMaria2, Scott P.

Presenters: John Knaff and Liqun Ma April 15, 2015

Mark DeMaria and John A. Knaff - NOAA/NESDIS/RAMMB, Fort Collins, CO

Cal/Val Activities at CIRA

Objective Methods for Tropical Cyclone Center

CIOSS Executive Board Al Powell Acting Director 30 August 2005

Validation of CIRA Tropical Cyclone Algorithms

Presentation transcript:

Lessons Learned from the Deployment and Integration of a Microwave Sounder Based Tropical Cyclone Intensity and Surface Wind Estimation Algorithm into NOAA NESDIS Satellite Product Operations. Scott P. Longmore 1 Andrea Schumacher 1, Jack Dostalek 1, Robert DeMaria 1, Galina Chirokova 1, John Knaff 2, Mark DeMaria 3, Dylan Powell 4, Angela Sigmund 4,Wei Yu 4 1 Colorado State University (CSU)/ Cooperative Institute for Research in the Atmosphere (CIRA)/ Regional and Mesoscale Meteorology Branch (RAMMB), Fort Collins, Colorado 2 National Oceanographic and Atmospheric Administration (NOAA)/ National Environmental Satellite, Data, and Information Service (NESDIS)/ Satellite Applications and Research (StAR), Fort Collins, Colorado 3 NOAA/National Weather Service (NWS) /National Hurricane Center (NHC)/ Technology and Science Branch (TSB), Miami, Florida 4 NOAA/NESDIS/Office of Systems Development (OSD)/ Satellite and Ground Systems Program (SGSP)/ System Engineering and Integration Division (SEID), Lockheed Martin, Silver Spring, Maryland

7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 1 Outline CIRA Microwave Sounder-based Tropical Cyclone (TC) Research Algorithms Operational Requirements and Real-Time frameworks Migration from Research to Operations Component and System Development Cycle Testing, Debugging and Integration at CIRA and NSOF Lessons Learned and Conclusions Future Development

CIRA Microwave Sounder TC Algorithms 7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 2 Estimates tropical cyclone intensity and surface wind radii for 34-, 50-, and 64-kt wind thresholds. Intensity and structure calculated from: Microwave temperature, moisture soundings from polar orbiting satellites Multiple linear regression statistics Hydrostatic and gradient balanced relationship Demuth et. al. (2004)

7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 3 CIRA Microwave Sounder TC Algorithms Purpose: To provide wind/intensity guidance to NWS National Hurricane Center, etc for winds/intensity between aircraft reconnaissance Initial version (AMSU_TC) utilizes Advance Microwave Sounder Unit (AMSU) soundings located on the polar orbiting satellites NOAA18, NOAA19, METOPA CIRA AMSU_TC system has been under development since 2002, it has been calibrated and tested extensively Latest version (NPP_TC) utilizes Advanced Technology Microwave Sounder (ATMS) located on the Suomi National Polar-orbiting Operational Environmental Satellite System (NPOESS) Preparatory Project (S-NPP or NPP) launched Oct, 28, 2011 CIRA NPP_TC has been recently developed from the AMSU_TC, and tested on limited NPP/ATMS test data.

7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 4 NPP/AMSU TC Algorithms Input data: Automated Tropical Cyclone Forecasts (ATCF/adeck) Global Forecast System model (boundary conditions) AMSU/ATMS (microwave sounding data) Products: ATCF format ATMS/AMSU TC fix text file (afx) t, u, v, z (p, lat, lon), surface [t, u, v, p] (lat, lon) text file (xya) Mean Azimuthal T, P, ρ, V gradient (range, Z) text file (rza) NPP_TC: V, z plots for 850,700,600,500,400,300,200,250 hPa References: Bessho, K., M. DeMaria, and J. A. Knaff, 2006: Tropical Cyclone Wind Retrievals from the Advanced Microwave Sounder Unit (AMSU): Application to Surface Wind Analysis. J. of Applied Meteorology. 45:3. Demuth, Julie L., M. DeMaria, and J. A. Knaff. 2006: Improvement of Advanced Microwave Sounding Unit tropical cyclone intensity and size estimation algorithms. Journal of Applied Meteorology & Climatology Demuth, Julie L., M. DeMaria, J.A. Knaff, & T. H. V. Haar, 2004: Evaluation of Advanced Microwave Sounding Unit tropical-cyclone intensity and size estimation algorithms. Journal of Applied Meteorology 43.2.

NPP/AMSU TC Algorithms Products 7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 5 AL, 18, , 30, ATMS, IPR,, 2822N, 7681W,, 1, 58, 2, 980, 2, MEAS, 34, NEQ, 211, 263, 192, 159,,,,, 2, 53,, L, NSOF, OPS,,,,,,, 980,, SNPP01, 34, NEQ, 211, 263, 192, 159,,,,,,,,,,,,, 2, storm center extrapolated from t=-12 and t=0 adeck AL, 18, , 30, ATMS, R,, 2822N, 7681W,, 1, 58, 2, 980, 2, MEAS, 50, NEQ, 87, 100, 81, 71,,,,, 2, 53,, L, NSOF, OPS,,,,,,, 980,, SNPP01, 50, NEQ, 87, 100, 81, 71,,,,,,,,,,,,, 2, storm center extrapolated from t=-12 and t=0 adeck AL, 18, , 30, ATMS, R,, 2822N, 7681W,, 1, 58, 2, 980, 2, MEAS, 64, NEQ, 0, 55, 0, 0,,,,, 2, 53,, L, NSOF, OPS,,,,,,, 980,, SNPP01, 64, NEQ, 0, 55, 0, 0,,,,,,,,,,,,, 2, storm center extrapolated from t=-12 and t=0 adeck

Storm Loop Satellite Loop NPP/AMSU TC Systems 7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 6 oparet - determines wind fields from MIRS satellite and GFS model shorttermrack – determines latest, 12hr previous TC track from ATCF satcenter - determines satellite scanline time coordinates closest to TC afdeck – determines latest TC time and coordinates grib2pack – converts GFS grib to internal CIRA pack format pool_query.py - retrieves MiRS data near storm track mirs_tc_input/validatePCF.py – retrieves/verifies ATCF, GFS, and MiRS data convert2netcdf.py – converts oparet output to netCDF main_read_plot_xya.py – plots oparet temp and wind fields Master Script – reads/sets environment, calls sub-programs, handles logging, errors and exceptions data_pool.py – subsets the required MiRS variables and sub-domain

NOAA Satellite Operations Facility (NSOF) 7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 7 Source: Standards for Science Algorithm Delivery and Integration using Delivered Algorithm Packages v1.4

NDE Data Handling System (DHS) 7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 8 Source: Standards for Science Algorithm Delivery and Integration using Delivered Algorithm Packages v1.4

NSOF SPSRB SA Standards 7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 9 Satellite Products and Services Review Board (SPSRB) Common Standards Working Group Fortran 90/95 Requirement Tiers: Standard - mandatory Guideline - encouraged Recommendation - optional Encapsulation i.e. Modules (guideline) Robustness Implicit None (standard) All variables declared and initialized (standard) Discourages Elements: Common/data statements to modules (standard) Goto statements to exit/cycle statements (standard) Headers/Comments (standard/guideline)

NDE SA Framework Conventions 7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 10 Process Control File (key=value) Working directory Job coverage start/end times Input data files i.e. ATCF, GFS, ATMS Other dynamic environment variables Process Status File List of NOAA Unique Product (NUP) files i.e. afx, nc, png NUP naming convention CLASS archive network Common Data Format (netCDF4) Climate and Forecast (CF) metadata convention CF standard naming (variables/attributes) Required NUP global, variable attributes Process Log File NDE log format: | | message NDE error, warning designators for diagnostics dump Return/Error Codes, unique for each error/warning condition

NDE NPP/NSOF AMSU TC Implementation 7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 11 NDE NPP TC DHS provides PCF, input data, invokes NPP_TC NSOF AMSU TC Supplemental to NDE NPP TC Purpose: Utilize all microwave sounder data for TC guidance Compare ATMS and AMSU, when applicable Real-time implementation different from NDE NPP TC Operational on stand alone server separate from NDE DHS ATCF, GFS, AMSU provided on NWS DDS data directories Determines and links latest input data vs reading NDE PCF Processes multiple satellites, NOAA18, NOAA18, METOPA Products: ATCF format fixes, XYA,RZA netCDF (no plots) Products distributed via DDS, archived on NOAA Comprehensive Large Array-data Stewardship System (CLASS) NPP/AMSU TC invoked every 2hrs

7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 12 CIRA Research to NSOF/NDE Operations NPP_TC CIRA NPP_TC System CIRA NPP_TC System SPSRB Standards Conversion CIRA Internal Testing NDE SA Conventions Upgrades CIRA Internal Testing NDE NPP_TC Development Server Installation and Testing NDE NPP_TC Test Server Installation and Testing NDE NPP_TC Operations AMSU_TC CIRA AMSU_TC System CIRA AMSU_TC System SPSRB Standards Conversion CIRA Internal Testing NSOF AMSU_TC Installation and Testing NSOF AMSU_TC Operations Current Status: NDE NPP_TC Test Server Testing NSOF Testing

CIRA NPP/AMSU TC Development Team 7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 13 Andrea Schumacher - PI (NSOF NDE NPP TC), scientist Proposal, development, calibration, validation, documentation oparet, grib2pack, shorttermtrack components Jack Dostalek - PI (NSOF AMSU TC), scientist Proposal, development, calibration, validation, documentation operet, afdeck, satcenter, master/sub-script components Scott Longmore, software engineer/scientist Development, Integration, Testing, Validation, Documentation Software version control, component, system and NSOF/NDE integration Component, system testing, case generation, testing, and validation F77 to F90 oparet conversion, mirs_tc_input, validatePCF, convert2netCDF, master/sub-script components Robert Demaria, software engineer Development, Validation data_pool, pool_query components Galina Chirokova, scientist Development, Validation main_read_plot_xya component

Integration Development Component Development NPP/AMSU TC Development and Integration Cycle 7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 14 Component Subversion Repository Component Staging Directory Hierarchy AMSU/NPP TC Subversion Source Directory Hierarchy AMSU/NPP TC Subversion Repository Component Source Directory Component Source Directory NSOF/NDE AMSU/NPP TC Development Server NSOF/NDE AMSU/NPP TC Development Server NSOF/NDE AMSU/NPP TC Test Server NSOF/NDE AMSU/NPP TC Test Server CheckOu t Tar/FTP Cop y Checkout CheckIn Tar/FTP

NPP/AMSU TC Testing 7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 15 CIRA Component testing used TC Sandy case (AMSU/ATMS) Each component iteration was compared to verified TC Sandy component output AMSU_TC system tested on ~3000 cases (Aug-Oct 2012/13) NPP_TC system tested on ~2000 cases (Aug-Oct 2012) Test run logs, sub-program output files checked for errors Tests products verified for select TC’s Set of test cases (13) built from verified TC’s NDE NPP_TC Tested within Development NDE on 13 test cases Migrated to Test NDE on 13 test cases, then real-time testing NSOF AMSU_TC Currently running in real-time on CIRA development server Migrated to Test/Operational NSOF standalone server

NPP/AMSU TC Debugging 7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 16 NPP_TC: Issues identified on NDE Development/Test servers replicated on CIRA Development server Component(s) identified and resolved in next Delivery Algorithm Package (DAP)/release Resolved issues: PCF, PSF, log formatting, conventions Product output naming Script/sub-program invocation and data path issues MiRS ATMS granule/data pool issue MiRS ATMS data type issue AMSU_TC: Issues resolved in NPP_TC, applied to AMSU_TC when applicable.

NPP/AMSU TC Debugging 7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 17 NPP_TC: Issues identified on NDE Development/Test servers replicated on CIRA Development server Component(s) identified and resolved in next Delivery Algorithm Package (DAP)/release Resolved issues: PCF, PSF, log formatting, conventions Product output naming Script/sub-program invocation and data path issues MiRS ATMS granule/data pool issue MiRS ATMS data type issue AMSU_TC: Issues resolved in NPP_TC, applied to AMSU_TC when applicable.

7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 18 NDE NPP TC System Releases 2014/03/06 v PCF validator, removes older adeck/gfs files, 2014/02/08 v netCDF bug fixes, minor updates - Error to Warning messages for sub-programs inside storm loop 2014/02/02 v netCDF converter for RZA files 2014/01/27 v netCDF converter for XYA files, MiRS file fix 2014/01/07: v ATCF (adeck), GFS grib file naming, product version in file names 2013/11/14: v Data test case inclusion 2013/11/08: v PCF formatting, required NDE and data input variable upgrades - Data, temporary sub-directory hierarchy handling - Static variables to master script, external and data variables to PCF - Date format/handling between NDE and master script - Executable/script location execution handling from working to operation directory - PSF output file location and naming is now absolute - LOG file location to top working directory 2013/10/31: v NDE framework upgrades: PCF, PSF, logging, exception handling - GFS grib to pack file converter 2013/06/25: v NDE initial version 2013/06/14: v Initial version

7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 19 NSOF AMSU TC System Releases 2014/03/03: v mirs_tc_input.py determines input files from data source directories - AMSU_TC_CRON.sh - determines run time, directory and calls AMSU_TC.sh - AMSU_TC.crontab - sample crontab for AMSU_TC 2014/02/10: v mirs_tc_input.py determines input files, time from working directory - AMSU_TC_ENV.sh - file added for environment variables - makefile flags/parameters moved to master makefile - XYA/RZA netCDF converter added - Updated AMSU_TC master and sub-scripts - Updated data_pool scripts 2012/11/27: v AMSU_TC Initial version

NPP/AMSU TC Lessons Learned 7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 20 Successes: CIRA/NSOF/NDE development and integration cycle for small development team and small number of components Larger system/team would require use of working and stable subversion software repositories Challenges: Unplanned development time and software required for migrating CIRA NPP_TC system to NSOF/NDE framework Implementing NDE framework standards, e.g. logging, time matching/manipulation challenging in bash control system Delays in MIRS ATMS data limited full testing of NPP system

NPP/AMSU TC Possible Development 7-11 April 2014 Third Annual Software Engineering Assembly Software Engineering Conference University Corporation for Atmospheric Research, Boulder, Colorado 21 NPP/AMSU systems migrated into one system Control system and data preprocessing converted to python Development of modular framework for NPOESS and other polar orbiting microwave sounder instruments Configuration built on JSON Refactoring of python input and netCDF converter