GFE in RFCs Tom LeFebvre ESRL/Global Systems Division

30 January 2007 Overview History History GFE Components GFE Components Strengths and Weaknesses Strengths and Weaknesses Performance Issues Performance Issues Tools Tools Climatology Climatology Inter-site Coordination Inter-site Coordination

30 January 2007 GFE History Project started in 1992 Project started in 1992 Worked closely with NWS field forecasters Worked closely with NWS field forecasters Rapid Prototype Project started in 1999 Rapid Prototype Project started in 1999 Prototype software delivered to the field Prototype software delivered to the field Feedback from users New Requirements/ Implementation Deploy Software

30 January 2007 GFE Components (GFESuite) GFE primarily a grid editor GFE primarily a grid editor GFE software also generates products GFE software also generates products NWS Legacy Text Products NWS Legacy Text Products Graphics Graphics Images (PNG) Images (PNG) Digital (netCDF, AsciiGrid) Digital (netCDF, AsciiGrid)

30 January 2007 Other GFESuite Components ifpInit – Converts model 3-D cubes into sensible surface weather elements ifpInit – Converts model 3-D cubes into sensible surface weather elements Inter-site Coordination (ISC) - Moves gridded data between sites to enhance collaboration Inter-site Coordination (ISC) - Moves gridded data between sites to enhance collaboration Graphical Hazards Generator (GHG) – Makes graphical versions of long-fused watches/warnings and formats text products with VTEC codes Graphical Hazards Generator (GHG) – Makes graphical versions of long-fused watches/warnings and formats text products with VTEC codes

30 January 2007 GFESuite - Strengths Highly configurable / flexible Highly configurable / flexible Weather elements Weather elements Domain / Projection / Resolution Domain / Projection / Resolution User Interface User Interface Products Products Fully Programmable Framework (SmartTools) Fully Programmable Framework (SmartTools) Interpretive language (Python) Interpretive language (Python) Easy to learn and use, very powerful Easy to learn and use, very powerful Over 500 tools in the Smart Tool Repository Over 500 tools in the Smart Tool Repository

30 January 2007 GFESuite - Weaknesses Architected more than 15 years ago Architected more than 15 years ago Core is C++ based Core is C++ based Compiled language, difficult to maintain Compiled language, difficult to maintain Performs inefficiently for some operations (e.g., ISC) Performs inefficiently for some operations (e.g., ISC) ifpServer is a bottleneck ifpServer is a bottleneck ifpServer GFE GFE GFE ifpInit ifpInit ISC Text Formatter

30 January 2007 Performance Issues GFE was originally designed for a 35 x 35 grid (20km) 1225 gridpoints GFE was originally designed for a 35 x 35 grid (20km) 1225 gridpoints WFOs now running at ~300x300 domain (2.5km) at most offices ~90,000 gridpoints WFOs now running at ~300x300 domain (2.5km) at most offices ~90,000 gridpoints RFCs want ~500x500 grid (2.5km) 250,000 grid points! RFCs want ~500x500 grid (2.5km) 250,000 grid points!

30 January 2007 AWIPS ifpInit Statistics

30 January 2007 Performance (cont.) Bottom Line: 250K grid won’t work on AWIPS hardware Bottom Line: 250K grid won’t work on AWIPS hardware Not enough memory (4Gb) Not enough memory (4Gb) CPUs not fast enough CPUs not fast enough

30 January 2007 Performance (cont.) However, better hardware exists! However, better hardware exists! 64 dual core, dual xeon (4 processors) 64 dual core, dual xeon (4 processors) 8 Gb memory 8 Gb memory Note: 64-bit 250K test used all 8GB and required 250Mb of swap space

30 January 2007 Performance (cont.) Recommendation: Recommendation: Use 64-bit processors (minimum of 4) Use 64-bit processors (minimum of 4) 12 Gb memory 12 Gb memory Option: Faster CPUs Option: Faster CPUs Limited testing demonstrated this hardware will likely support an RFC domain of ~250K grid points Limited testing demonstrated this hardware will likely support an RFC domain of ~250K grid points

30 January 2007 Performance (cont.) Text Formatters also exhibited performance problems Text Formatters also exhibited performance problems Hundreds of basins sampled Hundreds of basins sampled ~40 grids (6 hourly for 10 days) ~40 grids (6 hourly for 10 days) But…formatter logic is simple (just too many basins) But…formatter logic is simple (just too many basins) TextFormatter C++Samplers GriddedDatabase Data structures must be copied for every sample

30 January 2007 Performance (cont.) Possible Formatter Solution: Possible Formatter Solution: Rewrite formatters in Python as SmartTools Rewrite formatters in Python as SmartTools Avoid C++ sampling Avoid C++ sampling RFC products generally simple formats and don’t require text formatter infrastructure RFC products generally simple formats and don’t require text formatter infrastructure Potentially much better performance Potentially much better performance

30 January 2007 Tools GFE SmartTools perform reasonably well even on a 250K grid point domain GFE SmartTools perform reasonably well even on a 250K grid point domain AWIPS hardware ~ 2 seconds per grid AWIPS hardware ~ 2 seconds per grid 64-bit hardware < 1 second per grid 64-bit hardware < 1 second per grid

30 January 2007 Precipitation Climatology GFE contains monthly climatological data sets (CONUS only) GFE contains monthly climatological data sets (CONUS only) PRISM and NCDC PRISM and NCDC Precipitation Precipitation Maximum Temperature Maximum Temperature Minimum Temperature Minimum Temperature Spatial resolution: 4km for both PRISM and NCDC Spatial resolution: 4km for both PRISM and NCDC Baseline tools calculate daily values using spline technique. Baseline tools calculate daily values using spline technique.

30 January 2007 Precipitation Climatology PRISM Precip.

30 January 2007 Inter-site Coordination ISC will work at RFCs ISC will work at RFCs Small amount of configuration at both RFCs and WFOs Small amount of configuration at both RFCs and WFOs Problem: RFC domains overlap with WFO domains Problem: RFC domains overlap with WFO domains Solution: SmartTools that copy ISC grids into RFC forecast grids Solution: SmartTools that copy ISC grids into RFC forecast grids

30 January 2007 Conclusions RFCs present challenges for GFESuite software RFCs present challenges for GFESuite software Most, if not all, problems can be solved with new hardware and new software Most, if not all, problems can be solved with new hardware and new software Amount of software required is relatively small Amount of software required is relatively small Some configuration Some configuration A few tools to: A few tools to: Incorporate climatology in tools Incorporate climatology in tools Assimilate ISC data into the forecast Assimilate ISC data into the forecast Implement text formatters as SmartTools Implement text formatters as SmartTools