NOAA Response Tiger Team A NOAA Action Plan for UMAC 2015 Recommendations on Environmental Modeling 9 August 2016 NOAA Response Tiger Team Farida Adimi, Pat Burke, Jessie Carman, Mike Ek, Mike Farrar, Tim Schneider*, Vijay Tallapragada, Hendrik Tolman *Lead “What got you here, won’t get you there.” – Walt Childress, Executive Coach

Outline Process The Action Plan and the Traceability Matrix “Top 10” Issues

I. Process

Deliverables UMAC issued report to NCEP on December 7, 2015; it includes: 8 major findings 198 specific findings, recommendations, & observations A NOAA UMAC Action Plan (document) As of 9 August 2016: it is a pre-decisional draft Comprised of a document; a traceability matrix; and various supporting materials (plans, etc.)

Scope Complete, concise, clear, and actionable plan to address and/or implement the findings and recommendations in the UMAC report Includes a traceability matrix Identifies steps already taken Accomplishments, progress, and existing practices and policies It will not spell out specific solutions to particular challenges e.g. which regional modeling system to select, etc. Responds to specific organizational and technical recommendations Propose processes and practices Speaks to “organizational excellence”

Scope of the Action Plan, continued NWS/NCEP is at the “sharp end” Engagement across NOAA is critical: Engage: NWS; NOS; OAR; NESDIS Inform: OCIO; NMFS Initial focus primarily on the current NCEP Production Suite But with consideration of ‘environmental modeling’ more broadly

Goals A living document and process Outreach within the NOAA community To support the development of an integrated ‘strategic plan’ for modeling The Action Plan should be backed by cross-LO service-level agreements (either existing or new)

Timeline 7 March 2016: Coordination Call with UCACN and UMAC Chairs 23 February 2016: Kickoff Bill Lapenta provided Tim Schneider with charge to lead this effort 7 March 2016: Coordination Call with UCACN and UMAC Chairs Gary Lackman; Peter Nielly; Ricky Rood; Fred Carr April: Briefings for the AAs: Plans, expectations, timelines NWS: 8 Apr 2015 NOS: 15 Apr 2015 OAR: 18 Apr 2015 NESDIS TBD April 28: Briefing to full UMAC

Timeline, continued April – July: August September December Established an ad hoc NOAA group to engage across NOAA Line Offices Outreach and input Develop proposed Response August 9-11: Brief Preliminary findings to the UMAC Revise Action Plan September NOAA review and approval of Action Plan Begin developing a strategic plan for modeling December Deliver draft Strategic Plan to NOAA

Next Steps Post UMAC Meeting: Scope evolving Building a “community” Living process and plans Consider role of the ad hoc “NOAA Response Tiger Team”? Scope evolving Building a “community” Develop a Modeling Strategic Plan Develop a comprehensive, coordinated strategic plan and long term vision for modeling required to deliver a Weather Ready Nation 10-year vision for NCEP Production Suite: Where do we want to be 10 years from now… e.g. convection allowing global model with ensembles Broader NOAA: (NWS, OAR, NOS, NESDIS and NMFS) Use dialogue with UMAC at August meeting to inform Begin ASAP in August

NOAA UMAC Response Ad Hoc Tiger Team Tim Schneider Chair, Detail to NWS Vijay Tallapragada NWS POC Hendrik Tolman NWS Mike Farrar Mike Ek Jessie Carman OAR POC Pat Burke NOS POC Farida Adimi NWS, Project Manager Inform NESDIS NOAA UMAC Response ad hoc “Tiger Team.” The NWS, OAR and NOS have been directly involved. NESDIS has been briefly informed.  There has been no direct engagement with OCIO and NMFS yet. OCIO NMFS

II. Action Plan and the Traceability Matrix

Integrating Projects An organizing principle used in the Action Plan and the Traceability Matrix Unified Model Suite (UMS) NPS Requirements (REQ) Implementation Process Revision (IPR) Global Modeling (NGGPS) Hurricanes (HFIP) Meso Unification (MESO) Governance (GOV) Community Engagement and Coordination (CEC) Integrated Water (IW)

The “Top Ten” (1-5) Reduce the complexity of the NCEP Production Suite Rational evidence-driven approach towards decision-making and end-to-end modeling system development A unified, collaborative strategy for model development across NOAA Creation of a Chief Scientist position for Numerical Environmental and Weather Prediction Better leverage the capabilities of the external community Community Modeling (internal and external) Governance (internal and external)

The “Top Ten” (6-10) Continue to enhance HPC capabilities Develop a comprehensive and detailed vision document and strategic plan that maps out future development of national environmental prediction capabilities Execute strategic and implementation plans based on stakeholder requirements Integrated Water Future Considerations Note: 1-8 are from the UMAC 9-10 are from the NOAA Tiger Team

(1) Reduce the complexity of the NCEP Production Suite Top-level design of future production suite using five forecast ranges and nowcast range completed and vetted by users at December 2015 NPSR Top-level design of Unified Global Coupled Modeling systems similarly completed 5-10 year end state is clear Tentative global model transition work plans have been completed (present GFS, GEFS, CFS) Meso plans under development

Basic products Initial focus is on the weather side Starting with products: What forecast time ranges? Which reasonably imply: Run cadences Update cycle

Unified design (high level goal)

Meso-scale products (RRGS/WoFGS) 5-10y goal: Single-core ensemble 3km CAM products with ensemble hybrid DA. Hourly runs with 18h forecast Some runs extended to 30h forecast (FAA, small craft adv., etc) Some runs extended to 60h forecast (present NAM nest) Will replace/expand present HRRR, NAM-nest, HiRes Window. Driven by Global high-resolution ensemble which replaces / absorbs NAM parent, RAP, and SREF.

Meso-scale products (cont’d) 5-10y goal is well defined and generally agreed upon by all involved. Some key questions remain on how to get there: Risk vs. schedule? Do we operationalize an SSEO approach (multi-core ensembles) as Initial Operational Capability (as has been suggested by several stakeholders)? How does global-meso unification fit (‘re-tool’ meso teams once or twice)? Evidence to be considered Address present need for dual-core approach in SREF and SSEO Conflicting data: HRRR superior for short forecast ranges, but NAM-nest superior for longer forecast ranges Where is the evidence for suitability of single-core approach? Governance: need to get to a one-NOAA team approach

100% compute cost estimate for high-level production suite layout Cost of modeling Growing demands of mesoscale resources in the production suite: WoF not feasible as full-CONUS Needs to be nested and relocatable, on demand Linkages to hurricane approach 100% compute cost estimate for high-level production suite layout CGS (year) OGS (month) WGS (week) RRGS (day) WoFGS (hour) PFlop 0.5 0.8 12.5 23 230 Compute % (no WoFGS) 1.5 2.5 33 64 ---

(2) Rational evidence-driven approach towards decision-making and end-to-end modeling system development The implementation process for elements in the production suite is being re-designed with following key features Process starts of with stakeholder-input to target agreed-upon improvements Includes gated approach to selecting science considered for operational implementation, and defining final configuration(s) to be tested (as present practice with HWRF) Implementation decision will be based on evidence from extensive retrospective evaluation Full stakeholder evaluation and input No longer based on “30 day parallel” at the end of the implementation process NCO 30 day parallel testing becomes sanity check and IT stress test, but will no longer be the key decision point in the process

Implementation process The revision of the implementation process for individual models goes hand-in-hand with the development of a holistic implementation plan for the entire production suite: Resource allocation within the production suite will be based on explicit prioritization, not on a fixed “jig-saw puzzle” approach Will balance all products, not just those from NWS For the first time, this approach will also be applied to development resources. This has previously been done only at an ad-hoc bases within NCEP This will introduce a fixed mostly annual implementation scheduled for all main elements in the production suite More predictable for users Adjusting scope of implementations rather than implementation dates to improve predictability and management of resources

Implementation process Key issues remaining to be addressed A UMAC recommendation is to develop review boards for all major system. What is the proper composition of such boards, and what are the proper “internal” and “external” NWS/NCEP roles there? The NWS reorganization provides new governance for establishing requirements and setting priorities. NWS is starting to use this process, while details of the governance approach are still being developed

NWS will play an active role in the NOAA Unified Modeling Task Force (3) A unified, collaborative strategy for model development across NOAA Main players in NOAA are NWS and OAR. With the reorganization of NWS and the introduction of OSTI, the focus within the NWS is on building NWS-OAR teams for support and development of major models in the production suite Where needed, main partners may include other government agencies such as NASA and DoD (ocean, wave, aerosol) Where appropriate, Service Level Agreements will be developed (e.g., CFS development with CPO) NWS will play an active role in the NOAA Unified Modeling Task Force 4 members of the NOAA UMAC Response Tiger Team are members, one of which is a co-chair (Hendrik Tolman)

(4) Creation of a Chief Scientist position for Numerical Environmental and Weather Prediction This proposal was forwarded to senior NOAA leadership but was not accepted OSTI filled the Senior Advisor for Advanced Modeling Systems (SAAMS) position 7/11/16: Hendrik Tolman Position similar to the recommended position, but… Not at the organizational level suggested by the UMAC

(5) Better leverage the capabilities of the external community UMAC and NCEP both note that in some fields, this interaction is already strong (GSI, (H)WRF, Ocean, Waves, AQ, Space Weather) It is missing, however, in some of the flagship weather models Leveraging at two levels: Any research can help to make strategic decisions/identifying reasonable targets for improving operational models Efficient improvement of operations requires a O2R → R2O (community modeling) approach, strengthened by targeted funding Explicit recent actions for flagship models Community involvement in NGGPS global dycore selection Fall 2016 Community Modeling workshop for NGGPS

Community Modeling (internal and external) Community modeling is a key element for effective engagement with the research community inside and outside of the government. Many examples: WRF as a successful example of external community modeling efforts successfully incorporated in the production suite WAVEWATCH III, HWRF, CRTM and GSI as examples of internal NOAA efforts that have build a strong external community Many other models are mix of internal-external model origin with strong communities (MOM, HYCOM, CICE, HYSPLIT, GOCART, etc.) NEMS & NGGPS workshops in fall 2016 With DTC/GMTB and and other testbed involvement Effective community modeling considers and end-to-end process, including agreed-upon test plans and validation packages and well as the more elementary code sharing and management.

Governance (internal and external) Goes hand-in-hand with “Community” NCEP agrees with critical need for governance

(6) Continue to enhance HPC capabilities NOAA agrees NOAA 100% requirements effort lead by Brian Gross NCEP provided estimates for compute resources needed for tentative production suite layout under (1)

NWS/NCEP is developing this It will be closely coordinated with (7) Develop a comprehensive and detailed vision document and strategic plan that maps out future development of national environmental prediction capabilities NWS/NCEP is developing this It will be closely coordinated with NUOPC + ESPC (now the National-ESPC, or NESPC) – which provides a tentative backbone for this NOAA Unified Modeling Roadmap These are critical elements we have not developed yet, now is the time Some thoughts… An NCEP plan alone will be too limited, and will lack some credibility But these broader efforts will take more time Based on the OSTI drive to NOAA modeling teams, NOAA could be the starting point (preferred approach) ESMF / NUOPC / NEMS unified architecture approach National expansion can be reached by merging a NOAA plan in NESPC (consistent with present agency-focused NESPC)

(8) Execute strategic and implementation plans based on stakeholder requirements At individual model level, the revised implementation will address this, and will be in place for FY17 implementations (was applied to FY16 GFS) At the holistic production suite level, a 5-year incremental plan is also being developed On the strategic level, the plan under (7) needs to be developed in order to execute here

(9) Integrate Water There are well-defined plans for water issues, and clear Congressionally-driven for the National Water Center. But there is a need of additional integration: The National Water Model has clear 5-year plans (5-year cycles), but there is a risk that this will become a “stovepipe” if not integrated (coupled) with other components of the production suite (weather, climate) The Stormsurge Roadmap Team provided an integrated plan, but lacked governance and authority, resulting in organizations losing the integrated vision. How does the Nearshore Wave Prediction System (NWPS), which is intended to have a surge element, fit into the above two plans?

(10) Future Considerations UMAC identified Air Quality, Ecosystems and Space Weather as areas where they lacked expertise. Integrated water may also need expanded representation in UMAC. In NCEP’s unification of the production suite the position of some present models is not clear. Hurricanes: tentatively, this should be come a relocatable element in the global WGS, where this technology could be leveraged for WoF too. Due to run-cadence and attainable resolutions, it is likely that space weather will be a separate application for the intermediate future.

(10) Future Considerations, cont’d UMAC did not address NOAA testbeds. However, testbeds are to be addressed by UCACN for the present review cycle. Local modeling with WFO control has not been addressed by UMAC in the design of the unified modeling suite. Nearshore Wave Prediction System (NWPS) Local mesoscale modeling What is the future / strategy for local modeling?

Thank you! Questions?