1. Date: 20151207 UMAC Review Highlights at NCEP Production Suite Review (NPSR) UMAC committee Richard Rood and Frederick Carr, co-chairs December 7, 2015 UMAC Membership: https://www.earthsystemcog.org/projects/umac_model_advisory/members UMAC Website: https://www.earthsystemcog.org/projects/umac_model_advisory/

2. Date: 20151207 Committee Frederick Carr, co-chair Richard Rood, co-chair Alan Blumberg Chris Bretherton Andy Brown Eric Chassignet Brian Colle James Doyle Tom Hamill Anke Kamrath Jim Kinter Ben Kirtman Cliff Mass Peter Neilley Christa Peters-Lidard Ex Officio Gilbert Brunet William Kuo Tsengdar Lee NPSR Review December 7, 2015

3. Date: 20151207 Report Structure Executive Summary Introduction Overarching Findings and Recommendations Findings and Recommendations for Specific Modeling Themes – Global – Regional – Water-related – Ensembles – Reanalysis, Reforecast, Post-processing NCEP as an End-to-End system Appendices NPSR Review December 7, 2015 Introduction Overarching Findings and Recommendations Specific Recommendations

4. Date: 20151207 Timing On November 20, 2015 a draft was released to NCEP for fact checking Official release: December 7, 2015 NPSR Review December 7, 2015

5. Date: 20151207 from Overarching Findings and Recommendations UMAC notes that NOAA's organization of NEWP is unusual in many respects. No other peer environmental prediction service has chosen to separate its basic research and development from its applied, operational model implementation. No other national prediction service has produced such a diversity of prediction systems, most without the critical mass of resources to make them world-best. No other prediction service lacks top-level oversight spanning the research to the applied development. … NPSR Review December 7, 2015

6. Date: 20151207 from Overarching Findings and Recommendations The key finding is an optimistic one: U.S. Environmental Prediction has the potential to rapidly progress to world leadership. This requires a new level of organization and bold, evidence-driven decision-making. NPSR Review December 7, 2015

7. Date: 20151207 from Overarching Finding and Recommendations Reduce the complexity of the NCEP Production Suite. The NOAA environmental modeling community requires a rational, evidence- driven approach towards decision-making and end-to-end modeling system development. A unified, collaborative strategy for model development across NOAA is needed. NPSR Review December 7, 2015

8. Date: 20151207 from Overarching Finding and Recommendations Essential to effective planning and execution is the creation of a Chief Scientist position for Numerical Environmental and Weather Prediction (NEWP). NOAA needs to better leverage the capabilities of the external community NOAA must continue to enhance High Performance Computing (HPC) capabilities. NPSR Review December 7, 2015

9. Date: 20151207 from Overarching Finding and Recommendations NOAA must develop a comprehensive and detailed vision document and strategic plan that lay out future development of national environmental prediction capabilities. Execute strategic and implementation plans based on stakeholder requirements. NPSR Review December 7, 2015

10. Date: 20151207 The future will require Moving to more unified systems Addressing increasing scientific and data complexity Relying more broadly and more effectively on community research and assets NPSR Review December 7, 2015

11. Date: 20151207 Balanced, evidence-based decisions across products and systems Stakeholder requirements Scientific excellence Cost NPSR Review December 7, 2015

12. Date: 20151207 Highlighting Global ( Rood Take 1 ) The NGGPS and NEMS are very encouraging as scientific, software, and management advances. – Organizational focus needs to unite on these efforts – Data assimilation needs to be incorporated into these efforts – GFS, existing global systems, need to have a phase out plan to support NGGPS becoming operational NPSR Review December 7, 2015

13. Date: 20151207 Highlighting Regional ( Rood Take 1 ) Daunting complexity that is not scientifically justified. Needs to eliminate systems Needs to integrate better with global modeling Needs a vision – Unified systems – Convective-resolving U.S. ensembles – Software management NPSR Review December 7, 2015

14. Date: 20151207 Air Quality, Dispersion, Space Panel felt we are not properly staffed to address technical issues. – Endorse Space Weather integrated into NGGPS – Air Quality and Dispersion requirements / interface should be integrated with NGGPS and atmospheric systems – We note the community design and review of Air Quality (and Dispersion?) as a successful model of practice NPSR Review December 7, 2015

15. Date: 20151207 Highlighting ‘Water” ( Rood Take 1 ) Multi-organizational demands more management attention Requirements and resource demands for the evolving National Water Center need more attention and integration with EMC and NCEP Central Operations WAVEWATCH a successful model of community engagement NPSR Review December 7, 2015

16. Date: 20151207 Highlighting Ensembles ( Rood Take 1 ) NCEP should consolidate its ensemble prediction under unified dynamical cores using physically based stochastic parameterizations to treat model uncertainty in the ensembles Need to better integrate with global Scientific progress more likely to be made through robust investigation in a well managed system in a controlled environment than through algorithm and model diversity NPSR Review December 7, 2015

17. Date: 20151207 Highlighting ‘Reanalysis” ( Rood Take 1 ) Decisions on what computational and storage resources are allocated to R/R should be done in a systematic manner, based on how they help NOAA meet its requirements, as recommended elsewhere by UMAC Nonetheless, given the demonstrated value of carefully constructed reforecast data sets, post processing and the production of supporting data sets (reanalysis/reforecast, R/R) should be considered an integral part of NCEP’s future production suite and resourced accordingly Need more management, design and uniformity in grids used as stakeholder interfaces NPSR Review December 7, 2015

18. Date: 20151207 Highlight systems ( Rood take 1 ) UMAC calls for a “new-level of organization,” and “evidence-based decision making” This section is an analysis and description of the possible meaning of organizational recommendations – Project management – Software management – Organizational governance NPSR Review December 7, 2015

19. Date: 20151207 Highlight systems ( Rood take 1 ) Short-term to gain control over complexity – Document and maintain complete range of products and systems – Identify and publicize leads for all products and systems – Hire or identify software leads, with proven expertise in scientific software – Develop Change Review Boards for all products and systems – Develop a software release schedule for major systems on the order of 12-24 months – Replace all the code that EMC uses with code developed with formalized software management NPSR Review December 7, 2015

20. Date: 20151207 Final Remarks “a transformation to a science-based, service organization directed toward building a weather-ready nation” – Requires a new type of organization: More unified Better able to participate as member of community Recognize and manage complexity Requires commitment and continuity from leadership Requires buy in from entire organization Will be a transition of years, not simply responding to reports such as this one NPSR Review December 7, 2015

21. Date: 20151207 Acronyms NOAA – National Oceanographic and Atmospheric Administration NWS – National Weather Service NOS – National Ocean Service OAR – Oceanic and Atmospheric Research NCEP – National Centers for Environmental Prediction NWC – National Water Center UCACN - University Corporation for Atmospheric Research Community Advisory Committee for NCEP UMAC – UCACN Model Advisory Committee NGGPS: Next Generation Global Prediction Center NEMS: NOAA Environmental Modeling System CMC: Canadian Meteorological Center ECMWF: European Center for Medium- range Weather Forecasts NCAR: National Center for Atmospheric Research NRL: Naval Research Laboratory S2S: Sub-seasonal to seasonal S2A: Seasonal to Annual NPSR Review December 7, 2015