1. The Implementation Process1 / 8NPSR, December 8, 2015 Revising the Implementation Process Looking Forward, setting the stage for the panel discussion Hendrik L. Tolman Director, The Environmental Modeling Center NOAA / NWS / NCEP Hendrik.Tolman@NOAA.gov

2. The Implementation Process2 / 8NPSR, December 8, 2015 Outline  Background  The old way of doing business  Looking forward  Food for thought? CaRDS

3. The Implementation Process3 / 8NPSR, December 8, 2015 The old (present) process 3 EMC Change Control Board Scientific Integrity Product Quality EMC Mgmt Approval Implementation Phase SPA’s build NCO parallel from RFC’s 30-day NCO parallel  Test code stability  Test dataflow  Products to NCEP Centers and EMC code developers NCEP Centers  Evaluate impact  Assessments to NCEP OD R&D and Pre-Implementation Phase 30-day NCO parallel stable NCEP centers approve Briefing to NCEP Director for final approval Implementation Generate RFC’s Submit RFC’s to NCO

4. The Implementation Process4 / 8NPSR, December 8, 2015  Requirements definition ● Identified as a weakness by NCEP stakeholders and UMAC ● incomplete requirements may create false expectations ● NWS needs an improved process—is portfolio management the answer? Stakeholder input  Stakeholders--- need earlier access to information ● What changes are being made? ● What’s the rational? ● What characteristics of the tool will change? ● Stakeholder calibration methods need time and access to pre- implementation data in order to adapt (i.e., GEFS FY15 Upgrade) ● 30-day NCO parallel insufficient for customer assessment IMPROVE COMMUNICATION BETWEEN MODEL DEVELOPERS AND STAKEHOLDERS

5. The Implementation Process5 / 8NPSR, December 8, 2015 Process suggested recently to AA 5 Start of Development Cycle Conduct a workshop (modelers, field, academia, customers) Prioritize features to be improved How do you propose to improve them? How much will it cost (time=$, HPC) How will data be disseminated? Develop detailed test plan Create end-to-end charter Get appropriate approval to proceed Phase 1 of test plan (2-4 months) Assessment of Phase 1 results (2 weeks) Invite SOO’s to participate Phase 2 of test plan (2-4 months) Assessment of Phase 2 results (2 weeks) Invite SOO’s to participate Test Assess NCO Testing & Implementation Final Approval MEG !?

6. The Implementation Process6 / 8NPSR, December 8, 2015 Process in time lines …. Planning workshop Science testing of parts Selection of ready upgrades Run retrospectives (R/R) Science evaluation Implementation decision Downstream adjustments Hand off to NCO 30 day IT test Implementation Decision point / “gate” Work with partners “the funnel” Food for thought 1) “Fixed schedule” and skipping your slot 2) dealing with downstream 3) RR strategy

7. The Implementation Process7 / 8NPSR, December 8, 2015 Past – present - future  Past: ● This is following the established HWRF implementation process ➤ Fixed schedule, stringent project management. ● Last SREF implementation started with user defined scope.  Present: ● New GFS implementation decision will be made several weeks after retrospectives are done, WELL BEFORE NCO parallel. ● Starting off GEFS v12 with workshop to determine scope.  Future: ● Start with users, end with users. ● Project management per system, and for all suite. ● Stick to your schedule, or skip implementation ….

8. The Implementation Process8 / 8NPSR, December 8, 2015