1. Jamie K. Wolff 1, Louisa B. Nance 1, Brad S. Ferrier 2, Clifford F. Mass 3, Barbara G. Brown 1 and Ying-Hwa Kuo 1 Joint 46 th CMOS/25 th WAF/21 st NWP AMS Conference May 29, 2012 Bridging the Valley of Death: Defining a process for transitioning promising new mesoscale innovations from research to operations 1 National Center for Atmospheric Research/Research Applications Laboratory and Developmental Testbed Center 2 National Centers for Environmental Prediction/Environmental Modeling Center 3 University of Washington

2. DTC Mission The fundamental purpose of the DTC is to facilitate the interaction & transition of NWP technology between research & operations. DTC facilitates: O2R transition by making the operational NWP systems available to the research community & providing community user support R2O transition by performing testing & evaluation of new NWP innovations in a functionally similar operational environment over an extended period Interaction between research & operational NWP communities through the organization of community workshops/meetings on important topics of interest to the NWP community & hosting a DTC Visitor Program DTC strives to be an effective and efficient community facility for the transition of innovations in NWP between research and operations.

3. Testing Protocol Motivation Wide range of NWP science innovations under development in the research community Testing protocol imperative to advance new innovations through the research to operations (R2O) process efficiently and effectively. Three stage process: 1)Proving ground for research community 2)Comprehensive T&E performed by the DTC 3)Pre-Implementation testing at Operational Centers

4. Testing Protocol – Stage I Proving ground for research community Code development; Initial stage of testing Mesoscale Model Evaluation Testbed (MMET) Communicate results to the DTC; Nominate for Stage II testing Contribution of new technique into WRF repository encouraged Work with WRF Developers Committee (contact: wrfhelp@ucar.edu)wrfhelp@ucar.edu Apply for DTC Visitor Program support (see: http://www.dtcenter.org/visitors)http://www.dtcenter.org/visitors = –

5. Mesoscale Model Evaluation Testbed (MMET) Why: Assist the research community in efficiently demonstrating the merits of a new development Provide a common framework for testing; allow for direct comparisons What: Mechanism to assist research community with initial stage of testing Provide model input and observational datasets to utilize for testing Establish and publicize baseline results for select operational models Where: Hosted by the DTC; served through Repository for Archiving, Managing and Accessing Diverse DAta (RAMADDA) http://dtcenter.org/repository

6. RAMADDA View (http://dtcenter.org/repository)http://dtcenter.org/repository

7. MMET Cases Initial solicitation of cases from DTC Science Advisory Board Members and Physics Workshop Participants – great response and enthusiasm towards endeavor Target cases during initial year 20090228 – Mid-Atlantic snow storm where North American Mesoscale (NAM) model produced high QPF shifted too far north 20090311 – High dew point predictions by NAM over the upper Midwest and in areas of snow 20091007 –High-Resolution Window (HIRESW) runs underperformed compared to coarser NAM model 20091217 – “Snowapocalypse ‘09”: NAM produced high QPF over mid-Atlantic, lack of cessation of precipitation associated with decreasing cloud top over eastern North Carolina 20100428-0504 – Historic Tennessee flooding associated with an atmospheric river event 20110518-26 – Extended period of severe weather outbreak covering much of the mid- west and into the eastern states later in the period 20111128 – Cutoff low over SW US; NAM had difficulties throughout the winter of breaking down cutoff lows and progressing them eastward 20120203-05 – Snow storm over Colorado, Nebraska, etc.; NAM predicted too little precipitation in the warm sector and too much snow north of front (persistent bias)

8. “Snowpocalypse 2009” Nor’easter that became a major snowstorm for the East Coast of the U.S with record-breaking snowfall totals for several major cities, including D.C., Baltimore and Philadelphia. MMET case study for 20091217 12 UTC out to 84 hours

9. May 21-26, 2011 Tornado Outbreak Deadly outbreak across the Midwest and Southern regions of the U.S., including the Joplin tornado (May 22) and two “High Risk” days (May 24-25) Extended time period included in MMET to 20110518-26 in order to capture additional weaker warm-season convection prior to outbreak Confirmed Tornadoes TotalEF0EF1EF2EF3EF4EF5 2421089031832

10. Testing Protocol – Stage II Comprehensive T&E performed by the DTC Maintain a neutral position in order to provide a trusted, unbiased assessment Conduct comprehensive testing for a broad range of weather regimes Run end-to-end system composed of community codes Functionally similar to operational environment Evaluate based on extensive objective verification statistics Traditional scores New, relevant verification techniques (e.g., spatial methods) Statistical significance assessment

11. Testing Protocol – Stage III Pre-Implementation testing at Operational Centers Ultimate decision to proceed with pre-implementation testing is made by the Operational Centers and is based on a variety of factors, including: Forecast performance Computational requirements Testing specifics depend on the target production configuration, but may include: Complex data assimilation testing Initial condition diversity testing for ensemble members

12. Conclusion Establish data sets for nine initial MMET cases Publicize more information on R2O testing protocol and MMET at: http://dtcenter.org/eval/mmet Contact information for DTC MMET Team Jamie Wolff jwolff@ucar.edu Michelle Harrold harrold@ucar.edu Tara Jensenjensen@ucar.edu Tricia Slovacekslovacek@ucar.edujwolff@ucar.eduharrold@ucar.edujensen@ucar.eduslovacek@ucar.edu Acknowledgements: The DTC is funded by the National Oceanic and Atmospheric Administration, the Air Force Weather Agency, and the National Center for Atmospheric Science (NCAR). NCAR is sponsored by the National Science Foundation.