1. 1 INTRODUCTION TO DTC ENSEMBLE TESTBED (DET) Zoltan Toth 1 1 GSD/ESRL/OAR/NOAA 2 NCAR/RAL Team Members: Barbara Brown 2, Tara Jensen 2, Huiling Yuan 1, Isidora Jankov 1, Ed Tollerud 1, Linda Wharton 1, Paula McCaslin 1 Acknowledgements: Steve Koch, Jun Du, Yuejian Zhu, Geoff Dimego, Bill Lapenta DET website: http://www.dtcenter.org/det

2. 2

3. 3 About 3,220,000 results (0.28 seconds) Search Results Amazon.com: Gypsy Music From Hungary And Romania: Zoltan And His... Gypsy Music From Hungary And Romania by Zoltan And His Gypsy Ensemble. This product is manufactured on demand using CD-R recordable media....www.amazon.com ›... › Europe › Eastern Europe › Romania - Cached - Similar Amazon.com: Zoltan & His Gypsy Ensemble: Albums, Songs, Bios, Photos Visit Amazon.com's Zoltan & His Gypsy Ensemble Store to shop for Zoltan & His Gypsy Ensemble albums (CD, MP3, Vinyl) and other Zoltan & His Gypsy...www.amazon.com/Zoltan- His...Ensemble/e/B000APVOT6 - Cached - Similar Show more results from www.amazon.com Zoltan & His Gypsy Ensemble - Rhapsody Music Listen to Zoltan & His Gypsy Ensemble FREE on Rhapsody.com. Rhapsody lets you explore every style of music without paying per song.www.rhapsody.com/zoltan-his-gypsy-ensemble - Cached – Similar Zoltan and His Gypsy Ensemble – Free listening, concerts, stats... Listen free to Zoltan and His Gypsy Ensemble: Gypsy Tango, Hungarian Song And Dance & more, plus 1 picture. Gypsy, Hungarian and Balkan.www.last.fm/music/Zoltan+and+His+Gypsy+Ensemble – Cached Zoltan & His Gypsy Ensemble MP3 Downloads - Zoltan & His Gypsy... Zoltan & His Gypsy Ensemble on MP3.com - listen to audio streams, check out photos, watch videos, write a review, discover similar artists, find news and...www.mp3.com/artist/zoltan-and- his...ensemble/summary/ - Cached – Similar Zoltán Kodály – Háry János-Suite | for large orchestra - Universal... Zoltán Kodály 16.12.1882—06.03.1967. Picture – Zoltán Kodály.... Student ensemble Student ensemble also played the following 5 works:...www.universaledition.com/Zoltan-Kodaly/composers-and.../2721 - Cached [PDF] ON THE ECONOMIC VALUE OF ENSEMBLE BASED WEATHER FORECASTS Yuejian... File Format: PDF/Adobe Acrobat - Quick Viewby Y Zhu - 2001 - Cited by 110 - Related articlesON THE ECONOMIC VALUE OF ENSEMBLE BASED WEATHER FORECASTS. Yuejian Zhu1, Zoltan Toth1, Richard Wobus1, David Richardson2, and Kenneth Mylne3...www.emc.ncep.noaa.gov/gmb/ens/papers/econval.pdf

4. OUTLINE / SUMMARY Why and how DET was formed? –Community and NOAA ready –Sept 2009 Workshop Major tasks –Infrastructure, test & evaluation, transition Timeline –Planning, basic capabilities, benchmark, testing Questions –For WG discussion Workshop expectations –Recommendations for DET infrastructure & candidates for testing

5. FORMATION OF DET IN FY2010 Confluence of necessary ingredients Need –Strong interest from both community & agencies Knowledge base –Roadmap from Sept 2009 Workshop National Workshop on Mesoscale Probabilistic Prediction at NCAR Opportunity –Increase in funding for DTC from NOAA Initiative –DTC Director’s Office sets planning process in motion (Oct 09)

6. OBJECTIVES & LIMITATIONS “Provide an environment in which extensive testing and evaluation of ensemble-related techniques developed by the NWP community can be conducted such that the results are immediately relevant to the operational centers (e.g., NCEP/EMC and AFWA)” Link research & operational communities –R2O – Testable techniques from research to operational community –O2R – Operational codes and requirements from operational to research community Constraints –Limited funding ~ 3 FTEs – New task lead to be hired soon GSD & NCAR collaborative effort –Excited about working with community on new project –No funding for research Except via very limited visitor program Initial focus on mesoscales

7. DET MAJOR TASKS Develop and maintain DET infrastructure Establish NCEP operational system as benchmark Test and evaluate new community methods Transition successful methods to NCEP and other agencies Link up with ensemble work in other test-beds Input from revived WRF Ensemble WG Close collaboration with NCEP

8. INFRASTRUCTURE Objective –Assemble software and procedures to carry out main function of testing and evaluation of community methods Structure –Six modules identified Ensemble configuration Initial perturbationsEnsemble generation – meso-specific Model-related uncertainty Statistical post-processing Products / servicesUse of info – general, applies to all ens Verification Requirements –Modularity to facilitate transition to operations –Portability for ease of execution where computational resources are available –Flexibility to allow testing of new ideas

9. Module 1: Configuration Module 2: Initial Perturbations Module 4: Statistical Post- Processing External Input (HMT, HWT, HFIP, etc) Module 3: Module Perturbations Module 5: Product Generation Module 6: Verification DET MODULES

10. BENCHMARK Purpose –Ensemble system/module used as reference when testing and evaluating new community methods Definition –NCEP (or other center’s) operational ensemble system/module at any time Facilitate community’s easy access to and tools to work with benchmark Choice of first DET benchmark –Adopt current NCEP operational system/module By time task completed benchmark may be obsolete –Contribute to building system/module for next NCEP implementation In collaboration with NCEP –Accelerates NCEP implementation schedule Avoids falling behind at start Related tasks –Set it up, update, make it available to community

11. TEST AND EVALUATION Core DET task Carried out according to detailed protocol Independent from developers Use operational and other metrics First application –Test & evaluate benchmark system / module against current operational system as reference

12. TRANSITION TO OPERATIONS Objective –Offer NCEP & other agencies carefully tested and well performing modules IT solution –Use subversion control Potential NCEP actions –Possible modification Streamline, combine with other modifications, etc –Further test & evaluation –Final selection –Implementation into operations

13. LINK WITH OTHER TESTBEDS Objective –Leverage and coordinate DET work with other DTC testbeds and agency programs/projects for mutual benefits Examples – HMT, HWT, HFIP - Ensemble development & evaluation work DET to support ensemble work in various applications –Share infrastructure, best methods & practices, etc –GSI – Hybrid Variational / Ensemble-based data assimilation DET to focus on ensemble system (perturbations, etc) GSI task to focus on data assimilation aspect?

14. DET TIMELINE – STARTING MARCH 2010 First year deliverables – Planning, basic capabilities –Detailed plan for infrastructure (6 modules) –Basic capability for first two modules –Protocol for T&E –Plan for & commence collaboration with HMT Second year deliverables – Basic capabilities, benchmarks –Basic capability for all 6 modules –Benchmark for modules 2, 3, & 6 –T&E for Benchmarks 2 & 3 –Contribute to NCEP implementation for modules 2&3 (FY11-12) –Contribute to design and evaluation of HMT, HWT, & HFIP ensembles Third year deliverables – Benchmarks, T&E –Benchmark for modules 4-6 –T&E for Benchmark 4 –Contribute to NCEP implementation for module 4 (FY12-13) –TET of community methods for modules 2&3 Fourth year deliverables –TET of community methods for all modules –Update benchmarks

15. Module 1: Configuration Module 2: Initial Perturbations Module 4: Statistical Post- Processing External Input (HMT, HWT, HFIP, etc) Module 3: Module Perturbations Module 5: Product Generation Module 6: Verification DET MODULES

16. QUESTIONS Interest from agencies beyond NOAA? –AFWA, ONR? Expansion to global ensemble? –Needed? Expected benefits? –Resource implications? –NUOPC interagency interest? EnKF and similar DA applications tackled jointly with DTC GSI task? –DET – Creation of ensemble –GSI – Data assimilation step

17. QUESTIONS – CONT. Candidate methods for testing? –How to prioritize? NCEP involvement at all steps How to measure success for DET? –Define metrics for quality & performance –Improvement over benchmark of tested methods? Where to perform DET T&E? –Only limited resources on host (GSD, NCAR) computing facilities –In search of other opportunities… Community outreach –Alternate biannual workshops DET Ensemble Research / Development Workshop NCEP Ensemble User Workshop –User support via tutorials, what else?

18. WORKSHOP EXPECTATIONS Review ongoing research for potential testing by DET –Identify candidates and possible collaborative efforts Review DET activities / plans –Are community and NCEP needs met? –DET’s role in national mesoscale ensemble effort Breakout Group discussions on above WRF Ensemble WG meeting Friday am –Further discussion and recommendations

19. OUTLINE / SUMMARY Why and how DET was formed? –Community and NOAA ready –Sept 2009 Workshop Major tasks –Infrastructure, test & evaluation, transition Timeline –Planning, basic capabilities, benchmark, testing Questions –For WG discussion Workshop expectations –Recommendations for DET infrastructure & candidates for testing

20. BACKGROUND

21. EnKF sequential DA EnKF initial perturbations and forecasts GSI var. DA GSI-EnKF Hybrid Approach Analysis error estimate Ensemble-based forecast error covariance estimate Data assimilation step done twice Initial perturbations reflect uncertainty in ENKF, not GSI

22. Alternate Hybrid Approach ET (or other) initial perturbations and forecasts GSI var. DA Analysis error estimate Ensemble-based forecast error covariance estimate Only one (variational) Data assimilation Initial perturbations reflect uncertainty in GSI Challenge to create good estimate

23. HOW TO REPRESENT MODEL RELATED UNCERTAINTY? Multiple models or model versions – Pragmatic approach Different models / model versions being developed by community – Instead of competition, use them all in ensemble mode – Diversity of opportunity - Not by “design” Ad hoc approach, trial and error – try in ensemble, see if it works or not – Scientifically not well founded? If we wanted to intentionally create different models / model version – What design directives could we use? Stochastic modeling – Build a single model that can represent all model related uncertainty by design – Consider stochastic effect of subgrid-scale and other unresolved processes – Scientifically more appealing – Requires community to embrace new paradigm Focus on model development for ensemble use Different groups work on various aspects of same model?