1. Ensemble Task Isidora Jankov Collaborators: NOAA’s Environmental Modeling Center NOAA’s Earth System Research Laboratory 1

2. Ensemble Task AOP12 Activities ActivityStatus HWRF ensemble for GSI-Hybrid Applications methods Moved to the DA task Establish a functionally similar environment of the NCEP SREF Accomplished HMT Winter ExperimentDelayed Evaluate ensemble products using CAPS, AFWA, and NCEP algorithms On hold Ensemble Code RepositoryAccomplished Co-host semi-annual DTC Ensemble Task WorkshopAccomplished Maintain ensemble group wiki page and web siteOn-going Work done late in AOP11, DTC contribution to SREF post-processing 2

3. Work performed late in AOP11 NAEFS Bias Correction and Downscaling effort High-resolution version of SREF (SREFx) was recently implemented operationally at EMC. SREFx at 16km (NMM, NMMB)/17km (ARW) resolution Significant resolution upgrade, but still missing small scale variability needed for NWS forecast grids Downscaling to the resolution of the NDFD (~5km) should allow greater use of NCEP SREF output in the production of forecasts Courtesy of Brian Etherton, Jun Du and Tara Jensen 3

4. Bias Correction and Downscaling Bias Correction accomplished by taking the mean forecast of each model core (ARW, NMM, etc.) sub-ensemble of the NCEP SREF and comparing it to the NAM analysis valid at the same time. Downscale by comparing the RTMA analysis (GRID 259, like GRID 197, 1073x689, 5.079km) with the NAM analysis valid at the same time (10m wind,2m temperature and humidity). ARW NMM NMMB NAM RTMA ANALYSES FORECASTS BC DS Approach used in NAEFS 4 Courtesy of Brian Etherton, Jun Du and Tara Jensen

5. Bias Correction and Downscaling Recursive averaging method is used (~30 day mean) Bias-Correction (in operations) and Downscaling (DTC testing) from NAEFS Bias corrected and downscaled fields for each member Also: downscaled forecasts of ensemble min, max, mean, spread, 10 th, 50 th, and 90 th percentiles. ARW NMM NMMB NAM RTMA ANALYSES FORECASTS BC DS 5 Courtesy of Brian Etherton, Jun Du and Tara Jensen

6. Implementation of downscaling at EMC Downscaling code used for NAEFS ported to DTC Testing/evaluation of code application to existing NCEP SREF output showed forecast improvement DTC code sent to EMC (Jun Du) for application to the new 16km/17km NCEP SREF Code implemented by EMC in approximately 2 days, with downscaled forecasts being produced since December (Presentation at AMS Annual Meeting ) 6 Courtesy of Brian Etherton, Jun Du and Tara Jensen

7. Raw SREFx vs Bias corrected SREFx (Nov. 10 – 30, 2011, against NDAS) Ens mean fcst of T2m: RMSEProb fcst of T2m: RPSS (12km NAM as ref) Implementation of bias correction at EMC Courtesy of Jun Du, NOAA/EMC 7

8. Testing/Evaluation at DTC - Results As implemented at DTC, the bias correction reduces the RMSE of 2m T forecasts & the downscaling adds further improvement. ARW and NMM members of SREF 2011 – 0900 UTC Initialization Compared to RTMA Analyses Courtesy of Brian Etherton, Jun Du and Tara Jensen 8

9. Ensemble Task AOP12 Activities ActivityStatus HWRF ensemble for GSI-Hybrid Applications methods Moved to the DA task Establish a functionally similar environment of the NCEP SREF Accomplished HMT Winter ExperimentDelayed Evaluate ensemble products using CAPS, AFWA, and NCEP algorithms On hold Ensemble Code RepositoryAccomplished Co-host semi-annual DTC Ensemble Task WorkshopAccomplished Maintain ensemble group wiki page and web siteOn-going 9

10. NCEP SREF Functionally Similar Environment Activity description: Define & establish scalable, flexible, highly portable heterogeneous computational Functionally Similar Environment, (FSE) to NCEP SREF operations. Objectives and Accomplishments: 1.Develop FSE acceptance evaluation criteria in collaboration w/ NCEP 2.Establish portable integration of NCEP SREF components & DTC platforms 3.Participate in updates of NCEP NEMS code, SREF scripts & data cross management 4.Continually support research, DET T&E workflow & future R2O integrations 5.Analyze the differences & integrate NCEP operative & DET research workflows 6.Work closely w/ EMC to deploy computational & research innovations. Courtesy of Eugene Mirvis 10

11. Major Accomplishment: NCEP SREF v6.0.0 adapted by DTC DTC now has functionally similar environment for 4-5 of 6 SREF modules 11

12. New Generation of The Ensemble Testbed 5. MERGING OPS AND RESEARCH WORKFLOWS 3. PARTICIPATE IN UPDATES, OPS CODE & DATA MANAGEMENT 3. PARTICIPATE IN UPDATES, OPS CODE & DATA MANAGEMENT 6. ACADEMIA SUPPORT & INNOVATIONS 2. PORTABLE INTEGRATION of OPS & Research Computing Platforms 2. PORTABLE INTEGRATION of OPS & Research Computing Platforms 1.DEVELOPMENT OF THE FSE ACCEPTANCE CRITERIA ~ 60 UNIVERSITIES Conventions & Criteria Of Functional Similarity OPS ENVIRON -MENT TESTBED ENVIRON -MENT 12

13. Shared Physics NCEP NEMS – DTC Ensemble Workflows Merging Analysis NCEP NEMS – DTC Ensemble Workflows Merging Analysis External data processing External data processing Alternative Obs. data Standard Obs. data Standard Obs. data NEMS GSI NEMS/WRF Terrestrial DATA NEMS/WRF Terrestrial DATA Gridded Data: NAM,GFS, RUC, snow, ice, SST Gridded Data: NAM,GFS, RUC, snow, ice, SST Post Visualize & Verify Post Visualize & Verify RIP4 Grid-vs-Obs VSDB/FVS Ensemble Initialization Ensemble Initialization GrADS GEMPAK NCL Grid-vs-Grid VSDB/FVS - - not used yet at NCEP MET IC/LBC perturb Land surface perturb Cold start Cold start NEMS Apps Layer NEMS I/O Stochastic Forcing Perturbation Component NEMS entry Ensemble Post Ensemble Post UPPUPPUPPUPP Ensemble post and product generation BIASCorrect& downscale downscaleBIASCorrect& ? GLOB ? NPS Land surface init MESO ? YES NO 13

14. Ensemble Task AOP12 Activities ActivityStatus HWRF ensemble for GSI-Hybrid Applications methods Moved to the DA task Establish a functionally similar environment of the NCEP SREF Accomplished HMT Winter ExperimentDelayed Evaluate ensemble products using CAPS, AFWA, and NCEP algorithms On hold Ensemble Code RepositoryAccomplished Co-host semi-annual DTC Ensemble Task WorkshopAccomplished Maintain ensemble group wiki page and web siteOn-going 14

15. HMT Winter Experiment Activity description:  Use HMT Winter Experiment real-time system to test the impacts of alternate initial perturbation techniques (including the dynamical downscaling of global perturbations)  Test alternate ensemble perturbation techniques (cloud microphysical parameterizations  Presentation at NWA Annual. Accomplishments: Plans developed NWA presentation delivered Issues: This task has been affected by changes in staffing which resulted in a significant delay. 15

16. HMT Activities AOP12 2012 Tasks necessary to be completed for 2013 activities. HMT real time DTC 1DTC 2 MicrophysicsHMTSREF (ARW)HMT Initial Perturbations HMT SREF (ARW) Grid Spacing999 1 2 1.Evaluate the impact of changes in microphysics 2.Evaluate the impact of initial perturbation changes 16

17. Impact of Microphysics on 2010-2011 HMT  90% Intensity show over forecast of precip for ARW- Fer and ARW-MY members especially at higher thresholds – which means when it rains it pours in these members > 25.4 Difference (Fcst-Obs) in Near Peak Intensity in MODE Obj (mm) Object Threshold (mm) NMM ARW HMT NAM GFS Using Attributes from MODE Objects Color groups Different Microphysics Ferrier Milbrant-Yau Thompson Intense Cores Optimal Courtesy of Tara Jensen 17

18. 00Z06Z LAM Forecast Global Model Forecast Global Model Analysis interpolated on LAM grid Perturbation Legend Forecast Time Initial Perturbations Approach X a L = X f L + (X a G – X f G ) 18

19. Proposed Activities for 2013 19

20. Ensembles AOP 2013 Proposed Activities IDActivity HMT Four tests related to implementation of the HMT innovations into SREF EN3 Evaluate options for optimal combination of reference dynamic cores and corresponding analyses in SREF (currently beyond 100%) EN4 Evaluate the merits of using 2 vs. 3 dynamic cores in SREF – using ARW subset available from HMT activity (currently beyond 100%) 20

21. Ensemble Proposed AOP 2013 Activities 1.Evaluate impact of changes in initial conditions in context of SREF 2.Evaluate impact of HMT microphysics in context of SREF 3.Evaluate impact of increased resolution (16 vs. 9 km SREF) 4. Evaluate the impact of mixed microphysics by optimally combining HMT and SREF runs. HMT Real time DTC 1 (AOP12) DTC 2 (AOP12) DTC 3 (AOP13) NCEP Operations MicrophysicsHMTSREF (ARW)HMTSREF (ARW)SREF Initial Perturbations HMT SREF (ARW) SREF Grid Spacing999916 1 2&4 3 HMT CONTEXTSREF CONTEXT 21

22. Tressa L. Fowler Verification 22

23. Verification AOP 2012 Activities AOP 2012 ActivityStatus MET releases in July and JanuaryDone / Slightly Delayed Enhancement of software testingDone, more in progress New verification tools for T&E supportSome done, more in progress METViewer GUI enhancementsIn progress Initial release of MET TCSlated for 4.1 release LSM verificationDelayed TutorialFeb 4-5 Community presentationsMany completed MET helpongoing 23

24. Goals Promote easy, consistent, correct verification across the NWP community, especially in operations where there is the greatest potential for impact. Create a cohesive team of scientists and software engineers engaged in challenging, interesting, rewarding work. 24

25. Accomplishments Software Development MET TC MADIS data support Ensemble spread skill GRIB2 Series analysis tool Testing and Evaluation support HMT verification MMET cases Community support 25

26. Series Analysis Tool  Frequency Bias  Gilbert Skill Score Happy Users  26

27. Verification Support of HMT Capability was added to METViewer: User can constrain aggregation using Base Rate (aka, observed relative frequency). This feature helps users focus on model skill for meaningful events without having to constrain on an event-by-event basis Increases analysis speed and relevance No constraint Base Rate > 0.02 HMT Ensemble Mean 27

28. Confidence Tests Box-and-whiskers now on most FVS plots (thanks to Keith Brill of HPC) Verification Implementations 28 EMC Verification Website (intranet only): http://emc-ls-sand02.ncep.noaa.gov/EMC_VSDB_verif/index.cgi Slide excerpted from Geoff DiMego

29. Verification AOP 2013 Activities IDActivity Description VX2Verification support for T&E activities VX1Community Support: Release, tutorial, help VX3MET Development – External users VX4MET Development – Internal users VX5MET-TC Development (currently beyond 100%) 29

30. Next Year Software Development – Internal Enhancements to MET TC in support of HWRF testing Series analysis tool testing and enhancements METViewer GUI redesign Software Development – External Satellite data and cloud verification Complete MODE GUI Land Surface Model evaluation capability Testing and Evaluation support HMT Evaluation Use of MET TC for HWRF evaluations Additional MMET cases Radiation scheme evaluation 30

31. Community Support Yearly tutorial, with HWRF tutorial if funded Updated online tutorial MET Help Verification lectures at conferences and workshops 31

32. MET TC Development Community TC development Hurricane T&E development 32

33. MODE GUI Capability developed outside DTC but will be leveraged for inclusion in community software! 33

34. Track Error Rank of Candidate Model vs. 3 Operational Models 34

35. Leveraging SURFRAD Development of SURFRAD ingest underway Useful for radiation scheme evaluations Land surface model verification Solar forecast evaluation for DOE project 35

36. Conclusions Some work will carry over to 2013, due to staffing changes. 2013 will be a year of rebuilding. We accomplished much in 2012. Our 2013 planned work reaches into new areas More opportunities to assist with R2O More learning opportunities for our staff 36

37. Louisa Nance Director’s Office & Visitor Program 37

38. Director’s Office AOP 2013 Activities AOP 2012AOP 2013 Director’s Office Staff Non-salary expenses: travel, external meetings, etc… Non-salary expenses: travel, external meetings, etc… (reduced budget by 20 K) Director’s Reserve (65 K)Director’s Reserve (65 K) – 85-100% DTC Visitor ProgramDTC Visitor Program – 85-100% External communications (status: newsletter is in progress) External communications – below 100% CO1: Investigate requirements ECMWF- like environment) – 85-100% CO2: Post-doctoral Scientist Program – 85-100% 38

39. Director’s Office Staff AOP 2012 National Director (Kuo) 0.5FTE Assistant Director (Nance) 0.55 FTE JNT Management (Brown) 0.1 FTE Budget support (Jensen) 0.25 FTE Support staff for community activities (Johnson) 0.5 FTE GSD Management (Toth) 0.1 FTE Budget support (Etherton/Bernardet & Admin Asst) 0.25 FTE AOP 2013 National Director (Kuo) 0.5FTE Assistant Director (Nance) 0.55 FTE JNT Management (Brown) 0.1 FTE Budget support (Jensen) 0.25 FTE Support staff for community activities (Zarlingo) 0.3 FTE GSD Management (Toth) 0.3 FTE Budget support (Etherton/Bernardet & Admin Asst) 0.30 FTE 39

40. External Meetings AOP 2012 3 Workshops 3 Tutorials 4 Governance meetings AOP 2013 1 Workshop 1 Tutorial 4 Governance meetings 40

41. DTC Visitor Program Framework for engaging the research community in projects directed at improving NWP, with an eye towards eventually improving operational forecasts 41

42. DTC Visitor Program 42 Supports visitors to work w/ the DTC to test new forecasting & verification techniques, models & model components for NWP PI project – up to 2 months salary & travel & per diem Graduate student project - up to 1 yr temp living per diem stipend & travel expenses for student to work w/ DTC &/or one of its partners + travel & per diem for up to 2 2-wk visits to the location of the student by project PI

43. Current DTC Visitors PIInstitutionProject Title Jonathan VighNCAR Development of an HWRF Diagnostics Module to Evaluate Intensity & Structure Using Synthetic Flight Paths Through Tropical Cyclones Adam ClarkU. of Oklahoma Development & Application of 3-Dimensional Object Algorithms to High Resolution Forecasts Delia Arnold & Don Morton Technical U. of Catalonia / U. of Alaska –Fairbanks Moving The NEMS & NMM-B Into a Broader Community Resource Environment Robert Fovell / Travis Wilson UCLA Improvements to modeling persistent surface cold pools in WRF Sai RavelaMIT Deploying the MIT Field Alignment System & Test-bed (FAST) in DTC 43

44. Future DTC Visitors PIInstitutionProject Title Thomas Galarneau NCAR Diagnosing Tropical Cyclone Motion Forecast Errors in HWRF Robert Fovell / Huang UCLA Improving HWRF Track and Intensity Forecasts Via Model Physics Evaluation and Tuning Man ZhangCSU Impact Assessment of Cloud-Affected AMSU-A Radiance Assimilation in TC inner-Core Region using Hybrid Data Assimilation Approaches Marion Mittermaier UKMET Incorporating Observations Uncertainty to a Spatial Probabilistic Verification Framework for km-scale Models Adam Clark U. of Oklahoma Object-based Time-Domain Diagnostics for High- resolution Ensemble Forecasting and Evaluation in NOAA/HWT Spring Forecasting Experiments 44

45. AOP 2012 Visitor Funds 45 EC chair expressed concern over allocating all of the AOP 2012 funds budgeted for visitor program due to AOP 2013 funding uncertainties NSF funds cannot be reprogrammed to cover other DTC activities, so DTC moved forward with awarding 5 new visitor projects Estimate new projects will require only ~15-20 K from the NOAA AOP 2012 funding For continued NSF support, will be important to show NOAA matching, so plan to pursue off-cycle visitor proposals consistent with DTC priorities once we have a better picture of the AOP 2013 funding Demonstrating NOAA matching for visitor program will be important for future proposals to NSF for both the visitor program and potential Post-doctoral Scientist Program

46. Potential new activities Following new activities currently fall in the 85-100 % scenario 46

47. ECMWF-like Visitor Tools ($128K) Background: ECMWF has an interactive meteorological software application that allows researchers to easily prepare & execute research experiments. Scope: Investigate current capabilities & consider options for building & expanding capabilities to easily prepare & run research experiments using NCEP’s models. Requirements analysis: Identify user community (skills, needs) Identify system components adaptable to these needs Identify hardware, software & data needs & resources Identify documentation & training needs Preliminary system design Planning requires EMC staff participation in this info gathering stage! 47

48. Post-doctoral Scientist Program ($151K) Motivation: Mechanism for engaging next generation scientists in research focused on improving operational forecasts Proposed format: Gather information from EMC on key areas for which they are interested in contributions from the research community Applicants submit proposals for projects focused on an identified key area Successful applicant would reside at EMC in order to work closely with EMC staff and gain better understanding of operational requirements 48

49. Summary Activities that engage the NWP community and keep the community informed about DTC activities are important to full- filling the DTC mission Need to make sure we don’t overlook the importance of these activities and determine which types are most effective 49