1. Update on Dropout Team Work and Related COPC Action Items Presented by Dr. Bradley Ballish Co-Chair JAG/ODAA and Dropout Team* Member 16 November 2010 COPC Meeting AFWA *Including EMC’s J. Alpert, D. Carlis, Y. Ling, X. Su And NCO’s K. Kumar and J. Carr

2. November 16, 2010 COPC Dropout Update 2 Outline Recent AC scores with dropouts indicated COPC Action Item (AI) 2008-1.5 (real-time monitoring) COPC AI 2010-1.6 (data impact tests) NCEP activity summary Dropout related projects NCEP GFS implementation summary Summary

3. November 16, 2010 COPC Dropout Update 3 00Z 5-day AC Scores in September/October 2010 No GFS dropouts and GFS is number 2 in scores! FNMOC dropout Multiple dropouts for FNMOC and GFS in SHEM ECM run Scored.73

4. November 16, 2010 COPC Dropout Update 4 AI 2008-1.5 COPC Action Item 2008-1.5: Develop a monitoring system to analyze differences between the NCEP and FNMOC global models and the ECMWF global model in real- time and make this real-time system available to OPCs as a daily tool. Status Project management charter has been signed and work has started Initial operating capability at NCEP planned for February 2011 Recommend AI remain open

5. November 16, 2010 COPC Dropout Update 5 AI 2010-1.6 COPC Action Item 2010-1.6:The dropout team will provide a final summary report on the impact of observational data types suspected of causing model dropouts and make specific recommendations on data quality control and usage for operational implementation Status Team has conducted thorough analysis for many dropout occurrences More work and analysis will be needed well beyond the time frame of typical COPC action items Impact tests with satellite winds and radiances have been run (see the next four slides) –Satellite radiances generally have good impact –Satellite winds have shown some negative impact in dropout cases with GFS –Speed dependent QC and ECMWF filtering of satellite winds showed positive impact –FNMOC super obs of satellite winds show big positive impact

6. November 16, 2010 COPC Dropout Update 6 Impact of Conventional and Satellite Radiance Obs on 3-d and 5-d GFS Forecasts Retaining satellite radiance data shows small positive impact on NH 3-D and 5-D forecasts In the SH, conventional and non-radiance satellite observations (PREPB alone) show a large negative impact (8 points) in 5-D forecasts! -- another puzzle Addition of satellite radiance data to conventional and satellite observations has a positive impact on 5-D forecasts even for dropout cases yellow AMSUA along with PREPB conventional observations (yellow) show the largest positive impact in the NH and SH experiments and typically are correlated with the results of the CNTRL puzzle

7. November 16, 2010 COPC Dropout Update 7 Satellite Wind QC Experiments Satellite winds (SATWIND) are a data type of concern at EMC due to case studies that have shown negative impact by using SATWIND’s in the GDAS system. Su’s SATWND QC lowers the gross check asymmetrically, is planned for operations rather than Carlis’ QC, and its implementation is TBD Carlis’ Satwnd QC removes about 50% of the observations by deleting wind speeds that are less than the background guess CONTROL=0.804 SU’S QC=0.817 CARLIS’ QC=0.815 CONTROL=0.818 SU’S QC=0.824 CARLIS’ QC=0.823

8. November 16, 2010 COPC Dropout Update 8 By Riishojgaard, Jung and Velden Plan for operational SATWIND QC upgrades TBD

9. November 16, 2010 COPC Dropout Update 9 Notice big impact from Satellite winds FNMOC has big positive impact from satellite winds possibly from super obs

10. November 16, 2010 COPC Dropout Update 10 AI 2010-1.6 (continued) Recommend closing this AI and continue working on the data usage improvements listed below: Implement improved satellite wind QC and filtering (TBD) Aircraft data –Implement NRL track-checking code (Q2 2011)* and then improve –Aircraft temperature bias corrections –Improve waypoint dictionaries for AIREP data (Q1 2011)* –Test and implement aircraft moisture data Implement surface pressure bias corrections Use adjoint codes to analyze dropout cases and tune reject- list criterion Improve timeliness of adding bad data platforms to reject-list (Q1 2011)* Develop and implement use-list and improve methods of adding bad data platforms to reject-list * Indicates concrete implementation date

11. November 16, 2010 COPC Dropout Update 11 NCEP Activity Summary Dropouts –Data QC and data usage –Dropout analysis GFS implementation summary –GFS QC, data assimilation and model changes

12. November 16, 2010 COPC Dropout Update 12 Dropout Related Projects ProjectStatus Using Quikscat QC FlagsImplemented 2009 Dictionary CorrectionsUpper air changes in operations 2009 North American surface changes implemented in June 2010 Multicenter waypoint updates (Q1 2011)* Observational Data Impact TestsContinuous testing at NCEP plus Langland has started tests Adjoint Estimates of Observation Impact Langland has been running tests and NCEP has adjoint of GSI analysis running with help from NASA/Goddard Surface Pressure Bias CorrectionsPreliminary testing Data QC and Data Usage

13. November 16, 2010 COPC Dropout Update 13 Dropout Related Projects ProjectStatus Aircraft Temperature Bias CorrectionsWaiting for NRL aircraft QC upgrade - ECMWF to implement in 2010 Satellite Wind QC UpgradesBoth speed dependent QC and ECMWF filtering rules showed positive impact - NCEP implementation (TBD) Profile QCPreliminary investigation Improved Reject-listPreliminary investigation Use-list for Conventional ObsPreliminary testing Data QC and Data Usage

14. November 16, 2010 COPC Dropout Update 14 Dropout Related Projects (cont) Establish Dropout Investigation Procedures Is done and performed for dropout cases Dropout Relation to Data CountsRTDMS extended to 30 days, currently is in NCO testing for December 2010 implementation Dropout Climatology StudiesCommon origin areas found – Langland has shown preferred synoptic patterns ECM RunsJournal publication in progress – High resolution ECMWF input being tested Improved Diagnostics for Analysis Differences for Dropouts Preliminary investigation at both NCEP and FNMOC Dropout Analysis

15. November 16, 2010 COPC Dropout Update 15 NCEP GFS Implementation Summary GFS T574 Model BundleImplemented on 27 July 2010 Aircraft Track-CheckingImplementation of NRL code (Q2 2011)* GSI Analysis UpgradesOperational GSI updates (Q2 2011)* implementation QC, Analysis and Model Changes

16. November 16, 2010 COPC Dropout Update 16 Summary Observational data usage continues to be explored as a possible cause of dropouts –GFS model upgrade helped in NH, but less so in SH –GFS performance likely depends on the quality of each observation, not simply a particular class of data –Continue to work on improving and testing QC rules and metadata on a case by case basis Recommend AI 2010-1.6 to be closed Operational implementations –FY10 GFS model July 2010 Surface dictionary upgrades June 2010 –FY11 GSI upgrade (Q2 2011)* NRL Aircraft QC (Q2 2011)* –TBD Improved satellite wind QC and filtering implementations

17. November 16, 2010 COPC Dropout Update 17 Background Slides

18. November 16, 2010 COPC Dropout Update 18 Interagency Participants NCEP/EMC –Jordan Alpert –Yangrong Ling –DaNa Carlis NCEP/NCO –Bradley Ballish –Krishna Kumar –Joe Carr NCEP/HPC –James Cisco NRL –Rolf Langland FNMOC –Chuck Skupniewicz NESDIS –Kathy Kelly –Tom Renkevens The NCEP dropout team meets weekly with the EMC director, John Derber and sometimes with the JCSDA director and reports quarterly to the NCEP director

19. November 16, 2010 COPC Dropout Update 19 Latest Dropout Team Findings (continued) For more information on dropout research, see the reports presented at the AMS meetings: http://ams.confex.com/ams/89annual/techprogram/paper_142644.htm http://ams.confex.com/ams/89annual/techprogram/paper_142649.htm http://ams.confex.com/ams/23WAF19NWP/techprogram/paper_154268.htm http://ams.confex.com/ams/23WAF19NWP/techprogram/paper_154282.htm

20. November 16, 2010 COPC Dropout Update 20 Based on the code from the NASA/Goddard/GMAO GSI, the dropout team has the GSI analysis adjoint code running Further work is needed to develop an adjoint of the NCEP global model that with the analysis adjoint will produce estimates of how every observation helps or hurts the forecast skill We will determine if smart statistics from these promising new tools will give us more precision on how to better use observations, such as: –Satellite radiance data impact estimates by type, channel, surface conditions (water, ice, etc.) and more –Possibly better rules on filtering of satellite winds –Improved reject-list criterion –Possibly better analysis of dropout cases Adjoint Codes

21. November 16, 2010 COPC Dropout Update 21 AI 2008-1.5 Official Action Plan (Revised) NCEP will generate warnings on a restricted website when the GFS global analysis has extreme localized differences from ECMWF analyses on a 1x1 degree grid NCEP will develop a real-time warning system to alert us when the GFS global model 5-day forecast height correlations with the ECMWF exceed normal limits, which will give an early warning of a possible dropout NCEP website will have graphics to allow staff to study the divergence in forecasts in real time and show likely analysis problem areas