Ensemble-4DWX update: focus on calibration and verification

Slides:



Advertisements
Similar presentations
Chapter 13 – Weather Analysis and Forecasting
Advertisements

Report of the Q2 Short Range QPF Discussion Group Jon Ahlquist Curtis Marshall John McGinley - lead Dan Petersen D. J. Seo Jean Vieux.
A Brief Guide to MDL's SREF Winter Guidance (SWinG) Version 1.0 January 2013.
Statistical post-processing using reforecasts to improve medium- range renewable energy forecasts Tom Hamill and Jeff Whitaker NOAA Earth System Research.
Comparing and Contrasting Post-processing Approaches to Calibrating Ensemble Wind and Temperature Forecasts Tom Hopson Luca Delle Monache, Yubao Liu, Gregory.
Gridded OCF Probabilistic Forecasting For Australia For more information please contact © Commonwealth of Australia 2011 Shaun Cooper.
Instituting Reforecasting at NCEP/EMC Tom Hamill (ESRL) Yuejian Zhu (EMC) Tom Workoff (WPC) Kathryn Gilbert (MDL) Mike Charles (CPC) Hank Herr (OHD) Trevor.
1 Kalman filter, analog and wavelet postprocessing in the NCAR-Xcel operational wind-energy forecasting system Luca Delle Monache Research.
Recent performance statistics for AMPS real-time forecasts Kevin W. Manning – National Center for Atmospheric Research NCAR Earth System Laboratory Mesoscale.
Towards an Ensemble Forecast Air Quality System for New York State Michael Erickson 1, Brian A. Colle 1, Christian Hogrefe 2,3, Prakash Doraiswamy 3, Kenneth.
Brian Ancell, Cliff Mass, Gregory J. Hakim University of Washington
JEFS Status Report Department of Atmospheric Sciences University of Washington Cliff Mass, Jeff Baars, David Carey JEFS Workshop, August
Consortium Meeting June 3, Thanks Mike! Hit Rates.
Ensemble Post-Processing and it’s Potential Benefits for the Operational Forecaster Michael Erickson and Brian A. Colle School of Marine and Atmospheric.
A Regression Model for Ensemble Forecasts David Unger Climate Prediction Center.
Performance of the MOGREPS Regional Ensemble
Rank Histograms – measuring the reliability of an ensemble forecast You cannot verify an ensemble forecast with a single.
Tutorial. Other post-processing approaches … 1) Bayesian Model Averaging (BMA) – Raftery et al (1997) 2) Analogue approaches – Hopson and Webster, J.
NOAA’s National Weather Service National Digital Forecast Database: Status Update LeRoy Spayd Chief, Meteorological Services Division Unidata Policy Committee.
Ensemble Predictions: Understanding Uncertainties Lars Landberg Wind Energy Department Risø National Laboratory DTU Denmark Gregor Giebel, Jake Badger,
Mesoscale model support for the 2005 MDSS demonstration Paul Schultz NOAA/Earth System Research Laboratory Global Systems Division (formerly Forecast Systems.
MODEL OUTPUT STATISTICS (MOS) TEMPERATURE FORECAST VERIFICATION JJA 2011 Benjamin Campbell April 24,2012 EAS 4480.
Short-Range Ensemble Prediction System at INM José A. García-Moya SMNT – INM 27th EWGLAM & 12th SRNWP Meetings Ljubljana, October 2005.
Model Post Processing. Model Output Can Usually Be Improved with Post Processing Can remove systematic bias Can produce probabilistic information from.
2006(-07)TAMDAR aircraft impact experiments for RUC humidity, temperature and wind forecasts Stan Benjamin, Bill Moninger, Tracy Lorraine Smith, Brian.
Refinement and Evaluation of Automated High-Resolution Ensemble-Based Hazard Detection Guidance Tools for Transition to NWS Operations Kick off JNTP project.
Quantile regression as a means of calibrating and verifying a mesoscale NWP ensemble Tom Hopson 1 Josh Hacker 1, Yubao Liu 1, Gregory Roux 1, Wanli Wu.
© 2009 UCAR. All rights reserved. ATEC-4DWX IPR, 21−22 April 2009 National Security Applications Program Research Applications Laboratory Ensemble-4DWX.
1 NUOPC National Unified Operational Prediction Capability Update to COPC 4 – 5 Nov 2015 Dave McCarren, NUOPC DPM.
Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss A more reliable COSMO-LEPS F. Fundel, A. Walser, M. A.
An Examination Of Interesting Properties Regarding A Physics Ensemble 2012 WRF Users’ Workshop Nick P. Bassill June 28 th, 2012.
Fly - Fight - Win 2 d Weather Group Mr. Evan Kuchera HQ AFWA 2 WXG/WEA Template: 28 Feb 06 Approved for Public Release - Distribution Unlimited AFWA Ensemble.
Nathalie Voisin 1, Florian Pappenberger 2, Dennis Lettenmaier 1, Roberto Buizza 2, and John Schaake 3 1 University of Washington 2 ECMWF 3 National Weather.
NATIONAL CENTER FOR ATMOSPHERIC RESEARCH The NCAR/ATEC Operational Mesoscale Ensemble Data Assimilation and Prediction System – “Ensemble-RTFDDA” Yubao.
Bias Correction of RTFDDA Surface Forecasts Presented by: Daran Rife National Center for Atmospheric Research Research Applications Laboratory Boulder,
Judith Curry James Belanger Mark Jelinek Violeta Toma Peter Webster 1
Fly - Fight - Win 2 d Weather Group Mr. Evan Kuchera HQ AFWA 2 WXG/WEA Template: 28 Feb 06 Approved for Public Release - Distribution Unlimited AFWA Ensemble.
DOWNSCALING GLOBAL MEDIUM RANGE METEOROLOGICAL PREDICTIONS FOR FLOOD PREDICTION Nathalie Voisin, Andy W. Wood, Dennis P. Lettenmaier University of Washington,
VERIFICATION OF A DOWNSCALING SEQUENCE APPLIED TO MEDIUM RANGE METEOROLOGICAL PREDICTIONS FOR GLOBAL FLOOD PREDICTION Nathalie Voisin, Andy W. Wood and.
CWB Midterm Review 2011 Forecast Applications Branch NOAA ESRL/GSD.
1 Summary of CFS ENSO Forecast September 2010 update Mingyue Chen, Wanqiu Wang and Arun Kumar Climate Prediction Center 1.Latest forecast of Nino3.4 index.
ATEC-4DWX Project 6 October 2008 Jason Knievel, lead scientist Keith Searight, lead engineer Scott Swerdlin, project manager.
Findings and Recommendations from the Hydrologic Verification System Requirements Team Peter Gabrielsen, Julie Demargne, Mary Mullusky, Kevin Werner, Bill.
Predicting Intense Precipitation Using Upscaled, High-Resolution Ensemble Forecasts Henrik Feddersen, DMI.
Google Meningitis Modeling Tom Hopson October , 2010.
Tom Hopson, NCAR (among others) Satya Priya, World Bank
Ensemble Forecasting: Calibration, Verification, and use in Applications Tom Hopson.
Update on the Northwest Regional Modeling System 2013
University of Washington Ensemble Systems for Probabilistic Analysis and Forecasting Cliff Mass, Atmospheric Sciences University of Washington.
Verifying and interpreting ensemble products
Tom Hopson, Jason Knievel, Yubao Liu, Gregory Roux, Wanli Wu
Precipitation Products Statistical Techniques
Dan Petersen Bruce Veenhuis Greg Carbin Mark Klein Mike Bodner
Model Post Processing.
Northwest Modeling Consortium December 3, 2013
Winter storm forecast at 1-12 h range
Nathalie Voisin, Andy W. Wood and Dennis P. Lettenmaier
Post Processing.
CAPS Real-time Storm-Scale EnKF Data Assimilation and Forecasts for the NOAA Hazardous Weather Testbed Spring Forecasting Experiments: Towards the Goal.
Google Meningitis Modeling
The Importance of Reforecasts at CPC
New Developments in Aviation Forecast Guidance from the RUC
Integration of NCAR DART-EnKF to NCAR-ATEC multi-model, multi-physics and mutli- perturbantion ensemble-RTFDDA (E-4DWX) forecasting system Linlin Pan 1,
SPECS: Climate Prediction for Climate Services
Verification of Tropical Cyclone Forecasts
MOGREPS developments and TIGGE
Extra-Tropical Storm Surge (ETSS 2.1)
Forecast system development activities
Rapid Adjustment of Forecast Trajectories: Improving short-term forecast skill through statistical post-processing Nina Schuhen, Thordis L. Thorarinsdottir.
WRAP 2014 Regional Modeling
Presentation transcript:

Ensemble-4DWX update: focus on calibration and verification Tom Hopson, Josh Hacker, Jason Knievel, Yubao Liu, Gregory Roux, Wanli Wu National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, Aug 11-12 2009

Reminder of status in April 2009 Running WRF V3.0.1.1 Had just finished running E-4DWX for ATC during Dec 2008 and Jan 2009, then returned configuration to DPG Configured and ran E-4DWX for supporting UT Division of Air Quality Improved MYJ and YSU PBL height diagnosis and PBL mixing Added new graphics and improve post-processing flexibility (installation for GMOD and plotting historical case archive) and computing parallelisms Fully automated calibration just being tested Calibrated variables: 2-m T Aberdine Myj eta ; ysu korean scheme Gmod - global model on demand (nsap) National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, Aug 11-12 2009

Progress since April 2009 (1 of 2) Improved scripting to manage the computational resources Increased stations from 26 to 34 (regional) Calibration hindcast increased from roughly 600 to over 1000 points Increased hindcast size allows calibration of 2 additional lead-times (36hr, 42hr) Included new skill measure for ensemble skill-spread relationship Corrected over-dispersion of calibration process Calibrated variables: 2-m T, 2-m Td (new) National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, Aug 11-12 2009

Progress since April 2009 (2 of 2) Examined results from CAD study at ATC (earlier talk) Using performance of ensemble members to optimize DPG’s single-model configuration Submitted proposal for brief paper in BAMS but was rejected. Have not decide whether to revise and resubmit Continuing to work on scientific paper for MWR Downtime of HPC (recent): May 13-20, Agami4 failure June 16-18, SGI node July 11-19, power outage caused Agami4 failure July 29 - August 4, Agami4 failure Cad - clear air damming study National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, Aug 11-12 2009

Calibration and verification Ensemble calibration to correct predicted distribution Calibration is needed for users capable of decision making with probabilistic guidance. Will be needed for foreseeable future Verification of different ensemble characteristics is easily completed when performing calibration National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, Aug 11-12 2009

Calibration: review and recap “bias” obs Forecast PDF Probability Probability Forecast PDF obs “spread” or “dispersion” calibration Temperature [K] Temperature [K] Calibration (and verification) is now fully-automated Utilizes “persistence” if available 34 sites in and near DPG Full calibration for all sites ~ 1X per week for each weather variable Using lookup tables ~ every hour (8 hrs, was 1hr) National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, Aug 11-12 2009

3-hr dewpoint time series Station DPG S01 Before Calibration After Calibration National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, Aug 11-12 2009

42-hr dewpoint time series Station DPG S01 Before Calibration After Calibration National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, Aug 11-12 2009

PDFs: raw vs. calibrated Blue is “raw” ensemble Black is calibrated ensemble Red is the observed value Notice: significant change in both “bias” and dispersion of final PDF (also notice PDF asymmetries) obs ATEC-4DWX IPR, Aug 11-12 2009

Troubled rank histograms 0 10 20 30 Counts 0 10 20 30 Counts 1 2 3 4 5 6 7 8 9 10 Ensemble # 1 2 3 4 5 6 7 8 9 10 Ensemble # ATEC-4DWX IPR, Aug 11-12 2009

3-hr dewpoint rank histograms Station DPG S01 National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, Aug 11-12 2009

42-hr dewpoint rank histograms Station DPG S01 National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, Aug 11-12 2009

RMSE of ensemble members Station DPG S01 3hr Lead-time 42hr Lead-time National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, Aug 11-12 2009

Significant calibration regressors Station DPG S01 3hr Lead-time 42hr Lead-time National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, Aug 11-12 2009

Future plans Refine lookup table process to decrease computation time (required as calibration data size continues to increase and new weather variables are added) Implement for a) wind speed, b) wind direction, c) precipitation, d) surface pressure Diagnose most informative model set to use operationally Develop scheme for model points without surface observations over whole model domain National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, Aug 11-12 2009