1 Climate Test Bed Seminar Series 24 June 2009 Bias Correction & Forecast Skill of NCEP GFS Ensemble Week 1 & Week 2 Precipitation & Soil Moisture Forecasts.

Slides:



Advertisements
Similar presentations
Verification of NCEP SFM seasonal climate prediction during Jae-Kyung E. Schemm Climate Prediction Center NCEP/NWS/NOAA.
Advertisements

Seasonal Climate Predictability over NAME Region Jae-Kyung E. Schemm CPC/NCEP/NWS/NOAA NAME Science Working Group Meeting 5 Puerto Vallarta, Mexico Nov.
1 Seasonal Forecasts and Predictability Masato Sugi Climate Prediction Division/JMA.
Experimental Real-time Seasonal Hydrologic Forecasting Andrew Wood Dennis Lettenmaier University of Washington Arun Kumar NCEP/EMC/CMB presented: JISAO.
The NCEP operational Climate Forecast System : configuration, products, and plan for the future Hua-Lu Pan Environmental Modeling Center NCEP.
Seamless precipitation prediction skill in a global model: Actual versus potential skill Matthew Wheeler 1, Hongyan Zhu 1, Adam Sobel 2, and Debra Hudson.
MOS Performance MOS significantly improves on the skill of model output. National Weather Service verification statistics have shown a narrowing gap between.
1 Assessment of the CFSv2 real-time seasonal forecasts for Wanqiu Wang, Mingyue Chen, and Arun Kumar CPC/NCEP/NOAA.
The Long Journey of Medium-Range Climate Prediction Ed O’Lenic, NOAA-NWS-Climate Prediction Center.
Warm Season Precipitation Predictions over North America with the Eta Regional Climate Model Model Sensitivity to Initial Land States and Choice of Domain.
The North American Monsoon System: Recent Evolution and Current Status Update prepared by Climate Prediction Center / NCEP 05 August 2013.
Performance of the MOGREPS Regional Ensemble
Rongqian Yang, Ken Mitchell, Jesse Meng Impact of Different Land Models & Different Initial Land States on CFS Summer and Winter Reforecasts Acknowledgment.
CPC Unified Gauge – Satellite Merged Precipitation Analysis for Improved Monitoring and Assessments of Global Climate Pingping Xie, Soo-Hyun Yoo,
CPC Forecasts: Current and Future Methods and Requirements Ed O’Lenic NOAA-NWS-Climate Prediction Center Camp Springs, Maryland ,
1 The Asian-Australian Monsoon System: Recent Evolution, Current Status and Prediction Update prepared by Climate Prediction Center / NCEP April 01, 2013.
EUROBRISA Workshop – Beyond seasonal forecastingBarcelona, 14 December 2010 INSTITUT CATALÀ DE CIÈNCIES DEL CLIMA Beyond seasonal forecasting F. J. Doblas-Reyes,
Regional Climate Simulations of summer precipitation over the United States and Mexico Kingtse Mo, Jae Schemm, Wayne Higgins, and H. K. Kim.
1 The Asian-Australian Monsoon System: Recent Evolution, Current Status and Prediction Update prepared by Climate Prediction Center / NCEP May 31, 2011.
Combining CMORPH with Gauge Analysis over
Rongqian Yang, Kenneth Mitchell, Jesse Meng NCEP Environmental Modeling Center (EMC) Summer and Winter Season Reforecast Experiments with the NCEP Coupled.
Southern Hemisphere: Weather & Climate over Major Crops Areas Update prepared by Climate Prediction Center / NCEP 8 Mar 2011 For Real-time information:
. Outline  Evaluation of different model-error schemes in the WRF mesoscale ensemble: stochastic, multi-physics and combinations thereof  Where is.
1 Objective Drought Monitoring and Prediction Recent efforts at Climate Prediction Ct. Kingtse Mo & Jinho Yoon Climate Prediction Center.
3. Products of the EPS for three-month outlook 1) Outline of the EPS 2) Examples of products 3) Performance of the system.
Northern Hemisphere: Weather & Climate over Major Crop Areas Update prepared by Climate Prediction Center / NCEP 2 May 2011 For Real-time information:
Development of Precipitation Outlooks for the Global Tropics Keyed to the MJO Cycle Jon Gottschalck 1, Qin Zhang 1, Michelle L’Heureux 1, Peitao Peng 1,
NOAA’s Climate Prediction Center & *Environmental Modeling Center Camp Springs, MD Impact of High-Frequency Variability of Soil Moisture on Seasonal.
The lower boundary condition of the atmosphere, such as SST, soil moisture and snow cover often have a longer memory than weather itself. Land surface.
1 The Asian-Australian Monsoon System: Recent Evolution, Current Status and Prediction Update prepared by Climate Prediction Center / NCEP May 27, 2014.
Recent and planed NCEP climate modeling activities Hua-Lu Pan EMC/NCEP.
The North American Monsoon System: Recent Evolution and Current Status Update prepared by Climate Prediction Center / NCEP 03 June 2013.
The North American Monsoon System: Recent Evolution and Current Status Update prepared by Climate Prediction Center / NCEP 01 July 2013.
Exploring the Possibility to Forecast Annual Mean Temperature with IPCC and AMIP Runs Peitao Peng Arun Kumar CPC/NCEP/NWS/NOAA Acknowledgements: Bhaskar.
Seasonal Predictability of SMIP and SMIP/HFP In-Sik Kang Jin-Ho Yoo, Kyung Jin, June-Yi Lee Climate Environment System Research Center Seoul National University.
Nathalie Voisin 1, Florian Pappenberger 2, Dennis Lettenmaier 1, Roberto Buizza 2, and John Schaake 3 1 University of Washington 2 ECMWF 3 National Weather.
Intercomparison of US Land Surface Hydrologic Cycles from Multi-analyses & Models NOAA 30th Annual Climate Diagnostic & Prediction Workshop, 27 October,
Report on CTB CFS Test and Evaluation Team Activities Team Leads: Jae-Kyung Schemm and Shrinivas Moorthi CPC and EMC, NCEP/NWS/NOAA 32nd Climate Diagnostics.
1 The Asian-Australian Monsoon System: Recent Evolution, Current Status and Prediction Update prepared by Climate Prediction Center / NCEP March 12, 2012.
1 The Asian-Australian Monsoon System: Recent Evolution, Current Status and Prediction Update prepared by Climate Prediction Center / NCEP June 09, 2014.
Meteorology 485 Long Range Forecasting Friday, February 13, 2004.
Multi Model Ensembles CTB Transition Project Team Report Suranjana Saha, EMC (chair) Huug van den Dool, CPC Arun Kumar, CPC February 2007.
1 An Assessment of the CFS real-time forecasts for Wanqiu Wang, Mingyue Chen, and Arun Kumar CPC/NCEP/NOAA.
Huug van den Dool and Suranjana Saha Prediction Skill and Predictability in CFS.
Near-Surface Climate Extremes in the Past 50+ Years Yun Fan & Huug van den Dool CPC/NCEP/NOAA NOAA 32th Annual Climate Diagnostic & Prediction Workshop.
DOWNSCALING GLOBAL MEDIUM RANGE METEOROLOGICAL PREDICTIONS FOR FLOOD PREDICTION Nathalie Voisin, Andy W. Wood, Dennis P. Lettenmaier University of Washington,
VERIFICATION OF A DOWNSCALING SEQUENCE APPLIED TO MEDIUM RANGE METEOROLOGICAL PREDICTIONS FOR GLOBAL FLOOD PREDICTION Nathalie Voisin, Andy W. Wood and.
31 st CDPW Boulder Colorado, Oct 23-27, 2006 Prediction and Predictability of Climate Variability ‘Charge to the Prediction Session’ Huug van den Dool.
1 The Asian-Australian Monsoon System: Recent Evolution, Current Status and Prediction Update prepared by Climate Prediction Center / NCEP September 26,
1 Yun Fan, Huug van den Dool, Dag Lohmann, Ken Mitchell CPC/EMC/NCEP/NWS/NOAA Kunming, May, 2004.
1 An Assessment of the CFS real-time forecasts for Wanqiu Wang, Mingyue Chen, and Arun Kumar CPC/NCEP/NOAA.
1 The Asian-Australian Monsoon System: Recent Evolution, Current Status and Prediction Update prepared by Climate Prediction Center / NCEP July 12, 2010.
1 The Asian-Australian Monsoon System: Recent Evolution, Current Status and Prediction Update prepared by Climate Prediction Center / NCEP June 14, 2010.
Long-lead streamflow forecasts: 2. An approach based on ensemble climate forecasts Andrew W. Wood, Dennis P. Lettenmaier, Alan.F. Hamlet University of.
Land surface memory & hydrological cycle over the U.S. west coast states & monsoon region Yun Fan and Huug van del Dool CPC/NCEP/NOAA
Predictability of Monthly Mean Temperature and Precipitation: Role of Initial Conditions Mingyue Chen, Wanqiu Wang, and Arun Kumar Climate Prediction Center/NCEP/NOAA.
Seasonal Climate Forecasting for Applied Use in the Western USA Katherine Hegewisch 1, Renaud Barbero 2, John Abatzoglou 1 1 University of Idaho, Department.
1 A review of CFS forecast skill for Wanqiu Wang, Arun Kumar and Yan Xue CPC/NCEP/NOAA.
1 The Asian-Australian Monsoon System: Recent Evolution, Current Status and Prediction Update prepared by Climate Prediction Center / NCEP December 27,
1 Summary of CFS ENSO Forecast December 2010 update Mingyue Chen, Wanqiu Wang and Arun Kumar Climate Prediction Center 1.Latest forecast of Nino3.4 index.
1/39 Seasonal Prediction of Asian Monsoon: Predictability Issues and Limitations Arun Kumar Climate Prediction Center
1 The Asian-Australian Monsoon System: Recent Evolution, Current Status and Prediction Update prepared by Climate Prediction Center / NCEP June 27, 2011.
Mingyue Chen, Wanqiu Wang, and Arun Kumar
Nathalie Voisin, Andy W. Wood and Dennis P. Lettenmaier
Progress in Seasonal Forecasting at NCEP
Predictability of Indian monsoon rainfall variability
The Importance of Reforecasts at CPC
Precipitation variability over Arizona and
Raw plume forecast data
Environment Canada Monthly and Seasonal Forecasting Systems
Presentation transcript:

1 Climate Test Bed Seminar Series 24 June 2009 Bias Correction & Forecast Skill of NCEP GFS Ensemble Week 1 & Week 2 Precipitation & Soil Moisture Forecasts Yun Fan & Huug van den Dool Acknowledgement: Jae Schemm, John Janowiak, Doug Lecomte, Jin Huang, Pingping Xie, Viviane Silva, Peitao Peng, Vern Kousky, Wayne Higgins

2 Outline Motivation Methodology Performance of NCEP GFS Week1 & Week2 Ensemble Precipitation Forecasts Analysis of Week1 & Week2 Biases & Errors Application: land model forced with bias corrected week1 week2 P & T2m forecast Future Work

3 History of Soil Moisture “Dynamical” Outlook CPC Leaky Bucket Hydrological Model Forced With Week 1 & Week 2 GFS Forecasts Single member HR MRF (started around 1997 & CONUS) Ensemble GFS (started late 2001 & CONUS) Bias corrected Ensemble GFS (started late 2003 & CONUS) Bias corrected Ensemble GFS (started late 2007 & global land) : The prediction skill of soil moisture crucially depends on our ability to predict precipitation Early stage ( both good and bad comments )  Recent years ( more & more good comments ) So its time to verify & quantify: daily GFS ensemble week 1 & week 2 precip forecast skills & statistics

4 The quality of soil moisture prediction largely or almost entirely depends on the quality of precipitation prediction

5 Daily bias correction based on last 30 (or 7) day forecast errors Week1 Future Week2 Today Past Last 30 day 1/N Σ [ P f (week1) – Po (week1) ] = Bias1 1/N Σ [ P f (week2) – Po (week2) ] = Bias2 P f : GFS ensemble week1 & week2 precip forecast Po: Observed week1 & week2 precip from CPC daily global Unified Precip N = ( 30, 7..….)

6 North America Seasonal cycle with Large day to day fluctuation On 0.5x0.5 obs grid

7 South America On 0.5x0.5 obs grid

8 Asia-Australia On 0.5x0.5 obs grid

9 Africa On 0.5x0.5 obs grid

10 How good is GFS? On 0.5x0.5 obs grid Seasonal cycle with Large day to day fluctuation

11 On 0.5x0.5 obs grid How good is GFS? Seasonal cycle with Large day to day fluctuation

12 Comparison (based on last 30-day forecast errors) Obs grids (regrid model grids to 0.5x0.5 obs grids) Model grids (regrid obs grids to 2.5x2.5 model grids) Question: Does grid matter for skills assessment ?

13 Skill does not depend much on the grid

14 Comparison bias corrected skills (based on last 30-day forecast errors) bias corrected skills (based on last 7-day forecast errors) Question: Does the bias estimate influence skill ? Week1 Future Week2 Today Past Last 30 day 1/N Σ [ P f (week1) – Po (week1) ] = Bias1 1/N Σ [ P f (week2) – Po (week2) ] = Bias2 P f : GFS ensemble week1 & week2 precip forecast Po: Observed week1 & week2 precip from CPC daily global Unified Precip N = ( 30, 7..….)

15 1) Skill depends on the definition of bias 2) 30-day bias correction better than 7-day bias correction

16 Comparison bias corrected skills (based on last 30-day forecast errors) raw forecast skills (no bias correction applied) Question: Does bias correction improve skill in terms of Spatial Correlation and RMSE ?

17 Bias correction is time & location dependent

18 Bias correction helps everywhere

19 Table 1. Averaged (May 1, 2008 – June 7, 2009) spatial correlations over different monsoon regions Week 1Week 2 Bias Correction No Bias Correction Bias Correction No Bias Correction North America South America Asia Australia Africa Table 2. Averaged (May 1, 2008 – June 7, 2009) RMSE over different monsoon regions (unit: mm/week) Week 1Week 2 Bias Correction No Bias Correction Bias Correction No Bias Correction North America South America Asia Australia Africa The effectiveness of bias correction is mainly space dependent. Bias correction can correct spatial distribution of P f & reduce its error. Similarity of P f & P o Distance of P f & P o Reduced by 28% Reduced by 23% Increased by 80% Increased by 67%

20 CONUS 30-day running mean

21 In terms of Spatial Anomaly Correlation, bias correction helps : 1) very little over North America 2) considerably over South America & Africa 3) a little over Asia-Australia In terms of RMSE: Bias correction helps everywhere Questions: Why bias correction works but varies in space and time? What biases look like? Are biases removable & to what extent are they removable?

22 Temporal-spatial structures of last 30-day biases: Daily Bias1 & Bias2 used to correct GFS ensemble week1 & week2 forecasts Week1 Future Week2 Today Past Last 30 day 1/30 Σ [ P f (week1) – Po (week1) ] = Bias1 1/30 Σ [ P f (week2) – Po (week2) ] = Bias2

23 Annual Mean Bias or Raw Forecast Error Week-1 mean Bias Week-2 mean Bias

24 Mean Bias of Daily R2 & Observed Precip ( )

25 summer winter

26 winter summer

27 Temporal-spatial structures of last 30-day biases: Daily Bias1 & Bias2 used to correct GFS ensemble week1 & week2 forecasts Week1 Future Week2 Today Past Last 30 day 1/30 Σ [ P f (week1) – Po (week1) ] = Bias1 1/30 Σ [ P f (week2) – Po (week2) ] = Bias2 Large-scale & low-frequency (annual or semi-annual cycles) are prominent First two EOF modes of Bias1 & Bias2 explain about 60% total variances GFS has prominent annual cycle errors (lesson for model development?)

28 Temporal-spatial structures of real time raw forecast errors: Daily GFS week1 & week2 forecast errors without bias correction Week1 Future Week2 Today Past Last 30 day 1/30 Σ [ P f (week1) – Po (week1) ] = Bias1 1/30 Σ [ P f (week2) – Po (week2) ] = Bias2 P f (week1) – Po (week1) = Error1 P f (week2) – Po (week2) = Error2 No bias correction applied

29

30

31 Temporal-spatial structures of real time raw forecast errors: Daily GFS week1 & week2 forecast errors without bias correction Week1 Future Week2 Today Past Last 30 day 1/30 Σ [ P f (week1) – Po (week1) ] = Bias1 1/30 Σ [ P f (week2) – Po (week2) ] = Bias2 P f (week1) – Po (week1) = Error1 P f (week2) – Po (week2) = Error2 No bias correction applied Raw forecast errors are dominated by the 1 st, 2 nd or 3 rd EOFs in Bias1 & Bias2 First two EOF modes of Error1 & Error2 explain about 23~35% total variances At least this amount of error is removable. But so far bias correction was not done by EOF analysis

32 Temporal-spatial structures of real time forecast errors: GFS week1 & week2 forecast errors with last 30-day bias correction Week1 Future Week2 Today Past Last 30 day 1/30 Σ [ P f (week1) – Po (week1) ] = Bias1 1/30 Σ [ P f (week2) – Po (week2) ] = Bias2 P f (week1) – Po (week1) = Error1 P f (week2) – Po (week2) = Error2 Bias correction: Error1 = Error1 – Bias1 Error2 = Error2 – Bias2

33 Annual Mean Forecast Error after bias correction 5 times smaller than mean bias or raw forecast error Week-1 mean forecast error Week-2 mean forecast error

34

35

36 Temporal-spatial structures of real time forecast errors: GFS week1 & week2 forecast errors with last 30-day bias correction Week1 Future Week2 Today Past Last 30 day 1/30 Σ [ P f (week1) – Po (week1) ] = Bias1 1/30 Σ [ P f (week2) – Po (week2) ] = Bias2 P f (week1) – Po (week1) = Error1 P f (week2) – Po (week2) = Error2 Bias correction: Error1 = Error1 – Bias1 Error2 = Error2 – Bias2 Bias Corrected Forecast Errors are much more random (in time mainly, EOFs more “white”). Leading EOF modes of Bias1, Bias2, & Error1, Error2 Show that GFS has prominent large-scale & low- frequency errors or GFS has difficulty to reproduce those observed Precip patterns & their evolution. However, to some extent they can be corrected through bias correction, especially in winter season.

37 Application Soil Moisture “Dynamical” Outlook CPC Leaky Bucket Hydrological Model Forced With Week-1 & Week-2 GFS Ensemble Forecasts (Daily data from 01Nov2003 to present) All initial conditions & verification datasets are from leaky bucket model forced with daily observed P & T2m

38 Some Thoughts: Once this (SST, w) was the lower boundary…. Both SST and w have (high) persistence Old ‘standard’ in meteorology: If you cannot beat persistence ….. For instance: dw/dt = P – E - R = F or w(t+1)=w(t) + F Clearly if we do not know F with sufficient skill, the forecast loses against persistence (F=0).

39

40

41

42 30-day running mean P1=0.9511, C1= PR1=16.27, FR1=18.02 P2=0.9015, C2= PR2=23.67, FR2=26.56

43 30-day running mean Precip

44 Even moderate forecast skill at right time still help a lot

45 30-day running mean for week-2 Hybrid persistence = week-1 forecast persists to week-2

46 Moderate week-1 & week-2 GFS P forecast skills Last 30-d biases dominated by low-frequency & large-scale errors Bias corrections are time & location dependent Soil moisture forecast skill hardly beats its persistence over CONUS The inability to outperform persistence relates to the skill of precipitation not being above a threshold (AC>0.5 is required) Summary

47 Is PDF bias correction better? GFS Week3 & Week4 Precip Assessment GFS hindcasts? How about New CFSRR? Future Work