1. Precipitation Products Statistical Techniques
Tom Hopson, NCAR

2. Outline “Quantile-to-Quantile Mapping” (Q2Q) bias correction technique
“Quantile Regression” (QR) corrections
Multi-model ensemble generation
“Schaake Shuffle” for flood prediction applications
Preserving spatial covariances
Preserving temporal autocovariances

3. Forecast “calibration” or “post-processing”
“bias”
obs
Forecast
PDF
Probability
Probability
Forecast
PDF
obs
“spread” or “dispersion”
calibration
Flow rate [m3/s]
Flow rate [m3/s]
Post-processing has corrected:
the “on average” bias
as well as under-representation of the 2nd moment of the empirical forecast PDF (i.e. corrected its “dispersion” or “spread”)
Our approaches:
“quantile-to-quantile mapping” and under-utilized “quantile regression” approach
probability distribution function “means what it says”
daily variation in the ensemble dispersion directly relate to changes in forecast skill => informative ensemble skill-spread relationship

4. ECMWF 51-member Ensemble Precipitation Forecasts
5 Day Lead-time Forecasts
=> Lots of variability
2004 Brahmaputra Catchment-averaged Forecasts
black line satellite observations
colored lines ensemble forecasts
-Basic structure of catchment rainfall similar for both forecasts and observations
-But large relative over-bias in forecasts
Discharge forecasts employ a blend of precipitation forecasts (ECMWF) and near-real-time “observations” (GPCP or CMORPH)
Precipitation “observations” used to initialize discharge model conditions (soil-moisture, lagged subcatchment discharges, etc.) before the forecast period.
GPCP and CMORPHprecipitation “observations” based on IR and microwave satellite data, but distinctly different approaches. (Note: potential systematic error in GPCP in 2003 data shown here)
Adjust ECMWFcatchment-average ensemble forecasts to satisfy similar distribution as the “observations” using the criterion of the uniformity of the verification rank; method similar to that proposed method of Hamill and Colucci, 1997

5. Specific Necessity of Post-Processing Weather Forecasts for Hydrologic Forecasting Applications
Hydrologic forecast model calibration can often implicitly remove biases in input weather variables (i.e. precipitation)
However, if you use one product (i.e. satellite rainfall) to calibrate your hydrologic model, but use *more* than one product (i.e. satellite rainfall and numerical weather prediction rainfall) or weather forecasts at different lead-times (with different biases for each lead-time) to generate hydrologic forecasts, then biases *between* each product or forecast lead-time must be removed
This is because hydrologic model calibration cannot (implicitly) remove all biases of all input weather products simultaneously
Statistical post-processing can improve not only statistical (unconditional) accuracy of forecasts (as measured by reliability diagrams, rank histograms, etc), but also factors more related to “conditional” forecast behavior (as measured by, say, RPS, skill-spread relations,etc.)
Last point: inexpensive statistically-derived skill improvements can equate to significant NWP developments (expensive)

6. Forecast Bias Adjustment
done independently for each forecast grid
(bias-correct the whole PDF, not just the median)
Model Climatology CDF
“Observed” Climatology CDF
Pmax
Pmax
Precipitation
Pfcst
Padj
25th
50th
75th
100th
25th
50th
75th
100th
Quantile
Quantile
In practical terms …
ranked forecasts
ranked observations
Precipitation 1m
Precipitation 1m

7. Bias-corrected Precipitation Forecasts
Original Forecast
Brahmaputra Corrected Forecasts
Corrected Forecast
=> Now observed precipitation within the “ensemble bundle”

8. Outline “Quantile-to-Quantile Mapping” (Q2Q) bias correction technique
“Quantile Regression” (QR) corrections
Multi-model ensemble generation
“Schaake Shuffle” for flood prediction applications
Preserving spatial covariances
Preserving temporal autocovariances

9. Quantile Regression (QR)
Our application
Combining rainfall forecasts from 5 centers: CMA, CMC, CPTECH, ECMWF, NCEP conditioned on:
Ensemble mean of each center
Ranked forecast ensemble

10. PDFs: raw vs. calibrated
Blue is “raw” ensemble
Black is calibrated ensemble
Red is the observed value
Notice: significant change in both “bias” and dispersion of final PDF
(also notice PDF asymmetries)
obs

11. Outline “Quantile-to-Quantile Mapping” (Q2Q) bias correction technique
“Quantile Regression” (QR) corrections
Multi-model ensemble generation
“Schaake Shuffle” for flood prediction applications
Preserving spatial covariances
Preserving temporal autocovariances

12. A Cautionary Warning about using Probabilistic
Precipitation Forecasts in Hydrologic Modeling
(Importance of Maintaining Spatial and Temporal Covariances
for Hydrologic Forecasting => one option: “Schaake Shuffle”)
River catchtment A
ensemble1
ensemble2
ensemble3
subC
subB QC QB QA
QA same
For all 3 possible
ensembles
Scenario for
smallest possible
QA? No.
Scenario for
average QA?
Scenario for
largest possible
QA? No.

13. Early May 2011, floods in southwestern Africa
-- examine ens forecasts … NCEP GEFS 5day precip

14. Outline Overview of stations Regenerating time-series autocorrelation
“Schaake Shuffle”
Results for select stations
Difference histograms
Autocorrelation
Error histograms
Summary

15. Re-generating Temporal Autocorrelation of Ensembles Sampled from Analog PDF’s using Historical Observational Sequences
Slide from Demargne, Brown, Adams, Wood

16. Re-generating Temporal Autocorrelation of Ensembles Sampled from Analog PDF’s using Historical Observational Sequences (cont)
Slide from Demargne, Brown, Adams, Wood

17. “I have a very strong feeling that science exists to serve human welfare. It’s wonderful to have the opportunity given us by society to do basic research, but in return, we have a very important moral responsibility to apply that research to benefiting humanity.” Dr. Walter Orr Roberts (NCAR founder)
“I have a very strong feeling that science exists to serve human welfare. It’s wonderful to have the opportunity given us by society to do basic research, but in return, we have a very important moral responsibility to apply that research to benefiting humanity.” Walter Orr Roberts

18. Summary