1. 1 Kalman filter, analog and wavelet postprocessing in the NCAR-Xcel operational wind-energy forecasting system Luca Delle Monache lucadm@ucar.edu Research Applications Laboratory National Center for Atmospheric Research WORKSHOP ON ENVIRONMETRICS – 15 October 2010, NCAR

2. 2 ACKNOWLEDGMENTS NCAR: Aime Fournier, Yubao Liu, Gregory Roux UBC: Thomas Nipen, Roland Stull

3. 3 OUTLINE Ensemble and Kalman filtering (KF) to improve Numerical Weather and Air Quality Predictions New methods based on KF and an analog approach (ANKF, AN) Tests of KF, ANKF, and AN to correct 10-m wind speed Wavelet filtering for hub-height winds Summary

4. 4 A Kalman filter bias correction for deterministic and probabilistic ozone predictions Summer of 2004 (56 days) 8 photochemical models 360 ozone surface stations Sources: Delle Monache et al. (JGR, 2006b) Delle Monache et al. (Tellus B, 2008) Djalalova et al. (Atmospheric Environment, 2010)

5. 5 Time t = 0 day -1 day -2 day -6 day -5 day -4 day -3 day -7 OBS PRED KF-weight KF

6. 6 Ensemble averaging and Kalman Filtering effects on systematic and unsystematic RMSE components RMSE decomposition (Willmott, Physical Geography 1981):, a and b least-squares regression coefficients of P i and O i

7. 7 Kalman Filtering effects on probabilistic prediction reliability

8. 8 Kalman Filtering effects on probabilistic prediction resolution (6 %)(0.1 %)

9. 9 Time t = 0 day -1 day -2 day -6 day -5 day -4 day -3 day -7 OBS PRED KF-weight KF ANKF AN “Analog” Space day -4 day -7 day -5 day -3 day -2 day -1 day -6 PRED OBS farthest analog closest analog NOTE This procedure is applied independently at each observation location and for a given forecast time

10. 10 How to find analogs? (1) We can define a metric: Where, N var is the number of variable to compute the “distance” between and w var is the weight given to each variable while computing the metric is the standard deviation of the set with 0 ≤ t ≤ t 0 is the half-width of the time window over which differences are computed

11. 11 How to find analogs? (2) We can define a metric: tntn t n +1 t i -1 t 0 = 0 Time t n -1 titi t i +1

12. 12 How to find analogs? (3) We can define a metric: tntn t n +1 t n -1 titi t i +1

13. 13 How to find analogs?!? (4) We can define a metric: tntn t n +1 t n -1 titi t i +1

14. 14 WRF Model physics: Lin et al. microphysics YSU for PBL Monin-Oboukov for SL Kain-Fritsch CUP (Domain 1/2) Noah Land Surface Model Modeling settings for wind predictions Project: Wind energy predictions (Sponsor: Xcel Energy) D1: 30 km, 128x114 D2: 10 km, 253x232 D3: 3.3 km, 541x571 NOTE: 37 vertical levels, with 12 levels in the lowest 1-km D2 D1 D3 Source: Delle Monache et al. (Submitted to MWR, 2010)

15. 15 Analog-based methods: improvements relative to KF

16. 16 Wind speed: statistics as function of time 6 months period 500 surface stations Variables: u, v, T, P, Q @ surface

17. 17 Wind speed: Bias (m s -1 ) as function of space

18. 18 Sensitivity to data set length NWP model: MM5 1 year period 22 surface stations in NM Variables: u, v, T @ surface Courtesy of Josh Hacker (of NPS) and Daran Rife (of NCAR)

19. 19 Wavelet filtering Wavelet Components Time Series Raw Pred OBS Corrected Pred Sum Smoothest Component 15 min. 30 min. 1 h 2 h 4 h

20. Wavelet filtering: RMSE and Correlation at Cedar Creek Wind Farm 230 days 4 turbines Variables to search analogs: spd, dir, T, P, Q @ surface

21. 21 SUMMARY Ensemble and Kalman filtering (KF) to improve Numerical Predictions (AQ: ozone, PM; MET: wind, T, Td, sfc pressure, etc.) New methods based on KF and an analog approaches (ANKF, AN) Test KF, ANKF, and AN to correct 10-m wind speed –ANKF and AN beat KF over a range of metrics –ANKF and AN gain vs KF grows with length of data set The combination of ANKF and AN with a wavelet decomposition improve predictions when dealing with noisy data (@ a wind farm)

22. 22 Thanks!(lucadm@ucar.edu)

23. 23 Source: Djalalova et al. (Atmospheric Environment, 2010) A Kalman filter bias correction for deterministic and probabilistic PM 2.5 predictions

24. 24 f n is a forecast at time t n and at a given location, with t n > t 0 is a metric to measure the “distance” between f n and A i {A i } is a set of “analog” forecasts at a time t i, with t i < t 0 –{A i } are ordered with respect to d i : d i -1 > d i, and We can now introduce the Kalman filter bias correction procedure as follows: The true unknown forecast bias at time t n can me modeled by And the actual forecast error can be expressed as KF in analog space (1)

25. 25 The optimal recursive predictor of x t can be written as Where K, the “Kalman gain” is And p, the expected mean square error is NOTE: The system of equations is closed by: first running the filter for (with constant) KF in analog space (2)

26. 26 “Global” statistics

27. 27 Wind speed: CRMSE (m s -1 ) as function of space

28. 28 Wind speed: Rank correlation as function of space

29. 29 ANKF…how does it work?

30. 30