Gridded OCF Probabilistic Forecasting For Australia For more information please contact © Commonwealth of Australia 2011 Shaun Cooper and Timothy Hume Centre for Australian Weather and Climate Research Conclusions Acknowledgements Sloughter, J. M., Raftery, A., Gneiting, T., and Fraley, C. (2007). Probabilitistic quantitative precipitation forecasting using Bayesian model averaging. Monthly Weather Review, 135:3209–3320. Thanks to Philip Riley for advice and ideas for probabilistic rainfall forecasting. The original PME rainfall forecasts in the Bureau of Meteorology were developed by Elizabeth Ebert and Chermelle Engel. References Diagram illustrating the method used to produce probabilistic forecasts from the Gridded OCF system. Gridded Operational Consensus Forecasting (Gridded OCF) is an operational forecast guidance system used by the Australian Bureau of Meteorology. It consists of a Poor Man's Ensemble (PME) of Numerical Weather Prediction (NWP) output from a number of international centres. The PME members are bias corrected with respect to a gridded mesoscale analysis (MSAS) which covers the Australian domain (but precipitation forecasts are not currently bias corrected). A range of products are then generated from the PME members, including: Weighted average consensus forecasts (used as deterministic guidance by weather forecasters) Probabilistic forecasts of particular parameters exceeding specified thresholds This poster describes the probabilistic products produced by the Gridded OCF system. Gridded OCF: Introduction Probabilistic Forecasts: “Vote Counting” The “vote counting” method counts the number of PME members which satisfy a certain condition (e.g. rainfall exceeding a specified threshold), and express the probability as a proportion of the total number of members. These probabilities will not generally be calibrated. Probability of daily precipitation exceeding 1 mm computed using the “vote counting” method. 24 hour forecast valid at T00:00:00Z. Colours indicate percentages ranging from 0% (dark purple) to 100% (light pink). Probability of the wind speed exceeding 34 knots (gale force winds). 24 hour forecast valid at T00:00:00Z. Colours indicate percentages ranging from 0% (dark purple) to 100% (light pink) This method is easily applied to a range of variables. For example: Precipitation exceeding defined thresholds Wind speed exceeding critical speeds Temperature (cold events, heat waves and so on) Examples Calibration of Rainfall Forecasts Probabilistic forecasts generated using the “vote counting” method (see above) are not usually calibrated. That is, the observed frequency of an event turns out to be different than the forecast probability of the event occurring. To calibrate precipitation forecasts, the preceding year of uncalibrated forecasts covering the Australian states of Victoria and New South Wales are compared with daily rainfall analyses valid at the same time. A look-up table is constructed to convert uncalibrated probabilities to calibrated probabilities. The calibration tables are continuously updated using the most recent year of data. Comparison of un-calibrated (top panel) with calibrated (bottom panel) forecasts of the probability of daily precipitation exceeding 0.2 mm (made using the “vote counting method”). 24 hour forecast valid at T00:00Z. Colours indicate percentages ranging from 0% (dark purple) to 100% (light pink). Example of a daily rainfall analysis (constructed from rain gauge observations) for Analyses such as these are used for calibrating probabilities derived using the “vote counting” method (see panel above), and also in the QPF method (see panel to the right). Colours indicate the 24 hour accumulation from 0 mm (grey) to 80 mm (pink-white). No data available over the sea. Probabilistic Forecasts: QPF Method This method assigns probabilities of rainfall exceeding a threshold based on a quantitative precipitation forecast (QPF). The higher the QPF, the higher the probability assigned to exceeding a defined threshold. The method is based on work by Sloughter et al. (2007) which employs Bayesian Model Averaging. It can also be applied to a single deterministic rainfall forecast. In our work, we apply it to the PME average forecast; that is, we average the QPFs from each ensemble member and use the resulting average QPF to derive the probability that the daily precipitation will exceed defined thresholds. Example: The figure shows the relationship between the PME's average QPF and the probability of precipitation exceeding 0.2 mm over the Australian states of Victoria and New South Wales. The curve was derived by comparing one year of historical QPF forecasts with the corresponding daily rainfall analyses (taken as the “truth”) for the year. Probability of rainfall exceeding 0.2 mm computed using the QPF method. 24 hour forecast valid at T00:00Z. Colours indicate percentages ranging from 0% (dark purple) to 100% (light pink). Probabilistic forecasts generated using the QPF method are calibrated A continuous range of probabilities are generated, whereas the vote counting method produces a discrete number of possible probabilities based on the number of ensemble members Verification Reliability (top) and ROC (bottom) diagrams for 24 hour forecasts of probability of precipitation exceeding 0.2 mm (vote counting method) Reliability (top) and ROC (bottom) diagrams for 24 hour forecasts of probability of precipitation exceeding 0.2 mm (QPF method) Forecasts of the probability of precipitation exceeding 0.2mm over the Australian states of Victoria and New South Wales were prepared for the period 1 September 2009 through 29 May 2011 inclusive using both the calibrated “vote counting” and QPF methods. The probabilistic forecasts were verified against the operational daily rainfall analyses (see earlier panel), and reliability and relative operating characteristics (ROC) curves created. It is clear from the diagrams (vote counting results on the left, QPF method on the right) that both methods produce reasonably reliable forecasts. The area beneath the ROC curve is approximately the same for both forecast methods, indicating they have comparable skill. Two methods of generating probabilistic forecasts from a “Poor Man’s Ensemble” were investigated The “vote counting” method is simple, and can be easily applied to a variety of meteorological parameters such as rainfall, wind speed and temperature. A weakness of the “vote counting” method is that the probability forecasts are “quantised”. For example, if the PME has 4 members, only probabilities of 0, 25, 50, 75 and 100% are possible. The “vote counting” method does not usually produce calibrated probability forecasts. An extra calibration phase is required. The QPF method generates a continuous range of probabilities, even if there is only a single member in the PME. Forecasts made using the QPF method are calibrated. However, there is not much difference in forecast skill (as shown by the ROC and reliability diagrams) between the QPF and “vote counting” methods. Future Work Apply the probabilistic forecasting methods to other types of weather, such as the probability of extreme temperature events. Improved calibration techniques.