1. Towards an Ensemble Forecast Air Quality System for New York State Michael Erickson 1, Brian A. Colle 1, Christian Hogrefe 2,3, Prakash Doraiswamy 3, Kenneth Demerjian 3, Winston Hao 2, Mark Beauharnois 3, Jia-Yeong Ku 2, and Gopal Sistla 2 1 School of Marine and Atmospheric Sciences, Stony Brook University, Stony Brook, NY 2 New York State Department of Environmental Conservation, Albany, NY 3 Atmospheric Sciences Research Center, State University of New York at Albany, Albany, NY

2. Motivations and Goals -Project Goal: Develop an air quality ensemble forecast system to aid operational forecasters for New York State. -Motivation: Could errors in the atmospheric models impact air quality forecast simulations? Can these errors be corrected via post-processing? -Goal of this talk: Evaluate the air quality models (AQM) and Stony Brook (SBU) ensemble with a focus on similar biases and errors within each ensemble. -Future: Use a post-processing technique called Bayesian Model Averaging (BMA) to improve the deterministic and probabilistic forecasts within the ensembles.

3. Air Quality Model (AQM) Ensemble: CAMx, CAMQ Emissions Inventory: NYSDEC, EPA Ensemble of Air Quality Model Forecasts Ensemble Air Quality Model Flowchart Atmospheric Model Ensemble: SBU, NCEP NAM, ASRC, NYSDEC

4. AQM Operational Ensemble Members *Currently two SBU members are run in the operational AQM ensemble. Retrospective simulations used all SBU members except those with the Ferrier microphysics. **ASRC model was not run in the retrospective simulations. Member Name Met.Emis. Inv.AQMGrid Res Initial- ize Start Date NCEP_12zWRF- NMM EPACMAQv 4.6 12-km12z Summer 2004; Winter 2004- 2005; everyday since June 2005 NCEP_00zWRF- NMM EPACMAQv 4.6 12-km00z May 2008 SBU*MM5/ WRF NYSDECCMAQv 4.6 36-km, 12-km 00z June 2008 NYSDEC_3xWRF- NMM NYSDECCMAQv 4.6 12-km00z November 2008 ASRC**WRF- ARW NYSDECCAMxv 4.5.1 12-km00z March 2009

5. Synoptic Setup 8-hr Max. Ozone Daily Total PM 2.5 150 100 70 40 30 20 10 0 100 80 60 40 20 0 1-hr max PM 2.5 AQI Categories 1-hr Max. Ozone 50 40 30 20 10 0 130 115 100 85 70 55 30 Operational AQM Example - 8/18/2009 ASRC Member - http://asrc.albany.edu/research/aqf/aqfms/camx/mfb.php

6. Data and Methods Retrospective simulations of particulate matter 2.5 and ozone were verified over following time periods: - June 4, 2008 – July 22, 2008 - December 1, 2008 – February 28, 2009 Regions 1, 2, and 7 were selected to represent coastal, urban and inland New York, respectively. AQM output was compared against daily 8-hr maximum ozone and 24-hr average PM 2.5 model predictions from the AIRNOW database and official NYSDEC forecasts. To elucidate potential error sources in the AQM ensemble, the SBU 10-m wind speed and 2-m temperature were verified with ASOS observations over the same time period. NYSERDA Regions SBU Ensemble Domain

7. AQM Retrospective Simulations SBU Ensemble Members F2 and F9 were used to drive CMAQ forecasts each day since June 1, 2008. They were selected based on temperature and wind verification results for summer 2007 and operational considerations. Two additional SBU members use the Ferrier microphysics scheme that is currently not compatible with CMAQ.

8. Ozone Retrospective Simulations Time Series – 6/4/08 to 7/22/08 Model simulations generally track observations (in red) well. 100 80 60 40 20 0 100 80 60 40 20 0 100 80 60 40 20 0 NCEP NAM 12z NCEP NAM 00z NYSDEC 3x SBU MM5 BMMY-GFS SBU MM5 GRMRF-NAM SBU MM5 GRMY-NAM SBU MM5 GRBK-NGPS SBU MM5 KFMY-CMC SBU MM5 KFMRF-GFS SBU WRF KFMY-CMC SBU WRF BMMY-NAM SBU WRF KFMY-GFS SBU MM5 GRBK-GFS SBU WRF BMYSU-NGPS SBU WRF KFMY-GFS Ensemble Avg Ensemble Median DEC Forecast

9. Ozone Retrospective Simulations Bias and RMSE – 6/4/08 to 7/22/08 4 0 -4 -8 4 0 -4 -8 4 0 -4 -8 12 8 4 0 12 8 4 0 12 8 4 0 Ozone is underpredicted by SBU MM5 members and overpredicted by most remaining models. RMSE varies between members, with the ensemble mean/median outperforming individual members. NCEP NAM 12z NCEP NAM 00z NYSDEC 3x SBU MM5 BMMY-GFS SBU MM5 GRMRF-NAM SBU MM5 GRMY-NAM SBU MM5 GRBK-NGPS SBU MM5 KFMY-CMC SBU MM5 KFMRF-GFS SBU WRF KFMY-CMC SBU WRF BMMY-NAM SBU WRF KFMY-GFS SBU MM5 GRBK-GFS SBU WRF BMYSU-NGPS SBU WRF KFMY-GFS Ensemble Avg Ensemble Median DEC Forecast MM5 WRF Mean

10. PM 2.5 Retrospective Simulations Time Series – 12/1/08 to 2/28/09 Model simulations generally track observations (in red) well. 100 80 60 40 20 0 100 80 60 40 20 0 100 80 60 40 20 0 NCEP NAM 12z NCEP NAM 00z NYSDEC 3x SBU MM5 BMMY-GFS SBU MM5 GRMRF-NAM SBU MM5 GRMY-NAM SBU MM5 GRBK-NGPS SBU MM5 KFMY-CMC SBU MM5 KFMRF-GFS SBU WRF KFMY-CMC SBU WRF BMMY-NAM SBU WRF KFMY-GFS SBU MM5 GRBK-GFS SBU WRF BMYSU-NGPS SBU WRF KFMY-GFS Ensemble Avg Ensemble Median DEC Forecast

11. PM 2.5 Retrospective Simulations Bias and RMSE – 12/1/08 to 2/28/09 10 5 0 -5 10 5 0 -5 10 5 0 -5 15 10 5 0 15 10 5 0 15 10 5 0 PM is overpredicted for region 2 but underpredicted for region 7 and all other inland stations (not shown). NCEP members exhibit the least amount of bias overall. The WRF SBU members exhibit greater negative bias than the MM5 SBU. NCEP NAM 12z NCEP NAM 00z NYSDEC 3x SBU MM5 BMMY-GFS SBU MM5 GRMRF-NAM SBU MM5 GRMY-NAM SBU MM5 GRBK-NGPS SBU MM5 KFMY-CMC SBU MM5 KFMRF-GFS SBU WRF KFMY-CMC SBU WRF BMMY-NAM SBU WRF KFMY-GFS SBU MM5 GRBK-GFS SBU WRF BMYSU-NGPS SBU WRF KFMY-GFS Ensemble Avg Ensemble Median DEC Forecast MM5 WRF Mean

12. Ozone and PM forecasts are “L” shaped (biased) or “U” shaped (underdispersed). Biases and dispersion issues have also been noted in the SBU ensemble and may be negatively affecting the AQM. Therefore it is important to verify the SBU ensemble in juxtaposition with the AQM. Retrospective Simulations - Rank Histograms Winter Particulate MatterSummer Ozone

13. SBU/AQM Ensemble Comparison – Temperature Ozone and Bias – 6/4/08 to 7/22/08 SBU Ensemble AQI Ensemble The cooler, shallower and cloudier simulated PBL in the MM5 MY scheme is likely resulting in lower model ozone. This affect may be offset in one MY member by the KF convective scheme, which has been shown to decrease cloudiness and increase simulated ozone. (Tao et al. 2008). The MYJ WRF members have greater ozone concentrations than MY MM5, which could be the result of a higher PBL growth within the MYJ scheme. (Zielonka et al. 2008). oCoC oCoC oCoC

14. SBU/AQM Ensemble Comparison – Temperature PM 2.5 and Bias – 12/1/08 to 2/28/09 SBU Ensemble AQI Ensemble The MM5 members using the Reisner microphysics have more PM than those using Simple Ice. PM sensitivity to cloud microphysics schemes have also been noted in Meij et al. 2009. Lower WRF PM concentrations have been noted compared to MM5 (Meij et al 2009) due to the increase of vertical mixing within WRF caused by warmer surface temperatures. oCoC oCoC oCoC

15. SBU/AQM Ensemble Comparison – Rank Histogram Summer Ozone AQM Region 7 Winter PM 2.5 AQM Region 7 Winter Wind SBU Region 7 Summer Temp. SBU Region 7 After bias correction, the SBU ensemble is underdispersed for temperature and wind speed in all regions. The AQI ensemble also appears to be underdispersed in the absence of biases, suggesting that a lack of variability in atmospheric forecasts could affect the air quality models. Post-processing techniques, such as Bayesian Model Averaging (BMA), could help correct this lack of variability in ensemble forecasting.

16. Post-Processing - Bayesian Model Averaging  Bayesian Model Averaging (BMA, Raftery et al. 2005) has been shown to correct some model deficiencies associated with reliability and dispersion.  BMA creates a probability density function (PDF) for each ensemble member depending on the uncertainty in the model forecast and weights the result based on its performance and uniqueness in the recent past.  The main advantages of BMA appear to be with probabilistic skill, although deterministic skill is also increased.  An example using the 24 hour temperature forecast from the SBU ensemble will be presented. PDF for Temperature PDF for Wind Speed BMA weights each member based on past performance and assigns an uncertainty.

17. BMA Region 1 Region 7 Region 2 Region 1 Region 7 Region 2 Bias Corrected BMA BMA Example – Temperature Hour 24 Rank Histogram- Warm Season 2007-2009

18. BMA Example – Temperature Hour 24 Reliability > 295 K- Warm Season 2007-2009 Region 1 Region 7 Region 2

19. Conclusions An operational air quality forecast ensemble is currently being run using a variety of atmospheric models, air quality models (AQM) and pollutant emission inventories. Particulate matter and ozone simulations track observations reasonably well in the warm and cool seasons, although the ensemble exhibits systematic biases and underdispersion. Ensemble biases may be sensitive to the PBL parameterization, with the decreased (increased) vertical mixing within the MY (YSU) scheme resulting in lower (higher) ozone and higher (lower) PM forecasts. Bayesian model averaging (BMA) has been shown to correct dispersion and improve reliability for 2-m temperature and 10-m wind speed within the SBU ensemble. Therefore BMA could improve AQM forecasts through direct application or insertion of the post-processed SBU forecasts into the AQM ensemble.