Classificatory performance evaluation of air quality forecasting in Georgia Yongtao Hu 1, M. Talat Odman 1, Michael E. Chang 2 and Armistead G. Russell 1 1 School of Civil & Environmental Engineering, 2 Brook Byers Institute of Sustainable Systems Georgia Institute of Technology 9 th Annual CMAS Conference, October 12 th, 2010
Overview The Hi-Res air quality forecasting system O 3 and PM 2.5 performance for metro Atlanta New SOA module and its impact on PM 2.5 performance Classificatory performance evaluation: linking forecast performance to weather types and emissions conditions
Georgia Institute of Technology Hi-Res: forecasting ozone and PM 2.5 at 4-km resolution for metro areas in Georgia
Georgia Institute of Technology Hi-Res Forecast Products “Single Value” Report: tomorrow’s AQI, ozone and PM 2.5 by metro area in Georgia Air Quality Forecasts: AQI, ozone and PM 2.5, 48-hrs spatial plots and station profiles Meteorological Forecasts: precipitation, temperature and winds, 48-hrs spatial plots and station profiles Performance Evaluation: time series comparison and scatter plots for the previous day Snapshots from Hi-Res homepage:
Evolution of Hi-Res during Updated to latest release of WRF each year before the ozone season. WRF 2.1, 2.2, 3.0, 3.1 and 3.2 CMAQ is typically one version behind. CMAQ 4.6 with Georgia Tech extensions Projected NEI to current year in the very beginning of each year. Updated forecast products website each year before ozone season. Switched from single-cycle forecasting to two-cycles in Enlarged 4-km domain to cover the entire state of Georgia in Introduced Georgia Tech’s new SOA module in 2009.
Ambient Monitoring Sites for Performance Evaluation
Performance Metrics False AlarmsHits Correct Nonevents Missed Exceedences Forecast Observation NAAQS
Georgia Institute of Technology Overall Performance (Ozone Season): Atlanta Metro Ozone PM 2.5 MNB16% MNE23% MNB-20% MNE34%
Ozone Performance
Georgia Institute of Technology Forecast vs. Observed O MNB14% MNE18% MNB8.5% MNE19% MNB17% MNE23% MNB28% MNE30%
2009 O 3 Performance: Hi-Res vs. GA EPD’s Our 4-km Forecast EPD Ensemble Forecast MNB28% MNE30% MNB13% MNE21% Economic down turn?
2010 O 3 Performance: Hi-Res vs. GA EPD’s Our 4-km Forecast EPD Ensemble Forecast MNB14% MNE18% MNB9% MNE17% Economic recovery?
PM 2.5 Performance
Summer
Georgia Institute of Technology Forecast vs. Observed PM MNB-37% MNE44% MNB-38% MNE42% MNB8% MNE25% MNB4% MNE21%
2009 PM 2.5 Performance: 4-km vs. GA EPD’s Our 4-km ForecastEPD Ensemble Forecast MNB8% MNE25% MNB11% MNE24% Economic down turn? New SOA module?
2010 PM 2.5 Performance: 4-km vs. GA EPD’s Our 4-km ForecastEPD Ensemble Forecast MNB4% MNE21% MNB14% MNE30% New SOA module?
New SOA Module (Baek, J., Georgia Tech, 2009) SOA partitioned from anthropogenic VOCs’ oxidations (8 SVOCs) From monoterpenes (2 SVOCs) From isoprene (2 SVOCs added) From sesquiterpenes (1 SVOC added, gas phase oxidation reactions added for α-caryphyllene, β-humulene, and other sesquiterpenes) Aging of all semi-volatile organic carbons (SVOCs) added Included processes: SOA species in CMAQ: AORGAJ and AORGAI AORGBJ and AORGBI AORGBISJ and AORGBISI AORGBSQJ and AORGBSQI AORGAGJ and AORGAGI HSVOC LSVOC Aerosol Aging process: +OH,+O3
Forecast vs. Observed OC at South DeKalb 2009 Ozone Season May-September OC PM 2.5
Met. Performance
Georgia Institute of Technology Overall Performance (Ozone Season): Atlanta Metro Temperature Humidity MB-0.46K ME1.61K MB-0.67g/kg ME1.19g/kg
Georgia Institute of Technology Forecast vs. Observed Temperature MB-1.16K ME1.90K MB-0.80K ME1.94K MB0.37K ME1.47K MB-0.19K ME1.15K
Georgia Institute of Technology Forecast vs. Observed Humidity MB-0.59g/kg ME1.22g/kg MB-0.89g/kg ME1.33g/kg MB-0.36g/kg ME0.96g/kg MB-0.73g/kg ME1.03g/kg
Classificatory Performance Evaluation during Hu, Y., et al. 2010, "Using synoptic classification to evaluate an operational air quality forecasting system in Atlanta," Atmos. Pol. Res., 1,
Spatial Synoptic Classification (Sheridan 2002, : Atlanta Calendar (ozone season) Weather Types DP (dry polar): lowest in temp, clear and dry. DM (dry moderate): air is mild and dry. DT (dry tropical): hottest and driest. MP (moist polar): cloudy, humid and cool. MM (moist moderate): warmer and more humid. MT (moist tropical): warm and very humid. TR (transitional): one type yields to another, large shifts in P,Td,V. SSCDPDMDTMPMMMTTRMISSINGTOTAL Total
Forecasting error versus observation for spatial synoptic classifications SSC DMDTMMMTTR O3O3 MO (ppb) Number of days > 75 ppb MNE %18%39%27%14% MNE %8%48%26%19% PM 2.5 MO (μg m -3 ) Number of days > 35 μg m MNE %44%47%38%42% MNE %19%30%25%34% TemperatureMO (K) ME (K) HumidityMO (g kg -1 ) ME (g kg -1 ) Wind speedMO (m s -1 ) ME (m s -1 ) *Statistics are for summers where not specified.
Georgia Institute of Technology Observations Grouped by Weather Type Ozone PM 2.5
8-hr O 3 Performance Grouped by Weather Type
24-hr PM 2.5 Performance by Weather Type
GA EPD forecasting Performance by Weather Type Ozone PM 2.5
Linking Performance to Emissions Conditions Emissions Conditions Classification: Special weekdays (Monday and Friday) Typical weekdays Weekends/holidays Ozone PM 2.5
Georgia Institute of Technology Summary ozone forecasts are good. –Overall bias is +16% and error is 23% PM 2.5 forecasts are not very accurate. –May-September bias is -37% and error is 43% The new SOA module improved 2009 and 2010 PM 2.5 performances –May-September bias is 8% and error is 25% for 2009 –May-September bias is 4% and error is 21% for temperature and humidity performances are good. Better ozone performance and, during , better PM 2.5 performance are associated with dry weather types. Forecast performance is worse on weekend/holidays
Acknowledgements We thank Georgia EPD for funding the Hi-Res forecasts, Our former group member Dr. Jaemeen Baek for the new SOA module, and Dr. Carlos Cardelino of Georgia Tech for team forecasts.
Georgia Institute of Technology Forecast vs. Observed 2006 O3 Humidity Temperature PM 2.5 MNB11% MNE29% MNB-38% MNE43% MB0.50K ME1.57K MB-0.74g/kg ME1.39g/kg
Winter
Forecasted vs. Observed PM