AN EVALUATION OF THE ETA-CMAQ AIR QUALITY FORECAST MODEL AS PART OF NOAA’S NATIONAL PROGRAM CMAQ AIRNOW AIRNOW Brian Eder* Daiwen Kang * Ken Schere* Ken Schere* Robert Gilliam* Jonathan Pleim* Atmospheric Modeling Division Atmospheric Modeling Division Air Resources Laboratory, NOAA August 26,2003 August 26,2003 * On assignment to NERL EPA * On assignment to NERL EPA RTP, NC RTP, NC 27711

Forecast Configuration -Eta Meteorology -CBIV Mechanism -SMOKE Emissions (Offline) -12 km grid resolution -22 Vertical Layers 48 Hr. Forecast (12Z Initialization) 7 July – 31 September, July – 31 August (shown) 7 July – 31 August (shown) 48 Hr.Forecast (Corrected Land-use) August 48 Hr. Forecast (Corrected Land-use) August Domain Models-3 CMAQ

This evaluation used: Hourly O 3 concentrations (ppb) from EPA’s AIRNOW network 521 stations 7 July - 31 August A suite of statistical metrics for both: discrete forecasts and categorical forecasts for the: hourly, maximum 1-hr, maximum 8-hr O 3 simulations

Two Forecast / Evaluation Types - Discrete Forecasts [Observed] versus [Forecast] - Category Forecasts (Two Category) Observed Exceedances, Non-Exceedances versus Forecast Exceedances, Non-Exceedances

Discrete Forecast / Evaluation Statistics Statistics - Summary - Regression - Biases - Errors AIRNOW [Observed] versus [Forecast]

Category Forecast / Evaluation - Two Category Forecasts Observed Exceedances, Non-Exceedances versus Forecast Exceedances, Non-Exceedances ab cd Forecast Exceedance No Yes Observed Exceedance a b c d

Category Forecast A ccuracy Percent of forecasts that correctly predict event or non-event. B ias Indicates if forecasts are under-predicted (false negatives) or over-predicted (false positives ) F alse A larm R ate Percent of times a forecast of high ozone did not occur a b c d

C ritical S uccess I ndex How well the high ozone events were predicted. P robability O f D etection Ability to predict high ozone events Category Forecast a b c d

a c a b c d a= 151 b= 1 c= 24,227 d= 4 n= 24,383 CMAQ = (AIRNOW) Max 1-hr O 3 7 July – 31 August

Summary Statistics Discrete Evaluation Categorical Evaluation [ppb]CMAQAIRNOWCMAQ = (AIRNOW) Ozone  125 ppb Mean r0.60A99.4% SD n24,383B25.5 CV Max BIASES 95 th MB15.1FAR99.4% 75 th NMB26.9%CSI0.6% 50 th th ERRORS 5 th RMSE21.9POD16.7% Min01NME31.7% Max 1- hr O 3

Temporal Evaluation – Max 1 hr O 3 7 July 1 August 31 August

Spatial Evaluation Max 1- hr O 3 Correlation 0.00 – – – – 1.00 Mean = 0.60

Max 1- hr O 3 Mean Bias Spatial Evaluation -10 – – – 50 Mean = 15.1

Spatial Evaluation Max 1- hr O 3 Root Mean Square Error 0 – – – 50 Mean = 21.9

CMAQ = (AIRNOW) a b c d a= 3276 b= 149 c= 20,979 d= 65 n= 24,469 Max 8-hr O 3

Summary Statistics Discrete Evaluation Categorical Evaluation [ppb]CMAQAIRNOWCMAQ = (AIRNOW) Ozone  85 ppb Mean r0.57A86.3% SD n24,469B16.0 CV24.5%29.7% Max BIASES 95 th MB17.6FAR95.6% 75 th NMB35.8%CSI4.2% 50 th th ERRORS 5 th RMSE23.0POD69.6% Min01NME39.1% Max 8- hr O 3

Temporal Evaluation – Max 8 hr O 3 1 August 31 August 7 July

Spatial Evaluation Max 8- hr O 3 Correlation 0.00 – – – – 1.00 Mean = 0.57

Spatial Evaluation Max 8- hr O 3 Mean Bias -10 – – – 50 Mean = 17.6

Spatial Evaluation Max-8 hr O 3 Root Mean Square Error 0 – – – 50 Mean = 23.0

Land-Use Error Land-use fields associated with Eta were being post-processed incorrectly. As a result : - Most of the domain was classified as water. - Dry deposition was greatly under simulated This error was discovered/corrected by NCEP on Sept. 9 th. This error was discovered/corrected by NCEP on Sept. 9 th. - An eight day period (12-19 August) was re-simulated. - Positive biases were cut in half, errors reduced also.

Runr MB (ppb) NMB (%) RMSE (ppb) NME (%) A (%) BFAR (%) CSI (%) POD (%) Initial Corrected Max 1-hr O 3 Max 8-hr O 3 Comparison Between Initial and Corrected Simulations August 12 – Runr MB (ppb) NMB (%) RMSE (ppb) NME (%) A (%) BFAR (%) CSI (%) POD (%) Initial Corrected

Temporal Evaluation (Corrected August 12 –19) – Max 1 hr O 3 – Max 8 hr O 3

Summary The Eta-CMAQ modeling system performed reasonably well, in this, its first attempt at forecasting ozone concentrations: Correlation: Bias: 15.1 ppb (26.9%) ppb (31.7%) Error:21.9 ppb (31.7%) ppb (39.1%) Accuracy: % An error was discovered in Eta’s post processed land-use designation that resulted in the: – under-estimation of dry deposition and – hence over-simulation of concentrations Once corrected, the positive biases and errors were greatly reduced: Correlation: Bias:7.6 ppb (13.0%) ppb (20.1%) Error:16.6 ppb (21.7%) ppb (26.3%) Accuracy: %

Contact: Brian K. Eder