1. CMAQ Multi-Pollutant Response Surface Modeling: Applications of an Innovative Policy Support Tool CMAS Conference – October 17, 2006 Session 2: Analysis Methods and Tools USEPA/OAQPS - Sharon Phillips, Bryan Hubbell, Carey Jang, Pat Dolwick, Norm Possiel, Tyler Fox

2. Outline Overview of Response Surface Model (RSM)  Need for the RSM  Define RSM Development of multi-pollutant RSM using CMAQ Steps in designing CMAQ RSM applications  Experimental design  CMAQ – SMOKE interface development  Evaluation & Validation RSM outputs / Visual Policy Analyzer CMAQ RSM application results Next steps

3. Need for Response Surface Model Growing importance of AQ models in guiding and supporting policy analysis and implementation for complex AQ issues such as PM, O 3, and air toxics Enormous computational costs (time & resources) always present a challenge for time pressing need of policy analysis Current model operation in comparing the efficacy of various control strategies and policy scenarios is typically inefficient, if not ineffective An innovative policy support tool to address these issues in an economical manner is needed

4. What is a Response Surface Model? Response Surface Model (RSM) is a “reduced form” model of a complex air quality model (e.g. CMAQ) – “meta-model” Based on a systematically selected set of model runs, statistical techniques can be used to represent the relationship between model inputs and outputs (e.g. emissions control and concentrations of PM & ozone) Once the “response surface” has been generated, it can be used to simulate the functions of the more computationally expensive photochemical air quality model Cross-validation can then be conducted to examine the validity of RSM to represent model responses Can also be used to derive analytical representations of model sensitivities to changes in model inputs

5. How the RSM can be used Strategy design and assessment screening tool  Comparison of urban vs. regional controls  Comparison across sectors  Comparison across pollutants Optimization  Can be used to develop optimal combinations of controls to attain standards at minimum cost “What If?” Analyses  provide real-time predictions of model responses to model inputs  Quickly provide insights into questions for policy design, e.g. does is it matter whether regional controls are put in place before local controls? Model sensitivity  Can be used to systematically evaluate the relative sensitivity of modeled ozone and PM levels to changes in emissions/met inputs

6. Elements in the Development of a RSM Policy objectives Model requirements  Geographic scale  Output metrics Experimental design  Selection of policy factors  Specification of air quality model simulations Air quality modeling simulations Statistical analysis and development of predictive model Visualization software tool

7. Assess Policy Objectives Determine Output Metrics Determine Relevant Policy Factors Develop Experimental Design Run Modeling Experiments Select Base Inventory Develop Emissions Summaries Process Model Outputs Build Predictive Model Develop Response Surface Summary Metrics Evaluate Relative Effectiveness of Policy Factors Model Validation for PM Response Metrics Development Plan for Response Surface Model Development of Visual Policy Analysis Tool

8. RSM Pilot Applications: History RSM for O 3 using CAMx VOC controls vs. NOx controls RSM for PM 2.5 using REMSAD (Strategy Comparison) VOC controls NOx controls

9. An Initial RSM Application using CMAQ

10. Development of CMAQ RSM Applications Determine policy objectives Experimental design  Selection of policy factors Emission control factors Regional vs. urban control  Selection of air quality model simulations Continental U.S. modeling, 2010 CAIR Base, 36-km grid resolution 210 runs (in 3 stages) for 4 months (Feb., April, July, Dec.) CMAQ – SMOKE Interface Development  Develop a module within CMAQ to read directly the pre-merged SMOKE sector files (e.g., 3-D point, 2-D mobile, etc.)  Allow RSM to directly control % changes of (1) emissions and (2) specified areas Output Metrics  Response Surface Variables Validation and Evaluation  Cross-validation  Out-of-sample validation

11. Policy Objectives 1. Provide a modeling surrogate tool that can quickly simulate the PM and ozone impacts for a variety of control strategies for use in Regulatory Impact Analyses (e.g., PM NAAQS, O3 NAAQS)  For screening level estimates of the impacts of control strategies on NAAQS design values  For use in generating screening estimates of the health benefits of reductions in PM and ozone precursors 2. Provide air quality simulation tool for use in the Air Strategy Assessment Program (ASAP), a screening tool under development that evaluates the relative air quality impacts, costs, and health benefits of controlling emissions from different sources

12. Experimental Design: (1) Selection of control factors 12 emission control factors selected based on precursor emissions & source category relevant to policy analysis of interest Run # (1) NOx / EGU (2) NOx / Non EGU_ Point+ Area (3) NOx / Mobile (4) SOx / EGU (5) SOx / Non EGU_ Point (6) SOx / Area (7) VOC / All (8) NH3 / Area (9) NH3 / Mobile (10) POC& PEC / EGU+ Non EGU (11) POC& PEC / Mobile (12) POC& PEC / Area X1X2X3X4X5X6X7X8X9X10X11X12 11.000001.0000001.00000 1.0000001.00000 20.891771.0202860.830540.963020.8759180.902770.170620.174350.993880.3943870.908960.87647 30.575720.2660101.021900.556100.2476030.803280.617501.001130.024580.5016421.128490.52022 40.061340.4620600.505560.399240.9646531.137360.819320.669330.494750.6882371.081550.49674 5. 211 0.29622. 0.607127.. 0.74570. 0.06589.. 0.581318.. 0.43089. 1.05862. 1.07534. 0.63847. 0.913331.. 0.51173. 0.64017.

13. Factors Provide Reasonable Aggregation Source groupings with smaller contributions to emissions are grouped with similar larger source groupings for efficiency –NonEGU Area NOx and SO2 sources are primarily smaller industrial combustions sources such as coal, oil, and natural gas powered boilers and internal combustion engines –Agricultural area sources are only significant contributors to ammonia emissions VOC sources are lumped together because VOCs are not expected to influence PM levels significantly Combined Omitted Combined Omitted

14. Covers from zero to 120 percent of baseline emissions Staged Latin Hypercube (space filling design) 210 total runs, 120 runs in first stage, 60 runs in stage two and 30 boundary condition runs  Will allow testing of additional predictive power of additional model runs 30 additional model validation runs Experimental Design: (1) Selection of control factors

15. Regional vs. Urban control: independent response surfaces for 9 urban areas, as well as a generalized response surface for the rest of model domain  Nine urban areas include: NY/Philadelphia, Chicago, Atlanta, Dallas, San Joaquin, Salt Lake City, Phoenix, Seattle, and Denver  Selected so that ambient PM2.5 in each urban area is largely independent of the precursor emissions in all other included urban areas Experimental Design: (1) Selection of control factors

16. Run # (1) NOx / EGU (2) NOx / Non EGU+ Area (3) NOx / Mobile (4) SOx / EGU (5) SOx / Non EGU Point (6) SOx / Area (7) VOC / All (8) NH3 / Area (9) NH3 / Mobile (10) POC& PEC / EGU+ NonEGU (11) POC& PEC / Mobile (12) POC& PEC / Area X1X2X3X4X5X6X7X8X9X10X11X12 11.000001.0000001.00000 1.0000001.00000 20.891771.0202860.830540.963020.8759180.902770.170620.174350.993880.3943870.908960.87647 30.575720.2660101.021900.556100.2476030.803280.617501.001130.024580.5016421.128490.52022 40.061340.4620600.505560.399240.9646531.137360.819320.669330.494750.6882371.081550.49674 5. 211 0.29622. 0.607127. 0.74570. 0.06589. 0.581318. 0.43089. 1.05862. 1.07534. 0.63847. 0.913331. 0.51173. 0.64017. Run # (1) NOx / EGU (2) NOx / Non EGU+ Area (3) NOx / Mobile (4) SOx / EGU (5) SOx / Non EGU Point (6) SOx / Area (7) VOC / All (8) NH3 / Area (9) NH3 / Mobile (10) POC& PEC / EGU+ NonEGU (11) POC& PEC / Mobile (12) POC& PEC / Area X1X2X3X4X5X6X7X8X9X10X11X12 11.000001.0000001.00000 1.0000001.00000 20.891771.0202860.830540.963020.8759180.902770.170620.174350.993880.3943870.908960.87647 30.575720.2660101.021900.556100.2476030.803280.617501.001130.024580.5016421.128490.52022 40.061340.4620600.505560.399240.9646531.137360.819320.669330.494750.6882371.081550.49674 5. 211 0.29622. 0.607127. 0.74570. 0.06589. 0.581318. 0.43089. 1.05862. 1.07534. 0.63847. 0.913331. 0.51173. 0.64017. Region A (9 urban areas) Region B (rest of domain) CMAQ – SMOKE Interface: Timely & Efficient Development of Model-ready Emissions

17. CMAQ model simulations  CMAQ v4.4; 14 vertical layers  Domain = Continental U.S. 36-km CAIR modeling domain  4 months, one from each season, February, April, July, October (months selected to provide best prediction of quarterly mean) Baseline Emissions Data  CAIR 2010 Base Case  Includes Tier 2, Heavy Duty Diesel Engines, and Nonroad Diesel standards, as well as the NOx SIP Call and MACT standards Experimental Design: (2) Selection of Model Simulations CMAQ Modeling Domain

18. Output Metrics (Response Variables) Quarterly mean and annual 98 th percentile daily average: sulfate, nitrate, crustal, elemental carbon, organic carbon, ammonium PM2.5 annual and daily design values (at monitored locations) Annual/Seasonal nitrogen and sulfate deposition Visibility (light extinction): annual mean, average of 20% worst days, average of 20% best days Ozone summer averages for:  8hr max, 1hr max, 5hr avg, 8hr avg, 12hr avg, 24hr avg Ozone 8-Hour design values (at monitored locations)

19. RSM Validation and Evaluation Cross-validation  for each RSM iteration, one of the model runs was left out, the RSM is computed and used to predict the omitted run  RSM predicted changes in AQ are compared with CMAQ predictions and the mean square error (MSE) over all grid cells was computed for the run Out-of-sample validation  30 additional CMAQ runs were conducted (not part of the experimental design and were not used in developing RSM)  RSM predictions for these model runs were compared with the CMAQ predictions and the MSE over all grid cells was computed for each run

20. July total PM2.5 mass (sample of 700 grid cells) Cross-Validation: Comparison of RSM Predicted to CMAQ “true” Values for July PM 2.5 mass Performance Metric Cross Validation (n=121) MeanMinimumMaximum Mean Bias (μg/m 3 ) 0.000-0.0630.130 Mean Error (μg/m 3 ) 0.0270.0060.130 Mean Normalized Bias (%) 0.02%-1.58%2.96% Mean Normalized Error (%) 0.71%0.21%2.97% Mean Fractional Bias (%) 0.01%-1.61%2.87% Mean Fractional Error (%) 0.71%0.22%2.88% ** based on an evenly geographically distributed sub-sample of 700 grid cells, out of ~6,300 in the continental U.S.

21. October total PM2.5 mass (sample of 700 grid cells) Cross-Validation: Comparison of RSM Predicted to CMAQ “true” Values for October PM2.5 mass Performance Metric Cross Validation (n=121) MeanMinimumMaximum Mean Bias (μg/m 3 ) 0.000-0.1000.221 Mean Error (μg/m 3 ) 0.0470.0070.221 Mean Normalized Bias (%) 0.03%-2.70%6.40% Mean Normalized Error (%) 1.19%0.18%6.73% Mean Fractional Bias (%) 0.01%-1.61%6.40% Mean Fractional Error (%) 1.19%0.18%6.40% ** based on an evenly geographically distributed sub-sample of 700 grid cells, out of ~6,300 in the continental U.S.

22. Similarity of Geographic Patterns of Predicted PM2.5 (mean total) changes for October based on Run 120 CMAQRSM

23. RSM Graphical Tool: Visual Policy Analyzer Graphical analysis tool to allow for “real- time” RSM predictions of ozone, PM, visibility, and deposition Continuous improvements are implemented to the user interface and functions

24. 2-Way Response Surfaces for Chicago NR NOx NR VOC Onroad NOx Onroad VOC

25. RSM Visualization Tool: 3-D View Mode

26. Quick re-cap RSM can analyze air quality changes in 9 urban areas and associated counties independently of one another For each urban area:  Input to RSM: % local or regional reduction for one or more of the 12 factors  Output from RSM: Estimated changes in air quality at peak monitor in each county on an annual and daily basis Gridded air quality changes across urban area To estimate regional emission reductions reduces the regional emission reduction % in the entire rest of US, which is outside of the 9 urban areas

27. VPA example: Monitors with annual average PM 2.5 Post CAIR 2015 greater than 13 µg/m 3

28. VPA example: Monitors with annual average PM 2.5 Post CAIR 2015 greater than 13 µg/m 3 after applying 50 percent reduction in carbon

29. Example of Air Quality Impacts: “Regionality” of SO 2 vs. “Locality” of Carbon Carbon SO 2

30. Key Local Factors are Carbon, EGU SO2, NonEGU SO2, VOC, and NH3 Key Regional Factors are EGU SO2, NonEGU SO2, Area NH3, and Carbon Availability of controls may limit the ability to achieve the desired percent reductions in specific sources and pollutants that would result in reductions in ambient PM2.5 levels to meet the attainment targets.

31. Relative effectiveness per ton in reducing ambient PM2.5 levels is only one factor in determining the appropriateness of controls. Cost effectiveness per microgram is the more complete measure, and reflects both the atmospheric response and costs of the controls.

32. What types of reductions have the biggest local effect on PM2.5 in the East? For a given % reduction in factor RegionalLocal Largest local impact across most urban areas EGU SO2 Point & Area POC/PEC EGU SO2 Point, Mobile & Area POC/PEC Less impact Mobile POC/PEC Non-EGU SO2 Area NH3 Area SO2 Non-EGU SO2 Point source NOx Virtually zero local impact Mobile NOx, Mobile NH3 EGU & Non-EGU NOx VOC Area SO2 Mobile NOx, Mobile NH3 Area source NH3 VOC

33. Next Steps Planning for 12km “Local Scale” RSM for selected areas of concern (FY06/FY07) Implementation of multi-pollutant ASAP version using CMAQ RSM Use RSM results to investigate/guide sector based O3/PM analyses Collaboration & outreach to AQ community (RPOs, academic, international, etc.) to facilitate transfer of methods and development of RSM tools

34. Appendix

35. Assess Policy Objectives Determine Output Metrics (Response Variables) Determine Relevant Policy Factors Factor Elimination Process Universe of Potential Factors CMAQ/CAMx Limits Emissions Inventories Preliminary Modeling Time/Resources Tradeoff Matrix Develop Experimental Design(Battelle) Select Base Inventory (projection year + base control set) Select Air Quality Model Develop Emissions Summaries (Annual or Daily Emissions by Factor) Specify Modeling Domain (including nested subgrids) Select Model Grid Size (e.g. 12km or 36 km) Specify Modeling Periods (e.g. 4 months, one from each season) Evaluation of 2001 full year CMAQ results Run Modeling Experiments Process Model Outputs (compute response variables) Build Predictive Model Model Validation for Ozone and PM2.5 Response Metrics Develop Response Surface Summary Metrics Evaluate Relative Effectiveness of Policy Factors PM2.5 and Ozone Monitoring Data Develop Normalized Adjustment Ratios (SMAT technique for PM2.5, BenMAP eVNA for Ozone) Development of Visualization Tool (Batelle) And Integration into ASAP Evaluate Relative Cost-Effectiveness And Calculate optimal factor levels Development of CMAQ Response Surface Model DATA/INPUT PROCESS DECISION PREPARATION FLOWCHART KEY

36. PM2.5: Areas of Influence for All 9 Urban Locations February 2001 (monthly avg.) July 2001 (monthly avg.)

37. Chicago New York/Philadelphia Atlanta Small overlaps between Atlanta and Chicago influences in Western KY Small overlaps between Chicago and NY influences in Ohio and Western NY. No overlap between Atlanta and NY Areas of Influence for Selected Urban Locations PM2.5 July monthly avg.

38. Extent of Air Quality Influence Region for 9 Selected Urban Areas

39. CMAQ Application: Analyzing Illustrative Control Scenarios Analyze sequence of controls –Demonstrates ways in which states might meet the standard –Each bin contains multiple control options Iterate analysis to identify mix of local and regional controls –Use RSM to help optimize for cost and monetized human health benefits Baseline Modeling (e.g CAIR) (1) Local Controls (2) Non-EGU Regional Controls (3) Local-Scale Targeted EGU Controls as Proposed by States Initial Iteratation   Subsequent Iteration

40. Relative Impacts of 30 Percent Reductions in Precursor Emissions Across Source Categories Included in the Response Surface Model Chicago 2015 Annual Design Value = 16.9

41. Relative Impacts of 30 Percent Reductions in Precursor Emissions Across Source Categories Included in the Response Surface Model San Joaquin 2015 Annual Design Value = 21.7

42. 2-Way Response Surfaces for NY NR NOx NR VOC Onroad NOx Onroad VOC