The Use of AMET and Automated Scripts for Model Evaluation Kristen Foley K. Wyat Appel

Sharing evaluation tools through EPA’s CMAQ GitHub Repository As of October 2016, utility programs to process and prepare data for model evaluation are available on EPA’s GitHub repository for CMAQ: github.com/USEPA/CMAQ/tree/5.2Beta/POST

Sharing evaluation tools through EPA’s CMAQ GitHub Repository Last CMAQ release included 9 utility programs, including updated “sitecmp” and “sitcmp_dailyo3” programs for matching observed and model data. github.com/USEPA/CMAQ/tree/5.2Beta/POST

Sharing evaluation tools through EPA’s CMAQ GitHub Repository Each post processing program in the repo includes the source code, sample run scripts and documentation. As the CMAQ Team develops tools in-house to facilitate model evaluation, we will continue to add to this repository. We are seeking feedback on the types of utilities that would be most beneficial to the larger model evaluation community.

Sharing evaluation tools through EPA’s CMAQ GitHub Repository Current efforts include: Automatically generating evaluation plots after a simulation completes with and without a dependency on AMET (shell scripts, FORTRAN, R) Interactive spatial and time series plots for model evaluation (shell scripts, R) 3D model-obs pairing and evaluation plots for airborne and remote-sensed observations (shell scripts, R) Process analysis run script and standard evaluation plots (planned)

What is AMET? Atmospheric Model Evaluation Tool Two modules: Meteorology and Air Quality Combination of several open source software packages MySQL − R Fortran − Perl (will be removed in the next release) AMET is specifically designed to compare observations against meteorological (e.g. MM5, WRF) and air quality model (e.g. CMAQ, CAMx) predictions Evaluation options include: summary statistics, scatter plots, spatial plots, time series plots, stacked bar plots, many other analyses

Advantages of the AMET System Automated/interactive system Data stored in relational database Data from multiple simulations stored in a single location Allows data queries based on many factors Time period, geographic location, time of day, etc. Pre-generated analysis scripts Same analysis for multiple simulations Common analyses between different groups Open Source Easy to create new scripts (if you know R)

Examples of AMET output

Public Availability of AMET AMETv1.2 is currently available for download on the CMAS website Both Met and AQ versions available Script based version Met and AQ versions can be installed together or individually Includes users guide, tutorial data and example output plots

Future Improvements to AMET AMETv1.3 scheduled for release in June 2017 removes the need for PERL streamlined installation enhanced functionality, new analysis scripts, bug fixes, etc. AMET Batch post-processing single script that will create a set of pre-defined plots and other output files eliminates the need to run separate scripts for each analysis contains some basic level of customization Possible next steps: R and shell scripts available in GitHub repo Web based version?

Possible next steps for AMET Web based GUI for AMET Easy selection of plotting options and sub-setting of sites/locations Development will require resources outside of our team