1. The Use of AMET and Automated Scripts for Model Evaluation
Kristen Foley
K. Wyat Appel

2. Sharing evaluation tools through EPA’s CMAQ GitHub Repository
As of October 2016, utility programs to process and prepare data for model evaluation are available on EPA’s GitHub repository for CMAQ:
github.com/USEPA/CMAQ/tree/5.2Beta/POST

3. Sharing evaluation tools through EPA’s CMAQ GitHub Repository
Last CMAQ release included 9 utility programs, including updated “sitecmp” and “sitcmp_dailyo3” programs for matching observed and model data.
github.com/USEPA/CMAQ/tree/5.2Beta/POST

4. Sharing evaluation tools through EPA’s CMAQ GitHub Repository
Each post processing program in the repo includes the source code, sample run scripts and documentation.
As the CMAQ Team develops tools in-house to facilitate model evaluation, we will continue to add to this repository.
We are seeking feedback on the types of utilities that would be most beneficial to the larger model evaluation community.

5. Sharing evaluation tools through EPA’s CMAQ GitHub Repository
Current efforts include:
Automatically generating evaluation plots after a simulation completes with and without a dependency on AMET (shell scripts, FORTRAN, R)
Interactive spatial and time series plots for model evaluation (shell scripts, R)
3D model-obs pairing and evaluation plots for airborne and remote-sensed observations (shell scripts, R)
Process analysis run script and standard evaluation plots (planned)

6. What is AMET? Atmospheric Model Evaluation Tool
Two modules: Meteorology and Air Quality
Combination of several open source software packages
MySQL − R
Fortran − Perl (will be removed in the next release)
AMET is specifically designed to compare observations against meteorological (e.g. MM5, WRF) and air quality model (e.g. CMAQ, CAMx) predictions
Evaluation options include: summary statistics, scatter plots, spatial plots, time series plots, stacked bar plots, many other analyses

7. Advantages of the AMET System
Automated/interactive system
Data stored in relational database
Data from multiple simulations stored in a single location
Allows data queries based on many factors
Time period, geographic location, time of day, etc.
Pre-generated analysis scripts
Same analysis for multiple simulations
Common analyses between different groups
Open Source
Easy to create new scripts (if you know R)

8. Examples of AMET output

9. Public Availability of AMET
AMETv1.2 is currently available for download on the CMAS website
Both Met and AQ versions available
Script based version
Met and AQ versions can be installed together or individually
Includes users guide, tutorial data and example output plots

10. Future Improvements to AMET
AMETv1.3 scheduled for release in June 2017
removes the need for PERL
streamlined installation
enhanced functionality, new analysis scripts, bug fixes, etc.
AMET Batch post-processing
single script that will create a set of pre-defined plots and other output files
eliminates the need to run separate scripts for each analysis
contains some basic level of customization
Possible next steps:
R and shell scripts available in GitHub repo
Web based version?

11. Possible next steps for AMET
Web based GUI for AMET
Easy selection of plotting options and sub-setting of sites/locations
Development will require resources outside of our team