Nested Model Clinic Nested Model Co-Scientist Lin Zhang Yuxuan Wang Peking Univ. Yuxuan Wang Univ. of Houston With contributions by: Xiao Lu, Lin Nan, Melissa Sulprizio Thursday, May 9, 2019, Harvard University 1

Agenda How to run the nested-grid simulation FlexGrid (courtesy of Melissa Sulprizio)

How to run GEOS-Chem nested model? Step 1: Download (or prepare) the met. field For existing nested domains (see next slide), just download Step 2: Run the global simulation to generate boundary conditions Either 2o x 2.5o or 4o x 5o boundary conditions Step 3: Run the nested model (with boundary conditions from Step2 )

1. Decide which met field to use Two options: GEOS-FP and MERRA2 Check the horizontal grid first before you prepare the met field! http://wiki.seas.harvard.edu/geos-chem/index.php/GEOS-Chem_horizontal_grids Global BC 4°× 5° or 2°×2.5° Existing Nested Domains 1°× 1° (GEOS-3) (Y. Wang et al., 2004) (obsolete) 1/2°× 2/3° (GEOS-5) (Chen et al., 2009) (obsolete) 1/4°× 5/16°(GEOS-FP) 1/2°× 5/8° (MERRA-2) Africa (GEOS-5 and GEOS-FP) (E. Marais, U. Birmingham; M. Evans, U. York) Australia (GEOS-FP) (Jenny Fisher, U. Wollongong) ASIA (MERRA2 only) EU NA CH (GEOS-FP only) AF AUS

1. Decide which met field to use Two options: GEOS-FP and MERRA2 Check the horizontal grid first before you prepare the met field! http://wiki.seas.harvard.edu/geos-chem/index.php/GEOS-Chem_horizontal_grids Region Met field Resolution Lon-lat coverage US MERRA2 0.5x0.625 140-40W, 10-70N GEOS-FP 0.25x0.3125 130-60W, 9.75-60N Europe 30W-50E, 30-70N 15W-40E, 32.75-61.25N Asia 60-150E, 11S-55N China 70-140E, 15-55N

2. Run global simulation Everything is the same except you need to modify the nested grid menu in input.geos 1 2 3 3 1: set T to save BC for nested model 2: T for 2x2.5, F for 4x5 3: Set T for the region for nested simulation. A single global simulation can output boundary conditions for multiple nested grid regions

2. Run global simulation Everything is the same except you need to modify the nested grid menu in input.geos 4 4: You need to set the LL and UR box of BC region in input.geos ONLY IF you are running the nested-grid simulation over a custom region. Otherwise, the LL and UR boxes index will not affect your simulation. The first number in the LL and UR box refers to the longitude index of the global grid and the second number is the latitude index. As such, these index are different in 2x2.5 and 4x5 global grid.

2. Run global simulation BC will then be saved for daily output (3-hour frequency). Keep an eye on your storage! You can also use IDL -> GAMAP to check the BC file.

3. Create the run directory and run the nested model Modify your GEOS-Chem/UT/perl/CopyRunDirs.input http://wiki.seas.harvard.edu/geos-chem/index.php/GEOS-Chem_Makefile_Structure#Compiling_in_a_run_directory When creating a run directory from the GEOS-Chem Unit Tester, the input.geos and HEMCO_Config.rc file will automatically be set up for the nested grid domain that you specified.

3. Create the run directory and run nested model Make sure the model can find the BC from global simulations No need to change anything (should be already set when you create the run directory) except that you need to make sure the you have the correct BC pathway. Then you are ready to start the nested simulation.

4. Set up buffer zone if you only focus on a smaller domain J0_E I0_W I0_E J0_W I0_W, J0_W, I0_E, J0_E represent, respectively, the numbers of grid cells in left, bottom, right, top boundaries. They define a buffer zone that the model will skip transport/chemistry calculations for those grid cells and replace their concentrations with the boundary conditions. You can define the values to save some computational time.

Summary and references Checklist 1 Step 1: Have you decided and downloaded the met. fields for your simulation? 2 Step 2: Have you turned on and correctly save the boundary conditions in your global simulation? (input.geos in global simulation) 3 Step 3.1: Have you created the nested model run directory from UT? (UT/perls/Copyrundirs.input) 4 Step 3.2: Have you set the BC pathway correctly? (input.geos in nested simulation) http://wiki.seas.harvard.edu/geos-chem/index.php/GEOS-Chem_Nested_Model http://wiki.seas.harvard.edu/geos-chem/index.php/Setting_up_GEOS-Chem_nested_grid_simulations

Application: nested-grid VS coarse grid GEOS-Chem v11-02rc, GEOS-FP, surface MDA8 ozone from Mar-Oct 2017 in several Chinese cities Observation GEOS-Chem at 2x2.5 GEOS-Chem at 0.25x0.3125 Significant improvement of ozone simulation over complex topography with GESO-Chem nested model! Lu et al., 2019@ACPD

Application: nested-grid VS coarse grid GEOS-Chem v11-02rc, GEOS-FP, surface NO2, SO2 and O3 from May 2017 in the US NO2 SO2 O3 ppbv GEOS-Chem at 2x2.5 GEOS-Chem at 0.25x0.3125 Nested-grid provides better details of surface sources! Plots generated by Lin Nan and Elizabeth Klovenski

Melissa Sulprizio IGC9, 6-9 May 2019 FlexGrid Melissa Sulprizio IGC9, 6-9 May 2019

FlexGrid Overview Use HEMCO’s I/O interface and regridding/cropping capability to: Explicitly control meteorological variables at run time to perform sensitivity runs Set up nested-grid simulations over custom domains without changing source code Stage 1: Read meteorology fields through HEMCO Developer: Jiawei Zhuang (Harvard) Implemented in GEOS-Chem 12.1.0 Stage 2: Define custom grids at run time Developer: Melissa Sulprizio (Harvard) To be added in GEOS-Chem 12.4.0 or 12.5.0

FlexGrid Stage 1: Read met fields in HEMCO Provides consistency with other GEOS-Chem input. Emissions, non-emissions fields, restart fields, and now meteorology fields are all read in via HEMCO. Simplifies using new meteorology fields in GEOS-Chem. Allows users to do sensitivity tests and modify meteorology fields. Some examples include: Holding meteorology fields constant Applying scale factors Applying masks

FlexGrid Stage 1: Implementation Consolidate meteorology code. Remove geosfp_read_mod.F90, merra2_read_mod.F90 and use flexgrid_read_mod.F90 instead Add entries for met fields to HEMCO_Config.rc. Use tokens $MET, $met, $CNYR, $NC Get fields from HEMCO in routine Get_Met_Fields in hcoi_gc_main_mod.F90. Still have meteorology-specific code in HEMCO extensions (e.g. DustDead, LightNOx), but soon we will have grid-independent emission fields (expected in 12.4.0).

FlexGrid Stage 1: HEMCO_Config.rc

FlexGrid Stage 2: Custom nested grids Jiawei Zhuang’s original idea was to simplify how custom nested grids are defined in GEOS-Chem by taking advantage of HEMCO’s regridding and cropping capability. For consistency and transparency, the way all grids are defined has been updated. Instead of using compiler flags MET=, GRID= and NEST=, the meteorology fields, grid resolution, grid domain are defined at run time in input.geos. A new derived type object State_Grid stores the user-defined grid parameters and those computed by GEOS-Chem at run time Replaces fields previously in CMN_SIZE_mod.F (e.g. IIPAR, JJPAR) and gc_grid_mod.F90 (e.g. XMid, YMid, Area_M2) CPP statements throughout code are replaced with IF statements, so changing meteorology field or grid doesn’t require recompiling.

FlexGrid Stage 2: Defining met and grid A Grid Menu has been added to input.geos: Theoretically, any meteorology field, grid resolution, and grid domain may be defined but the first step is to validate the existing grids supported in GEOS-Chem.

FlexGrid Stage 2: Nested grids Boundary conditions are now saved out to netCDF files using the History component. In HISTORY.rc: NOTE: Currently only global BC files can be saved out, but the capability to save out regional BC files will eventually be added to History.

FlexGrid Stage 2: Nested grids Boundary conditions are read in via HEMCO. In HEMCO_Config.rc: Get BCs from HEMCO in routine Get_Boundary_Conditions in hcoi_gc_main_mod.F90. BCs are applied to the buffer zone defined in the Grid Menu of input.geos.

FlexGrid Stage 2: Status FlexGrid Stage 2 is currently being validated to ensure: The structural code updates don’t impact model output. All existing metrology fields (GEOS-FP, MERRA-2), grid resolutions (4°x5°, 2°x2.5°, 0.5°x0.625°, 0.25°x0.3125°), and nested domains (China/Asia, Europe, North America) still work as expected. The GCST will need support from the user community to experiment with other meteorology fields and grid defitions for further refinement to FlexGrid. FlexGrid is slated for inclusion in GEOS-Chem 12.4.0 or 12.5.0 (expected late spring).