1. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Inherently mass-conservative semi-Lagrangian transport scheme and global hydrostatic atmospheric model V. Shashkin 1,2 (vvshashkin@gmail.com), M. Tolstykh 1,2, R. Fadeev 2 August 21, 2014 1 - Hydrometeorological centre of Russia 2 - Institute of Numerical Mathematics, Russian Academy of Sciences

2. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Outline Motivation (Why inherently mass-conservative SL?) Transport scheme design (Cell integrated SL scheme) Advection tests (DCMIP advection tests result) Inherently mass-conservative (IMC) SL model design IMC SL model tests (Held&Suarez, Baroclinic instability, Medium-range NWP)

3. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Motivation More invariants = more trust in model (and hopely more credible results) in long- period simulations Why mass-conservation? Why SL? Operationaly verified model at Hydrometcentre of Russia (SL-AV, Tolstykh 2010) Multi-tracer efficient – benefit for using with atmospheric chemistry block Large CFL permitting – better computational performance at lower number of cores Why no global mass-correction? Mass-correction – aggravates locality problems, how to deal with chemical species?

4. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Integral formulation of transport equation: Air density Lagrangian air volume Tracer mass conservation provided no physical sources/sinks - Arrival volume = Grid cell - Departure volume Prognostic variable: Tracer density Time discretization : Tracer specific concentration Transport scheme design – Cell Integrated Semi-Lagrangian type

5. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Transport scheme design – Departure volume integration - Integral over departure volume Approximation of departure volume geometry O(Δx 2 ) Tracer density approximation O(Δx 3 ) (PPM, Colella & Woodward, 1984 & monotonic options) 3D integral 3 x 1D integrals (remappings) (using cascade approach - CCS) (2D – Nair et al, MWR, 2002, 3D – Shashkin, HMC Proc, 2012) Works with reduced lat-lon grid

6. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Transport scheme design – Parallel computation & Scalability Parallel computations: 1D MPI decomposition (in latitude) OpenMP longitude cycles innermost vertical cycle 90 -90 0 360 MPI-0 MPI-1 MPI-2 MPI-3 OMP-0 OMP-1 OMP-2OMP-3 Scalability: 1600x756 (points lon x lat) x 60 levs x 5 tracers:1024 cores (256 MPI x 4 OMP) eff. 62% 1600x800 (points lon x lat) x 60 levs x 5 tracers:800 cores (200 MPI x 4 OMP) eff. 78% 1600x800 (points lon x lat) x 60 levs x 5 tracers:1600 cores (200 MPI x 8 OMP) eff. 53% Probably more cores but no supercomputer to test it Rather moderate scalabity. Do we really need more?

7. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Advection tests – DCMIP 1-1 (Kent et al, QJRMS 2014) @ 1 0 x1 0 x60 levs, time-step 1800s Unlimited Monotonic Exact=Initial T=6 days T=12 days T=0 and12 days l1l1 l2l2 l∞l∞ max Unlim.159.132.275.078 Monot.150.140.285 -0.015 MCore.177.155.263 CAM-FV.121.0998.192

8. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Inherently mass-conservative (IMC) SL hydrostatic atmospheric model Basic model: SL-AV (Semi-Lagrangian Absolute Vorticity): Operational at Hydrometeorological center of Russia (medium-range and seasonal forecasts) Dyn.Core of own development (Institute of Numerical Mathematics & HMC) Physics – ALADIN-LACE Global, Hydrostatic Non mass-conservative (a posteriori mass correction) => Development of Inherently mass-conservative version for the longer term forecasts Model equations & discretization (Tolstykh, JCP, 2002 & Tolstykh and Shashkin, JCP, 2012): Semi-implicit semi-Lagrangian, SETTLS, pseudo-second order decentering Regular (and reduced) lat-lon grid, sigma-coordinate in vertical Vorticity-Divergence formulation (Z-grid) in horizontal Fourier representation in longitude

9. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Inherently mass-conservative (IMC) SL hydrostatic atmospheric model Basic version continuity equation: - orography geopotential - ref. temperature - 2d divergence - horizontal wind IMC version continuity equation (Shashkin and Tolstykh, GMD, 2014) : - reference pressure - Lagrangian volume Basic version cont. eq. also kept for discretization of divergence equation IMC version tracer transport equation: - specific concentration - phys. source/sink Thermodynamic equation: Consistent with Basic or IMC cont. eq.

10. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 IMC-SLAV tests Standard dynamics tests: Baroclinic instability (Jablonowski and Williamson, QJRMS, 2006) Held&Suarez (Held and Suarez, 1994) Test with physics: Medium range weather forecast

11. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Baroclinic instability test Baroclinic wave evolution day 6-9 IMC-SLAV 0.9 0 x0.72 0 x28 levs Jablonowski and Williamson, QJRMS, 2006 initial conditions

12. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Baroclinic instability test Day 9 p s (left) and T 850 hPa (right) convergence. Resolution (lon x lat) from top to bottom: 0.9x0.72, 0.5625x0.45, 0.45x0.36, 0.3x.24, 28 levs all

13. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Baroclinic instability test Day 9 relative vorticity (RV) Good agreement with reference results (Jablonowski and Williamson, NCAR TN, 2006) despite some minor details. RV amplitude is easily tuned by diffusion. 0.9x0.72 0.5625x0.45 0.45x0.36 0.3x.24

14. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Held & Suarez testcase Held and Suarez, 1994: Initially atmosphere at rest, small perturbations Temperature forcing (idealized solar radiation) and Railey wind damping 200 days for spin-up and 1000 days – control period 1000 days average zonal mean Temperature (left) and zonal wind speed (right)

15. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Held & Suarez testcase Held and Suarez, 1994: Initially atmosphere at rest, small perturbations Temperature forcing (idealized solar radiation) and Releygh wind damping 200 days for spin-up and 1000 days – control period 1000 days average zonal mean Temperature (left) and zonal wind speed (right) 32 m/s -12,5 m/s Strongly affected by diffusion

16. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Held & Suarez testcase 1000 days average zonal mean Eddy Kinetic energy (1) Eddy heat flux (2) Eddy momentum flux (3) Temperature second moment (4) Good agreement with IFS, GM, GME and other models

17. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Medium range weather forecast Real physics, real orography Real flow configuration Computationally cheap Ideal for preliminary testing Model configuration: Regular latitude-longitude grid 0.72 0 x0.9 0, 28 levels Initial data – HMCR operational analysis Jan 1- Jan 31 2012, Jul 1 – Jul 31 2012 Setup (physics, diffusion etc) identical to basic SLAV operational version, except IMC dynamical core Results: Reasonable verification scores (BIAS, RMS, Absolute err against analysis) essentialy similar to basic SLAV results. Note no tuning was performed One apparent flaw – spurious orographic resonance stronger (as compared to basic version) – gradient error verification score suffers Precipitation maxima generally weaker (as compared to basic version)

18. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Medium range weather forecast Examples of spurious orographic resonance. Left IMC-SLAV, right basic SLAV. 72 hours forecast of 500 hPa geopotential height over and downstream Himalayas. IMC-SLAV basic-SLAV

19. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Medium range weather forecast Examples of spurious orographic resonance. Left IMC-SLAV, right basic SLAV. 72 hours forecast of 500 hPa geopotential height over and downstream Himalayas. IMC-SLAV basic-SLAV

20. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Medium range weather forecast Example of 24 hours forecast of 6 hours accumulated precipitation over Europe. Typical case: IMC and basic SLAV give generally similar results, but IMC-SLAV precipitation pattern is slightly broader and less intensive.

21. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Summary and outlook Summary: Locally mass-conservative CISL transport scheme performs well (as compared to DCMIP advection test cases reference results) Inherently mass conservative SLAV atmospheric model gives reasonable results in both idealized test cases and real flows It seems reasonable to continue with CISL-based IMC SL technique Future development: To do something with spurious orographic resonance (any ideas?) Seasonal forecast and longer simulations testing Consistent tracer transport Improved scalability

22. V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014 Thank you for attention!