GungHo! A new dynamical core for the Unified Model Nigel Wood, Dynamics Research, UK Met Office © Crown copyright Met Office

Outline Unified Model – where we are now & the need for change GungHo! Some results from each workpackage Summary © Crown copyright Met Office © Crown copyright Met Office 2

Current Unified Model “New Dynamics” Davies et al. (2005) Regular lat/lon grid. Non-hydrostatic dynamics with a deep atmosphere. Semi-implicit time integration with 3D semi-Lagrangian advection. Atmospheric tracer advection Physics: Spectral band radiation Diagnostic or prognostic cloud Mixed-phase ppn Mass flux convection Boundary layer Gravity wave schemes Coupling possible to non-atmospheric components: Land surface model Ocean model Sea ice model Chemistry/aerosol model © Crown copyright Met Office

Relative performance 3 day Northern Hemisphere surface pressure errors 2003 2011 3 day Northern Hemisphere surface pressure errors © Crown copyright Met Office

T24/TN Scalability (17km) Nodes Perfect scaling 24 nodes Same plot but as a scaling plot. 24 nodes Nodes (1 node=32 processors) © Crown copyright Met Office

The finger of blame… At 25km resolution, grid spacing near poles = 75m At 10km reduces to 12m! © Crown copyright Met Office © Crown copyright Met Office 6 6

Challenges! Scalability – remove the poles! © Crown copyright Met Office © Crown copyright Met Office

Challenges! Scalability – remove the poles! Speed – cannot sacrifice this for low resolution moderate core counts Accuracy – need to maintain standing of model Space weather  600km deep model… Danger: Everything to everyone…or Nothing to anyone? © Crown copyright Met Office © Crown copyright Met Office

GungHo! Globally Uniform Next Generation Highly Optimized “Working together harmoniously” © Crown copyright Met Office

5 Year Project Split into two phases: 2 years “research” (2011-13) “To research, design and develop a new dynamical core suitable for operational, global and regional, weather and climate simulation on massively parallel computers of the size envisaged over the coming 20 years.” Split into two phases: 2 years “research” (2011-13) 3 years “development” (2013-2016)

UK Collaboration of GFD, numerical and computational scientists 5 FTEs from Met Office (Dynamics Research; HPC Optimisation; UM infrastructure) 5 FTEs from NERC (Bath, Exeter, Imperial, Leeds, Manchester, Reading, Warwick) 2 FTEs from STFC (Hartree Centre)

GungHo Issues How to maintain accuracy of current model on a GungHo grid? Principal points about current grid are: Orthogonality C-grid These provide a number of good numerical properties (Staniforth & Thuburn QJ 2012) Challenge is to retain those on a non-orthogonal grid © Crown copyright Met Office

Some workpackage results © Crown copyright Met Office 13

Grids Good dispersion Minimal grid imprinting No computational modes C-grid dispersion relations Grids Exact Low order FEM Frequency FD Good dispersion Minimal grid imprinting No computational modes Finite element approach Focus on: Cubed-sphere; possibly triangles Higher order FEM Partially mass lumped FEM Group velocity Cotter (Imperial), Melvin & Staniforth (MetO) © Crown copyright Met Office Nondimensional wavenumber

Grids Higher order FEM Good dispersion Minimal grid imprinting No computational modes Finite element approach Focus on: Cubed-sphere; possibly triangles U V Φ Partially mass lumped FEM Cotter (Imperial), Melvin & Staniforth (MetO) © Crown copyright Met Office

Recent results Williamson Test Case 5 with 160K d.o.f.s (320x160) 9 m Thuburn (Exeter) Williamson Test Case 5 with 160K d.o.f.s (320x160) 9 m 6 m FEM Hexagonal FEM Cubed-sphere 10 m 11 m h background = 5000 m. Reference solution = 1024x512. Rotated = by 45 degrees. Reference solution cubically interpolated to each grid. SISL not SLICE. ENDGame lat-lon ENDGame rotated lat-lon © Crown copyright Met Office

Are implicit schemes viable? Weak horizontal scaling for a 3D Helmholtz problem Baseline resolution = 64x64 Nz=128 Grid cells per processor = 520K Cs*Dt/Dx=const=8.4 One side of cubed-sphere Algebraic Multi-grid Conjugate Gradient Geometric Multi-grid Mueller & Scheichl (Bath) Hector © Crown copyright Met Office

What to do if not… Horizontally Explicit – Vertically Implicit (HEVI) Computational modes arise from multistep schemes  Examine range of Runge-Kutta Implicit-Explicit (IMEX) schemes Weller (Reading) & Lock (Leeds) © Crown copyright Met Office HEVI Implicit

Test cases Finite difference scheme applied on a variety of grids Simple solid body rotation (Williamson test case 2) Height and velocity errors after 5 days Weller, Thuburn and Cotter, MWR, 2012 Weller (Reading), Thuburn (Exeter) & Cotter (Imperial) © Crown copyright Met Office

Computational Science Ham (Imperial), Ford & Pickles (STFC), Riley (Manchester) Vertical loop inner most Indirect addressing for horizontal F2003 © Crown copyright Met Office

Transport Mass conservation = #1 user requirement! Inherent part of mimetic approach But want to maintain non-split approach of current SL scheme OK in horizontal (CFL<1 on uniform mesh) – see previous simulations Challenge is in vertical… © Crown copyright Met Office

Timetable… Further development and testing of horizontal [2013] Testing of proposals for code architecture [2013] Vertical discretization [2013] 3D prototype development [2014-2015] Operational…by 2020  Long term step change in scalability © Crown copyright Met Office

Thank you! Questions? © Crown copyright Met Office