LFRic: A new model for the Met Office PowerPoint guidelines. Our refreshed PowerPoint style matches our refreshed corporate values. These guidelines ensure we achieve consistent, professional-looking presentations. Font: Arial only Bullets: Arial round Front page slide Headline text point size 53 Subtitle text point size 20 Presenter, location and date point size 12 Divider slides Should be used to break up subjects or when changing content Headline text point size 40 Content slides Headline text size: minimum 40 First Level Bullet Points 24 Subsequent Level; Bullet Points 20 Body text size: minimum 16 Printing Please select greyscale when printing, this will remove the backgrounds and save on ink. Colour Web safe green #CFF33 (R204, G255, B51), can be used to highlight important words or phrases. Web safe red #ED2939 (R237, G41, B57), can be used to highlight severe weather warnings Campaign presentations If your presentation is part of a campaign or event then please ask the studio for assistance LFRic: A new model for the Met Office Steve Mullerworth Bath DG Workshop 1st June 2016 © Crown copyright Met Office

Met Office’s Unified Model Unified Model (UM) supports: Operational forecasts at Mesoscale (resolution approx. 12km → 4km → 1km) Global scale (resolution approx. 17km) Global and regional climate predictions (global resolution around 100km, run for 10-100 years) Seasonal predictions + Research mode (1km - 10m) and single column model 26 years old this year © Crown copyright Met Office

Operational NWP Models: Jun 2016 Global Operational: 17km Climate: ~100km 10-100 year simulations Seasonal: 50km Euro 4 Operational 4km 70 levels Our current suite of operational NWP models are shown here along with future configurations. The numbers refer to typical horizontal grid spacings in mid-latitudes. This grid spacing is often referred to as resolution. UKV Operational: 1.5 km 70 levels © Crown copyright Met Office © Crown copyright Met Office 3 3

Scalability (March 2010) © Crown copyright Met Office

The finger of blame… At 25km resolution, grid spacing near poles = 75m At 10km reduces to 12m! Our current suite of operational NWP models are shown here along with future configurations. The numbers refer to typical horizontal grid spacings in mid-latitudes. This grid spacing is often referred to as resolution. © Crown copyright Met Office © Crown copyright Met Office 5 5

GungHo – 2011-2016 Met Office, NERC, STFC partnership involving Imperial College, STFC, Universities of Bath, Exeter, Manchester, Leeds, Reading and Warwick Develop a dynamical core that is Scientifically as good as current “ENDGame” dynamics Scales on future architecture Used for future dynamics research Formulation Uses finite element/volume methods (FEM/FV) Supports a “higher order” schemes Supports quads (cubed- or diamond-sphere) or triangles © Crown copyright Met Office

LFRic: Replacement of the UM Develop a replacement for the UM by around 2020 Project name LFRic after Lewis Fry Richardson Replacement to go operational mid-2020s Based on GungHo dynamics and GungHo computational science recommendations © Crown copyright Met Office

GungHo Single Model Architecture © Crown copyright Met Office

Driver Algorithm PSy Kernel Set-up Distributed memory Time-step control IO Coupling (internal and external) Driver Modular science components Field operations (local partition) Concurrent running? Algorithm Auto-generated Break field into chunks for kernels Distributed memory updates Shared memory concurrency PSy Small chunk of field-data Contiguous memory data Currently, vertical column Kernel © Crown copyright Met Office

Looping up the column Unstructured mesh requires indirect mapping with dof-map Dof-map addresses bottom layer of field Data in vertical column of dofs is contiguous in memory Therefore, to get data for a cell on level K, add (K-1) to values in the dof-map of cell at base of column By looping up column we hope to balance out cost of dof-map lookup Vectorisation Cache re-use © Crown copyright Met Office

Current status of implementation Infrastructure provides basic driver layer and classes for: Algorithms and kernels GungHo fields on different function spaces Flexible choice of (quad-based) mesh, and FEM order Support for distributed memory parallel and shared-memory colouring PSyclone Auto-generates PSy-layer parallel code Being used for development of GungHo science Support being added for finite difference physics © Crown copyright Met Office

Mixed FEM Function spaces. Lowest order Higher order W3 W0 W2 © Crown copyright Met Office

LFRic Object Stack (parallel) UGRID file Global Mesh Partition Function Space Field Run Init

Cell ordering for concurrent comms & computation Inner n+1 cells Inner n cells ... Inner 1 cells Edge cells Halo 1 cells Halo n-1 cells Halo n cells Halo n+1 cells

Dynamo and PSyclone Prototype GungHo implementation developed at Met Office Comprises both LFRic infrastructure and GungHo science Originally hand-written PSy-layer: PSyKAl-lite Named Dynamo PSyclone currently developed at STFC Daresbury Written to clone the hand-written PSy-layer code © Crown copyright Met Office

Dynamo kernel example ... From matrix_vector_kernel_mod.F90

Dynamo algorithm post PSyclone

Going MPI and OpenMP Parallel Dynamo infrastructure and Psyclone support PSyclone script > python generator.py -oalg alg.f90 -opsy psy.f90 file.f90 -s script.py No Scientists were harmed

PSyclone generated code MPI halo swap Colouring and OMP Halo is not up to date

Spot the difference – first parallel run

Dynamo Initial MPI results

Dynamo Initial OpenMP results

Xeon and Xeon Phi

Current Software Dynamo prototype comprises PSyclone BSD-style licence LFRic infrastructure GungHo dynamics implementation F2003 PSyclone PSy-layer code generation Python BSD-style licence © Crown copyright Met Office

Immediate plans Support for multigrid solver development Support for coupling to physics Separate software infrastructure and science Extend PSyclone optimisations Resolve preliminary scalability issues © Crown copyright Met Office

High level Schedule Support scientific evaluation Early 2017: Dynamical core trials – with simple physics Early 2018: Aquaplanet (no land) 2020: Operational and coupled configurations 2020-2X: Trials and deployment © Crown copyright Met Office

Credits & Questions PSyKAl structure and earliest developments based on GungHo collaboration computational science recommendations NERC, STFC and Met Office collaboration Imperial College, Bath, Reading, Leeds, Manchester, Warwick Current development Met Office STFC University of Manchester Mike Rezny at Monash University © Crown copyright Met Office