© Crown copyright Met Office LFRic Coupling Requirements 3rd Workshop on Coupling Technologies for Earth System Models Steve Mullerworth April 22 nd 2015

© Crown copyright Met Office Gung Ho – Met Office, NERC, STFC partnership involving Imperial College, STFC, Universities of Bath, Exeter, Manchester, Leeds, Reading and Warwick Develop a dynamical core that is Scientifically as good as ENDGame Scales on future architecture Used for future dynamics research Gung Ho will probably: Use finite element/volume methods (FEM/FV) Support a “higher order” schemes Support quads (cubed- or diamond-sphere) or triangles

© Crown copyright Met Office LFRic: Replacement of the UM Develop a replacement for the UM by around 2019 Project name LFRic after Lewis Fry Richardson Replacement to go operational around 2022 This is distinctly a Met Office project, but we would clearly like to continue and extend our collaborations High-level plan with 3 distinct phases (dates of phases are indicative)

© Crown copyright Met Office Phases for LFRic Phase 1: Delivery of Gung Ho dynamical core Design of LFRic infrastructure Phase 2: Delivery of an LFRic atmosphere Results must be promising enough to commit to operational implementation stage Phase 3: ? Move to operational implementation Ramp down Unified Model development to zero

© Crown copyright Met Office Gung Ho Single Model Architecture

© Crown copyright Met Office Driver Algorithm PSy Kernel Set-up Distributed memory Time-step control IO Coupling (internal and external) Modular science components Field operations (local partition) Concurrent running? Auto-generated Break field into chunks for kernels Distributed memory updates Shared memory concurrency Small chunk of field-data Contiguous memory data Currently, vertical column

Current status of implementation Dynamo package provides basic driver layer and classes for: Algorithms and kernels GungHo fields Flexible choice of mesh Flexible choice of order of FEM PSyclone Auto-generates PSy-layer Being used for development of GungHo science © Crown copyright Met Office

Finite Element Methods: the basics! Fields represented by functions Function parameters defined by values held on mesh entities (cells, edges, vertices and faces) Higher order implies multiple degrees of freedom (dofs) per mesh entity Mixed finite element scheme

Function spaces To fully represent the field, need to know Value and location of degrees of freedom The basis functions © Crown copyright Met Office

Degrees of Freedom and dof-maps © Crown copyright Met Office

3D Cell example © Crown copyright Met Office Arrows represent dofs of a vector field

Looping up the column Dof-map addresses bottom layer of field Data in a column of dofs is contiguous in memory Therefore, to get data for a cell on level K, add (K-1) to values in the dof-map of cell at base of column By looping up column we hope to balance out cost of dof-map lookup Vectorisation Cache re-use © Crown copyright Met Office

Coupling to Physics Schemes Still to be explored PSyKAl design should support physics schemes Physics is commonly column-based Initially will transform any high-order GungHo field to low order Numerical issues may need to be solved Scope for extending the design. E.g. Bigger stencils for some schemes Change direction of loop ordering to level first, if appropriate for performance

High to lowest order transform – 2D example © Crown copyright Met Office A field within a cell, using quadratic basis functions (3x3 dofs) Transformed to 4 cells with piecewise constant basis functions

Coupling to other models and subsystems Meshes and order Support for changing semi-unstructured meshes (between runs, not adaptive meshes) Support flexibility of change of order How to couple fields which have basis functions? IO support – data formats for high-order fields. © Crown copyright Met Office

Time-stepping Concurrent versus sequential models Potential need for flexible deployment options Flexible time-step length Model time-step may relate to CFL limit which will vary © Crown copyright Met Office

Questions © Crown copyright Met Office Key Dynamo developers: LFRic team: Chris Maynard, Mike Hobson, Mike Rezny, Matthew Hambley, Maff Glover, Ricky Wong GungHo Computational Science: Rupert Ford (PSyclone), Graham Riley, Paul Slavin, Eike Mueller GungHo science: Tom Melvin, Iva Kavcic