1. LFRic: A new model for the Met Office
PowerPoint guidelines.
Our refreshed PowerPoint style matches our refreshed corporate values. These guidelines ensure we achieve consistent, professional-looking presentations.
Font: Arial only
Bullets: Arial round
Front page slide
Headline text point size 53
Subtitle text point size 20
Presenter, location and date point size 12
Divider slides
Should be used to break up subjects or when changing content
Headline text point size 40
Content slides
Headline text size: minimum 40
First Level Bullet Points 24
Subsequent Level; Bullet Points 20
Body text size: minimum 16
Printing
Please select greyscale when printing, this will remove the backgrounds and save on ink.
Colour
Web safe green #CFF33 (R204, G255, B51), can be used to highlight important words or phrases.
Web safe red #ED2939 (R237, G41, B57), can be used to highlight severe weather warnings
Campaign presentations
If your presentation is part of a campaign or event then please ask the studio for assistance
LFRic: A new model for the Met Office
Steve Mullerworth
Bath DG Workshop 1st June 2016
© Crown copyright Met Office

2. Met Office’s Unified Model
Unified Model (UM) supports:
Operational forecasts at
Mesoscale (resolution approx. 12km → 4km → 1km)
Global scale (resolution approx. 17km)
Global and regional climate predictions (global resolution around 100km, run for years)
Seasonal predictions
+ Research mode (1km - 10m) and single column model
26 years old this year
© Crown copyright Met Office

3. Operational NWP Models: Jun 2016
Global
Operational: 17km
Climate: ~100km
year simulations
Seasonal: 50km
Euro 4
Operational 4km 70 levels
Our current suite of operational NWP models are shown here along with future configurations. The numbers refer to typical horizontal grid spacings in mid-latitudes. This grid spacing is often referred to as resolution.
UKV
Operational: 1.5 km 70 levels
© Crown copyright Met Office
© Crown copyright Met Office 3 3

4. Scalability (March 2010)
© Crown copyright Met Office

5. The finger of blame… At 25km resolution, grid spacing near poles = 75m
At 10km reduces to 12m!
Our current suite of operational NWP models are shown here along with future configurations. The numbers refer to typical horizontal grid spacings in mid-latitudes. This grid spacing is often referred to as resolution.
© Crown copyright Met Office
© Crown copyright Met Office 5 5

6. GungHo – 2011-2016 Met Office, NERC, STFC partnership involving
Imperial College, STFC, Universities of Bath, Exeter, Manchester, Leeds, Reading and Warwick
Develop a dynamical core that is
Scientifically as good as current “ENDGame” dynamics
Scales on future architecture
Used for future dynamics research
Formulation
Uses finite element/volume methods (FEM/FV)
Supports a “higher order” schemes
Supports quads (cubed- or diamond-sphere) or triangles
© Crown copyright Met Office

7. LFRic: Replacement of the UM
Develop a replacement for the UM by around 2020
Project name LFRic after Lewis Fry Richardson
Replacement to go operational mid-2020s
Based on GungHo dynamics and GungHo computational science recommendations
© Crown copyright Met Office

8. GungHo Single Model Architecture
© Crown copyright Met Office

9. Driver Algorithm PSy Kernel Set-up Distributed memory
Time-step control IO
Coupling (internal and external)
Driver
Modular science components
Field operations (local partition)
Concurrent running?
Algorithm
Auto-generated
Break field into chunks for kernels
Distributed memory updates
Shared memory concurrency
PSy
Small chunk of field-data
Contiguous memory data
Currently, vertical column
Kernel
© Crown copyright Met Office

10. Looping up the column
Unstructured mesh requires indirect mapping with dof-map
Dof-map addresses bottom layer of field
Data in vertical column of dofs is contiguous in memory
Therefore, to get data for a cell on level K, add (K-1) to values in the dof-map of cell at base of column
By looping up column we hope to balance out cost of dof-map lookup
Vectorisation
Cache re-use
© Crown copyright Met Office

11. Current status of implementation
Infrastructure provides basic driver layer and classes for:
Algorithms and kernels
GungHo fields on different function spaces
Flexible choice of (quad-based) mesh, and FEM order
Support for distributed memory parallel and shared-memory colouring
PSyclone
Auto-generates PSy-layer parallel code
Being used for development of GungHo science
Support being added for finite difference physics
© Crown copyright Met Office

12. Mixed FEM Function spaces.
Lowest order
Higher order W3 W0 W2
© Crown copyright Met Office

13. LFRic Object Stack (parallel)
UGRID
file
Global
Mesh
Partition
Function
Space
Field
Run
Init

14. Cell ordering for concurrent comms & computation
Inner n+1 cells
Inner n cells
...
Inner 1 cells
Edge cells
Halo 1 cells
Halo n-1 cells
Halo n cells
Halo n+1 cells

15. Dynamo and PSyclone
Prototype GungHo implementation developed at Met Office
Comprises both LFRic infrastructure and GungHo science
Originally hand-written PSy-layer: PSyKAl-lite
Named Dynamo
PSyclone currently developed at STFC Daresbury
Written to clone the hand-written PSy-layer code
© Crown copyright Met Office

16. Dynamo kernel example ...
From matrix_vector_kernel_mod.F90

17. Dynamo algorithm post PSyclone

18. Going MPI and OpenMP Parallel
Dynamo infrastructure and Psyclone support
PSyclone script
> python generator.py -oalg alg.f90 -opsy psy.f90 file.f90 -s script.py
No Scientists were harmed

19. PSyclone generated code
MPI halo swap
Colouring and OMP
Halo is not up to date

20. Spot the difference – first parallel run

21. Dynamo Initial MPI results

22. Dynamo Initial OpenMP results

25. Xeon and Xeon Phi

26. Current Software Dynamo prototype comprises PSyclone BSD-style licence
LFRic infrastructure
GungHo dynamics implementation
F2003
PSyclone
PSy-layer code generation
Python
BSD-style licence
© Crown copyright Met Office

27. Immediate plans Support for multigrid solver development
Support for coupling to physics
Separate software infrastructure and science
Extend PSyclone optimisations
Resolve preliminary scalability issues
© Crown copyright Met Office

28. High level Schedule Support scientific evaluation
Early 2017: Dynamical core trials – with simple physics
Early 2018: Aquaplanet (no land)
2020: Operational and coupled configurations
2020-2X: Trials and deployment
© Crown copyright Met Office

29. Credits & Questions
PSyKAl structure and earliest developments based on GungHo collaboration computational science recommendations
NERC, STFC and Met Office collaboration
Imperial College, Bath, Reading, Leeds, Manchester, Warwick
Current development
Met Office
STFC
University of Manchester
Mike Rezny at Monash University
© Crown copyright Met Office