1. © Crown copyright Met Office Technical developments at the Met Office Matthew Glover

2. © Crown copyright Met Office

3. Overview Data Centre DC power Cooling Coupled models Components Coupling Unified Model IO server NEMO-CICE Direct coupling

4. © Crown copyright Met Office The Met Office National Weather Service for the UK Climate Prediction (Met Office Hadley Centre) Operational and Research activities About 1,700 staff in 60 locations around the world

5. Collaboration

6. © Crown copyright Met Office The same model formulation is used for all models from climate scale to mesoscale The Unified Model Climate modelling: input into IPCC reports (Coupled Atmosphere-Ocean models) Seasonal forecasting: For commercial and business customers NWP: Public Weather Service WAFC, Commercial ……

7. © Crown copyright Met Office Data Centre

8. © Crown copyright Met Office Energy efficiency Supercomputer on DC power Free cooling Cold aisle containment Efficient cooling at lower power - motor efficiency

9. © Crown copyright Met Office IBM Phase2 Timescales: Frames accepted

10. © Crown copyright Met Office Coupling

11. © Crown copyright Met Office Coupling Schematic Simple example – 2 component system Atmos OASIS3 Ocean

12. © Crown copyright Met Office HadGEM3 Schematic UM Atmos OASIS3 NEMO Ocean CICE Sea ice

13. © Crown copyright Met Office Pseudo Parallel OASIS3 You can use more than one instance of OASIS3 Work is divided by field – i.e. TASK PARALLELISM e.g. instance 0 deals with SST and heat flux instance 1 deals with U, V and PME instance 2 deals with ice thickness and river runoff NEMO-CICE Atmos OASIS3

14. © Crown copyright Met Office Couplers

15. © Crown copyright Met Office Unified Model - IO Server

16. © Crown copyright Met Office N512 scalability – Nov 2010 Figure 1

17. © Crown copyright Met Office N512 scalability – Cray XE6 Thanks: Pier Luigi Vidale, NCAS. Climate model (atmosphere only) Preparation for real science on PRACE XE6

18. © Crown copyright Met Office Data output inhibits scaling Thanks to Martyn Foster for this plot

19. © Crown copyright Met Office Older I/O I/O

20. © Crown copyright Met Office I/O Server I/O

21. © Crown copyright Met Office Key design considerations Output units distributed over servers Server is threaded “Listener” receives data & puts in queue “Writer” processes queue including packing Ensures asynchronous behaviour First-in, first-out (FIFO) queue Preserves instruction order Metadata/Data split Data initially stored on compute processes Enables more data “in-flight” Data of same type combined into large messages

22. © Crown copyright Met Office How data gets output ComputeI/O ListenerWriter Thread 0 Thread 1

23. © Crown copyright Met Office Overloaded servers

24. © Crown copyright Met Office Well tuned…

25. © Crown copyright Met Office What savings do we get? 19%28%33%10%

26. © Crown copyright Met Office Nemo-Cice coupling

27. © Crown copyright Met Office Sea ice decomposition

28. © Crown copyright Met Office Nemo-Cice decomposition

29. © Crown copyright Met Office Direct coupling

30. © Crown copyright Met Office Direct vs Indirect – timings

31. © Crown copyright Met Office Direct vs Indirect – scaling

32. © Crown copyright Met Office Summary Data Centre DC power Free cooling; Cold aisle containment; Cooling motor efficiency Unified Model IO server Encouraging scaling on Cray XE6 NEMO-CICE Direct coupling Coupling Short term: Oasis3 Longer: continue with Oasis4?

33. © Crown copyright Met Office Questions and answers