1. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 0

2. © Fujitsu Laboratories of Europe 2009 Chaste Workshop University of Oxford 24 March 2009 HPC and Chaste: Towards Real- Time Simulation James Southern Fujitsu Laboratories of Europe

3. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 2 The petascale challenge Petascale computing has been achieved in the last 12 months. 10 15 (one million billion) floating point operations per second There will be many challenges in building and using the next-generation computer systems. One of the biggest is developing applications, algorithms and numerical libraries that will make efficient use of the computer. Fujitsu aims to contribute through collaboration with world-leading modelling and simulation groups.

4. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 3 First petascale system: IBM’s Road Runner June 2008

5. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 4 “World’s most powerful supercomputer for science” November 2008 Cray XT5 (Opteron) Oak Ridge The world’s second petascale computer

6. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 5 Petascale computing and the heart The MEXT Next-Generation Supercomputer Project will provide a ‘leadership’ 10 petaflop/s resource for Japan by 2012. 10 times faster than the world’s current fastest computer. A budget of £500 million. Fujitsu to be one of the main contractors One of the main application fields to be targeted is the life sciences. Real-time simulation of a beating heart is a life science application that requires petascale computing.

7. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 6 Parallel solution of the bidomain equations Current simulation methods are >100,000 times slower than real-time for a realistic mesh on a single processor. Large scale parallel computing will be needed if we are to realize real-time heart simulations in the near future. This leads to a series of additional factors that must be considered when solving the bidomain equations. How do we minimize communication between processors? How do we ensure each processor does the same amount of work? It is important that we balance these factors with the scientific computing issues that also have an impact.

8. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 Making the code run faster Improve performance of existing methods. Profile and benchmark the code to identify parts of the code that could be improved. Which parts of the code take most of the time? Are some processors spending a lot of time waiting for others to finish executing code? Develop new, improved methods. More efficient solution of the bidomain equations. Better linear solvers, preconditioners. Improved ODE solvers. Getting the same level of accuracy for less computation. Adaptivity. 7

9. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 Benchmarking and profiling Benchmarking tells us how fast Chaste is in comparison to similar codes. Difficult to get a precise measure as we often can’t run exactly the same simulations. Meshes used by others not available to us, not always possible to get hold of other applications to run comparison. Indications are that Chaste is very competitive. Profiling showed that partitioning was causing a load imbalance when we used large meshes. Some processors doing a lot more work than others. Solution: re-distribute mesh nodes over the processors by optimizing with METIS library. 8

10. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 9 Improved load balancing

11. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 10 Improved load balancing

12. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 Parallel scaling Good load balancing is key to good parallel performance. Allows us to minimize the time each processor spends waiting for synchronization with others. Introducing METIS has significantly improved load balancing in Chaste. Also important to limit the amount of time processors have to spend communicating with one another. This time could be better used doing calculations. How good is Chaste’s parallel performance currently? For a perfectly load balanced code with no communication overhead we’d expect linear speedup. i.e. Twice as many processors  Half the execution time 11

13. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 12 Parallel scaling

14. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 13 Parallel efficiency

15. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 Summary Chaste has been designed to work with HPC. Should not impact performance on smaller systems. We regularly benchmark and profile the code both sequentially and in parallel. Performance improvements can be rapidly implemented to address any issues raised by profiling. Chaste’s parallel performance is very good (up to at least 32 processors). We will be working to ensure that this scaling is carried forward onto even larger systems. 14

16. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 15

17. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 16 A flagship eHealth project in FP7. Computational frameworks and ICT-based tools for multiscale models of the human anatomy, physiology and pathology. Libraries of data and toolbox for simulation and visualisation. A key facilitator for: Personalized (patient-specific) healthcare solutions. Early diagnosis and predictive medicine. Assessment of safety/efficacy of drugs using patient-specific computational models. Innovative personalized drugs. Virtual Physiological Human

18. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 17 preDiCT Collaborative project to create a framework for in silico testing of drugs, using Chaste as the base software. Systems-based approach, using multi-scale and multi- physics techniques to integrate components from different levels of biological organization. Partners from high-performance computing, academic institutions and the pharmaceutical industry. Ultimate goal is to facilitate real time in silico testing of the cardiac toxicity of drugs. FLE leads the Computational Tools and Methodologies work package of the project.

19. © Fujitsu Laboratories of Europe 2009 HPC and Chaste: Towards Real-Time Simulation 24 March 2009 18 preDiCT partners