The GATE-LAB system Sorina Camarasu-Pop, Pierre Gueth, Tristan Glatard, Rafael Silva, David Sarrut VIP Workshop December 2012

1 GATE-LAB ? What is GATE-LAB ? It is one module in VIP, dedicated to GATE What is GATE ? A software for Monte-Carlo simulations in medical physics Particles tracking Both for imaging (PET, SPECT, X/proton-radiography, …) and radiation therapy (dose computation) Based on the Geant4 library fro particles physic (CERN) Open source Developed by the OpenGate collaboration Community of users (estimated to 1000 worldwide) First release of GATE in May 2004 ; 18 releases since 2 reference articles : [Phys Med Biol 2004 and 2011] (highly cited)

2 One example GATE simulation of proton cancer treatment Goal: study the dose distribution inside patient data Protontherapy treatment room CT patient data

3 One example Proton beam (complex) Track every particles inside biological material Store deposited energy Around 10 8 particles to track + secondaries About 7 days computing time brute-force, reference Proton beam intensity, energy position orientation

4 One example : results Example of dose distribution Overlaid on the CT data Computed with GATE More than 1 month comp time Done in few hours VIP/GateLab Speed up = 290

5 GATE-LAB specificities Why ? In general MC simulations can be very long (up to days) Access to grid or cluster need skills, not straightforward To provide easy and efficient access to Gate GATE-LAB started before VIP Started since 2008 Test version Open to public since Feb 2012 (beta) Tight integration between GATE and GATE-LAB GATE code has been modified to monitor the progress of the simulations to stop simulation on demand VIP-GATE-LAB has been adapted To manage all GATE output types Very high performance computer

6 What is the GateLab ? Design your simulations Click to load macro file in web browser Monitor simulation Click to download (merged) results Computing (x 10 3 jobs) Computing (x 10 3 jobs) GATE-LAB Upload all files Split simulation Handle errors Load balancing Merge results

7 One example From the GATE-LAB (VIP): Specific input parser Submission options: mode + GATE release Specific job monitoring Specific output merger

8 One example Specific input parser Load the main macro file Parse the macro file to find all needed files Archive and zip together all files Upload files on the GATE-LAB server

9 Launch simulation screen Simulation name Estimated time Gate release Type of submission Go !

10 Monitor simulation screen Total # of particles Current # of particles

11 One example Specific output merger GATE simulations can have multiple results Each result is split into several files (nb of jobs) The merger gather all files and merge them according to the file types One of the most critical point Could be long Incremental merger in development

12 N jobs, p particles. Two modes Static mode : Each job simulates p/N particles Dynamic mode : Jobs simulate particles until the system stops them Static vs Dynamic mode Static Dynamic Two times faster

13 Submission modes Which mode to use ? Dynamic : each job & events must be strictly equivalent If time is involved : use static mode For radiation therapy applications : dynamic For PET/SPECT, one run: static For PET/SPECT, multiple run : not (yet) possible Need to split according to time rather than Events, need refactoring

14 Limits of the GateLab No scripting possible (launch one simulation at a time) Difficult to manage large files Typically needed for phase-space Either as input or output Difficulty for simulation involving time (with jobs not equivalent, not interruptible)

15 Results - GateLab Currently, about 228 registered users About 25 active user/months Active user = launch at least 3 simulations About 200 simulations per months (except in summer...) Global average speedup around 50 (increasing !) Max speedup of 350 (taking queuing & merging time into account)

16 Conclusion GateLab allows easy and free high performance computation for Gate simulations Work in progress New splitting procedure for PET simulations New more robust merger Improved interface Feedback welcome ! Acknowledgments : Sorina Camarasu, Tristan Glatard, Rafael Ferreira da Silva, Pierre Gueth, David Sarrut