1. GATE simulations on the Eumed Grid infrastructure N. Alioui-Bendjaballah Laboratoire des Sciences Nucléaires et Interaction Rayonnement-Matière Field: Medical Physics USTHB, Algiers, Algeria EPIKH Workshop Algiers, 15 July 2010

2. Medical physicists: Quality Assurance in hospital services using radiations (Staff+Patients)‏ Medical Physics domain (Training & research): Nuclear medecine: radiodiagnostic Radiotherapy : cancer treatment

3. Why we need to use the grid? Steps followed for the radiodiagnostic : 1-Patient preparation with isotope injection : *Longer Decay Times: Technecium99m:6h, Iode123:13h, Xenon133:5d **Shorter Decay Times: Fluor18:108min,Oxygen:~2min,Carbon11:20min 2-Patient scanned with the Detector system: *SPECT **TEP 3-Use Algorithms to reconstruct images 4-Diagnostic done Tomography Emission Positron (TEP)‏

4. Why we need to use the grid? Steps followed for the radiotherapy treatement 1-CT simulations with patient in treatment position: Slices (DICOM) Binary matrix + transcription of Hounsfield units in atomic composition and density 2- Data machine used to treatment: Cobalt60 teletherapy unit, LINear AccleratorNAC (LINAC) 3-Treatment planning system (TPS) using algorithms to evaluate: -Dose delivred to target volume with high protection of the organs at risk -Time Exposition(Unity Monitoring)‏ 4-Treatment done LINAC

5. Why we need to use the grid? Imagery Medica l Radiotherapy Algorithms Analytic methods Advantages: Short Time calculation, adapted for clinical routine Disadvantages: use approximations, precision limited

6. Why we need to use the grid? Using algorithms based on Monte Carlo Method (Statistical method)‏ Random Number Generator (RNG)‏ Data cross sections Interaction in medium described by Probability densities functions Advantage High precision in dose computation Disadvantages Large Time calculation : Big statistical to converge to the scale Not suitable for clinical routine

7. Why we need to use the grid? To reduce the Monte Carlo calculation times to acceptable levels The Grid is the computing infrastructure able to do this job Need Large number CPUs : High performance Computation in parallel Improve the reconstruction image and dose computation algorithms Improve System Detection make Monte Carlo as the standard approach to radiotherapy treatment planning

8. Several codes based on Monte Carlo Method  Geant4: Geometry ANd Tracking http://geant4.web.cern.ch  EGS4: Electron Gamma Shower  PENELOPE: PENetration and Energy Loss of Positrons and Electrons  MCNP4: Monte Carlo N Particles

9. Specific code GATE (Geant4 Application for Tomographic Emission) http://www.opengatecollaboration.org

10. A GATE simulation generating a lot of particles in matter could take a very long time to run on a single processor Hundreds Millions of particles are divided into little ones: many macros files are created with different RNG (avoid to repeat the history of particles)‏ Use many CPUs which runs in parallel: time computation decrease, statistical increase U se EUMEDgrid Infrastructure

11. GEANT4 source code(C++ Language): geant4 ‐ 9.2.p03 ‐ 1gate.i386.rpm geant4 ‐ G4EMLOW ‐ 6.2 ‐ 1gate.noarch.rpm geant4 ‐ G4NDL ‐ 3.13 ‐ 1gate.noarch.rpm geant4 ‐ G4PhotonEvaporation ‐ 2.0 ‐ 1gate.noarch.rpm geant4 ‐ G4RadioactiveDecay ‐ 3.2 ‐ 1gate.noarch.rpm geant4 ‐ G4ABLA ‐ 3.0 ‐ 1gate.noarch.rpm CLHEP libraries: clhep ‐ 2.0.4.2_ ‐ 1gate.i386.rpm Gate source code: gate ‐ 6.0.0grid ‐ 1.i386.rpm LMF(List Mode Format) libraries : lmf ‐ 3.0 ‐ 1gate.i386.rpm ECAT libraries: ecat-7.0-1gate RootSoftware: root ‐ 5.24.00.Linux-slc4-gcc4.3.tar.gz GATE Installation:

12. Installation GATE(version 6.0.0) on the EUMEDgrid Directory : “VO_EUMED_SW_DIR/gate-6.0.0” env/env_gate.sh & env_geant4.sh lmf/ ecat/ geant4/ bin/ config/ data/ environments/ include/ lib/ source/ /tmp root/ clhep/ gate/ lib/ tmp/ benchmarks/ GNUmakefile bin/ examples/ source/

13. Middlware of the EUMED : gLite Machines and services part of the grid:. User Interface (UI): gateway to the EUMEDgrid services. Computing Element (CE): built on a farm computing nodes called Worker Nodes (WN)‏. Storage Element (SE) :provides uniform access and services to large storage spaces. Information System: provides information about the grid resources and their status. Data management

14. Requirements to access the grid: -Obtaining a certificate: https://security.fi.infn.it/CA/en/mgt/getCA.php -Be part of Virtual Organisation (VO) EUMED: able to generate a proxy certificate to launch jobs Lifetime limited: 12 h voms-proxy-init --vo eumed Lifetime longer: 7 days myproxy-init --voms eumed -s myproxy.ct.infn.it -d

15. Gate running on the EUMEDgrid Example : compute the energy deposited in water phantom(size field 20x20cm2) and in head Patient (cavum)‏ Input files: 1- Introduce the telecobalt head geometry in details generate file macro Telecobalt head is still used at radiotherapy service “ Centre Pierre et Marie Curie” (CPMC) of Algiers: Co60 source (2 photons: 1.17 MeV,1.33MeV) for treatment tumors in the cavum, brain, breast,... 2- File GateMaterials.db: material components and densities 3-SlicesCT(DICOM) converted to Analyze75 Format: 2 files are generated.hdr &.img each pixel value density The size files are big Store them in SE 4-Modelling Source Cobalt

16. U I WMS I S (Information System)‏ S E (Storage Element)‏ C E & W N (Computing Element & Worker Nodes)‏ InputSandbox OutputSandbox UI JDL Our Job flow in EUMEDgrid Store input datafile:.hdr &.img Download input User Interface Workload Management System

17. Jdl file for computing energy deposited in water phantom: simulated 2 Millions particles Type = "normal"; Type = "Job"; Executable = "/bin/sh"; Arguments = "./script.sh"; StdOutput = "msg.out"; StdError = "msg.err"; OutputSandbox = {"msg.out","msg.err","f20x20.root"}; InputSandbox = {"script.sh","field20x20.mac",”GateMaterials.db”}; RetryCount = 3; Requirements = RegExp("ce-01.roma3.infn.it",other.GlueCEUniqueID); Rank=(other.GlueCEStateWaitingJobs == 0 ? other.GlueCEStateFreeCPUs : - other.GlueCEStateWaitingJobs);

18. Script.sh: #!/bin/sh echo "=================VO_EUMED_SW_DIR/gate===========" ls -l $VO_EUMED_SW_DIR/gate-6.0.0 echo "===============VO_EUMED_SW_DIR/gate/root======" ls -l $VO_EUMED_SW_DIR/gate-6.0.0/root echo "==============GATE BINARY==================" ls -l $VO_EUMED_SW_DIR/gate-6.0.0/gate/bin/Linux-g++ echo "=============ENVIRONMENT FILE=============" source $VO_EUMED_SW_DIR/gate-6.0.0/env/env_gate.sh echo "==============EXECUTE MACRO FILE========================" $VO_EUMED_SW_DIR/gate-6.0.0/gate/bin/Linux-g++/Gate field20x20.mac

19. Jdl file for computing energy deposited in Head Patient (cavum): 3 macros sent in one job to increase statistical (3x 2 Millions particles)‏ JobType = "Parametric"; Executable = "/bin/sh"; Arguments = "scriptparam.sh cavu_PARAM_.mac"; InputSandbox = {"scriptparam.sh","cavu_PARAM_.mac"}; Parameters= 4; ParameterStep =1; ParameterStart = 1; StdOutput = "msg_PARAM_.out"; StdError = "msg_PARAM_.err"; OutputSandbox = {"msg_PARAM_.out","msg_PARAM_.err","dose_PARAM_- Dose.hdr","dose_PARAM_-Dose.img"}; RetryCount = 3; Requirements = RegExp("ce-01.roma3.infn.it",other.GlueCEUniqueID); Rank=(other.GlueCEStateWaitingJobs == 0 ? other.GlueCEStateFreeCPUs : - other.GlueCEStateWaitingJobs)‏

20. Script example to the parametrical job #!/bin/sh hostname -f export LCG_GFAL_INFOSYS=wms-01.eumedgrid.eu:2170 export LCG_CATALOG_TYPE=lfc export LFC_HOST=lfc.ulakbim.gov.tr lcg-cp --vo eumed lfn:/grid/eumed/nouria/data_input.tar file://$PWD/data_input.tar tar xvf data_input.tar ls -a echo "=================VO_EUMED_SW_DIR/gate===========" ls -l $VO_EUMED_SW_DIR/gate-6.0.0 echo "===============VO_EUMED_SW_DIR/gate/root======" ls -l $VO_EUMED_SW_DIR/gate-6.0.0/root echo "==============GATE BINARY==================" ls -l $VO_EUMED_SW_DIR/gate-6.0.0/gate/bin/Linux-g++ echo "=============ENVIRONMENT FILE=============" source $VO_EUMED_SW_DIR/gate-6.0.0/env/env_gate.sh echo "==============EXECUTE MACRO FILE========================" $VO_EUMED_SW_DIR/gate-6.0.0/gate/bin/Linux-g++/Gate $1 Store data_input.tar SE

21. Pick up results: Outputjob in tmp directory 1-Energy deposited Edep in tank water: file f20x20.root analyzed with root z Water tank Telecobalt head 20x20 cm2 y

23. Pick up results: Outputjob in tmp directory (3 nodes)‏ 2-Energy deposited Edep in Head Patient: 3 files (.hdr/.img )‏ Cavum centre Gantry Telecobalt head Position: 90° field size:12x16 cm2 Cavum centre Energy Distribution Histogramm deposited in Cavum centre

24. Statistical is poor: need to simulate hundreds millions particules to provide well distribution dose Cavum centre

25. Time computation on the grid ~ Time computation on the single computer/x if we send x files macros in a parametric job

26. Conclusion My participation to EPIKH school allows me by using the resource of the grid EUMED to: My participation to EPIKH school allows me by using the resource of the grid EUMED to: 1- have access to CE in and outside Algeria; 1- have access to CE in and outside Algeria; 2- have access to programs implemented; 2- have access to programs implemented; 3- decrease time in computing!! 3- decrease time in computing!! To do in the futur with CERIST collaboration: -Install cluster in university and translate the application Gate -Install cluster in university and translate the application Gate

27. AKNOWLEDGMENTS MANY THANKS TO : Roberto BARBERA Giuseppe La Rocca & Valeria Ardizzone all the staff of CERIST specially Aouaouche El-Maouhab