Simbuca [1], using a graphics card to simulate Coulomb interactions in a Penning trap Simon Van Gorp [1]: S. Van Gorp et al. (2011), Nuclear Instruments and Methods in Physics research A, 638,

Simbuca motivation The WITCH experiment: searches for physics beyond the Standard Model by measuring the recoil energy distribution of the nucleus after b- decay. This by comparing the expected simulated energy distribution (of recoil ions after  decay) with the experimentally obtained energy distribution -> Any mismatch would hint to new physics. [2]: S. Van Gorp et al. (2012), to be submitted to PRC (PRL?) Without Simbuca (i.e. assuming a perfectly cooled ion cloud) the Standard Model value was found to be 4  off its expected value [2]. Simulations are important! Therefore the source of ions in the Penning trap ( stored there for up to a few seconds) has to be simulated properly. -> The duration of a computer simulation is dominated by the Coulomb interaction calculation Simon Van Gorp EFTMS 20122nd of April, /13

Simbuca overview Simbuca is a modular Penning trap simulation package. Reading external fieldmaps Comsol SIMION Trap excitations 3 different integrators 3 buffer gas routines Can run on CPU and GPU Compile with g++ or icpc Several analysis tools are provided A Makefile is provided Support by me Simon Van Gorp EFTMS 20122nd of April, /13

Integrators and buffer gas models Integrators: 4 th and 5 th order Runga Kutta with adaptive step size and error control 1 st order (predictor corrector) Gear method Boris algorithm ? [3] Buffer gas models: Langevin or polarizability model (= for all mases) Ion mobility based model ( ≈ for all mases) HS1 SIMION model IonCool model ? [4] [3]: Boris J.P. (1970), Proc. 4th Conf. Num. Sim. Plasmas, 3-67 [4]: Schwarz S. et al. (2006), NIM A 566 2, Simon Van Gorp EFTMS 20122nd of April, /13

Coulomb interactions Simulation time scales with O(N 2 ) Tree methods (Barnes Hut, PM, P 3 M, PIC, FMM) reduces this to O(N log N) Space is divided in nodes. Which are subdivided A node has the total charge and mass, and is located on the centre of mass. Approx. long range force by aggregating particles into one particle and use the force of this one particle Scaled Coulomb force puts more weight to the charge of one ion to simulate more ions. Works well [5,6,7]. [5]: D. Beck et al. (2001), Hyperfine Interactions, 132, 469–474 [6]: S. Sturm et al. (2009), AIP Conference Proceedings, 1114(1), 185–190 [7]: S. Van Gorp (2012), PhD thesis, Leuven Simon Van Gorp EFTMS 20122nd of April, /13

Benefits of a GPU 1. Parallellism due to multiple stream processors 2. SIMD structure (pipelining) 3. Very fast floating point calculations 4. CUDA programming language (2007) 8 x 16 stream processors ≈ each comparable to a single processor = comparable to a conveyor belt with the threads being the workers Geforce 8800 GTX Simon Van Gorp EFTMS 20122nd of April, /13

Chamomile scheme The Chamomile scheme by Hamada and Iitaka (2007) [8] calculated the gravitational interaction between entities precisely Each set of i-particles is coupled to its own conveyor belt j-particles are sequentually presented to each conveyor belt At the end the result of all conveyor belts is being summed to obtain the force between the particles [8]: T. Hamada and T. Iitaka (2007), arXiv.org:astro-ph/ Simon Van Gorp EFTMS 20122nd of April, /13

Chamomile scheme: practical usage Black box function provided by Hamada and Iitaka: Gravitational force ≈ Coulomb Force Conversion coefficient: Needed: - 64 bit linux - NVIDIA Graphics Card that supports CUDA - CUDA environment v Not needed: - CUDA knowledge - … Simon Van Gorp EFTMS 20122nd of April, /13

GPU vs CPU GPU blows the CPU away. The effect becomes more visible with even more particles simulated. Simulated is a quadrupole excitation for N ions, moving 100 ms with buffer gas. This takes 3 days with a GPU compared to 3-4 years with a CPU! GPU improvement factorCPU and GPU simulation time Simon Van Gorp EFTMS 20122nd of April, /13

Simbuca: usage by other groups 1. WITCH Behavior of large ion clouds Energy and position distribution 2. Smiletrap (Stockholm) Highly charged ions Stochastic cooling processes 3. ISOLTRAP (CERN) In-trap decay [9] Investigate the influence of Coulomb repulsion between ions in a Penning trap 4. Piperade (Orsay and MPI Heidelberg) Simulate mass separation of ion clouds 5. ISOLTRAP (Greifswald) isobaric buncher, mass separation and negative mass effect [10] 6. CLIC accelerator (CERN) Simulate bunches of the beam [9]: A. Herlert, S. Van Gorp et al. Recoil-ion trapping for precision mass measurements, to be published [10]: Wolf, R et al. (2011). Hyperfine Interactions, 199, 115–122 Simon Van Gorp EFTMS 20122nd of April, /13

Tree codes on a GPU Octgrav v1.7 = first tree code on the GPU [11] Under construction, close to being finished [12] The step size (  t) defines the speed of the program (  t ~  c -1 ). Bring B out the equation -> If  c is constant ->  t schrinks [3,13,14] MPI-ing the code: just started by parallellizing the force calculation Improvements [11]: Gaburov, E et al. (2010). Procedia Computer Science, 1(1), 1119 –1127. [12]: Iitaka, T. (2012). A novel tree code for the gpu. private communication. [13]: Spreiter, Q. & Walter, M. (1999). Journal of Computational Physics, 152(1), 102 – 119. [14]: Herfurth, F et al. (2006). Hyperfine Interactions, 173, 93–101. Simon Van Gorp EFTMS 20122nd of April, /13

An example without Coulomb with Coulomb 50  s When trapping a large amount of ions, the cloud’s own electric field will create an E x B drift force for the ions with Simon Van Gorp EFTMS 20122nd of April, /13 Y X

1. A versatile Penning trap simulation package Simbuca is presented 2. The first program that uses a GPU to calculate Coulomb interactions 3. GPU computing is a new field of which we barely scratched the surface 4. Simbuca will continue to develop in the future Conclusion Simon Van Gorp EFTMS 20122nd of April, /13

Thank you for your attention

Starting up a simulation (1) Download program on a linux/windows PC ( Compile the program …. 15/18 Simon Van Gorp – MPI Heidelberg –

Simon Van Gorp – MPI Heidelberg /24 Change variables in main.cpp according to what simulation you want to do Starting up a simulation (2)

Starting up a simulation (3) Change variables in main.cpp 17/18 Simon Van Gorp – MPI Heidelberg –

Compile Change variables in Makefile compile the program 18/18 Simon Van Gorp – MPI Heidelberg –

execute Go to the directory with executable and execute the program check the logfile check the outputfile 19/18 Simon Van Gorp – MPI Heidelberg –

Run the simulation Post-process with functions parser.cpp or with root / linux bash 20/18 Simon Van Gorp – MPI Heidelberg –

21/18 Simon Van Gorp – MPI Heidelberg – Backup slides

22/18 Simon Van Gorp – MPI Heidelberg – Backup slides

23/18 Simon Van Gorp – MPI Heidelberg – Backup slides

24/18 Simon Van Gorp – MPI Heidelberg – Backup slides

Simulation Motivation WITCH compares the simulated spectra with an experimentally expected spectra to determine the  angular correlation coefficient a. The source of ions in the Penning trap has to be simulated properly. Separating ion species in a Penning trap is strongly distorted by the mutual Coulomb repulsion: both broadening and a shift of the excitation frequency has been observed [3] 25/18 The duration of a computer simulation is dominated by the Coulomb interaction calculation [2]: S. Van Gorp et al., to be submitted to PRC (PRL?) [3]: A. Herlert et al., Hyperfine Interactions, 199, 211–220 Without Simbuca (i.e. assuming a perfectly cooled ion cloud) the Standard Model value was found to be 4  off its expected value [2]. Simon Van Gorp – MPI Heidelberg –

Cs 1+ ions Helium Buffer gas (p=10 -4 mbar T=293 K) With Coulomb interaction. Scaled Coulomb factor of > mimic 10 7 ions 1. Dipole excitation (0.5 V amplitude. 5 ms duration and  - frequency) ms cooling Movie time 26/18 Simon Van Gorp – MPI Heidelberg –