Building an Electron Cloud Simulation using Bocca, Synergia2, TxPhysics and Tau Performance Tools Phase I Doe SBIR Stefan Muszala, PI DOE Grant No DE-FG02-08ER85152 Tech-X Corporation Boulder, CO

Tech-X Corporation Particle accelerator programs play a significant role in 14 out of 28 DOE laboratories which span a number of DOE offices such as the Offices of High Energy Physics (HEP), Nuclear Physics (NP) and Basic Energy Sciences (BES) (Facilities for the Future of Science) Accelerator simulation is required throughout the life- cycle of accelerators in four areas -Design -Analysis -Optimization -Upgrading Accelerator simulations play vital near-, medium-, and long-term roles

Tech-X Corporation Software reuse and common interfaces Ability to compose simulations Portability Mixed language programming interoperability Performance analysis of composed simulations High-performance accelerator software should allow complex applications while promoting good software engineering practices

Tech-X Corporation The Electron Cloud Effect (ECE) is one of the most pervasive issues in accelerator modeling and is part of the COMPASS SciDAC project An ECE simulation combines beam dynamics with cloud generation codes Electrons bouncing off of beam walls often emit more electrons due to secondary emission and eventually build into a cloud. The ECE is important to particle accelerator simulations since the cloud causes the proton beam to degrade Community Petascale Project for Accelerator Science and Simulation (COMPASS) –COMPASS goal :: developing “a comprehensive set of interoperable components for beam dynamics, electromagnetics, electron cooling, and advanced accelerator modeling” (Spentzouris, Community Petascale Project for Accelerator Science and Simulation (COMPASS). FNAL DOCDB, CD-doc- 2098, version 1, 2007) –Is a SciDAC2 project funded through HEP, NP, BES and ASCR Synergia2 provides the beam dynamics code (Python, F90, C++) TxPhysics provides the cloud generation code (C )

Tech-X Corporation CCA Ecloud addresses ECE simulation shortcomings and provides direct support to the COMPASS mission by using the Common Component Architecture (CCA) Software engineering hurdles specific to ECE simulations -Beam dynamics (Synergia2) and cloud generation (TxPhysics) codes are often written in different programming languages ==> Need to solve language interoperability problem -Timescales of particles in beam dynamics involve movement over km (10^5 cm) while those of cloud generation involve movement over cm ==> Requires solution to software interoperation problem so codes may be easily coupled and composed The Common Component Architecture (CCA) addresses these hurdles by providing a framework and language interoperability (Babel) TAU performance tools provide necessary performance monitoring

Tech-X Corporation CCA Ecloud uses the BOCCA and the CCA framework to drive a simulation using TxPhysics and Synergia2 components A)Synergia2 for Beam Dynamics (calculates x,y,z,v of simulated particles) B)TxPhysics for Electron Cloud Generation a. Number of secondary electrons produced in each impact b. Energy spectrum of those electrons

Tech-X Corporation Bringing good computer science practices to physics applications: svn and trac

Tech-X Corporation Initial work involved providing CCA components to TxPhysics and testing ports and interfaces. All CCA TxPhysics components TxIonPack component testing

Tech-X Corporation The full CCA Ecloud simulation uses TxPhysics, Synergia2 and Tau Performance Tools components.

Tech-X Corporation Comparison of original and CCA versions of the number of electrons produced after each bunch crossing shows similar behavior.

Tech-X Corporation Comparison of original and CCA versions of beta_{x,y,z} vs. {x,y,z} show similarity after multiple bunch crossings. Cross=1 Cross=9 ORIGINAL CCA

Tech-X Corporation Initial profiles indicate that integrators and solvers are computationally most expensive

Tech-X Corporation The granularity of components is an important design question synergia2/synergia/propagate.py def propagate(…) S2_fish (solver for space charge simulation) gourmetWrapPropagate S2_fish.apply_space_charge_kick(…) Current CCA Component granularity CCA Component granularity if we want to interchange solvers

Tech-X Corporation CQoS requires understanding a number of parameters Computer Architecture (L1,L2,TLB miss rate FP vs. Int instrs., etc…) Compiler optimization (What works Best for A particular Code+Arch) Programming language (Inner loops in C/C++ or Python?!?) Coarse and Fine Grain Components (Overhead with component calls) Ease of use and will people really use this? Does it pay to swap a component? (time to swap vs. time to execute) CQoS

Tech-X Corporation Future work involves continuing on current paths and focusing specifically on CQoS work. Future Paths –Streamline Synergia2 build system (recall port to x86_64) to work with BOCCA (which makes using CCA dramatically easier) –Define interfaces on all levels with regards to Babel rules –Implement different versions of components –Move CCA Ecloud code to FNAL computing facilities –Apply CQoS dynamically in a full operating environment (Lois Curfman McInnes, Boyana Norris) –Harden ports and interfaces for TxPhysics (Peter Stoltz, Seth Veitzer) –Test full coupling capability between multiple TxPhysics and Synergia2 components (Jim Amundson, Paul Lebrun) –Update documentation and current work on Trac Site Future Questions –What is the role of Boost now that Babel is also available? –Can we reduce outside software dependencies? –How do we educate a user to want to build and use these tools?

Tech-X Corporation CCA Ecloud involves multiple institutions and collaborators all of whom will provide project feedback and guidance Personnel –Jim Amundson (Fermi National Accelerator Laboratory) –Lois Curfman McInnes (Argonne National Labs) –Paul Lebrun (Fermi National Accelerator Laboratory) –Boyana Norris (Argonne National Labs) –Peter Stoltz (Tech-X) –Seth Veitzer (Tech-X) Institutions and URLs –COMPASS (Community Petascale Project for Accelerator Science and Simulation), SciDAC-2 program, –Common Component Architecture (CCA), Bocca Ccaffeine Babel CQoS (Computational Quality of Service), –Tuning and Analysis Utilities (TAU) for Performance Analysis and CQoS, –TASCS (The Center for Technology for Advanced Scientific Component Software),