1. DOE HEP Physics Program Review June 14-16, 2005 @SLAC Advanced Computations Department Kwok Ko * Work supported by U.S. DOE ASCR & HEP Divisions under contract DE-AC02-76SF00515

2. ACD Mission  Develop new simulation capability to support accelerator R&D at SLAC & accelerator facilities across SC,  Advance computational science to enable ultra-scale computing on SC’s flagship computers (NERSC, ORNL)  Share resources with community and educate/train future computational scientists. Support: Base program, SciDAC, Accelerator projects, SBIR + others Personnel:15 people/13 FTE (5 computational physicists, 7 computer scientists, 2 graduate students, 1 admin/technical assistant) Output: 3 PhD thesis, 5 papers, 3 reports, 30 talks/posters (2003-05) Formed in 2000 to focus on high performance computing with the mission to:

3. ACD R&D Overview & SciDAC Simulation and Modeling Computational Science Parallel Code Development H High Performance Computing (NERSC, ORNL) Accelerators SLAC FNAL ANL Jlab MIT DESY KEK PSI ACD Accelerator Modeling Computational Mathematics Computing Technologies SciDAC LBNL LLNL SNL Stanford UCD RPI, CMU Columbia UWisconsin D SBIR - STAR Inc Modeling and Simulation

4. Electromagnetic Modeling NLC Cell Design Refinement Performance Optimization Visualization  CAD/Meshing PartitioningSolvers Analysis Elements of Computational Science Large-scale electromagnetic modeling is enabled by advancing all elements through SciDAC collaborations

5. SciDAC ESS Team ISICs (TSTT, TOPS, PERC) and SAPP UCD K. Ma, H. Yu Z. Bai RPI M. Shephard, A. Brewer, E. Seol SNL P. Knupp, K. Devine. L. Fisk, J. Kraftcheck LBNL E. Ng, W. Gao, X. Li, C, Yang P. Husbands, A. Pinar, D. Bailey, D. Gunter LLNL L. Diachin, D. Brown, D. Quinlan, R. Vuduc Stanford G. Golub Columbia D. Keyes CMU O. Ghattas V. Akcelik UWisconsin T. Tautges, H. Kim, Computational Mathematics L. Lee, L. Ge, E. Prudencio, S. Chen (Stanford), Accelerator Modeling K. Ko, V. Ivanov, A. Kabel, Z. Li, C. Ng,, L. Xiao, A. Candel (PSI) Computing Technologies N. Folwell, G. Schussman, R. Uplenchwar, A. Guetz (Stanford) SLAC/ACD “Electromagnetic Systems Simulation”

6. Parallel Code Development Electromagnetics (SciDAC funded) Beam Dynamics (SLAC supported) “Unstructured Grid and Parallel Computing” Omega3P Tau3P/T3P S3P Time Domain Simulation With Excitations Frequency Domain Mode Calculation Scattering Matrix Evaluation Finite-Element Discretization Track3P – Particle Tracking with Surface Physics Generalized Yee Grid V3D – Visualization/Animation of Meshes, Particles & Fields Weak-strong Beam-beam Strong-strong Beam-beam TrafiC4 - CSR

7. Achievements in Accelerator Science (Electromagnetics & Beam Dynamics)

8. Omega3P: Sum over eigenmodes NLC DDS Wakefields NLC 55-cell DDS Omega3P/Tau3P computed the long-range wakefields in the 55-cell Damped Detuned Structure to verify the NLC design in wakefield suppression by damping and detuning. Tau3P: Direct beam excitation Omega3P Wakefields Tau3P Wakefields Tau3P: Direct beam excitation

9. NLC Dark Current Dark current @ 3 pulse risetimes Track3P Data -- 10 nsec -- 15 nsec -- 20 nsec Dark current pulses were simulated for the 1 st time in a 30- cell X-band structure with Track3P and compared with data. Simulation shows increase in dark current during pulse risetime due to field enhancement from dispersive effects. Track3P: Dark current simulation Red – Primary particles, Green – Secondary particles Track3P: Dark current simulation

10. ILC Cavity Design An international collaboration (DESY, KEK, SLAC, FNAL, Jlab) is working on a Low- Loss cavity (23% lower cryogenic loss) as a viable option for the ILC linac. SLAC is calculating the HOM damping & multipacting for the DESY and KEK designs. ILC LL 9-cell Cavity Design

11. ILC Cavity HOM Damping Partitioned Mesh of LL Cavity Complex Omega3P is being used to calculate the Q ext of dipole modes in the DESY and KEK LL cavity designs. DESY KEK

12. PEP-II Vertex Bellows Damping Ceramic tile absorber Bellows mode Dielectric loss Omega3P was used to study the effectiveness of ceramic tiles mounted on the bellows convolution to damp localized modes that contribute to HOM heating of the bellows. Bellows modes can be damped to very low Qs (~20-50) Bellows Modes PEP-II Vertex Bellows

13. LCLS RF Gun Cavity Design ACD provided the dimensions for the LCLS RF Gun cavity that meet two important requirements:  minimized dipole and quadrupole fields via a racetrack dual-feed coupler design,  reduced pulse heating by rounding of the z coupling iris. Quad  (  βr)/mm A new parallel Particle-In-Cell (PIC) capability is being developed in T3P for self-consistent modeling of RF guns needed for the LCLS upgrade, future light sources and FELs. Quad

14. LCLS CSR Effects LCLS Bunch Compressor (with P. Emma): Predict FEL performance in the self-consistent Coherent Synchrotron Radiation (CSR) regime for different compressor settings Coherent Edge Radiation: Field viewer module for TraFiC4 allows study of the spatial & temporal behaviour of the detector signal Slice Saturation Power Slice Gain Length

15. Tevatron Beam-Beam Simulation Tevatron (with Y. Cai and T. Sen): Calculate actual lifetimes and lifetime signatures for the machine at injection and collision for different machine parameters New version of parallel beam-beam framework PLIBB: Allows billions of particle-turns Resolves ~100h lifetime (collision case!) Handles chromaticity exactly Strong-strong being integrated Lifetime enhancement with lowered chromaticity Example result - Low particle loss rates at collision PLIBB Results

16. PSI Cyclotron HOM Analysis 1 st ever eigenmode analysis of an entire ring cyclotron as part of a PhD research (L. Stingelin) to investigate the beam-cavity interactions in the existing machine and future upgrade. CAVITY VACUUM CHAMBER MIXED MODES (NEW)

17. Advances in Computational Science (SciDAC)

18. Parallel Meshing (SNL, UWisconsin) Processor : 1 2 3 4 To be able to model multiple ILC cavities a parallel meshing capability has been developed in collaboration with SNL and UWisconsin (PhD thesis) to facilitate the generation of VERY LARGE meshes on the supercomputer directly to overcome the memory limitation of desktops.

19. Omega3P Lossless Lossy Material Periodic Structure External Coupling ESIL ISIL w/ refinement Implicit Restarted Arnoldi SOAR Self-Consistent Loop WSMPMUMPSSuperLUKryov Subspace Methods Domain-specific preconditioners Eigensolvers (LBL, UCDavis, Stanford) With LBL, UCD and Stanford, a comprehensive capability has been under development for solving large, complex RF cavities to accuracies previously not possible. The parallel eigensolver Omega3P has been successfully applied to numerous accelerator cavities and beamline components.

20. Mesh Refinement (RPI) In modeling RIA’s RFQs, Adaptive Mesh Refinement (AMR) provided accuracy gain of 10 and 2 in frequency and wall loss calculations with Omeg3P over standard codes, while using a fraction of CPU time compared to the case without AMR. Wall Loss on AMR Mesh More accurate f and Q predictions reduce the number of tuners and tuning range, and allow for better cooling design AMR speeds up convergence thereby minimizing computing resources Frequency Convergence Qo Convergence

21. Omega3P Sensitivity meshing sensitivity optimization geometricmodel Omega3Pmeshing (only for discrete sensitivity) Shape Optimization (CMU, SNL, LBNL) An ongoing SciDAC project is to develop a parallel shape optimization tool to replace the existing manual process of optimizing a cavity design with direct computation. The capability requires the expertise from SciDAC’s ISICs.

22. Visualization (UCDavis) New graphics tools for rendering LARGE, multi-stream, 3D unstructured data have been developed, to be supported by a dedicated visualization cluster to help in analyzing cavity design, such as mode rotation in the ILC cavity. Graphics tools for rendering LARGE, 3D multi-stream, unstructured data have been developed and a visualization cluster soon be installed, both to support accelerator analysis Mode rotation (in space and time) exhibited by the two polarizations of a damped dipole mode in ILC cavity

23. Dissemination  HEP/SBIR: STAR Inc and ACD are developing the GUIs to interface SLAC’s parallel codes which are in use at e.g. FNAL and KEK. These codes potentially can replace use of commercial software (MAFIA, HFSS) at DOE sites to save costs ~million+ $ per year in leases.  USPAS: SciDAC codes and capabilities are shared regularly with the community via the course “Computational Methods in Electromagnetism” USPAS sponsored by the Cornell University held in Ithaca, NY June 20 - July 1, 2005 Cornell University http://uspas.fnal.gov/

24. Education/Training PhDs completed in ACD ; Yong Sun, SCCM, Stanford University, March 2003 “The Filter Algorithm for Solving Large-Scale Eigenvalue Problems from Accelerator Simulations” Greg Schussman, Computer Science, UCDavis, December 2003 “Interactive and Perceptively Enhanced Visualization of Large, Complex Line-based Datasets” Lukas Stingelin, Physics, Ecole Polytechnique Lausanne, December 2004 “Beam-cavity Interactions in High Power Cyclotrons” PhDs in progress ; Adam Guetz, ICME, Stanford University Sheng Chen, ICME, Stanford University Summer interns – Grad/Undergrad

25. ACD Goals  Continue to support Accelerator Science across SC  Continue SciDAC collaborations in Computational Science  Involve in Astroparticle Physics & Photon Science ILC BPM & Wakefields in LCLS Undulator XFEL SC RF Gun MIT PBG Cavity for Jlab 12 GeV Upgrade ILC LL Cavity & Cryomodule