Technische Universität Darmstadt, Fachbereich Elektrotechnik und Informationstechnik Schloßgartenstr. 8, Darmstadt, Germany - URL: Technische Universität Darmstadt, Fachbereich Elektrotechnik und Informationstechnik Schloßgartenstr. 8, Darmstadt, Germany - URL: S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Large Scale 3D Wakefield Simulations with PBCI S. Schnepp, W. Ackermann, E. Arevalo, E. Gjonaj, and T. Weiland "Wake Fest 07 - ILC wakefield workshop at SLAC” December 2007

1 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Outline IntroductionIntroduction Numerical MethodNumerical Method Parallelization StrategyParallelization Strategy Modal Termination of Beam PipesModal Termination of Beam Pipes PBCI Simulation ExamplesPBCI Simulation Examples

2 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Introduction 1.A new generation of LINACs with ultra-short electron bunches a.bunch size for ILC: 300 μm b.bunch size for LCLS: 20 μm 2.Geometry of tapers, collimators… far from rotational a.8 rectangular collimators at ILC-ESA in the design process b.30 rectangular-to-round transitions in the undulator of LCLS 3.Many (semi-) analytical approximations become invalid a.based on rotationally symmetric geometry b.low frequency assumptions (Yokoya, Stupakov) c.detailed physics needed for high frequency wakes (Bane) Motivation for PBCI:

3 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Introduction mm 38.1mm 15.05mm 2.75mm 17.65mm 38.05mm ILC-ESA collimator #8 ILC-ESA collimator #3 Courtesy: N. Watson, Birmingham 1 m bunch length300μm collimator length~1.2m catch-up distance~2.4m beam view side view

4 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Introduction bunch length1cm taper length50cm tube shielding outgoing pipe step vacuum vessel gap PITZ diagnostics double cross Tapered III Courtesy: R. Wanzenberg, DESY bunch length2.5mm bunch width2.5mm structure length325mm vacuum slots elliptic pipe

5 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Introduction 3D-codes 1.Wake field simulations in arbitrary 3D-geometry 3D-codes (quasi-) dispersionless codes 2.Accurate numerical solutions for high frequency fields (quasi-) dispersionless codes parallelized codes 3.Utilizing large computational resources for ultra-short bunches parallelized codes moving window codes 4.Specialized algorithms for long accelerator structures moving window codes There is an actual demand for:

6 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Introduction DimensionsNondispersiveParallelizedMoving window BCI / TBCI2.5DNo Yes NOVO2.5DYesNo ABCI2.5DNo Yes MAFIA2.5/3DNo Yes GdfidL3DNoYes Tau3P3DNoYesNo ECHO2.5/3DYesNoYes CST Particle Studio 3DNo PBCI3DYes NEKCEM3DQuasiYesNo Time 20 years An (incomplete) survey of available codes 5 years

7 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Outline IntroductionIntroduction Numerical MethodNumerical Method Parallelization StrategyParallelization Strategy Modal Termination of Beam PipesModal Termination of Beam Pipes PBCI Simulation ExamplesPBCI Simulation Examples

8 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Numerical Method The FIT discretization FIT semidiscrete energy conservation semidiscrete charge conservation Topology of FIT:

9 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Numerical Method Using the conventional leapfrog time integration Behavior of numerical phase velocity vs. propagation angle Stable but large dispersionNo dispersion but unstable

10 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Numerical Method Implementing a dispersion-free scheme leads to this: Numerical phase velocity and amplification vs. propagation angle Neutrally stable No longitudinal dispersion

11 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Outline IntroductionIntroduction Numerical MethodNumerical Method Parallelization StrategyParallelization Strategy Modal Termination of Beam PipesModal Termination of Beam Pipes PBCI Simulation ExamplesPBCI Simulation Examples

12 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Parallelization Strategy A balanced domain partitioning approach total computational domain intermediate subdomains active subdomains processor #1 processor #2 processor #3 Equal loads assigned to each node: binary tree leafs

13 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Parallelization Strategy Processor Number Number of Grid Points moving grid window Distribution of grid points Example: Tapered transition for PETRA III Domain partitioning pattern for 7 processors Grid points Total Min Max Dev.< 1.0%

14 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Ideal 1E+06 cells Parallelization Parallel performance tests

15 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Ideal 10E+06 cells Parallelization Parallel performance tests

16 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Ideal 50E+06 cells Parallelization Parallel performance tests

17 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Ideal 100E+06 cells Parallelization Parallel performance tests

18 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Ideal 200E+06 cells Parallelization Parallel performance tests

19 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Parallelization Strategy TEMF Cluster: 20 INTEL 3.4GHz, 8GB RAM, 1Gbit/s Ethernet Network Parallel performance tests

20 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Parallelization Strategy Parallel performance tests TEMF Cluster: 20 INTEL 3.4GHz, 8GB RAM, 1Gbit/s Ethernet Network

21 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Outline IntroductionIntroduction Numerical MethodNumerical Method Parallelization StrategyParallelization Strategy Modal Termination of Beam PipesModal Termination of Beam Pipes PBCI Simulation ExamplesPBCI Simulation Examples

22 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Modal Termination of Pipes 1.Indirect integration of potential for 2D-structures (Weiland 1983, Napoly 1993) direct indirect Indirect integration schemes irrotational 2.Generalization for 3D-structures (A. Henke and W. Bruns, EPAC’06, July 2006, Edinburgh, UK)

23 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Modal Termination of Pipes directmodal Modal approach n-th (TM) mode contribution spectral coefficient of n-th (TM) mode - ~ 1982 Robert Siemann - “Indirect methods for wake potential integration”, I. Zagorodnov, PRSTAB 9 ‘06 - “Eigenmode expansion method in the indirect calculation of wake potential in 3D structures”, X. Dong, E. Gjonaj, ICAP’06

24 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Modal Termination of Pipes 1.Time domain integration in the inhomogeneous sections: 2.Modal analysis at z = 0: 3.Compute spectral coefficients (FFT): 4.Compute wake potential contribution per mode (IFFT): 5.Compute wake potential transition in the outgoing pipe:

25 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) lowest eigenmode5 eigenmodes10 eigenmodes15 eigenmodes 20 eigenmodes Modal Termination of Pipes Using FD reconstruction in long intermediate pipes bunch full Time Domain bunch E z / [kV /m] injector section diagnostics cross 1mm step FD analysis reconstruction In an accurate simulation ~300 modes

26 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Outline IntroductionIntroduction Numerical MethodNumerical Method Parallelization StrategyParallelization Strategy Modal Termination of Beam PipesModal Termination of Beam Pipes PBCI Simulation ExamplesPBCI Simulation Examples

27 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) ILC-ESA collimator ILC-ESA collimator #8 Convergence vs. grid step bunch size300μm no. of grid points~450M no. of processors24 simulation time85hrs Moving window: 3 mm length

28 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) ILC-ESA collimator ILC-ESA collimator #8 Direct and transition wakes m m mm 38.1 mm mm 2.75 mm mm bunch

29 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) TESLA / HOM coupler TESLA 9-cell cavity E z / [kV /m] bunch length1mm bunch charge1nC cavity length1.5m no. of grid points~760M no. of processor cores 408 simulation time~40hrs bunch

30 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) TESLA / HOM coupler TESLA 9-cell cavity Longitudinal wake potential

31 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) TESLA / HOM coupler HOM / HOM-RF coupler (present DESY design) Beam view Courtesy: I. Zagorodnov Upstream coupler HOM Downstream coupler RF + HOM

32 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) TESLA / HOM coupler ECHO3D PBCI Upstream coupler

33 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) TESLA / HOM coupler ECHO3D Downstream coupler PBCI

34 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) TESLA / HOM coupler ECHO3D PBCI

35 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) old new I. Zagorodnov, M. Dohlus TESLA / HOM coupler

36 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) + old new old new I. Zagorodnov, M. Dohlus + TESLA / HOM coupler

37 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) TESLA / HOM coupler Transverse wake potential Present DESY Design Beam view Preliminary

38 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) + TESLA / HOM coupler W x,sum = W x,up + W x,down W y,sum = W y,up + W y,down Assumption of linearity not valid + 

39 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) TESLA / HOM coupler Proposed DESY Design (Dohlus, Zagorodnov) Beam view (symmetrical coupler positioning) Transverse wake potential Preliminary

40 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) TESLA / HOM coupler proposed present proposed present Preliminary

41 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Tapered Transition PETRA III PETRA III No convergence with MAFIA due to memory limitations and dispersion Complex geometry “Wake Computations for Undulator Vacuum Chambers of PETRA III”, R. Wanzenberg, PAC’07

42 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) PITZ Photoinjector Low-Emittance Injector Development DESY/Zeuthen

43 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) PITZ Photoinjector Optimization studies performed

44 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) PITZ Photoinjector longitudinal current profile longitudinal wake potential

45 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) PITZ Photoinjector

46 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) PITZ Photoinjector

47 S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) PITZ Photoinjector A minimum of the transverse kick was found at 8mm distance.

Technische Universität Darmstadt, Fachbereich Elektrotechnik und Informationstechnik Schloßgartenstr. 8, Darmstadt, Germany - URL: Technische Universität Darmstadt, Fachbereich Elektrotechnik und Informationstechnik Schloßgartenstr. 8, Darmstadt, Germany - URL: S. Schnepp et al. Institut für Theorie Elektromagnetischer Felder (TEMF) Large Scale 3D Wakefield Simulations with PBCI S. Schnepp, W. Ackermann, E. Arevalo, E. Gjonaj, and T. Weiland "Wake Fest 07 - ILC wakefield workshop at SLAC” December 2007