Presented by XGC: Gyrokinetic Particle Simulation of Edge Plasma CPES Team Physics and Applied Math Computational Science

2 Klasky_XGC_0611 Computational Science California Institute of Technology J. Cummings Lawrence Berkeley National Laboratory Shoshani Oak Ridge National Laboratory R. Barreto, S. Klasky, P. Worley Princeton Plasma Physics Laboratory S. Ethier, E. Feibush Rutgers M. Parashar, D. Silver University of California - Davis B. Ludäscher, N. Podhorszki University Tennessee - Knoxville M. Beck University of Utah S. Parker CPES Team Physics and Applied Math New York University Chang, Greengard, Ku, Park, Strauss, Weitzner, Zorin Oak Ridge National Laboratory Schultz, D’Azevedo, Maingi Princeton Plasma Physics Laboratory Hahm, Lee, Stotler, Wang, Zweben Columbia University Adams, Keyes Lehigh University Bateman, Kritz, Pankin University of Colorado Parker, Chen University of California - Irvine Lin, Nishimura Massachusetts Institute of Technology Sugiyama, Greenwald Hinton Associates Hinton

3 Klasky_XGC_0611 Physics in tokamak plasma edge  Plasma turbulence  Turbulence suppression (H-mode)  Edge localized mode and ELM cycle  Density and temperature pedestal  Diverter and separatrix geometry  Plasma rotation  Neutral collision ITER ( Edge turbulence in NSTX 100,000 frames/s) Diverted magnetic field

4 Klasky_XGC_0611 XGC development roadmap Black: Achieved Blue: In progress Red: To be developed Full-f neoclassical ion root code (XGC-0) Buildup of pedestal along ion root by neutral ionization Full-f ion-electron electrostatic code (XGC-1) - Whole edge Neoclassical solution Turbulence solution Full-f electromagnetic code (XGC-2) Study L-H transition Multi-scale simulation of pedestal growth in H-mode XGC-MHD coupling for pedestal-ELM cycle

5 Klasky_XGC_0611 XGC 1 code  Particle-in-cell code  5-dimensional (3-D real space + 2-D velocity space)  Conserving plasma collisions (Monte Carlo)  Full-f ions, electrons, and neutrals  Gyrokinetic Poisson equation for neoclassical and turbulent electric field  PETSc library for Poisson solver  MPI for parallelization  Realistic magnetic geometry containing X-point  Particle source from neutral ionization

6 Klasky_XGC_0611 RoutineTime %Peak Performance Total100%6% Poisson7%~1.5% Pushing37%~8% Charging50%~4% Peak performance of XGC1 on Jaguar  131M ions and 131M electrons, 200K nodes  Peak performance with 2048 cores, using strong scaling results  Working with team members to increase peak performance to 18%

7 Klasky_XGC_0611 Scalability of XGC1 on Jaguar: Near linear scaling for strong, linear scaling for weak scaling XGC Strong Scaling : 131M ions and electrons, 200K grid XGC Weak Scaling : 50K ions and electrons/core 1001,00010,000 1.E+06 1.E+07 1.E+08 1.E+09 Speed (# of particle x step/s) Number of Cores 1.E+04 1.E+05 1.E+06 1.E ,00010,000 Number of Cores Speed (# of particle/s)

8 Klasky_XGC_0611 Neoclassical potential and flow of edge plasma from XGC1 Electric potential Parallel flow and particle positions

9 Klasky_XGC_0611 Phs-0: Simple coupling: with M3D and NIMROD XGC-0 grows pedestal along neoclassical root MHD checks instability and crashes the pedestal The same with XGC-1 and 2 Phs-2: Kinetic coupling: MHD performs the crash XGC supplies closure information to MHD during crash Phs-3: Advanced coupling: XGC performs the crash M3D supplies the B crash information to XGC during the crash XGC-MHD coupling plan Black: Developed Red: To be developed

10 Klasky_XGC_0611 Data replication XGC-M3D code coupling Code coupling framework with Kepler-HPC XGC on Cray XT3 End-to-end system 160p, M3D runs on 64P Monitoring routines here Ubiquitous and transparent data access via logistical networking User monitoring Data replication Post-processing 40 Gb/s Data archiving

11 Klasky_XGC_0611 M3D equilibrium and linear simulations new equilibrium from eqdsk, XGC profiles Equilibrium poloidal magnetic flux Linear perturbed poloidal magnetic flux, n = 9 Linear perturbed electrostatic potential

12 Klasky_XGC_0611 Contact Scott A. Klasky Lead, End-to-End Solutions Center for Computational Sciences (865) Klasky_XGC_0611