TerascaleSimulation Tools and Technologies Spectral Elements for Anisotropic Diffusion and Incompressible MHD Paul Fischer Argonne National Laboratory.

Slides:



Advertisements
Similar presentations
NONLINEAR COMPUTATION OF LABORATORY DYNAMOS DALTON D. SCHNACK Center for Energy and Space Science Science Applications International Corp. San Diego, CA.
Advertisements

Dynamo Effects in Laboratory Plasmas S.C. Prager University of Wisconsin October, 2003.
Control of Magnetic Chaos & Self-Organization John Sarff for MST Group CMSO General Meeting Madison, WI August 4-6, 2004.
Outline: I. Introduction and examples of momentum transport II. Momentum transport physics topics being addressed by CMSO III. Selected highlights and.
Madison 2006 Dynamo Fausto Cattaneo ANL - University of Chicago Stewart Prager University of Wisconsin.
Magnetic Chaos and Transport Paul Terry and Leonid Malyshkin, group leaders with active participation from MST group, Chicago group, MRX, Wisconsin astrophysics.
CMSO 2005 Simulation of Gallium experiment * § Aleksandr Obabko Center for Magnetic-Self Organization Department of Astronomy and Astrophysics.
Outline Dynamo: theoretical General considerations and plans Progress report Dynamo action associated with astrophysical jets Progress report Dynamo: experiment.
EXTENDED MHD SIMULATIONS: VISION AND STATUS D. D. Schnack and the NIMROD and M3D Teams Center for Extended Magnetohydrodynamic Modeling PSACI/SciDAC.
Particle acceleration in a turbulent electric field produced by 3D reconnection Marco Onofri University of Thessaloniki.
Algorithm Development for the Full Two-Fluid Plasma System
Design Constraints for Liquid-Protected Divertors S. Shin, S. I. Abdel-Khalik, M. Yoda and ARIES Team G. W. Woodruff School of Mechanical Engineering Atlanta,
MUTAC Review April 6-7, 2009, FNAL, Batavia, IL Mercury Jet Target Simulations Roman Samulyak, Wurigen Bo Applied Mathematics Department, Stony Brook University.
Momentum transport and flow shear suppression of turbulence in tokamaks Michael Barnes University of Oxford Culham Centre for Fusion Energy Michael Barnes.
Network and Grid Computing –Modeling, Algorithms, and Software Mo Mu Joint work with Xiao Hong Zhu, Falcon Siu.
The Stability of Internal Transport Barriers to MHD Ballooning Modes and Drift Waves: a Formalism for Low Magnetic Shear and for Velocity Shear The Stability.
Chamber Dynamic Response Modeling Zoran Dragojlovic.
James Sprittles ECS 2007 Viscous Flow Over a Chemically Patterned Surface J.E. Sprittles Y.D. Shikhmurzaev.
Brookhaven Science Associates U.S. Department of Energy MUTAC Review March 16-17, 2006, FNAL, Batavia, IL Target Simulations Roman Samulyak Computational.
Temperature Gradient Limits for Liquid-Protected Divertors S. I. Abdel-Khalik, S. Shin, and M. Yoda ARIES Meeting (June 2004) G. W. Woodruff School of.
LES of Turbulent Flows: Lecture 3 (ME EN )
Non-disruptive MHD Dynamics in Inward-shifted LHD Configurations 1.Introduction 2.RMHD simulation 3.DNS of full 3D MHD 4. Summary MIURA, H., ICHIGUCHI,
© 2011 Autodesk Freely licensed for use by educational institutions. Reuse and changes require a note indicating that content has been modified from the.
1 Hantao Ji Princeton Plasma Physics Laboratory Experimentalist Laboratory astrophysics –Reconnection, angular momentum transport, dynamo effect… –Center.
Massively Parallel Magnetohydrodynamics on the Cray XT3 Joshua Breslau and Jin Chen Princeton Plasma Physics Laboratory Cray XT3 Technical Workshop Nashville,
SIMULATION OF A HIGH-  DISRUPTION IN DIII-D SHOT #87009 S. E. Kruger and D. D. Schnack Science Applications International Corp. San Diego, CA USA.
Hybrid WENO-FD and RKDG Method for Hyperbolic Conservation Laws
Kinetic Effects on the Linear and Nonlinear Stability Properties of Field- Reversed Configurations E. V. Belova PPPL 2003 APS DPP Meeting, October 2003.
Version, Date, POC Name 1 Purpose: To investigate multiscale flow discretizations that represent both the geometry and solution variable using variable-order.
Overview of MHD and extended MHD simulations of fusion plasmas Guo-Yong Fu Princeton Plasma Physics Laboratory Princeton, New Jersey, USA Workshop on ITER.
Hybrid Simulations of Energetic Particle-driven Instabilities in Toroidal Plasmas Guo-Yong Fu In collaboration with J. Breslau, J. Chen, E. Fredrickson,
Physics of Convection " Motivation: Convection is the engine that turns heat into motion. " Examples from Meteorology, Oceanography and Solid Earth Geophysics.
Bin Wen and Nicholas Zabaras
Stability Properties of Field-Reversed Configurations (FRC) E. V. Belova PPPL 2003 International Sherwood Fusion Theory Conference Corpus Christi, TX,
BGU WISAP Spectral and Algebraic Instabilities in Thin Keplerian Disks: I – Linear Theory Edward Liverts Michael Mond Yuri Shtemler.
DIII-D SHOT #87009 Observes a Plasma Disruption During Neutral Beam Heating At High Plasma Beta Callen et.al, Phys. Plasmas 6, 2963 (1999) Rapid loss of.
Nonlinear interactions between micro-turbulence and macro-scale MHD A. Ishizawa, N. Nakajima, M. Okamoto, J. Ramos* National Institute for Fusion Science.
© Fluent Inc. 11/24/2015J1 Fluids Review TRN Overview of CFD Solution Methodologies.
The Magneto-Rotational Instability and turbulent angular momentum transport Fausto Cattaneo Paul Fischer Aleksandr Obabko.
M. Onofri, F. Malara, P. Veltri Compressible magnetohydrodynamics simulations of the RFP with anisotropic thermal conductivity Dipartimento di Fisica,
Numerical Simulation of Dendritic Solidification
STUDIES OF NONLINEAR RESISTIVE AND EXTENDED MHD IN ADVANCED TOKAMAKS USING THE NIMROD CODE D. D. Schnack*, T. A. Gianakon**, S. E. Kruger*, and A. Tarditi*
ITP 2008 MRI Driven turbulence and dynamo action Fausto Cattaneo University of Chicago Argonne National Laboratory.
Gas-kineitc MHD Numerical Scheme and Its Applications to Solar Magneto-convection Tian Chunlin Beijing 2010.Dec.3.
Partially-relaxed, topologically-constrained MHD equilibria with chaotic fields. Stuart Hudson Princeton Plasma Physics Laboratory R.L. Dewar, M.J. Hole.
NIMROD Simulations of a DIII-D Plasma Disruption
QAS Design of the DEMO Reactor
Some slides on UCLA LM-MHD capabilities and Preliminary Incompressible LM Jet Simulations in Muon Collider Fields Neil Morley and Manmeet Narula Fusion.
Ergodic heat transport analysis in non-aligned coordinate systems S. Günter, K. Lackner, Q. Yu IPP Garching Problems with non-aligned coordinates? Description.
ANGULAR MOMENTUM TRANSPORT BY MAGNETOHYDRODYNAMIC TURBULENCE Gordon Ogilvie University of Cambridge TACHOCLINE DYNAMICS
Resistive Modes in CDX-U J. Breslau, W. Park. S. Jardin, R. Kaita – PPPL D. Schnack, S. Kruger – SAIC APS-DPP Annual Meeting Albuquerque, NM October 30,
1 LES of Turbulent Flows: Lecture 7 (ME EN ) Prof. Rob Stoll Department of Mechanical Engineering University of Utah Spring 2011.
Direct numerical simulation has to solve all the turbulence scales from the large eddies down to the smallest Kolmogorov scales. They are based on a three-dimensional.
NIMROD Simulations of a DIII-D Plasma Disruption S. Kruger, D. Schnack (SAIC) April 27, 2004 Sherwood Fusion Theory Meeting, Missoula, MT.
U NIVERSITY OF S CIENCE AND T ECHNOLOGY OF C HINA Influence of ion orbit width on threshold of neoclassical tearing modes Huishan Cai 1, Ding Li 2, Jintao.
Reconnection Process in Sawtooth Crash in the Core of Tokamak Plasmas Hyeon K. Park Ulsan National Institute of Science and Technology, Ulsan, Korea National.
A V&V Overview of the 31st Symposium on Naval Hydrodynamics
POSTPROCESSING Review analysis results and evaluate the performance
Large Eddy Simulation of Mixing in Stratified Flows
Spectral and Algebraic Instabilities in Thin Keplerian Disks: I – Linear Theory Edward Liverts Michael Mond Yuri Shtemler.
Finite difference code for 3D edge modelling
Convergence in Computational Science
Adaptive Grid Generation for Magnetically Confined Plasmas
Numerical Simulation of Dendritic Solidification
POSTPROCESSING Review analysis results and evaluate the performance
The Effects of Magnetic Prandtl Number On MHD Turbulence
Objective Numerical methods Finite volume.
20th IAEA Fusion Energy Conference,
Low Order Methods for Simulation of Turbulence in Complex Geometries
Presentation transcript:

TerascaleSimulation Tools and Technologies Spectral Elements for Anisotropic Diffusion and Incompressible MHD Paul Fischer Argonne National Laboratory

2 Terascale Simulation Tools and Technologies Goal: Enable high-fidelity calculations based on multiple coupled physical processes and multiple physical scales –Adaptive methods –Advanced meshing strategies –High-order discretization strategies Technical Approach: –Develop interchangeable and interoperable software components for meshing and discretization –Push state-of-the-art in discretizations

3 Terascale Simulation Tools and Technologies Goal: Enable high-fidelity calculations based on multiple coupled physical processes and multiple physical scales –Adaptive methods –Advanced meshing strategies –High-order discretization strategies Technical Approach: –Develop interchangeable and interoperable software components for meshing and discretization –Push state-of-the-art in discretizations

4 Outline Driving physics SEM overview –Costs –MHD formulation –Convective transport properties of SEM Anisotropic Diffusion Brief summary of current results on MHD project Conclusions

5 Driving Physics Anisotropic Diffusion: ( S. Jardin, PPPL) –Central to sustaining high plasma temperatures –Want to avoid radial leakage –Need to understand significant instabilities Incompressible MHD: ( F. Cattaneo UC, H. Ji PPPL, … ) –Several liquid metal experiments are under development to understand momentum transport in accretion disks Magneto rotational instability (MRI) is proposed as a mechanism to initiate turbulence capable of generating transport The magnetic Prandtl number for liquid metals is ~  Re=10 6 & Rm=10 for experiments Numerically, we can achieve Re=10 4 & Rm=10 3 ( 2005 INCITE award ) –Free-surface MHD ( H. Ji, PPPL ) Proposed as a plasma-facing material for fusion Desire to understand the effect of B on the free surface

6 Spectral Element Overview Spectral elements can be viewed as a finite element subset Performance gains realized by using: –tensor-product bases (quadrilateral or brick elements) –Lagrangian bases collocated with GLL quadrature points Operator Costs:Standard FEM * SEM # memory accesses: O( EN 6 ) O( EN 3 ) # operations: O( EN 6 ) O( EN 4 ) N = order, E = number of elements, EN 3 = number of gridpoints Dramatic cost reductions for large N ( > 5 ) 2D basis function, N=10

7 Computational Advantages of Spectral Elements Exponential convergence with N minimal numerical dispersion / diffusion for anisotropic diffusion, lements of ker(A || ) well-resolved – sharp decoupling of isotropic and anisotropic modes Matrix-free form matrix-vector products cast as efficient matrix-matrix products number of memory accesses identical to 7-pt. finite difference no additional work or storage for anisotropic diffusion tensor

8 SEM Computational Kernel on Cached-Based Architectures –matrix-matrix products, C=AB : 2N 3 ops for 2N 2 memory references –much of the additional work of the SEM is covered by efficient use of cache – e.g., time for A*B vs A+B for N=10 is: 2.0 x on DEC Alpha, 1.5 x on IBM SP A*B mxm vs m+m A*B A+B

9 Incompressible MHD t — plus appropriate boundary conditions on u and B Typically, Re >> Rm >> 1 Semi-implicit formulation yields independent Stokes problems for u and B

10 Incompressible MHD in a Nutshell t Convection: –dominates transport –dominates accuracy requirements often the challenging part of the discretization –treated explicitly in time Diffusion: –“easy” ( ?? ) Projection: div u = 0 div B = 0 –dominates work –isotropic SPD operator multiple right-hand side information scalable multilevel Schwarz methods ( 1999 GB award ) SE multigrid ( Lottes & F 05 )

11 High-Order Methods for Convection-Dominated Flows Phase Error for h vs. p Refinement: u t + u x = 0 h-refinement p-refinement

12 High-Order Methods for Convection-Dominated Flows Fraction of accurately resolved modes (per space direction) is increased only through increased order –Savings cubed in R 3 Rate of convergence is extremely rapid for high N –Important for multiscale / multiphysics problems ( Q: Why do we want 10 9 gridpoints? ) Stability issues are now largely understood –stabilization via DG, filtering, etc. –dealiasing Still, must resolve structures (no free lunch … ) –Computational costs are somewhat higher –Data access costs are equivalent to finite differences

13 c = (-x,y) c = (-y,x) Stabilizing convective problems: Models of straining and rotating flows: –Rotational case is skew-symmetric. –Filtering attacks the leading-order unstable mode. –Dealiasing ( high-order quadrature ) yields imaginary eigenvalues – vital for MHD N=19, M=19 N=19, M=20 straining field rotational field

14 Diffusion – easy ??

15 CEMM Challenge Problems S. Jardin, PPPL 1.Anisotropic diffusion in a toroidal geometry 2.Two-dimensional tilt mode instability 3.Magnetic reconnection in 2D Provides a problem suite that –captures essential physics of fusion simulation –stresses traditional numerical approaches –identifies pathways for next generation fusion codes Excellent vehicle for initiating SciDAC interatctions.

16 Anisotropic Diffusion in Toroidal Domains b – normalized B-field, helically wrapped on toroidal surfaces thermal flux follows b.

17 High degree of anisotropy creates significant challenges For this problem is more like a (difficult) hyperbolic problem than straightforward diffusion. This is reflected in the variational statement for the steady case with

18 Numerical challenges: –radial diffusion, –nearly singular, with large (but finite) null space –avoid grid imprinting –unsteady case constitutes a stiff relaxation problem –preconditioning nearly singular systems CEMM challenge: –establish spatial convergence for steady state case –check unsteady energy conservation when –investigate the behavior of the tearing mode instability High degree of anisotropy creates significant challenges

19 total number of gridpoints centerpoint error Steady State Error – 3D,  || = 10 8 It is advantageous to use few elements of high order Fewer gridpoints are required CPU time proportional to number of gridpoints ( N odd, 3-7) ( N even, 2-6)

20 High Anisotropy Demands High Accuracy A =  || A || + A I A I controls radial diffusion –A || must be accurately represented when  || >> 1 –Error must scale as ~ 1 /  || Steady-State L 2 – error over a range of discretizations E N

21 High Anisotropy Demands High Accuracy Difficulty stems from high-frequency content in null space of A || High-order discretizations are able to accurately represent these functions. N=16 k N=2 k k 18 th mode in circular geometry,  || = 10 8

22 Error vs. t T vs. r,t N=14 Error vs. r, N=12 Evolution of Gaussian Pulse for –minimal radial diffusion –no grid imprinting –careful time integration required (e.g., adaptive DIRK4 )  -averaged temperature vs. time Evolution of Gaussian pulse for

23 SEM is able to identify critical physics – Tearing mode instability –radial perturbation: b = b 0 +  cos(m  -n  ) r –field lines do not close on m-n rational surface –magnetic island results, with significant increase in radial conductivity. –island width scales as: W ~ in accord with asymptotic theory Note: this is a subtle effect! W W Island width vs.  || at onset. W

24 Outstanding Challenges for Anisotropic Diffusion Simulation Preconditioning –need null-space control –condition number scales as  || Non-aligned grids predict early island formation  -averaged temperature vs. time a 32 max dT/dr

25 Incompressible MHD Results

26 Axisymmetric Hydro Simulations of Taylor-Couette w/ Rings Re=620 steady Re=6200 unsteady Axisymmetric MHD simulations are being carried out now. Starting point for 3D simulations, which are being compared with experiments at PPPL. Computation by Obabko, Fischer, & Cattaneo Normalized Torque Vorticity inner cylinder outer cylinder Re=6200

27 Computational MRI: preliminary results Computations Fischer, Obabko & Cattaneo Nonlinear development of Magneto-Rotational Instability Cylindrical geometry similar to Goodman-Ji experiment Hydrodynamically stable rotation profile Weak vertical field Use newly developed spectral element MHD code Try to understand differences between experiments and simulations Simulations Re  Rm (moderate). Experiments Re>>Rm (Rm smallish)

28 Summary & Conclusions Block-structured SEM provides an efficient path to high-order –accurate treatment of challenging physics –fast cache-friendly operator evaluation ( N 4 vs. N 6 ) For anisotropic diffusion –effects of grid imprinting are minimized –able to capture physics of high anisotropy MRI experiment –MRI has been observed with axial periodicity at Re=Rm=1000. –preliminary axisymmetric results indicate hydrodynamic unsteadiness at Re=6000 for two-ring boundary configuration, may be mitigated in 3D… Free-surface MHD –Free-surface NS is working –Coupling with full MHD is underway

29

30