1. Particle Transport Methods in Parallel Environments
Prof. A. Haghighat
Nuclear & Radiological Engineering Department
University of Florida
202 Nuclear Sciences Building
Gainesville, FL 32611, USA
[Prepared for a meeting with Dell/HPC visitors, Dec. 13, 2002]

2. Objective
Simulation of particle transport of real-life nuclear systems (power reactors, medical devices, detection devices) for design and optimization

3. Particle Transport Theory
Expected number of particles in a phase space (dVdEd) at time t:
n(r, E, Ω, t) dVdEd z Ω dE dΩ dv r y x
Definitions
Angular flux - Scalar flux -
Reaction rate density = macroscopic cross-section

4. Particle Transport Theory Approaches
Deterministic
Statistical Monte Carlo

5. Deterministic Transport Theory Approach
Linear Boltzmann equation (steady-state)
Balance equation for expected number of particles in a phase space (dVdEd);
Streaming
Collision
Scattering
Independent source
Fission

6. Sn Method
The Sn balance equation for a spatial grid, group g, direction m is given by
where
There are 6 independent variables, e.g., x,y,z, E,  and 

7. Issues
Need for large computer memory, e.g., for a typical shielding problem with
100x100x100 (space) x 80 (directions)x 50 (groups)
memory required = 32 GB !
Need for significant computation time because of slow convergence

8. PENTRANTM (Parallel Environment Neutral-particle TRANsport) and its application to Real-Life Nuclear Systems 8
UFTTG

9. PENTRANTM (1)
Parallel Environment Neutral-particle TRANsport developed from scratch in 1996:
ANSI FORTRAN F77/f90 with MPI library, over 33,000 lines
Industry standard FIDO input
Solves 3-D Cartesian, multigroup, anisotropic transport problems
Forward and adjoint mode
Fixed source, criticality eigenvalue problems
Parallel processing algorithms
Full phase-space decomposition: Parallel in angle, energy, and spatial variables 9
UFTTG

10. PENTRANTM (2) (continued) Numerical formulations Parallel I/O
Partitioned memory for memory intensive arrays (angular fluxes, etc)
Builds MPI processor communicators
Automatic scheduling using a decomposition weighting vector
Numerical formulations
Adaptive Differencing Strategy
Diamond Zero (DZ)
Directional Theta-Weighted differencing (DTW)
Exponential-Directional Weighted (EDW) 10
UFTTG

11. PENTRANTM (4) (continue)
Allows for a fully discontinuous variable meshing between coarse meshes: Uses a novel higher order mesh coupling scheme: Taylor Projection Mesh Coupling (TPMC)
Acceleration
Spatial Two-grid (“/”) with TPMC
Angular multigrid (AMG) formulations with TPMC
PCR with a zoned rebalance acceleration
AMG + PCR
Iterative techniques
Multigroup & One-level SI schemes 11
UFTTG

12. PENTRANTM (5) Red-Black and Block Jacobi iteration
Anisotropic scattering via Legendre moments through P7,
Angular quadrature set:
Level symmetric (up to S20) with ordinate splitting (OS)
Pn-Tn with OS
Vacuum, reflective, and albedo boundaries
Volumetric & planar angular sources 12
UFTTG

13. PENTRAN Performance
Achieved high parallel fractions of 90-98% for solving real-life problems such as
a BWR core shroud; a PWR cavity dosimetry;
CT scan; X-ray room
Compared well with theoretical predictions

14. Monte Carlo Methods
Simulation of a physical process on a computer by sampling PDFs of basic physical processes using Random Numbers
Issues
Large computation time
Achieving small variance
Solution
Parallel Monte Carlo (it is straightforward; embarrassingly parallel)
Variance reduction methods (developed automated adjoint-based algorithms)

15. Activities/interests
Parallel computing
PTDC laboratory at NRE contains 2 PC clusters:
8 machines with 2 GB memory dedicated to LBE
6 machines with 1 GB memory dedicated to Monte Carlo
Web-based Computing
PENTRAN code system is available on the web
High Performance Computing graduate minor/certificate
We are preparing a proposal