Workshop finale dei Progetti Grid del PON "Ricerca" Avviso Febbraio 2009 Catania Abstract In the contest of the S.Co.P.E. italian national project, within a multi-disciplinary collaboration between computational and chemical engineering scientists, we are concerned with the deployment of three dimensional simulations of fluido-dynamic processes. The computing environment relies on PETSc (Portable, Extensible Toolkit for Scientific Computation [1]) components integrated with the TFEM (Toolkit for Finite Element Method [2]) software toolkit, for discretization and solution of the partial differential equation–based model. We will present some experiments, numerical results and performance on the basis of selected test cases. Bibliography: [1] S. Balay, K. Bushelman, W. Gropp, D. Kaushik, M. Knepley, L. Curfman McInnes, B. Smith, H. Zhang, Petsc Users Manual, ANL-95/11- Revision 2.1.3, Argonne National Laboratory, [2] M.A. Hulsen, TFEM: A toolkit for the finite element method, Userguide, [3] G. D'Avino, M.A. Hulsen, F. Snijkers, J. Vermant, F. Greco, P.L. Maffettone, Rotation of a sphere in a viscoelastic liquid subjected to shear flow. Part I: Numerical results, J. Rheol. 52(6), Affiliations: 1 CNR 2 Dipartimento di Matematica ed Applicazioni-Università degli Studi di Napoli Federico II 3 Dipartimento di Ingegneria Chimica-Università degli Studi di Napoli Federico II 4 Consorzio SPACI 1 2 Mathematical model Numerical model - Algorithm 6 3 Computational simulations of 3D fluido-dynamic processes in High Performance Computing Environment L. Carracciuolo 1, D. Casaburi 2, L. D'Amore 2, G. D'Avino 3, P.L. Maffettone 3 and A. Murli 4 PETSc ( P ortable, E xtensible T oolkit for S cientific C omputation) 5 TFEM ( T oolkit for the F inite E lement M ethod) Initialization (definition of mesh, etc.) Computation of x 0 for t = 1 to num_time_steps do Computation of A t and b t using x t-1 Solution of the system A t x t = b t endfor PETSc Parallel TFEM Balance equations: Hydrodynamic interactions: Constitutive equations (Giesekus): Discretization method: Finite Elements Methods using Galerkin schema A t : sparse non-symmetric matrix Algorithm outline: Plot of the mesh and solution for poisson3 TFEM example BlasLapack MPI 4 Problem Dimension The dimension of linear systems A t x t = b t is at least of order O (10 6 ) in the case of 3D simulations. A simulation can require O(10 10 ) Flop and a big amount of memory (the order is of 10 GB ) Design and Implementation of software in the HPC environment Software for the discretization of partial differential equations by finite elements methods,using Galerkin-schema. Also it provides various tools for mesh generation and data display. A suite of data structures and routines for the scalable parallel solution of scientific applications modeled by partial differential equations. Software development in HPC enviroment Development of a parallel version of TFEM Solution of a linear system using iterative methods based on Krylov's subspaces and preconditioned algebraic multigrid Initialization (definition of mesh, etc.) Computation of x 0 for t= 1 to num_time_steps do Computation of A t and b t using x t-1 Solution of the system A t x t= b t endfor A suite of data structures and routines for the scalable parallel solution of scientific applications modeled by partial differential equations. Algorithm outline: Development Environment Abstraction Level Problem: The dynamics of spheres suspended in a viscoelastic fluid In many industrial applications, solid particles are added into fluids to improve final material properties. The suspending liquid often shows a viscoelastic rheological behaviour which further complicates the complessive rheological response of the material. In such materials we also find the occurrence of phenomena such as particle migration and aggregation. The aim of this work is a 3D numerical simulation of the dynamics of solid spheres suspended in viscoelastic fluid, subjected to shear flow. Such simulation is perfomed by discretizing the mathematical model through the Finite Element Method [3]. Problem: The dynamics of spheres suspended in a viscoelastic fluid : stress tensor u : velocity vector p: pressure s : solvent viscosity D : rate-of-deformation tensor p : polymer stress tensor F i : total force on i particle T i : torque on i particle : relaxation time : constitutive parameter p : polymer viscosity