Automatic Parameterisation of Parallel Linear Algebra Routines Domingo Giménez Javier Cuenca José González University of Murcia SPAIN Algèbre Linéaire et Arithmétique: Calcul Numérique, Symbolique et Paralèle Rabat, Maroc Mai 2001

Outline Current Situation of Linear Algebra Parallel Routines (LAPRs) Objective Approach I: Analytical Model of the LAPRs Application: Jacobi Method on Origin 2000 Approach II: Exhaustive Executions Application: Gauss elimination on networks of processors Validation with the LU factorization Conclusions Future Works

 Linear Algebra: highly optimizable operations  Optimizations are Platform Specific  Traditional method: Hand-Optimization for each platform Current Situation of Linear Algebra Parallel Routines (LAPRs)

 Time-consuming  Incompatible with Hardware Evolution  Incompatible with changes in the system (architecture and basic libraries)  Unsuitable for dynamic systems  Misuse by non expert users Problems of traditional method

ATLAS, FLAME, I-LIB  Analyse platform characteristics in detail  Sequential code  Empirical results of the LAPR + Automation  High Installation Time Current approaches

Develop a methodology for obtaining Automatically Tuned Software Execution Environment Auto-tuning Software Our objective

 Routines Parameterised: System parameters, Algorithmic parameters  System parameters obtained at installation time Analytical model of the routine and simple installation routines to obtain the system parameters A reduced number of executions at installation time  Algorithmic parameters obtained at running time From the analytical model with the system parameters obtained in the installation process From the file with information generated in the installation process Methodology

 System parameters obtained at installation time Analytical model of the routine and simple installation routines to obtain the system parameters  Algorithmic parameters obtained at running time From the analytical model with the system parameters obtained in the installation process Analytical modelling

The behaviour of the algorithm on the platform is defined T exec = f (SPs, n, APs)  SPs = f(n, APs)System Parameters  APsAlgorithmic Parameters  nProblem Size Analytical Model

System Parameters (SPs): Hardware Platform  Physical Characteristics  Current Conditions Basic libraries How to estimate each SP? 1º.- Obtain the kernel of performance cost of LAPR 2º.- Make an Estimation Routine from this kernel Two Kinds of SPs: Communication System Parameters (CSPs) Arithmetic System Parameters (ASPs) Analytical Model LAPRs Performance

Arithmetic System Parameters (ASPs): t c arithmetic cost but using BLAS: k 1 k 2 and k 3. Computation Kernel of the LAPR  Estimation Routine  Similar storage scheme  Similar quantity of data Analytical Model

Communication System Parameters (CSPs): t s start-up time t w word-sending time Communication Kernel of the LAPR  Estimation Routine  Similar kind of communication  Similar quantity of data Analytical Model

Algorithmic Parameters (APs) Values chosen in each execution b block size pnumber of processors r  c logical topology grid configuration (logical 2D mesh) Analytical Model

Pre-installing (manual): 1º Make the Analytical Model: T exec = f (SPs, n, APs) 2º Write the Estimation Routines for the SPs Installing on a Platform (automatic): 3º Estimate the SPs using the Estimation Routines of step 2 4º Write a Configuration File, or include the information in the LAPR: for each n APs that minimize T exec Execution: The user executes LAPR for a size n: LAPR obtains optimal APs The Methodology. Step by step:

LAPR: One-sided Block Jacobi Method to solve the Symmetric Eigenvalue Problem.  Message-passing with MPI  Logical Ring & Logical 2D-Mesh Platform:SGI Origin 2000 Application Example

Application Example. Algorithm Scheme B WD 00 b n/r n

Application Example: Pre-installing. 1º Make the Analytical Model: T exec = f (SPs,n,APs)

Application Example: Pre-installing. 2º Write the Estimation Routines for the SPs k 3 matrix-matrix multiplication with DGEMM k 1 Givens Rotation to 2 vectors with DROT t s communications along the 2 directions of the 2D-mesh t w

Application Example: I nstalling 3º Estimate the SPs using the Estimation Routines k µs µsb = 32 k µsb = µsb = 128 t s 20 µs t w 0.1 µs

Comparison of execution times using different sets of Execution Parameters (4 processors) Application Example: Executing

Comparison of execution times using different sets of Execution Parameters (8 processors) Application Example: Executing

LAPR: One-sided Block Jacobi Method Algorithmic Parameters: block size mesh topology Platform: SGI Origin 2000 with message-passing System Parameters:arithmetic costs communication costs Satisfactory Reduction of the Execution Time: from 25% higher than the optimal to only 2% Application Example: Executing

Outline Current Situation of Linear Algebra Parallel Routines (LAPRs) Objective Approach I: Analytical Model of the LAPRs Application: Jacobi Method on Origin 2000 Approach II: Exhaustive Executions Application: Gauss elimination on networks of processors Validation with the LU factorization Conclusions Future Works

 System parameters obtained at installation time Installation routines making a reduced number of executions at installation time  Algorithmic parameters obtained at running time From the file with information generated in the installation process Exhaustive Execution

The behaviour of the algorithm on the platform is defined (as in Analytical Modelling) T exec = f (SPs, n, APs)  SPs = f(n, APs)System Parameters  APsAlgorithmic Parameters  nProblem Size Exhaustive Execution

Identify Algorithmic Parameters (APs) (as in Analytical Modelling) Values chosen in each execution b block size pnumber of processors r  c logical topology grid configuration (logical 2D mesh) Exhaustive Execution

Pre-installing (manual): 1º Determine the APs 2º Decide heuristics to reduce execution time in the installation process Installing on a Platform (automatic): 3º Decide (the manager) the problem sizes to be analysed 4º Execute and write a Configuration File, or include the information in the LAPR: for each n APs that minimize T exec Execution: The user executes LAPR for a size n: LAPR obtains optimal APs The Methodology. Step by step:

LAPR: Gaussian elimination.  Message-passing with MPI  Logical Ring, rowwise block-cyclic striped partitioning Platform:networks of processors (heterogeneous system) Application Example

Application Example: Pre-installing. 1º Determine the APs logical ring, rowwise block-cyclic striped partitioning pnumber of processors b block size for the data distribution different block sizes in heterogeneous systems b0b0 b1b1 b2b2 b0b0 b1b1 b2b2 b0b0 b1b1 b2b2 b0b0

Application Example: Pre-installing. 2º Decide heuristics to reduce execution time in the installation process u Execution time varies in a continuous way with the problem size and the APs u Consider the system as homogeneous u Installation can finish:  When Analytical and Experimental predictions coincide  When a certain time has been spent on the installation

 Homogeneous Systems: 3º The manager decides the problem sizes 4º Execute and write a Configuration File, or include the information in the LAPR: for each n APs that minimize T exec  Heterogeneous Systems: 3º The manager decides the problem sizes 4º Execute: write a Configuration File, for each n APs that minimize T exec write a Speed File, with the relative speeds of the processors in the system Application Example: I nstalling

 RI-THE:Obtains p and b from the formula.  RI-HOM:Obtains p and b through a reduced number of executions.  RI-HET:1º. As RI-HOM. 2º. Obtains b i for each processor Application Example: I nstallation Routines

Three different configurations: PLA_HOM: 5 SUN Ultra-1 PLA_HYB: 5 SUN Ultra-1 1 SUN Ultra-5 PLA_HET: 1 SUN Ultra-1 1 SUN Ultra-5 1 SUN Ultra-1 (manages the file system) Application Example: Systems

Experimental results in PLA-HOM: Quotient between the execution time with the parameters from the Installation Routine and the optimum execution time Application Example: Executing

Experimental results in PLA-HYB: Quotient between the execution time with the parameters from the Installation Routine and the optimum execution time Application Example: Executing

Experimental results in PLA-HET: Quotient between the execution time with the parameters from the Installation Routine and the optimum execution time Application Example: Executing

Two techniques for automatic tuning of Parallel Linear Algebra Routines: 1. Analytical Modelling For predictable systems (homogeneous, static,...) like Origin Exhaustive Execution For less predictable systems (heterogeneous, dynamic,...) like networks of workstations  Transparent to the user  Execution close to the optimum Comparison

Outline Current Situation of Linear Algebra Parallel Routines (LAPRs) Objective Approach I: Analytical Model of the LAPRs Application: Jacobi Method on Origin 2000 Approach II: Exhaustive Executions Application: Gauss elimination on networks of processors Validation with the LU factorization Conclusions Future Works

To validate the methodology it is necessary to experiment with:  More routines: block LU factorization  More systems: u Architectures: IBM SP2 and Origin 2000 u Libraries: reference BLAS, machine BLAS, ATLAS Validation with the LU factorization

Sequential LU Analytical Model: T exec = f (SPs,n,APs) SPs: cost of arithmetic operations of different levels: k 1, k 2, k 3 APs: block size b LUES UM b

Quotient between different execution times and the optimum execution time Sequential LU. Comparison in IBM SP2

Quotient between the execution time with the parameters provided by the model and the optimum execution time, with different basic libraries. In SUN 1 Sequential LU. Model execution time/optimum execution time

Parallel LU Analytical Model: T exec = f (SPs,n,APs) SPs: cost of arithmetic operations: k 1, k 2, k 3 cost of communications: t s, t w APs: block size b, number of processors p, grid configuration r  c b

Quotient between the execution time with the parameters provided by the model and the optimum execution time. In the sequential case, and in parallel with 4 and 8 processors. Parallel LU. Comparison in IBM SP2

Quotient between the execution time with the parameters provided by the model and the optimum execution time. In the sequential case, and in parallel with 4 and 8 processors. Parallel LU. Comparison in Origin 2000

 The modelling of the algorithm provides satisfactory results in different systems Origin 2000, IBM SP2 reference BLAS, machine BLAS, ATLAS  The prediction is worse in some cases: When the number of processors increases In multicomputers where communications are more important (IBM SP2)  Exhaustive Executions Parallel LU. Conclusions

If the manager installs the routine for sizes 512, 1536, 2560, and executions are performed for sizes 1024, 2048, 3072, the execution time is well predicted The same policy can be used in the installation of other software: Quotient between the execution time with the parameters provided by the installation process and the optimum execution time. With ScaLAPACK, in IBM SP2 Parallel LU. Exhaustive Execution

 Parameterisation of Parallel Linear Algebra Routines enables development of Automatically Tuned Software  Two techniques can be used: Analytical Modelling Exhaustive Executions or a combination of both  Experiments performed in different systems and with different routines Conclusions

 We try to develop a methodology valid for a wide range of systems, and to include it in the design of linear algebra libraries: it is necessary to analyse the methodology in more systems and with more routines  Architecture of an Automatically Tuned Linear Algebra Library  At the moment we are analysing routines individually, but it could be preferable to analyse algorithmic schemes Future Works

Architecture of an Automatically Tuned Linear Algebra Library Installation file Installation routines Basic routines library SP file AP file Library Basic routines declaration manager Installation Compilation designer manager

Architecture of an Automatically Tuned Linear Algebra Library Installation routines Library designer

Architecture of an Automatically Tuned Linear Algebra Library Installation routines Basic routines library Library Basic routines declaration designer manager

Architecture of an Automatically Tuned Linear Algebra Library Installation file Installation routines Basic routines library Library Basic routines declaration manager Installation designer manager

Architecture of an Automatically Tuned Linear Algebra Library Installation file Installation routines Basic routines library SP file AP file Library Basic routines declaration manager Installation designer manager

Architecture of an Automatically Tuned Linear Algebra Library Installation file Installation routines Basic routines library SP file AP file Library Basic routines declaration manager Installation Compilation designer manager