1. Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dušan Gajić, University of Niš Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dr. Dušan B. Gajić dusan.b.gajic@gmail.com CIITLab, Dept. of Computer Science Faculty of Electronic Engineering University of Niš, Serbia 15 th Workshop on Software Engineering Education and Reverse Engineering Bohinj, Slovenia, August 23 - 30, 2015

2. Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dušan Gajić, University of Niš Outline 1.Heterogeneous parallel computing 2.Computing model, trends, and optimizations 3.Case study: Hybrid CPU/GPU computation of the Galois field transform 4.Conclusions Part I Part II

3. Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dušan Gajić, University of Niš Heterogeneous Parallel Computing Switch to heterogeneous systems is a milestone in high-performance computing (HPC) and computing in general Homogeneous computing – one or more processors of the same architecture used to execute programs Heterogeneous computing – a set of different processor architectures (CPUs, GPUs, FPGAs, DSPs) used to execute programs Each processor intended for different tasks and, therefore, is based on different design philosophy Applying each task to the best-suited architecture leads to improved performance in terms of time and energy, but requires novel programming techniques GPUs have become dominant additions to CPUs – GPU computing or General-purpose computing on GPUs ( GPGPU )

4. Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dušan Gajić, University of Niš CPU and GPU Throughput

5. Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dušan Gajić, University of Niš CPU and GPU Bandwidth

6. Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dušan Gajić, University of Niš Computing Model 1 1 2 2 Device executes kernels with high parallelism 3 3 4 4 input output input buffer output buffer New programming methods

7. Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dušan Gajić, University of Niš Trends More computation for less energy Integration of CPU and GPU to avoid the PCIe bottleneck Appereance of Intel Xeon Phi, Adapteva Parallela… ArchitectureFermiKeplerMaxwell CUDA cores51215362048 Frequency (GHz)1.51.01.1 SP per SM32192128 Bandwidth (GBs/s)192288336 TDP (W)244195165

8. Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dušan Gajić, University of Niš Program Optimizations High throughput at the price of extended latency covered by very large number of threads Focus on memory optimizations Memory transfers – coalesced access, page-locked memory… Explicit memory region definition Registers per thread – all SPs on an SM share the same register field Size of the computation grid (no. of threads/block, no. blocks/grid…) Problem of task scheduling gets more complicated – tasks are (optimally) scheduled to the best-suited processor Parallel programming patterns (scatter/gather, map/reduce, stencil…)

9. Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dušan Gajić, University of Niš CIITLab has a small group performing research on spectral methods and linear algebra on GPUs (Fourier and related transforms, matrix computations…) Heterogenous parallel computing @ FEE: Bachelor Digital Signal Processing – VII semester Pattern Recognition - VIII semester Master Heterogenous Methods for DSP Spectral Methods FEE founded in 1960, part of University of Niš with 28.000 students Faculty of Electronic Engineering Niš 480 students in the common 1 st year, 180 major in CS (from the 2 nd year)

10. Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dušan Gajić, University of Niš Case Study – Hybrid CPU/GPU computation of GFT Galois field expressions are a generalization of the Reed-Muller expressions from binary to the multiple-valued logic case Perform computation on different parts of the vector in parallel on the CPU and the GPU Basic idea: Different parts of the same task on different processors? Fast algorithms based on the factorization of the transfom matrix into sparse matrices

11. Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dušan Gajić, University of Niš Transform matrix for GF(4): Cooley-Tukey factorization:

12. Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dušan Gajić, University of Niš

13. Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dušan Gajić, University of Niš “Pure” MPI is the fastest for small functions (n ≤ 6) MPI/CUDA the fastest for all functions with (n ≥ 7) Use of page-locked memory is 3× faster that standard read-write “Pure” MPI is the fastest for small functions (n ≤ 6) MPI/CUDA the fastest for all functions with (n ≥ 7) Use of page-locked memory is 3× faster that standard read-write

14. Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dušan Gajić, University of Niš Speedup of MPI/CUDA over CUDA n = 114 % n = 1216,3 % n = 13 19,9 % Speedup of MPI/CUDA over CUDA n = 114 % n = 1216,3 % n = 13 19,9 % Experimental Results Speedup over CUDA increases with the size of the function Speedup is limited by memory transfers and division granularity Speedup over CUDA increases with the size of the function Speedup is limited by memory transfers and division granularity

15. Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dušan Gajić, University of Niš Conclusions Heterogeneous parallel computing is both the present and the future of high-performance computing (at least until we have something better ) Switch to heterogeneous systems is a milestone in computing Case study – hybrid CPU/GPU computation of GF(4) expressions Challenges in education for heterogeneous parallel computing: Making a shift in thinking from the“CPU-only” programming Adjusting course topics to address recent developments More computation for less energy with heterogeneous processors integrated around common memory

16. Program Optimizations and Recent Trends in Heterogeneous Parallel Computing Dušan Gajić, University of Niš Thank you for your time and attention! Dr. Dušan B. Gajić dusan.b.gajic@gmail.com CIITLab, Dept. of Computer Science Faculty of Electronic Engineering University of Niš, Serbia 15 th Workshop on Software Engineering Education and Reverse Engineering Bohinj, Slovenia, August 23 - 30, 2015