Presentation is loading. Please wait.

Presentation is loading. Please wait.

Ashwin M. Aji, Lokendra S. Panwar, Wu-chun Feng …Virginia Tech Pavan Balaji, James Dinan, Rajeev Thakur …Argonne.

Published byMarjory Stanley Modified over 8 years ago

Similar presentations

Presentation on theme: "Ashwin M. Aji, Lokendra S. Panwar, Wu-chun Feng …Virginia Tech Pavan Balaji, James Dinan, Rajeev Thakur …Argonne."— Presentation transcript:

1 http://www.vt.edu http://synergy.cs.vt.edu Ashwin M. Aji, Lokendra S. Panwar, Wu-chun Feng …Virginia Tech Pavan Balaji, James Dinan, Rajeev Thakur …Argonne National Lab. Feng Ji, Xiaosong Ma …N. C. State University Milind Chabbi, Karthik Murthy, John Mellor-Crummey …Rice University Keith R. Bisset Presenter: Ashwin M. Aji Ph.D. Candidate, Virginia Tech …Virginia Bioinformatics Inst. On the Efficacy of GPU-Integrated MPI for Scientific Applications

2 http://www.vt.edu http://synergy.cs.vt.edu Overview We describe the hybrid design and optimization strategies of today’s MPI+GPU applications. We show how GPU-integrated MPI helps to expand this design and optimization space. We evaluate and discuss the tradeoffs of the new design and optimizations … – … using real application case studies from epidemiology and seismology domains. – … on HokieSpeed, a 212 TFlop CPU-GPU cluster at Virginia Tech. Ashwin M. Aji (aaji@cs.vt.edu) 2

3 http://www.vt.edu http://synergy.cs.vt.edu Accelerator-Based Supercomputers Accelerator Vendors/Models NVIDIA 2050/2070/2090/K20x Intel Xeon Phi (MIC) IBM PowerXCell 8i AMD Firepro/Radeon Ashwin M. Aji (aaji@cs.vt.edu) 3

4 http://www.vt.edu http://synergy.cs.vt.edu CPU MPI Programming HPC Systems Mantra: Overlap communication with computation Ashwin M. Aji (aaji@cs.vt.edu) 4

5 http://www.vt.edu http://synergy.cs.vt.edu CPU MPI GPU PCIe GPU Programming HPC Systems Mantra(s) Overlap CPU-CPU communication with CPU computation Overlap CPU computation with GPU computation Overlap CPU-CPU communication with CPU-GPU communication Overlap xPU-xPU communication with xPU computations Mantra(s) Overlap CPU-CPU communication with CPU computation Overlap CPU computation with GPU computation Overlap CPU-CPU communication with CPU-GPU communication Overlap xPU-xPU communication with xPU computations Ashwin M. Aji (aaji@cs.vt.edu) 5

6 http://www.vt.edu http://synergy.cs.vt.edu What is GPU-integrated MPI? GPU device memory GPU device memory CPU main memory CPU main memory PCIe Network MPI Rank = 0MPI Rank = 1 if(rank == 0) { GPUMemcpy(host_buf, dev_buf, D2H) MPI_Send(host_buf,....) } if(rank == 1) { MPI_Recv(host_buf,....) GPUMemcpy(dev_buf, host_buf, H2D) } Ashwin M. Aji (aaji@cs.vt.edu) 6

7 http://www.vt.edu http://synergy.cs.vt.edu What is GPU-integrated MPI? GPU device memory GPU device memory CPU main memory CPU main memory PCIe Network MPI Rank = 0MPI Rank = 1 int processed[chunks] = {0}; for(j=0;j<chunks;j++) { /* Pipeline */ cudaMemcpyAsync(host_buf+offset, gpu_buf+offset, D2H, streams[j],...); } numProcessed = 0; j = 0; flag = 1; while (numProcessed < chunks) { if(cudaStreamQuery(streams[j] == cudaSuccess)) { /* start MPI */ MPI_Isend(host_buf+offset,...); numProcessed++; processed[j] = 1; } MPI_Testany(...); /* check progress */ if(numProcessed < chunks) /* next chunk */ while(flag) { j=(j+1)%chunks; flag=processed[j]; } MPI_Waitall(); Performance vs. Programmability tradeoff Multiple optimizations for different...  …GPUs (AMD/Intel/NVIDIA)  …programming models (CUDA/OpenCL)  …library versions (CUDA v3/CUDA v4) Performance vs. Programmability tradeoff Multiple optimizations for different...  …GPUs (AMD/Intel/NVIDIA)  …programming models (CUDA/OpenCL)  …library versions (CUDA v3/CUDA v4) Ashwin M. Aji (aaji@cs.vt.edu) 7

8 http://www.vt.edu http://synergy.cs.vt.edu What is GPU-integrated MPI? GPU device memory GPU device memory CPU main memory CPU main memory PCIe Network MPI Rank = 0MPI Rank = 1 if(rank == 0) { MPI_Send(any_buf,....) } if(rank == 1) { MPI_Recv(any_buf,....) } Examples: MPI-ACC, MVAPICH, Open MPI Programmability: multiple accelerators and prog. models (CUDA, OpenCL) Performance: system-specific and vendor-specific optimizations (Pipelining, GPUDirect, pinned host memory, IOH affinity) Performs great on benchmarks! How about in the context of complex applications? Examples: MPI-ACC, MVAPICH, Open MPI Programmability: multiple accelerators and prog. models (CUDA, OpenCL) Performance: system-specific and vendor-specific optimizations (Pipelining, GPUDirect, pinned host memory, IOH affinity) Performs great on benchmarks! How about in the context of complex applications? Ashwin M. Aji (aaji@cs.vt.edu) 8

9 http://www.vt.edu http://synergy.cs.vt.edu Outline What is GPU-integrated MPI? Programming CPU-GPU systems using simple MPI and GPU-integrated MPI Evaluating the efficacy of GPU-integrated MPI Conclusion Ashwin M. Aji (aaji@cs.vt.edu) 9

10 http://www.vt.edu http://synergy.cs.vt.edu CPU GPU-MPI GPU Using GPU-integrated MPI Ashwin M. Aji (aaji@cs.vt.edu) 10

11 http://www.vt.edu http://synergy.cs.vt.edu A Programmer’s View CPU GPU-MPI GPU CPU MPI GPU Ashwin M. Aji (aaji@cs.vt.edu) 11 Provides a clean and natural interface to communicate with the intended communication target All the communication optimization details are hidden under the MPI layer Computation model still remains the same – GPU kernel offload from the CPU Provides a clean and natural interface to communicate with the intended communication target All the communication optimization details are hidden under the MPI layer Computation model still remains the same – GPU kernel offload from the CPU

12 http://www.vt.edu http://synergy.cs.vt.edu Using GPU-integrated MPI (2) CPU MPI GPU Preprocess Ashwin M. Aji (aaji@cs.vt.edu) 12

13 http://www.vt.edu http://synergy.cs.vt.edu Using GPU-integrated MPI (2) CPU GPU-MPI GPU Preprocess Ashwin M. Aji (aaji@cs.vt.edu) 13 It enables the programmer to naturally consider even the GPU for preprocessing, which is usually against the norms. o GPU preprocessing need not always be a good idea, but at least that option can easily be explored and evaluated. It enables the programmer to naturally consider even the GPU for preprocessing, which is usually against the norms. o GPU preprocessing need not always be a good idea, but at least that option can easily be explored and evaluated.

14 http://www.vt.edu http://synergy.cs.vt.edu Using GPU-integrated MPI (2) CPU GPU-MPI GPU Preprocess Ashwin M. Aji (aaji@cs.vt.edu) 14

15 http://www.vt.edu http://synergy.cs.vt.edu Capabilities of GPU-integrated MPI Capabilities of MPI – Simultaneously execute multiple processes. – Overlap communication with computation. Additional capabilities of GPU-integrated MPI – Overlap CPU-CPU communication with CPU-GPU communication (internal pipelining). – Overlap xPU-xPU communication with CPU computation and GPU computation. – Provides a natural interface to move data and choose CPU or GPU for the next task at hand. – Efficiently manages resources while providing optimal performance Ashwin M. Aji (aaji@cs.vt.edu) 15

16 http://www.vt.edu http://synergy.cs.vt.edu Outline What is GPU-integrated MPI? Programming CPU-GPU systems using simple MPI and GPU-integrated MPI Evaluating the efficacy of GPU-integrated MPI Conclusion Ashwin M. Aji (aaji@cs.vt.edu) 16

17 http://www.vt.edu http://synergy.cs.vt.edu Evaluation and Discussion Case Studies – Epidemiology Simulation (EpiSimdemics) – Seismology Simulation (FDM-Seismology) Experimental Setup – HokieSpeed: CPU-GPU cluster at Virginia Tech – Used up to 128 nodes, where each node has two hex-core Intel Xeon E5645 CPUs two NVIDIA Tesla M2050 GPUs – CUDA toolkit v4.0 and driver v270. Ashwin M. Aji (aaji@cs.vt.edu) 17

18 http://www.vt.edu http://synergy.cs.vt.edu EpiSimdemics: Infection propagation simulator N m Person: N Location: m Location b Receive Visit Messages Preprocess Data Compute Interactions (on GPU) Compute Interactions (on GPU) Ashwin M. Aji (aaji@cs.vt.edu) 18

19 http://www.vt.edu http://synergy.cs.vt.edu EpiSimdemics Algorithm …… Persons MPILocations a bc d e f g a.person computes visits b.person sends visit messages c.location receives visits d.location computes infections e.location sends outcomes f.person receives outcomes g.person combines outcomes CPU 1 CPU n Offload Ashwin M. Aji (aaji@cs.vt.edu) 19

20 http://www.vt.edu http://synergy.cs.vt.edu Case Study for GPU-MPI: Epidemiology Network PE i (Host CPU) GPU i (Device) 1. Copy to GPU 2. Preprocess on GPU MPI+CUDA for(j=0;j<chunks;j++) { MPI_Irecv(host_chunk[j],...); } MPI_Waitall(); gpuMemcpy (dev_buf, host_buf, H2D); gpuPreprocess (dev_buf); gpuDeviceSynchronize(); gpuMemcpy NOT an overhead here gpuPreprocess is an overhead can be pipelined gpuMemcpy NOT an overhead here gpuPreprocess is an overhead can be pipelined Ashwin M. Aji (aaji@cs.vt.edu) 20

21 http://www.vt.edu http://synergy.cs.vt.edu Case Study for GPU-MPI: Epidemiology GPU i (Device) PE i (Host CPU) Network 1a. Pipelined data transfers to GPU 1b. Overlapped Preprocessing with internode CPU-GPU communication MPI+CUDA (Advanced) Ashwin M. Aji (aaji@cs.vt.edu) 21 allocate(host_chunks[N]); allocate(dev_chunks[N]); for(j=0;j<chunks;j++) { MPI_Irecv(host_chunk[j],...); } while(condition) { /* Pipeline ALL data transfers and preprocessing kernels */ if( MPI_Testany(request[i],...) ) { gpuMemcpyAsync(dev_chunk[i], host_chunk[i], &event[i]); } if( gpuEventQuery(event[k]) ) { gpuPreprocess (dev_chunk[k]); } } gpuDeviceSynchronize(); Good Overlapped CPU-CPU communication with CPU-GPU communication Overlap communication with GPU Preprocessing Bad Pinned memory footprint increases with number of processes (unless you write your own memory manager as well…) o The data movement pattern in EpiSimdemics is nondeterministic Code can get verbose Good Overlapped CPU-CPU communication with CPU-GPU communication Overlap communication with GPU Preprocessing Bad Pinned memory footprint increases with number of processes (unless you write your own memory manager as well…) o The data movement pattern in EpiSimdemics is nondeterministic Code can get verbose

22 http://www.vt.edu http://synergy.cs.vt.edu Case Study for GPU-MPI: Epidemiology GPU i (Device) PE i (Host CPU) Network 1a. Pipelined data transfers to GPU 1b. Overlapped Preprocessing with internode CPU-GPU communication GPU-MPI allocate(dev_chunks[N]); for(j=0;j<chunks;j++) { MPI_Irecv(dev_chunk[j],...); } while(condition) { /* Pipeline data transfers and preprocessing kernels */ if( MPI_Testany(request[i],...) ) { gpuPreprocess (dev_chunk[i]); } } gpuDeviceSynchronize(); Good Overlapped CPU-CPU communication with CPU-GPU communication Overlap communication with GPU Preprocessing Pinned memory that is used for staging has constant footprint o Memory manager is implemented within GPU-MPI o Created during MPI_Init and destroyed at MPI_Finalize Code is almost like a CPU-only MPI program! o More natural way of expressing the computational and/or communication target Good Overlapped CPU-CPU communication with CPU-GPU communication Overlap communication with GPU Preprocessing Pinned memory that is used for staging has constant footprint o Memory manager is implemented within GPU-MPI o Created during MPI_Init and destroyed at MPI_Finalize Code is almost like a CPU-only MPI program! o More natural way of expressing the computational and/or communication target Ashwin M. Aji (aaji@cs.vt.edu) 22

23 http://www.vt.edu http://synergy.cs.vt.edu Case Study for GPU-MPI: Epidemiology Network PE i (Host CPU) GPU i (Device) 1. Copy to GPU 2. Preprocess on GPU GPU i (Device) PE i (Host CPU) Network 1a. Pipelined data transfers to GPU 1b. Overlapped Preprocessing with internode CPU-GPU communication MPI+CUDAMPI+CUDA Adv. & GPU-MPI Ashwin M. Aji (aaji@cs.vt.edu) 23 Preprocessing overhead Pipelined preprocessing Memory management? Pipelined preprocessing Memory management?

24 http://www.vt.edu http://synergy.cs.vt.edu EpiSimdemics: Experimental Results 6.3% better than MPI+CUDA on average 14.6% better than MPI+CUDA for lesser nodes 24.2% better than MPI+CUDA Adv. on average 61.6% better than MPI+CUDA Adv. for larger nodes 6.3% better than MPI+CUDA on average 14.6% better than MPI+CUDA for lesser nodes 24.2% better than MPI+CUDA Adv. on average 61.6% better than MPI+CUDA Adv. for larger nodes Ashwin M. Aji (aaji@cs.vt.edu) 24

25 http://www.vt.edu http://synergy.cs.vt.edu Cost of GPU Preprocessing (MPI+CUDA vs. GPU-MPI) Preprocessing on GPU is completely hidden Scope for optimization lessens for more nodes because of strong scaling Preprocessing on GPU is completely hidden Scope for optimization lessens for more nodes because of strong scaling Ashwin M. Aji (aaji@cs.vt.edu) 25

26 http://www.vt.edu http://synergy.cs.vt.edu Cost of Pinned Memory Management (MPI+CUDA Adv. vs. GPU-MPI) Ashwin M. Aji (aaji@cs.vt.edu) 26

27 http://www.vt.edu http://synergy.cs.vt.edu HPCToolkit Analysis: MPI+CUDA Ashwin M. Aji (aaji@cs.vt.edu) 27

28 http://www.vt.edu http://synergy.cs.vt.edu HPCToolkit Analysis: GPU-MPI Ashwin M. Aji (aaji@cs.vt.edu) 28

29 http://www.vt.edu http://synergy.cs.vt.edu Overlapping CPU-GPU Communication with GPU Preprocessing Ashwin M. Aji (aaji@cs.vt.edu) 29

30 http://www.vt.edu http://synergy.cs.vt.edu Case Study for GPU-MPI: FDM-Seismology GPU-MPI Ashwin M. Aji (aaji@cs.vt.edu) 30

31 http://www.vt.edu http://synergy.cs.vt.edu FDM-Seismology: Experimental Results ~43% overall improvement Ashwin M. Aji (aaji@cs.vt.edu) 31

32 http://www.vt.edu http://synergy.cs.vt.edu FDM-Seismology: Experimental Results As the number of nodes increase (strong scaling)… cudaMemcpy time decreases and benefits of hiding it also decreases (strong scaling) xPU-xPU MPI communication becomes significant As the number of nodes increase (strong scaling)… cudaMemcpy time decreases and benefits of hiding it also decreases (strong scaling) xPU-xPU MPI communication becomes significant Ashwin M. Aji (aaji@cs.vt.edu) 32

33 http://www.vt.edu http://synergy.cs.vt.edu Outline What is GPU-integrated MPI? Programming CPU-GPU systems using simple MPI and GPU-integrated MPI Evaluating the efficacy of GPU-integrated MPI Conclusion Ashwin M. Aji (aaji@cs.vt.edu) 33

34 http://www.vt.edu http://synergy.cs.vt.edu Efficacy of GPU-integrated MPI Natural interface to express communication/computation targets Overlapped xPU-xPU communication with xPU computation for better resource utilization Efficient buffer management techniques for improved scalability Convenient for programmers to explore and evaluate newer design/optimization spaces – GPU-driven data partitioning (read the paper) – GPU-driven data marshaling Contact: Ashwin Aji (aaji@cs.vt.edu) Prof. Wu Feng (feng@cs.vt.edu) Questions?

35 http://www.vt.edu http://synergy.cs.vt.edu BACKUP SLIDES Ashwin M. Aji (aaji@cs.vt.edu) 35

36 http://www.vt.edu http://synergy.cs.vt.edu Intra-node Topology Awareness Graphics Processing Units (GPUs) – Many-core architecture – Prog. Models: CUDA, OpenCL, OpenACC – Explicitly managed global memory and separate address spaces CPU clusters – Most popular parallel prog. model: Message Passing Interface (MPI) – Host memory only Disjoint Memory Spaces! Keeneland Node Architecture Ashwin M. Aji (aaji@cs.vt.edu) 36

Download ppt "Ashwin M. Aji, Lokendra S. Panwar, Wu-chun Feng …Virginia Tech Pavan Balaji, James Dinan, Rajeev Thakur …Argonne."

Similar presentations

Instructor Notes This lecture describes the different ways to work with multiple devices in OpenCL (i.e., within a single context and using multiple contexts),

Instructor Notes This lecture describes the different ways to work with multiple devices in OpenCL (i.e., within a single context and using multiple contexts),
Shredder GPU-Accelerated Incremental Storage and Computation

Shredder GPU-Accelerated Incremental Storage and Computation
The Development of Mellanox - NVIDIA GPUDirect over InfiniBand A New Model for GPU to GPU Communications Gilad Shainer.

The Development of Mellanox - NVIDIA GPUDirect over InfiniBand A New Model for GPU to GPU Communications Gilad Shainer.
Scalable Multi-Cache Simulation Using GPUs Michael Moeng Sangyeun Cho Rami Melhem University of Pittsburgh.

Scalable Multi-Cache Simulation Using GPUs Michael Moeng Sangyeun Cho Rami Melhem University of Pittsburgh.
1 A GPU Accelerated Storage System NetSysLab The University of British Columbia Abdullah Gharaibeh with: Samer Al-Kiswany Sathish Gopalakrishnan Matei.

1 A GPU Accelerated Storage System NetSysLab The University of British Columbia Abdullah Gharaibeh with: Samer Al-Kiswany Sathish Gopalakrishnan Matei.
GPU Virtualization Support in Cloud System Ching-Chi Lin Institute of Information Science, Academia Sinica Department of Computer Science and Information.

GPU Virtualization Support in Cloud System Ching-Chi Lin Institute of Information Science, Academia Sinica Department of Computer Science and Information.
Multi-GPU System Design with Memory Networks

Multi-GPU System Design with Memory Networks
GPU System Architecture Alan Gray EPCC The University of Edinburgh.

GPU System Architecture Alan Gray EPCC The University of Edinburgh.
BWUPEP2011, UIUC, May 29 - June 10 2011 1 Blue Waters Undergraduate Petascale Education Program May 29 – June 10 2011 Hybrid MPI/CUDA Scaling accelerator.

BWUPEP2011, UIUC, May 29 - June Blue Waters Undergraduate Petascale Education Program May 29 – June Hybrid MPI/CUDA Scaling accelerator.
HPCC Mid-Morning Break High Performance Computing on a GPU cluster Dirk Colbry, Ph.D. Research Specialist Institute for Cyber Enabled Discovery.

HPCC Mid-Morning Break High Performance Computing on a GPU cluster Dirk Colbry, Ph.D. Research Specialist Institute for Cyber Enabled Discovery.
PVOCL: Power-Aware Dynamic Placement and Migration in Virtualized GPU Environments Palden Lama, Xiaobo Zhou, University of Colorado at Colorado Springs.

PVOCL: Power-Aware Dynamic Placement and Migration in Virtualized GPU Environments Palden Lama, Xiaobo Zhou, University of Colorado at Colorado Springs.
Sparse LU Factorization for Parallel Circuit Simulation on GPU Ling Ren, Xiaoming Chen, Yu Wang, Chenxi Zhang, Huazhong Yang Department of Electronic Engineering,

Sparse LU Factorization for Parallel Circuit Simulation on GPU Ling Ren, Xiaoming Chen, Yu Wang, Chenxi Zhang, Huazhong Yang Department of Electronic Engineering,
Exploring Efficient Data Movement Strategies for Exascale Systems with Deep Memory Hierarchies Heterogeneous Memory (or) DMEM: Data Movement for hEterogeneous.

Exploring Efficient Data Movement Strategies for Exascale Systems with Deep Memory Hierarchies Heterogeneous Memory (or) DMEM: Data Movement for hEterogeneous.
University of Michigan Electrical Engineering and Computer Science Transparent CPU-GPU Collaboration for Data-Parallel Kernels on Heterogeneous Systems.

University of Michigan Electrical Engineering and Computer Science Transparent CPU-GPU Collaboration for Data-Parallel Kernels on Heterogeneous Systems.
Development of a track trigger based on parallel architectures Felice Pantaleo PH-CMG-CO (University of Hamburg) Felice Pantaleo PH-CMG-CO (University.

Development of a track trigger based on parallel architectures Felice Pantaleo PH-CMG-CO (University of Hamburg) Felice Pantaleo PH-CMG-CO (University.
A Scalable Heterogeneous Parallelization Framework for Iterative Local Searches Martin Burtscher 1 and Hassan Rabeti 2 1 Department of Computer Science,

A Scalable Heterogeneous Parallelization Framework for Iterative Local Searches Martin Burtscher 1 and Hassan Rabeti 2 1 Department of Computer Science,
PARALLEL PROCESSING COMPARATIVE STUDY 1. CONTEXT How to finish a work in short time???? Solution To use quicker worker. Inconvenient: The speed of worker.

PARALLEL PROCESSING COMPARATIVE STUDY 1. CONTEXT How to finish a work in short time???? Solution To use quicker worker. Inconvenient: The speed of worker.
GPUs on Clouds Andrew J. Younge Indiana University (USC / Information Sciences Institute) UNCLASSIFIED: 08/03/2012.

GPUs on Clouds Andrew J. Younge Indiana University (USC / Information Sciences Institute) UNCLASSIFIED: 08/03/2012.
OpenFOAM on a GPU-based Heterogeneous Cluster

OpenFOAM on a GPU-based Heterogeneous Cluster
2009/04/07 Yun-Yang Ma.  Overview  What is CUDA ◦ Architecture ◦ Programming Model ◦ Memory Model  H.264 Motion Estimation on CUDA ◦ Method ◦ Experimental.

2009/04/07 Yun-Yang Ma.  Overview  What is CUDA ◦ Architecture ◦ Programming Model ◦ Memory Model  H.264 Motion Estimation on CUDA ◦ Method ◦ Experimental.

Similar presentations

Ads by Google