Parallel Computing Lecture

Why Parallel Want to do more per unit of time Because we care for performance We want to better exploit/use HW resources Divide and conquer Load Balancing (be careful)

Why Parallel Solve large problems Regular PC using 1 core  (X time) Regular PC under Y cores, where (Z time) < (X time) Regular PC under Y cores, + additional features  (P time) << Z time (P time) <<< X time

Speedup

Speedup Example 1: Objective Convert Seq. Code to Parallel Code Conditions  the % of time that is spent in the part that can be parallelized is 30% . Assume that you can reach/achieve a 100x speedup on the parallel portion Question 1 What is the Total Speedup? Question 2  in what % the execution time decreases? Question 3  assume NOW that you can reach an infinite speedup on the parallel version, in what % the execution time decreases? Question 4 What is the Total Speedup ?

Speedup Example 2: Objective Convert Seq. Code to Parallel Code Conditions  the % of time that is spent in the part that can be parallelized is 99% . Assume that you can reach/achieve a 100x speedup on the parallel portion Question 1 What is the Total Speedup? Question 1  in what % the execution time decreases?

Homogeneous Multicore Architectures.

Heterogeneous Multicore Architectures.

Heterogeneous Multicore Architectures. The CBE Cell Broadband Engine Have I used one before ? Quite Possible

Heterogeneous Multicore Architectures. SIMD  Single Instructions Multiple Data

Heterogeneous Multicore Architectures. SIMD  Single Instructions Multiple Data

Heterogeneous Multicore Architectures. SIMD  Single Instructions Multiple Data

SIMT  Single Instructions Multiple Thread Nvidia GPU’s. SIMT  Single Instructions Multiple Thread

SIMT  Single Instructions Multiple Thread Nvidia GPU’s. SIMT  Single Instructions Multiple Thread THIS IS A G80 SP = Streaming Processor SM = Streaming Multiprocessor 2 SM = 1 Building Block 128 SP, grouped as follows 16 SM, each one with 8 SP 768 Threads Per SM 768 Threads* 16 SM = 12288 Threads for Chip THIS IS A GT200 1024 Threads per SM ~ 30K threads