Carlo del Mundo Department of Electrical and Computer Engineering Ubiquitous Parallelism Are You Equipped To Code For Multi- and Many- Core Platforms?

Agenda Introduction/Motivation Why Parallelism? Why now? Survey of Parallel Hardware CPUs vs. GPUs Conclusion How Can I Start? 2

Talk Goal Encourage undergraduates to answer the call to the era of parallelism Education Software Engineering 3

Why Parallelism? Why now? You’ve already been exposed to parallelism Bit Level Parallelism Instruction Level Parallelism Thread Level Parallelism 4

Why Parallelism? Why now? Single-threaded performance has plateaued Silicon Trends Power Consumption Heat Dissipation 5

Why Parallelism? Why now? 6

Power Chart: P = CV 2 F 7

Heat Chart (Feature Size) 8

Why Parallelism? Why now? Issue: Power & Heat Good: Cheaper to have more cores, but slower Bad: Breaks hardware/software contract 9

Why Parallelism? Why now? Hardware/Software Contract Maintain backwards-compatibility with existing codes 10

Why Parallelism? Why now? 11

Agenda Introduction/Motivation Why Parallelism? Why now? Survey of Parallel Hardware CPUs vs. GPUs Conclusion How Can I Start? 12

Personal Mobile Device Space 13 iPhone 5 Galaxy S3

Personal Mobile Device Space 14 2 CPU cores/ 3 GPU cores iPhone 5 Galaxy S3

Personal Mobile Device Space 15 2 CPU cores/ 3 GPU cores 4 CPU cores/ 4 GPU cores iPhone 5 Galaxy S3

Desktop Space 16

Desktop Space CPU cores AMD Opteron 6272 Rare To Have “Single Core” CPU Clock Speeds < 3.0 GHz Power Wall Heat Dissipation

Desktop Space GPU Cores AMD Radeon 7970 General Purpose Power Efficient High Performance Not All Problems Can Be Done on GPU

Warehouse Space (HokieSpeed) 19 Each node: 2x Intel Xeon 5645 (6 cores each) 2x NVIDIA C2050 (448 GPUs each)

Warehouse Space (HokieSpeed) 20 Each node: 2x Intel Xeon 5645 (6 cores each) 2x NVIDIA C2050 (448 GPUs each) 209 nodes

Warehouse Space (HokieSpeed) 21 Each node: 2x Intel Xeon 5645 (6 cores each) 2x NVIDIA C2050 (448 GPUs each) 209 nodes ★ 2508 CPU cores ★ GPU cores ★ 2508 CPU cores ★ GPU cores

All Spaces 22

Convergence in Computing Three Classes: Warehouse Desktop Personal Mobile Device Main Criteria Power, Performance, Programmability 23

Agenda Introduction/Motivation Why Parallelism? Why now? Survey of Parallel Hardware CPUs vs. GPUs Conclusion How Can I Start? 24

What is a CPU? CPU SR71 Jet Capacity 2 passengers Top Speed 2200 mph 25

What is the GPU? GPU Boeing 747 Capacity 605 passengers Top Speed 570 mph 26

CPU vs. GPU 27 Capacity (passengers) Speed (mph) Throughput (passengers * mph) “CPU” Fighter Jet “GPU” ,860

CPU Architecture Latency Oriented (Speculation) 28

GPU Architecture 29

APU = CPU + GPU Accelerated Processing Unit Both CPU + GPU on the same die 30

CPUs, GPUs, APUs How to handle parallelism? How to extract performance? Can I just throw processors at a problem? 31

CPUs, GPUs, APUs Multi-threading (2-16 threads) Massive multi-threading (100,000+) Depends on Your Problem 32

Agenda Introduction/Motivation Why Parallelism? Why now? Survey of Parallel Hardware CPUs vs. GPUs Conclusion How Can I Start? 33

How Can I start? CUDA Programming You most likely have a CUDA enabled GPU if you have a recent NVIDIA card 34

How Can I start? CPU or GPU Programming Use OpenCL (your laptop could potentially run) 35

How Can I start? Undergraduate research Senior/Grad Courses: CS 4234 – Parallel Computation CS 5510 – Multiprocessor Programming ECE 4504/5504 – Computer Architecture CS 5984 – Advanced Computer Graphics 36

In Summary … Parallelism is here to stay How does this affect you? How fast is fast enough? Are we content with current computer performance? 37

Thank you! Carlo del Mundo, Senior, Computer Engineering Website: Previous

Appendix 39

Programming Models pthreads MPI CUDA OpenCL 40

pthreads A UNIX API to create and destroy threads 41

MPI A communications protocol “Send and Receive” messages between nodes 42

CUDA Massive multi- threading (100,000+) Thread- level parallelism 43

OpenCL Heterogeneous programming model that is catered to several devices (CPUs, GPUs, APUs) 44

Comparisons pthreadsMPICUDAOpenCL Number Threads ,000+2 – 100,000+ PlatformCPU onlyAny PlatformNVIDIA OnlyAny Platform Productivity † EasyMediumHard Parallelism through ThreadsMessagesThreads † Productivity is subjective and draws from my experiences

Parallel Applications Vector Add Matrix Multiplication 46

Vector Add 47

Vector Add Serial Loop N times N cycles † Parallel Assume you have N cores 1 cycles † 48 † Assume 1 add = 1 cycle

Matrix Multiplication 49

Matrix Multiplication 50

Matrix Multiplication 51

Matrix Multiplication Embarassingly Parallel Let L be the length of each side L^2 elements, each element requires L multiplies and L adds 52

Performance Operations/Second (FLOPS) Power (W) Throughput (# things/unit time) FLOPS/W 53

Puss In Boots 54 Renders that took hours now take minutes - Ken Mueseth, Effects R&D Supervisor DreamWorks Animation

Computational Finance Black-Scholes – A PDE which governs the price of an option essentially “eliminating” risk 55

Genome Sequencing Knowledge of the human genome can provide insights to new medicine and biotechnology E.g.: genetic engineering, hybridization 56

Applications 57

Why Should You Care? Trends: CPU Core Counts Double Every 2 years 2006 – 2 cores, AMD Athlon 64 X – 8-12 cores, AMD Magny Cours Power Wall 58

Then And Now Today’s state-of-the-art hardware is yesterday’s supercomputer 1998 – Intel TFLOPS supercomputer 1.8 trillion floating point ops / sec (1.8 TFLOP) 2008 – AMD Radeon 4870 GPU x trilliion floating point ops / sec (2.4 TFLOP) 59