An Execution Model for Heterogeneous Multicore Architectures Gregory Diamos, Andrew Kerr, and Sudhakar Yalamanchili Computer Architecture and Systems Laboratory Center for Experimental Research in Computer Systems School of Electrical and Computer Engineering Georgia Institute of Technology

SCHOOL OF ELECTRICAL AND COMPUTER ENGINEERING | GEORGIA INSTITUTE OF TECHNOLOGY Software Challenges of Heterogeneity Programming Model Execution Model Portability Performance

SCHOOL OF ELECTRICAL AND COMPUTER ENGINEERING | GEORGIA INSTITUTE OF TECHNOLOGY System Space System Size and Configuration Level of Abstraction Multi-node Multi GPU Multicore CPU Single GPU Multicore CPU Runtime Execution Model (Harmony) Runtime Translation of Data-Parallel IR (Ocelot)

SCHOOL OF ELECTRICAL AND COMPUTER ENGINEERING | GEORGIA INSTITUTE OF TECHNOLOGY Scalable Portable Execution – Harmony Runtime Local Memory Cache ACC DMA FIFO Local Memory Cache Local Memory Cache ACC DMA FIFO ACC DMA FIFO Network (e.g., Hypertransport, QPI, PCIe) CPU Harmony Run-time chunk InputsOutputsInputsOutputs Memory Transparent scheduling, execution management of chunks Binary compatibility across system sizes Cap Model 3 readInputs(); computeInvariants(); for all chunks { simulateChunk(); } generateResults(); Minimize/avoid retuning and porting applications as you add accelerators Advanced optimizations Speculation, performance prediction, kernel fusion kernel

SCHOOL OF ELECTRICAL AND COMPUTER ENGINEERING | GEORGIA INSTITUTE OF TECHNOLOGY

Emerging Environment Run Time (Harmony) CUDA JIT LLVM I/F Emulator GPGPU Simulator Supported ISAs (MIPS, SPARC, x86, etc.) Language Front End Kernel IR Language Front End Ocelot Prof. H. Kim Status Status: Single node/multi-GPU Status Status: Test and Debug Status Status: In progress (Fall 2009) Status Status: Summer 2009 With Prof. Nate Clark Datalog CUDA/OpenCL

SCHOOL OF ELECTRICAL AND COMPUTER ENGINEERING | GEORGIA INSTITUTE OF TECHNOLOGY Emerging HVM Platform Architecture With K. Schwan and A. Gavrilovska

SCHOOL OF ELECTRICAL AND COMPUTER ENGINEERING | GEORGIA INSTITUTE OF TECHNOLOGY Problem Scaling – Risk Analysis Application With latest CPUs (2x faster) and GPUs(4x faster), GPU advantage should grow by 2x Measured execution times GPU interactive overhead dominates

SCHOOL OF ELECTRICAL AND COMPUTER ENGINEERING | GEORGIA INSTITUTE OF TECHNOLOGY Other Applications

SCHOOL OF ELECTRICAL AND COMPUTER ENGINEERING | GEORGIA INSTITUTE OF TECHNOLOGY Execution Group GPU Compilation Flow Abstract Syntax Tree (Datalog Clauses) GPU (EU) GPU (EU) GPU (EU) GPU CoreCPU Core Data StructuresCompute Kernels P P Runtime Clauses to Execution Units Execution Units to Algorithms (Kernels) Predicates to Data Structures Runtime Mapping of Kernels to Cores