A Case for Core-Assisted Bottleneck Acceleration in GPUs Enabling Flexible Data Compression with Assist Warps Nandita Vijaykumar Gennady Pekhimenko, Adwait Jog, Abhishek Bhowmick, Rachata Ausavarangnirun, Chita Das, Mahmut Kandemir, Todd C. Mowry, Onur Mutlu In this work we observe that different bottlenecks and imbalances in execution leave different GPU resources idle Our goal in this work is employ these idle resources to accelerate different bottlenecks in execution In order to do this we propose to employ light-weight helper threads However, there are some key challenges that need to be addressed to enable helper threading in GPUs We propose a new framework CABA that effectively addresses these challenges The framework is flexible enough to be applied to a wide range of use case In this work, the apply the framework to enable flexible data compression We find that CABA shows that improves the performance of a wide range of GPGPU applications evaluated

Observation Imbalances in execution leave GPU resources underutilized Memory Hierarchy Saturated Memory Bandwidth Cores Register File Idle! Idle! Full! I’ll start by summarizing our work in a nutshell. GPU Threads GPU Streaming Multiprocessor

Our Goal Employ idle resources to do something useful: accelerate the bottleneck − using helper threads Memory Hierarchy Cores Register File Running Helper Threads I’ll start by summarizing our work in a nutshell. GPU Threads GPU Streaming Multiprocessor

Challenge How do you manage and use helper threads in a throughput-oriented architecture?

Our Solution: CABA A new framework to enable helper threading in GPUs CABA (Core-Assisted Bottleneck Acceleration) Wide set of use cases Compression, prefetching, memoization, … Flexible data compression using CABA alleviates the memory bandwidth bottleneck 41.7% performance improvement