Fine-grain Task Aggregation and Coordination on GPUs † § Fine-grain Task Aggregation and Coordination on GPUs Marc S. Orr†§, Bradford M. Beckmann§, Steven K. Reinhardt§, David A. Wood†§ ISCA, June 16, 2014

Executive Summary SIMT languages (e.g. CUDA & OpenCL) restrict GPU programmers to regular parallelism Compare to Pthreads, Cilk, MapReduce, TBB, etc. Goal: enable irregular parallelism on GPUs Why? More GPU applications How? Fine-grain task aggregation What? Cilk on GPUs

Outline Background Our Work Results/Conclusion GPUs Cilk Channel Abstraction Our Work Cilk on Channels Channel Design Results/Conclusion

+ Maps well to SIMD hardware - Limits fine-grain scheduling GPUs Today GPU tasks scheduled by control processor (CP)— small, in-order programmable core Today’s GPU abstractions are coarse-grain GPU SIMD SIMD CP CP SIMD SIMD System Memory + Maps well to SIMD hardware - Limits fine-grain scheduling

Cilk Background Cilk extends C for divide and conquer parallelism Adds keywords spawn: schedule a thread to execute a function sync: wait for prior spawns to complete 1: int fib(int n) { 2: if (n <= 2) return 1; 3: int x = spawn fib(n - 1); 4: int y = spawn fib(n - 2); 5: sync; 6: return (x + y); 7: }

Dynamic aggregation enables “CPU-like” scheduling abstractions on GPUs Prior Work on Channels CP, or aggregator (agg), manages channels Finite task queues, except: User-defined scheduling Dynamic aggregation One consumption function GPU SIMD SIMD Agg Agg SIMD SIMD System Memory channels Dynamic aggregation enables “CPU-like” scheduling abstractions on GPUs

Outline Background Our Work Results/Conclusion GPUs Cilk Channel Abstraction Our Work Cilk on Channels Channel Design Results/Conclusion

Enable Cilk on GPUs via Channels Step 1 Cilk routines split by sync into sub-routines 1: int fib (int n) { 2: if (n<=2) return 1; 3: int x = spawn fib (n-1); 4: int y = spawn fib (n-2); 5: sync; 6: return (x+y); 7: } 1: int fib (int n) { 2: if (n<=2) return 1; 3: int x = spawn fib (n-1); 4: int y = spawn fib (n-2); 5: } 6: int fib_cont(int x, int y) { 7: return (x+y); 8: } “pre-sync” “continuation”

Enable Cilk on GPUs via Channels Step 2 Channels instantiated for breadth-first traversal Quickly populates GPU’s tens of thousands of lanes Facilitates coarse-grain dependency management 2 1 “pre-sync” task ready “pre-sync” task done 3 3 3 2 2 1 “continuation” task 4 4 4 3 3 3 A B task A spawned task B A B task B depends on task A 5 5 5 fib channel fib_cont channel stack: top of stack

Bound Cilk’s Memory Footprint Bound memory to the depth of the Cilk tree by draining channels closer to the base case The amount of work generated dynamically is not known a priori We propose that GPUs allow SIMT threads to yield Facilitates resolving conflicts on shared resources like memory 5 4 3 2 1

Channel Implementation Our design accommodates SIMT access patterns + array-based + lock-free + non-blocking See Paper

Outline Background Our Work Results/Conclusion GPUs Cilk Channel Abstraction Our Work Cilk on Channels Channel Design Results/Conclusion

Methodology Implemented Cilk on channels on a simulated APU Caches are sequentially consistent Aggregator schedules Cilk tasks

Cilk scales with the GPU Architecture More Compute Units  Faster execution

Conclusion We observed that dynamic aggregation enables new GPU programming languages and abstractions We enabled dynamic aggregation by extending the GPU’s control processor to manage channels We found that breadth first scheduling works well for Cilk on GPUs We proposed that GPUs allow SIMT threads to yield for breadth first scheduling Future work should focus on how the control processor can enable more GPU applications

Backup

Divergence and Channels Branch divergence Memory divergence + Data in channels good Pointers to data in channels bad

GPU NOT Blocked on Aggregator

GPU Cilk vs. standard GPU workloads Cilk is more succinct than SIMT languages Channels trigger more GPU dispatches   LOC reduction Dispatch rate Speedup Strassen 42% 13x 1.06 Queens 36% 12.5x 0.98 Same performance, easier to program

Disclaimer & Attribution The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of such revisions or changes. AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. ATTRIBUTION © 2014 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo and combinations thereof are trademarks of Advanced Micro Devices, Inc. in the United States and/or other jurisdictions. SPEC is a registered trademark of the Standard Performance Evaluation Corporation (SPEC). Other names are for informational purposes only and may be trademarks of their respective owners.