1. Fine-grain Task Aggregation and Coordination on GPUs†
Fine-grain Task Aggregation and Coordination on GPUs
Marc S. Orr†§, Bradford M. Beckmann§, Steven K. Reinhardt§, David A. Wood†§
ISCA, June 16, 2014

2. Executive Summary
SIMT languages (e.g. CUDA & OpenCL) restrict GPU programmers to regular parallelism
Compare to Pthreads, Cilk, MapReduce, TBB, etc.
Goal: enable irregular parallelism on GPUs
Why? More GPU applications
How? Fine-grain task aggregation
What? Cilk on GPUs

3. Outline Background Our Work Results/Conclusion GPUs Cilk
Channel Abstraction
Our Work
Cilk on Channels
Channel Design
Results/Conclusion

4. + Maps well to SIMD hardware - Limits fine-grain scheduling
GPUs Today
GPU tasks scheduled by control processor (CP)— small, in-order programmable core
Today’s GPU abstractions are coarse-grain
GPU
SIMD
SIMD CP CP
SIMD
SIMD
System Memory
+ Maps well to SIMD hardware
- Limits fine-grain scheduling

5. Cilk Background Cilk extends C for divide and conquer parallelism
Adds keywords
spawn: schedule a thread to execute a function
sync: wait for prior spawns to complete
1: int fib(int n) { 2: if (n <= 2) return 1; 3: int x = spawn fib(n - 1); 4: int y = spawn fib(n - 2); 5: sync; 6: return (x + y); 7: }

6. Dynamic aggregation enables “CPU-like” scheduling abstractions on GPUs
Prior Work on Channels
CP, or aggregator (agg), manages channels
Finite task queues, except:
User-defined scheduling
Dynamic aggregation
One consumption function
GPU
SIMD
SIMD
Agg
Agg
SIMD
SIMD
System Memory
channels
Dynamic aggregation enables “CPU-like” scheduling abstractions on GPUs

7. Outline Background Our Work Results/Conclusion GPUs Cilk
Channel Abstraction
Our Work
Cilk on Channels
Channel Design
Results/Conclusion

8. Enable Cilk on GPUs via Channels
Step 1
Cilk routines split by sync into sub-routines
1: int fib (int n) {
2: if (n<=2) return 1;
3: int x = spawn fib (n-1);
4: int y = spawn fib (n-2);
5: sync;
6: return (x+y);
7: }
1: int fib (int n) {
2: if (n<=2) return 1;
3: int x = spawn fib (n-1);
4: int y = spawn fib (n-2);
5: }
6: int fib_cont(int x, int y) {
7: return (x+y);
8: }
“pre-sync”
“continuation”

9. Enable Cilk on GPUs via Channels
Step 2
Channels instantiated for breadth-first traversal
Quickly populates GPU’s tens of thousands of lanes
Facilitates coarse-grain dependency management 2 1
“pre-sync” task ready
“pre-sync” task done 3 3 3 2 2 1
“continuation” task 4 4 4 3 3 3 A B
task A spawned task B A B
task B depends on task A 5 5 5
fib channel
fib_cont channel stack:
top of
stack

10. Bound Cilk’s Memory Footprint
Bound memory to the depth of the Cilk tree by draining channels closer to the base case
The amount of work generated dynamically is not known a priori
We propose that GPUs allow SIMT threads to yield
Facilitates resolving conflicts on shared resources like memory 5 4 3 2 1

11. Channel Implementation
Our design accommodates SIMT access patterns
+ array-based
+ lock-free
+ non-blocking
See Paper

12. Outline Background Our Work Results/Conclusion GPUs Cilk
Channel Abstraction
Our Work
Cilk on Channels
Channel Design
Results/Conclusion

13. Methodology Implemented Cilk on channels on a simulated APU
Caches are sequentially consistent
Aggregator schedules Cilk tasks

14. Cilk scales with the GPU Architecture
More Compute Units  Faster execution

15. Conclusion
We observed that dynamic aggregation enables new GPU programming languages and abstractions
We enabled dynamic aggregation by extending the GPU’s control processor to manage channels
We found that breadth first scheduling works well for Cilk on GPUs
We proposed that GPUs allow SIMT threads to yield for breadth first scheduling
Future work should focus on how the control processor can enable more GPU applications

16. Backup

17. Divergence and Channels
Branch divergence
Memory divergence
+ Data in channels good
Pointers to data in channels bad

18. GPU NOT Blocked on Aggregator

19. GPU Cilk vs. standard GPU workloads
Cilk is more succinct than SIMT languages
Channels trigger more GPU dispatches
LOC reduction
Dispatch rate
Speedup
Strassen
42%
13x
1.06
Queens
36%
12.5x
0.98
Same performance, easier to program

20. Disclaimer & Attribution
The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of such revisions or changes. AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. ATTRIBUTION © 2014 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo and combinations thereof are trademarks of Advanced Micro Devices, Inc. in the United States and/or other jurisdictions. SPEC is a registered trademark of the Standard Performance Evaluation Corporation (SPEC). Other names are for informational purposes only and may be trademarks of their respective owners.