1. Optimizing MapReduce for GPUs with Effective Shared Memory Usage Department of Computer Science and Engineering The Ohio State University Linchuan Chen and Gagan Agrawal

2. Outline  Introduction  Background  System Design  Experiment Results  Related Work  Conclusions and Future Work

3. Introduction  Motivations  GPUs Suitable for extreme-scale computing Cost-effective and Power-efficient  MapReduce Programming Model Emerged with the development of Data-Intensive Computing  GPUs have been proved to be suitable for implementing MapReduce  Utilizing the fast but small shared memory for MapReduce is chanllenging Storing (Key, Value) pairs leads to high memory overhead, prohibiting the use of shared memory

4. Introduction  Our approach  Reduction-based method Reduce the (key, value) pair to the reduction object immediately after it is generated by the map function Very suitable for reduction-intensive applications  A general and efficient MapReduce framework Dynamic memory allocation within a reduction object Maintaining a memory hierarchy Multi-group mechanism Overflow handling

5. Outline  Introduction  Background  System Design  Experiment Results  Related Work  Conclusions and Future Work

6. MapReduce K1: v, v, v, vK2:vK3:v, vK4:v, v, vK5:v M M Group by Key K1:v k1:v k2:vK1:vK3:v k4:vK4:v k5:vK4:vK1:v k3:v M M M M M M M M M M M M M M R R R R R R R R R R

7. MapReduce  Programming Model  Map() Generates a large number of (key, value) pairs  Reduce() Merges the values associated with the same key  Efficient Runtime System  Parallelization  Concurrency Control  Resource Management  Fault Tolerance  … …

8. GPUs Host Kern el 1 Kern el 2 Device Grid 1 Block (0, 0) Block (1, 0) Block (2, 0) Block (0, 1) Block (1, 1) Block (2, 1) Grid 2 Block (1, 1) Thread (0, 1) Thread (1, 1) Thread (2, 1) Thread (3, 1) Thread (4, 1) Thread (0, 2) Thread (1, 2) Thread (2, 2) Thread (3, 2) Thread (4, 2) Thread (0, 0) Thread (1, 0) Thread (2, 0) Thread (3, 0) Thread (4, 0) (Device) Grid Constant Memory Texture Memory Device Memory Block (0, 0) Shared Memory Local Memor y Thread (0, 0) Register s Local Memor y Thread (1, 0) Register s Block (1, 0) Shared Memory Local Memor y Thread (0, 0) Register s Local Memor y Thread (1, 0) Register s Host Processing ComponentMemory Component

9. Outline  Introduction  Background  System Design  Experiment Results  Related Work  Conclusions and Future Work

10.  Traditional MapReduce map(input) { (key, value) = process(input); emit(key, value); } grouping the key-value pairs (by runtime system) reduce(key, iterator) { for each value in iterator result = operation(result, value); emit(key, result); } System Design

11.  Reduction-based approach map(input, reduction_object) { (key, value) = process(input); reduction_object->insert(key, value); } reduce(value1, value2) { value1 = operation(value1, value2); } Reduces the memory overhead of storing key-value pairs Makes it possible to effectively utilize shared memory on a GPU Eliminates the need of grouping Especially suitable for reduction-intensive applications System Design

12. Chanllenges  Result collection and overflow handling  Maintain a memory hierarchy  Trade off space requirement and locking overhead  A multi-group scheme  To keep the framework general and efficient  A well defined data structure for the reduction object

13. Memory Hierarchy CPU GPU Reduction Object 0 Reduction Object 1 Block 0’s Shared Memory Reduction Object 0 Reduction Object 1 Block 0’s Shared Memory … Device Memory Reduction Object Result Array Host Memory Device Memory

14. Reduction Object  Updating the reduction object  Use locks to synchronize  Memory allocation in reduction object  Dynamic memory allocation  Multiple offsets in device memory reduction object

15. Reduction Object KeyIdx[0]ValIdx[0] … Key SizeVal Size Key Data Val Data Memory Allocator Key SizeVal Size Key Data KeyIdx[1] ValIdx[1]

16. Multi-group Scheme  Locks are used for synchronization  Large number of threads in each thread block  Lead to severe contention on the shared memory RO  One solution: full replication  every thread owns a shared memory RO  leads to memory overhead and combination overhead  Trade-off  multi-group scheme  divide threads in each thread block into multiple sub-groups  each sub-group owns a shared memory RO  Choice of groups numbers  Contention overhead  Combination overhead

17. Overflow Handling  Swapping  Merge the full shared memory ROs to the device memory RO  Empty the full shared memory ROs  In-object sorting  Sort the buckets in the reduction object and delete the unuseful data  Users define the way of comparing two buckets

18. Discussion  Reduction-intensive applications  Our framework has a big advantage  Applications with few or no reduction  No need to use shared memory  Users need to setup system parameters  Develop auto-tuning techniques in future work

19. Extension for Multi-GPU  Shared memory usage can speed up single node execution  Potentially benefits the overall performance  Reduction objects can avoid global shuffling overhead  Can also reduce communication overhead

20. Outline  Introduction  Background  System Design  Experiment Results  Related Work  Conclusions and Future Work

21. Experiment Results  Applications used  5 reduction-intensive  2 map computation-intensive  Tested with small, medium and large datasets  Evaluation of the multi-group scheme  1, 2, 4 groups  Comparison with other implementations  Sequential implementations  MapCG  Ji et al.'s work  Evaluating the swapping mechanism  Test with large number of distinct keys

22. Evaluation of the Multi-group Scheme

23. Comparison with Sequential Implementations

24. Comparison with MapCG  With reduction-intensive applications

25. Comparison with MapCG  With other applications

26. Comparison with Ji et al.'s work

27. Evaluation of the Swapping Mechamism  VS MapCG and Ji et al.’s work

28. Evaluation of the Swapping Mechamism  VS MapCG

29. Evaluation of the Swapping Mechamism  swap_frequency = num_swaps / num_tasks

30. Outline  Introduction  Background  System Design  Experiment Results  Related Work  Conclusions and Future Work

31. Related Work  MapReduce for multi-core systems  Phoenix, Phoenix Rebirth  MapReduce on GPUs  Mars, MapCG  MapReduce-like framework on GPUs for SVM  Catanzaro et al.  MapReduce in heterogeneous environments  MITHRA, IDAV  Utilizing shared memory of GPUs for specific applications  Nyland et al., Gutierrez et al.  Compiler optimizations for utilizing shared memory  Baskaran et al. (PPoPP '08), Moazeni et al. (SASP '09)

32. Conclusions and Future Work  Reduction-based MapReduce  Storing the reduction object on the memory hierarchy of the GPU  A multi-group scheme  Improved performance compared with previous implementations  Future work: extend our framework to support new architectures

33. Thank you!