1. An Effective GPU Implementation of Breadth-First Search Lijuan Luo, Martin Wong and Wen-mei Hwu Department of Electrical and Computer Engineering, UIUC From DAC 2010

2. Outline Introduction  Why BFS and why GPU? Previous works  IIIT-BFS and a matrix-vector based BFS The proposed GPU solution  Architecture of GPU, CUDA  Hierarchical queue and kernel  Synchronization  Examples Experimental results and conclusions

3. Introduction The graphic processing unit (GPU) has become popular in parallel processing for its cost-effectiveness. Due to the architecture of GPU, implementing the fastest CPU algorithm on it directly may cause huge overhead. Hence, the speed-up is not always meaningful.

4. Introduction (cont.)

5. Breadth-First Search has been widely used in EDA.  Maze routing, circuit simulation, STA, etc. Previous works are slower than the fastest CPU program in some certain types of graphs.

6. Breadth-First Search (BFS) Given a graph G=(V,E) and a distingui- shed source vertex s. BFS explores the edges of G to discover every vertex that is reachable from s. Produce a breadth-first tree with root s that contains all reachable vertices.

7. Example

8. BFS (cont.) Traditional BFS algorithms use a queue to store the frontiers. Complexity is O(V+E)

9. Previous works IIIT-BFS [2] is the first work implementing BFS algorithm on GPU. [3] uses matrix-vector multiplication way to represent and do BFS.

10. IIIT-BFS Point out that to maintain the frontier queue can cause a huge overhead on the GPU. For each level, IIIT-BFS exhaustively check every vertex to see whether it belongs to the current frontier. O(VL+E), where L is the total number of levels. In sparse graph, L=E=O(V) and hence O(VL+E)=O(V 2 )

11. BFS in Matrix-vector multiplication Accelerated a matrix-based BFS algorithm for sparse graphs. Each frontier propagation can be transformed into a matrix-vector multiplication. O(V+EL), where L is the number of levels. In sparse graph, L=E=O(V) and hence O(V+EL)=O(V 2 )

12. The proposed GPU solution To propagate from all the frontier vertices in parallel.  Since lots of EDA problems are formulated as sparse graphs, the number of neighbors of each frontier vertex is less Hierarchical queue and kernel. Same complexity as the traditional CPU implementation.

13. Architecture of Nvidia GTX280 A collection of 30 multiprocessors, with 8 streaming processors each. The 30 multiprocessors share one off-chip global memory.  Access time: about 300 clock cycles Each multiprocessor has a on-chip memory shared by that 8 streaming processors.  Access time: 2 clock cycles

14. Architecture diagram

15. Memory coalescing Several memory transactions can be coalesced into one transaction when consecutive threads access consecutive memory locations. Due to access time of global memory is relatively large, it is important to achieve this.

16. CUDA programming Compute Unified Device Architecture The CPU code does the sequential part. Highly parallelized part usually implement in the GPU code, called kernel. Calling GPU function in CPU code is called kernel launch.

17. Hierarchical Queue Management Hierarchical frontiers structure

18. Hierarchical Queue Management (cont.) G-Frontier: the frontier vertices shared by all the threads of a grid. B-Frontier: the frontier vertices common to a whole block. W-Frontier: the frontier vertices only accessed by certain threads from a warp.

19. Collision Collision means more than one thread are accessing the same queue at the same time.  suppose only one queue and each SP has a thread that is returning the new frontier vertices  8 threads are accessing the same queue—collision happens

20. Hierarchical Queue Management (cont.) Each W-Frontier maintains 8 queues so that no collision will happen in a W-Frontier. The scheduling unit – warp.  contains 32 threads each  four 8-thread groups

21. Synchronization issues Correct results require thread synchronization at the end of each level of queue. General solution: launch one kernel for each level of queue and implement a global barrier between two launched kernels. If we do that, the kernel-launch overhead will be huge. CUDA only provides barrier function to synchronize threads in a block.

22. Hierarchical Kernel Arrangement Only the highest layer uses this expensive synchronization method (global barrier) and the others use more efficient GPU synchronization. Using intra-block synchronization (provide by CUDA) to synchronize the threads in a block. Using inter-block synchronization [10] to synchronize threads between different blocks.  These two are GPU synchronization techniques that the kernel does not need to terminated.

23. Example Intra-block sync. Inter-block sync. Global barrier (kernel sync.)

24. Another example Assume that there are 100 vertices in the queue. First, launch a kernel. It will create a grid with a block with 512 thread (only 100 non- empty threads). 1……100 Vertex 1 ~ 100 Empty threads Block Grid

25. Another example (cont.) Threads in that block will synchronize by intra-block synchronization. Threads 8 W-Frontiers and one B-Frontier Intra-block sync.

26. Another example (cont.) Assume that after the computation, we get 1000 new frontier vertices. 1000>512. It will use G-Frontier queue to handle the whole vertices and continue. Threads in different blocks will synchronize by inter- block synchronization. Global memory G-Frontier Threads Inter-block sync.

27. Another example (cont.) Once the new frontier vertices is larger than 15360, the kernel is terminated and re-launch a kernel with 15360 threads until this BFS-level is finished.

28. Experimental results Environment  A dual socket, dual core 2.4 GHz Opteron processor  8 GB of memory  A single NVIDIA GeForce GTX280 GPU

29. Experimental results (cont.) The results on degree-6 regular graph.  grid-based graph like

30. Experimental results (cont.) The results on real world graphs  Average deg(V)=2, maximum deg(V)=8 or 9

31. Experimental results (cont.) The results on scale-free graphs  0.1% of the vertices have degree 1000  Other vertices have average degree of 6, maximum degree of 7

32. Conclusions This ideas proposed were never used on other architecture. Most suitable for sparse and near- regular graphs that is often used in EDA. Hierarchical queue management and hierarchical kernel arrangement.