1. INTEL CONFIDENTIAL Parallel Decomposition Methods Introduction to Parallel Programming – Part 2

2. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Review & Objectives Previously: Defined parallel computing Explained why parallel computing is becoming mainstream Explained why explicit parallel programming is necessary At the end of this part you should be able to: Identify opportunities for parallelism in code segments and applications Describe three methods of dividing independent work 2

3. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Methodology Study problem, sequential program, or code segment Look for opportunities for parallelism Try to keep all cores busy doing useful work 3

4. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Ways of Exploiting Parallelism Domain decomposition Task decomposition Pipelining 4

5. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition First, decide how data elements should be divided among cores Second, decide which tasks each core should be doing Example: Vector addition 5

6. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 6 Find the largest element of an array

7. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 7 Find the largest element of an array Core 0Core 1Core 2Core 3

8. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 8 Find the largest element of an array Core 0Core 1Core 2Core 3

9. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 9 Find the largest element of an array Core 0Core 1Core 2Core 3

10. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 10 Find the largest element of an array Core 0Core 1Core 2Core 3

11. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 11 Find the largest element of an array Core 0Core 1Core 2Core 3

12. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 12 Find the largest element of an array Core 0Core 1Core 2Core 3

13. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 13 Find the largest element of an array Core 0Core 1Core 2Core 3

14. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 14 Find the largest element of an array Core 0Core 1Core 2Core 3

15. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 15 Find the largest element of an array Core 0Core 1Core 2Core 3

16. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 16 Find the largest element of an array Core 0Core 1Core 2Core 3

17. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 17 Find the largest element of an array Core 0Core 1Core 2Core 3

18. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Task (Functional) Decomposition First, divide problem into independent tasks Second, decide which data elements are going to be accessed (read and/or written) by which tasks Example: Event-handler for GUI 18

19. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Task Decomposition 19 s() r() q() h() g() f()

20. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Task Decomposition 20 s() r() q() h() g() Core 0 Core 2 Core 1 f()

21. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Task Decomposition 21 s() r() q() h() g() Core 2 Core 1 f() Core 0

22. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Task Decomposition 22 s() r() q() h() g() Core 2 Core 1Core 0 f()

23. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Task Decomposition 23 s() r() q() h() g() Core 2 Core 1Core 0 f()

24. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Task Decomposition 24 s() r() q() h() g() Core 2 Core 1Core 0 f()

25. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Special kind of task decomposition “Assembly line” parallelism Example: 3D rendering in computer graphics 25 RasterizeClipProjectModel Input Output

26. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing One Data Set (Step 1) 26 RasterizeClipProjectModel

27. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing One Data Set (Step 2) 27 RasterizeClipProjectModel

28. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing One Data Set (Step 3) 28 RasterizeClipProjectModel

29. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing One Data Set (Step 4) 29 RasterizeClipProjectModel The pipeline processes 1 data set in 4 steps

30. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing Two Data Sets (Step 1) 30 RasterizeClipProjectModel

31. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing Two Data Sets (Time 2) 31 RasterizeClipProjectModel

32. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing Two Data Sets (Step 3) 32 RasterizeClipProjectModel

33. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing Two Data Sets (Step 4) 33 RasterizeClipProjectModel

34. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing Two Data Sets (Step 5) 34 RasterizeClipProjectModel The pipeline processes 2 data sets in 5 steps

35. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Five Data Sets (Step 1) 35 Data set 0 Data set 1 Data set 2 Data set 3 Data set 4 Core 0Core 1Core 2Core 3

36. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Five Data Sets (Step 2) 36 Data set 0 Data set 1 Data set 2 Data set 3 Data set 4 Core 0Core 1Core 2Core 3

37. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Five Data Sets (Step 3) 37 Data set 0 Data set 1 Data set 2 Data set 3 Data set 4 Core 0Core 1Core 2Core 3

38. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Five Data Sets (Step 4) 38 Data set 0 Data set 1 Data set 2 Data set 3 Data set 4 Core 0Core 1Core 2Core 3

39. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Five Data Sets (Step 5) 39 Data set 0 Data set 1 Data set 2 Data set 3 Data set 4 Core 0Core 1Core 2Core 3

40. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Five Data Sets (Step 6) 40 Data set 0 Data set 1 Data set 2 Data set 3 Data set 4 Core 0Core 1Core 2Core 3

41. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Five Data Sets (Step 7) 41 Data set 0 Data set 1 Data set 2 Data set 3 Data set 4 Core 0Core 1Core 2Core 3

42. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Five Data Sets (Step 8) 42 Data set 0 Data set 1 Data set 2 Data set 3 Data set 4 Core 0Core 1Core 2Core 3

43. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. References Richard H. Carver and Kuo-Chung Tai, Modern Multithreading: Implementing, Testing, and Debugging Java and C++/Pthreads/ Win32 Programs, Wiley-Interscience (2006). Robert L. Mitchell, “Decline of the Desktop,” Computerworld (September 26, 2005). Michael J. Quinn, Parallel Programming in C with MPI and OpenMP, McGraw-Hill (2004). Herb Sutter, “The Free Lunch is Over: A Fundamental Turn Toward Concurrency in Software,” Dr. Dobb’s Journal 30, 3 (March 2005). 43