INTEL CONFIDENTIAL Parallel Decomposition Methods Introduction to Parallel Programming – Part 2
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Review & Objectives Previously: Defined parallel computing Explained why parallel computing is becoming mainstream Explained why explicit parallel programming is necessary At the end of this part you should be able to: Identify opportunities for parallelism in code segments and applications Describe three methods of dividing independent work 2
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Methodology Study problem, sequential program, or code segment Look for opportunities for parallelism Try to keep all cores busy doing useful work 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Ways of Exploiting Parallelism Domain decomposition Task decomposition Pipelining 4
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition First, decide how data elements should be divided among cores Second, decide which tasks each core should be doing Example: Vector addition 5
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 6 Find the largest element of an array
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 7 Find the largest element of an array Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 8 Find the largest element of an array Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 9 Find the largest element of an array Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 10 Find the largest element of an array Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 11 Find the largest element of an array Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 12 Find the largest element of an array Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 13 Find the largest element of an array Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 14 Find the largest element of an array Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 15 Find the largest element of an array Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 16 Find the largest element of an array Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Domain Decomposition 17 Find the largest element of an array Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Task (Functional) Decomposition First, divide problem into independent tasks Second, decide which data elements are going to be accessed (read and/or written) by which tasks Example: Event-handler for GUI 18
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Task Decomposition 19 s() r() q() h() g() f()
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Task Decomposition 20 s() r() q() h() g() Core 0 Core 2 Core 1 f()
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Task Decomposition 21 s() r() q() h() g() Core 2 Core 1 f() Core 0
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Task Decomposition 22 s() r() q() h() g() Core 2 Core 1Core 0 f()
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Task Decomposition 23 s() r() q() h() g() Core 2 Core 1Core 0 f()
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Task Decomposition 24 s() r() q() h() g() Core 2 Core 1Core 0 f()
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Special kind of task decomposition “Assembly line” parallelism Example: 3D rendering in computer graphics 25 RasterizeClipProjectModel Input Output
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing One Data Set (Step 1) 26 RasterizeClipProjectModel
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing One Data Set (Step 2) 27 RasterizeClipProjectModel
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing One Data Set (Step 3) 28 RasterizeClipProjectModel
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing One Data Set (Step 4) 29 RasterizeClipProjectModel The pipeline processes 1 data set in 4 steps
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing Two Data Sets (Step 1) 30 RasterizeClipProjectModel
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing Two Data Sets (Time 2) 31 RasterizeClipProjectModel
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing Two Data Sets (Step 3) 32 RasterizeClipProjectModel
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing Two Data Sets (Step 4) 33 RasterizeClipProjectModel
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Processing Two Data Sets (Step 5) 34 RasterizeClipProjectModel The pipeline processes 2 data sets in 5 steps
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Five Data Sets (Step 1) 35 Data set 0 Data set 1 Data set 2 Data set 3 Data set 4 Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Five Data Sets (Step 2) 36 Data set 0 Data set 1 Data set 2 Data set 3 Data set 4 Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Five Data Sets (Step 3) 37 Data set 0 Data set 1 Data set 2 Data set 3 Data set 4 Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Five Data Sets (Step 4) 38 Data set 0 Data set 1 Data set 2 Data set 3 Data set 4 Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Five Data Sets (Step 5) 39 Data set 0 Data set 1 Data set 2 Data set 3 Data set 4 Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Five Data Sets (Step 6) 40 Data set 0 Data set 1 Data set 2 Data set 3 Data set 4 Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Five Data Sets (Step 7) 41 Data set 0 Data set 1 Data set 2 Data set 3 Data set 4 Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pipelining Five Data Sets (Step 8) 42 Data set 0 Data set 1 Data set 2 Data set 3 Data set 4 Core 0Core 1Core 2Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. References Richard H. Carver and Kuo-Chung Tai, Modern Multithreading: Implementing, Testing, and Debugging Java and C++/Pthreads/ Win32 Programs, Wiley-Interscience (2006). Robert L. Mitchell, “Decline of the Desktop,” Computerworld (September 26, 2005). Michael J. Quinn, Parallel Programming in C with MPI and OpenMP, McGraw-Hill (2004). Herb Sutter, “The Free Lunch is Over: A Fundamental Turn Toward Concurrency in Software,” Dr. Dobb’s Journal 30, 3 (March 2005). 43