1. INTEL CONFIDENTIAL OpenMP for Task Decomposition Introduction to Parallel Programming – Part 8

2. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Review & Objectives Previously: Defined deadlock and explained ways to prevent it At the end of this part you should be able to: Describe how the OpenMP task pragma is different from the for pragma Code a task decomposition solution with the OpenMP task construct 2

3. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Pragma: single Denotes block of code to be executed by only one thread First thread to arrive is chosen Implicit barrier at end 3 #pragma omp parallel { DoManyThings(); #pragma omp single { printf(“Many Things done\n”); } // threads wait here for single DoManyMoreThings(); }

4. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. New Addition to OpenMP Tasks – Main change for OpenMP 3.0 Allows parallelization of irregular problems unbounded loops recursive algorithms producer/consumer 4

5. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. What are tasks? Tasks are independent units of work Threads are assigned to perform the work of each task Tasks may be deferred Tasks may be executed immediately The runtime system decides which of the above Tasks are composed of: code to execute data environment internal control variables (ICV) 5 SerialParallel

6. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; while (p) { process(p); p = p->next; } 6 data next head data next data next data next data next

7. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; while (p) { process(p); p = p->next; } 7 data next head data next data next data next data next p

8. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; while (p) { process(p); p = p->next; } 8 data next head data next data next data next data next p

9. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; while (p) { process(p); p = p->next; } 9 data next head data next data next data next data next p

10. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; while (p) { process(p); p = p->next; } 10 data next head data next data next data next data next p

11. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; while (p) { process(p); p = p->next; } 11 data next head data next data next data next data next p

12. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; while (p) { process(p); p = p->next; } 12 data next head data next data next data next data next p

13. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Task Construct – Explicit Task View node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; } 13 A team of threads is forked at the omp parallel construct A single thread, T0, executes the while loop Each time T0 crosses the omp task construct it generates a new task Each task runs in a thread All tasks complete at the barrier at the end of the parallel region’s single construct

14. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; } 14 data next head data next data next data next data next p

15. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; } 15 data next head data next data next data next data next p process() p

16. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; } 16 data next head data next data next data next data next p process() p process() p

17. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; } 17 data next head data next data next data next data next p process() p

18. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; } 18 data next head data next data next data next data next p process() p process() p

19. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; } 19 data next head data next data next data next data next p process() p

20. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; } 20 data next head data next data next data next data next p process() p process() p

21. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; } 21 data next head data next data next data next data next p process() p

22. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; } 22 data next head data next data next data next data next process() p process() p p

23. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; } 23 data next head data next data next data next data next process() p p

24. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. A Linked List Example node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; } 24 data next head data next data next data next data next p

25. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. When are tasks gauranteed to be complete? Tasks are gauranteed to be complete: At thread or task barriers At the directive: #pragma omp barrier At the directive: #pragma omp taskwait 25

26. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Example: Naive Fibonacci Calculation Recursion typically used to calculate Fibonacci number Widely used as toy benchmark Easy to code Has unbalanced task graph 26 long SerialFib( long n ) { if( n < 2 ) return n; else return SerialFib(n-1) + SerialFib(n-2); }

27. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Example: Naive Fibonacci Calculation We can envision Fibonacci computation as a task graph 27 SerialFib(4) SerialFib(3) SerialFib(2) SerialFib(1) SerialFib(2) SerialFib(1) SerialFib(0) SerialFib(2) SerialFib(1) SerialFib(0) SerialFib(3) SerialFib(2) SerialFib(1) SerialFib(0) SerialFib(1) SerialFib(0)

28. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Fibonacci - Task Spawning Solution long ParallelFib(long n) { long sum; #pragma omp parallel { #pragma omp single FibTask(n,&sum); } return sum; } Write a helper function to set up parallel region Call FibTask() to do computation Use sum return parameter in FibTask() 28

29. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Fibonacci - Task Spawning Solution void FibTask(long n, long* sum) { if( n < CutOff ) { *sum = SerialFib(n); } else { long x, y; #pragma omp task FibTask(n-1,&x); #pragma omp task FibTask(n-2,&y); #pragma omp taskwait *sum = x+y; } Thread will first check the value of n against CutOff If the cutoff hasn’t been reached, the thread will create two new tasks One to compute the n-1 Fib value One to compute the n-2 Fib value The computed values for these tasks will be returned through the private variables x and y, respectively The #pragma omp taskwait is required to make sure that the values for x and y have been computed before they are added together into sum 29

30. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Fibonacci Task Solution Example 30 long x, y; FibTask(7,&x); FibTask(6,&y); FibTask(8,*sum) *sum = x + y;

31. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Fibonacci Task Solution Example 31 long x, y; FibTask(7,&x); FibTask(6,&y); FibTask(8,*sum) *sum = x + y; long x, y; FibTask(6,&x); FibTask(5,&y); FibTask(7,*sum) *sum = x + y;

32. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Fibonacci Task Solution Example 32 long x, y; FibTask(7,&x); FibTask(6,&y); FibTask(8,*sum) *sum = x + y; long x, y; FibTask(6,&x); FibTask(5,&y); FibTask(7,*sum) *sum = x + y; long x, y; FibTask(5,&x); FibTask(4,&y); FibTask(6,*sum) *sum = x + y;

33. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Fibonacci Task Solution Example 33 long x, y; FibTask(7,&x); FibTask(6,&y); FibTask(8,*sum) *sum = x + y; long x, y; FibTask(6,&x); FibTask(5,&y); FibTask(7,*sum) *sum = x + y; long x, y; FibTask(5,&x); FibTask(4,&y); FibTask(6,*sum) *sum = x + y; long x, y; FibTask(5,&x); FibTask(4,&y); FibTask(6,*sum) *sum = x + y;

34. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Fibonacci Task Solution Example 34 long x, y; FibTask(7,&x); FibTask(6,&y); FibTask(8,*sum) *sum = x + y; long x, y; FibTask(6,&x); FibTask(5,&y); FibTask(7,*sum) *sum = x + y; long x, y; FibTask(5,&x); FibTask(4,&y); FibTask(6,*sum) *sum = x + y; long x, y; FibTask(5,&x); FibTask(4,&y); FibTask(6,*sum) *sum = x + y;

35. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Fibonacci Task Solution Example 35 long x, y; FibTask(7,&x); FibTask(6,&y); FibTask(8,*sum) *sum = x + y; long x, y; FibTask(6,&x); FibTask(5,&y); FibTask(7,*sum) *sum = x + y; long x, y; FibTask(5,&x); FibTask(4,&y); FibTask(6,*sum) *sum = x + y; long x, y; FibTask(5,&x); FibTask(4,&y); FibTask(6,*sum) *sum = x + y;

36. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Fibonacci Task Solution Example 36 long x, y; FibTask(7,&x); FibTask(6,&y); FibTask(8,*sum) *sum = x + y; long x, y; FibTask(6,&x); FibTask(5,&y); FibTask(7,*sum) *sum = x + y; long x, y; FibTask(5,&x); FibTask(4,&y); FibTask(6,*sum) *sum = x + y; long x, y; FibTask(5,&x); FibTask(4,&y); FibTask(6,*sum) *sum = x + y;

37. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Fibonacci Task Solution Example 37 long x, y; FibTask(7,&x); FibTask(6,&y); FibTask(8,*sum) *sum = x + y; long x, y; FibTask(6,&x); FibTask(5,&y); FibTask(7,*sum) *sum = x + y; long x, y; FibTask(5,&x); FibTask(4,&y); FibTask(6,*sum) *sum = x + y;

38. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Fibonacci Task Solution Example 38 long x, y; FibTask(7,&x); FibTask(6,&y); FibTask(8,*sum) *sum = x + y; long x, y; FibTask(6,&x); FibTask(5,&y); FibTask(7,*sum) *sum = x + y; long x, y; FibTask(5,&x); FibTask(4,&y); FibTask(6,*sum) *sum = x + y;

39. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Fibonacci Task Solution Example 39 long x, y; FibTask(7,&x); FibTask(6,&y); FibTask(8,*sum) *sum = x + y; long x, y; FibTask(6,&x); FibTask(5,&y); FibTask(7,*sum) *sum = x + y;

40. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. Fibonacci Task Solution Example 40 long x, y; FibTask(7,&x); FibTask(6,&y); FibTask(8,*sum) *sum = x + y;

41. Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners. References OpenMP API Specification, www.openmp.org 41