1. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Principles of Parallel Programming First Edition by Calvin Lin Lawrence Snyder Chapter 6: Programming with Threads

2. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-2 Code Spec 6.1 pthread_create(). The POSIX Threads thread creation function.

3. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-3 Code Spec 6.2 pthread_join(). The POSIX Threads rendezvous function.

4. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-4 Code Spec 6.3 pthread_self(). The POSIX Threads function to fetch a thread’s ID.

5. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-5 Code Spec 6.4 pthread_equal(). The POSIX Threads function to compare two thread IDs for equality.

6. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-6 Code Spec 6.5 pthread_exit(). The POSIX Threads thread termination function.

7. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-7 Code Spec 6.6 pthread attributes. An example of how thread attributes are set in the POSIX Threads interface.

8. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-8 Code Spec 6.7 The POSIX Threads routines for acquiring and releasing mutexes.

9. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-9 Code Spec 6.8 The POSIX Threads routines for dynamically creating and destroying mutexes.

10. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-10 Code Spec 6.9 An example of how dynamically allocated mutexes are used in the POSIX Threads interface.

11. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-11 Figure 6.1

12. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-12 Code Spec 6.10 pthread_cond_wait(). The POSIX Thread routines for waiting on condition variables.

13. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-13 Code Spec 6.11 pthread_cond_signal(). The POSIX Threads routines for signaling a condition variable.

14. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-14 Figure 6.2

15. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-15 Figure 6.3 Bounded buffer example using condition variables nonempty and nonfull.

16. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-16 Figure 6.4 Example of why a signaling thread needs to be protected by a mutex.

17. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-17 Figure 6.5

18. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-18 Code Spec 6.12 The POSIX Threads routines for dynamically creating and destroying condition variables.

19. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-19 Figure 6.6 Example of thread-specific data in POSIX Threads. Thread-specific data are accessed by keys, which map to different memory locations in different threads.

20. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-20 Code Spec 6.13 Example of how thread-specific data is used. Once initialized with this code, any procedure can access the value of my_index.

21. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-21 Code Spec 6.14 pthread_key_create(). POSIX Thread routine for creating a key for thread- specific data.

22. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-22 Code Spec 6.15 pthread_key_delete(). POSIX Thread routine for deleting a key.

23. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-23 Code Spec 6.16 pthread_setspecific(). POSIX Thread routine for setting the value of thread- specific data.

24. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-24 Code Spec 6.17 pthread_getspecific(). POSIX Thread routine for getting the value of some thread-specific data.

25. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-25 Figure 6.7 Deadlock example. Threads T1 and T2 hold locks L1 and L2, respectively, and each thread attempts to acquire the other lock, which cannot be granted.

26. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-26 Figure 6.8 Monitors provide an abstraction of synchronization in which only one thread can access the monitor’s data at any time. Other threads are blocked either waiting to enter the monitor or waiting on events inside the monitor.

27. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-27 Figure 6.9 Monitor implementation in C++.

28. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-28 Figure 6.9 Monitor implementation in C++. (cont.)

29. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-29 Figure 6.10 Monitors and invariants. The shaded circles represent program states in which the invariants may be violated. The empty circles represent program states in which the invariants are assumed to be true.

30. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-30 Figure 6.11 A program to check the invariants in the bounded buffer program, Figure 6.9.

31. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-31 Figure 6.12 Multiple readers, single writer support routines.

32. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-32 Figure 6.13 Multiple readers, single-writer support routines based on a single-condition variable, but subject to spurious wake-ups.

33. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-33 Code Spec 6.18 POSIX Thread routine for setting thread scheduling attributes.

34. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-34 Figure 6.14 A 2D relaxation replaces—on each iteration—all interior values by the average of their four nearest neighbors.

35. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-35 Figure 6.15

36. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-36 Figure 6.16 2D Sucessive over-relaxation program written using POSIX Threads.

37. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-37 Figure 6.16 2D Sucessive over-relaxation program written using POSIX Threads. (cont.)

38. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-38 Figure 6.16 2D Sucessive over-relaxation program written using POSIX Threads. (cont.)

39. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-39 Figure 6.16 2D Sucessive over-relaxation program written using POSIX Threads. (cont.)

40. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-40 Figure 6.17 It’s often profitable to do useful work while waiting for some long-latency operation to complete.

41. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-41 Figure 6.18 A split-phase barrier allows a thread to do useful work while waiting for the other threads to arrive at the barrier.

42. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-42 Figure 6.19 A 1D over-relaxation replaces—on each iteration—all interior values by the average of their two nearest neighbors.

43. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-43 Figure 6.20 Program for 1D successive over-relaxation using a single-phase barrier.

44. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-44 Figure 6.21 Program for 1D successive over- relaxation using a split-phase barrier.

45. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-45 Figure 6.21 Program for 1D successive over- relaxation using a split-phase barrier. (cont.)

46. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-46 Figure 6.22 Initial split-phase barrier implementation that keeps a count of the number of arrivals.

47. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-47 Figure 6.22 Initial split-phase barrier implementation that keeps a count of the number of arrivals. (cont.)

48. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-48 Figure 6.23 Deadlock with our initial implementation of a split-phase barrier; Thread0 and Thread1 each waits for different instances of the barrier.

49. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-49 Figure 6.24 A correct barrier implementation that keeps track of the correct phase.

50. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-50 Figure 6.24 A correct barrier implementation that keeps track of the correct phase. (cont.)

51. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-51 Figure 6.25 Program for 1D successive over- relaxation solution of Figure 6.19 with the split- phase barrier code.

52. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-52 Figure 6.26 Deadlock does not occur with our new split-phase barrier.

53. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-53 Code Spec 6.19 Java’s Thread class methods.

54. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-54 Figure 6.27 Count 3s solution in Java.

55. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-55 Figure 6.27 Count 3s solution in Java. (cont.)

56. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-56 Figure 6.27 Count 3s solution in Java. (cont.)

57. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-57 Code Spec 6.20 Examples of Java’s atomic objects.

58. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-58 Figure 6.28

59. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-59 Code Spec 6.21 OpenMP parallel for statement.

60. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-60 Code Spec 6.22 OpenMP reduce operation. The choice comes from the accompanying table.

61. Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 6-61 Code Spec 6.23 Atomic specification in OpenMP.