1. Multicore Programming Summer 2008

2. Daily Schedule 09:00am - 10:30am – Lecture 10:30am - 10:50pm - Break 10:50am - 12:20pm - Lecture 12:20pm - 01:20pm - Lunch 01:20pm - 03:20pm – Class work 03:20pm - 03:30pm - Break 03:30pm - 05:00pm - Lecture Herlihy-Shavit The Art of Multiprocessor Programming 2

3. Introduction Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

4. Art of Multiprocessor Programming4 Moore’s Law Clock speed flattening sharply Transistor count still rising

5. Art of Multiprocessor Programming5 Still on some of your desktops: The Uniprocesor memory cpu

6. Art of Multiprocessor Programming6 In the Enterprise: The Shared Memory Multiprocessor (SMP) cache Bus shared memory cache

7. Art of Multiprocessor Programming7 Your New Desktop: The Multicore Processor (CMP) cache Bus shared memory cache All on the same chip Sun T2000 Niagara

8. Art of Multiprocessor Programming8 Multicores Are Here “Intel's Intel ups ante with 4-core chip. New microprocessor, due this year, will be faster, use less electricity...” [San Fran Chronicle] “AMD will launch a dual-core version of its Opteron server processor at an event in New York on April 21.” [PC World] “Sun’s Niagara…will have eight cores, each core capable of running 4 threads in parallel, for 32 concurrently running threads. ….” [The Inquierer]

9. Art of Multiprocessor Programming9 Why do we care? Time no longer cures software bloat –The “free ride” is over When you double your program’s path length –You can’t just wait 6 months –Your software must somehow exploit twice as much concurrency

10. Art of Multiprocessor Programming10 Traditional Scaling Process User code Traditional Uniprocessor Speedup 1.8x 7x 3.6x Time: Moore’s law

11. Art of Multiprocessor Programming11 Multicore Scaling Process User code Multicore Speedup 1.8x7x3.6x Unfortunately, not so simple…

12. Art of Multiprocessor Programming12 Real-World Scaling Process 1.8x 2x 2.9x User code Multicore Speedup Parallelization and Synchronization require great care…

13. Art of Multiprocessor Programming13 Multicore Programming: Course Overview Fundamentals –Models, algorithms, impossibility Real-World programming –Architectures –Techniques

14. Art of Multiprocessor Programming14 Multicore Programming: Course Overview Fundamentals –Models, algorithms, impossibility Real-World programming –Architectures –Techniques We don’t necessarily want to make you experts…

15. Art of Multiprocessor Programming 15 Sequential Computation memory object thread

16. Art of Multiprocessor Programming 16 Concurrent Computation memory object threads

17. Art of Multiprocessor Programming17 Asynchrony Sudden unpredictable delays –Cache misses (short) –Page faults (long) –Scheduling quantum used up (really long)

18. Art of Multiprocessor Programming18 Model Summary Multiple threads –Sometimes called processes Single shared memory Objects live in memory Unpredictable asynchronous delays

19. Art of Multiprocessor Programming19 Road Map We are going to focus on principles first, then practice –Start with idealized models –Look at simplistic problems –Emphasize correctness over pragmatism –“Correctness may be theoretical, but incorrectness has practical impact”

20. Art of Multiprocessor Programming20 Concurrency Jargon Hardware –Processors Software –Threads, processes Sometimes OK to confuse them, sometimes not.

21. Art of Multiprocessor Programming21 Parallel Primality Testing Challenge –Print primes from 1 to 10 10 Given –Ten-processor multiprocessor –One thread per processor Goal –Get ten-fold speedup (or close)

22. Art of Multiprocessor Programming22 Load Balancing Split the work evenly Each thread tests range of 10 9 … … 10 910 2·10 9 1 P0P0 P1P1 P9P9

23. Art of Multiprocessor Programming 23 Procedure for Thread i void primePrint { int i = ThreadID.get(); // IDs in {0..9} for (j = i*10 9 +1, j<(i+1)*10 9 ; j++) { if (isPrime(j)) print(j); }

24. Art of Multiprocessor Programming24 Issues Higher ranges have fewer primes Yet larger numbers harder to test Thread workloads –Uneven –Hard to predict

25. Art of Multiprocessor Programming25 Issues Higher ranges have fewer primes Yet larger numbers harder to test Thread workloads –Uneven –Hard to predict Need dynamic load balancing rejected

26. Art of Multiprocessor Programming 26 17 18 19 Shared Counter each thread takes a number

27. Art of Multiprocessor Programming 27 Procedure for Thread i int counter = new Counter(1); void primePrint { long j = 0; while (j < 10 10 ) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); }

28. Art of Multiprocessor Programming 28 Counter counter = new Counter(1); void primePrint { long j = 0; while (j < 10 10 ) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); } Procedure for Thread i Shared counter object

29. Art of Multiprocessor Programming29 Where Things Reside cache Bus cache 1 shared counter shared memory void primePrint { int i = ThreadID.get(); // IDs in {0..9} for (j = i*10 9 +1, j<(i+1)*10 9 ; j++) { if (isPrime(j)) print(j); } code Local variables

30. Art of Multiprocessor Programming 30 Procedure for Thread i Counter counter = new Counter(1); void primePrint { long j = 0; while (j < 10 10 ) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); } Stop when every value taken

31. Art of Multiprocessor Programming 31 Counter counter = new Counter(1); void primePrint { long j = 0; while (j < 10 10 ) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); } Procedure for Thread i Increment & return each new value

32. Art of Multiprocessor Programming 32 Counter Implementation public class Counter { private long value; public long getAndIncrement() { return value++; }

33. Art of Multiprocessor Programming 33 Counter Implementation public class Counter { private long value; public long getAndIncrement() { return value++; } OK for single thread, not for concurrent threads

34. Art of Multiprocessor Programming 34 What It Means public class Counter { private long value; public long getAndIncrement() { return value++; }

35. Art of Multiprocessor Programming 35 What It Means public class Counter { private long value; public long getAndIncrement() { return value++; } temp = value; value = value + 1; return temp;

36. Art of Multiprocessor Programming 36 time Not so good… Value… 1 read 1 read 1 write 2 read 2 write 3 write 2 232

37. Art of Multiprocessor Programming 37 Is this problem inherent? If we could only glue reads and writes… read write read write

38. Art of Multiprocessor Programming 38 Challenge public class Counter { private long value; public long getAndIncrement() { temp = value; value = temp + 1; return temp; }

39. Art of Multiprocessor Programming 39 Challenge public class Counter { private long value; public long getAndIncrement() { temp = value; value = temp + 1; return temp; } Make these steps atomic (indivisible)

40. Art of Multiprocessor Programming 40 Hardware Solution public class Counter { private long value; public long getAndIncrement() { temp = value; value = temp + 1; return temp; } ReadModifyWrite() instruction

41. Art of Multiprocessor Programming 41 An Aside: Java™ public class Counter { private long value; public long getAndIncrement() { synchronized { temp = value; value = temp + 1; } return temp; }

42. Art of Multiprocessor Programming 42 An Aside: Java™ public class Counter { private long value; public long getAndIncrement() { synchronized { temp = value; value = temp + 1; } return temp; } Synchronized block

43. Art of Multiprocessor Programming 43 An Aside: Java™ public class Counter { private long value; public long getAndIncrement() { synchronized { temp = value; value = temp + 1; } return temp; } Mutual Exclusion

44. Art of Multiprocessor Programming 44 Mutual Exclusion or “Alice & Bob share a pond” AB

45. Art of Multiprocessor Programming 45 Alice has a pet AB

46. Art of Multiprocessor Programming 46 Bob has a pet AB

47. Art of Multiprocessor Programming 47 The Problem AB The pets don’t get along

48. Art of Multiprocessor Programming48 Formalizing the Problem Two types of formal properties in asynchronous computation: Safety Properties –Nothing bad happens ever Liveness Properties –Something good happens eventually

49. Art of Multiprocessor Programming49 Formalizing our Problem Mutual Exclusion –Both pets never in pond simultaneously –This is a safety property No Deadlock –if only one wants in, it gets in –if both want in, one gets in. –This is a liveness property

50. Art of Multiprocessor Programming50 Simple Protocol Idea –Just look at the pond Gotcha –Trees obscure the view

51. Art of Multiprocessor Programming51 Interpretation Threads can’t “see” what other threads are doing Explicit communication required for coordination

52. Art of Multiprocessor Programming52 Cell Phone Protocol Idea –Bob calls Alice (or vice-versa) Gotcha –Bob takes shower –Alice recharges battery –Bob out shopping for pet food …

53. Art of Multiprocessor Programming53 Interpretation Message-passing doesn’t work Recipient might not be –Listening –There at all Communication must be –Persistent (like writing) –Not transient (like speaking)

54. Art of Multiprocessor Programming 54 Can Protocol cola

55. Art of Multiprocessor Programming 55 Bob conveys a bit AB cola

56. Art of Multiprocessor Programming 56 Bob conveys a bit AB cola

57. Art of Multiprocessor Programming57 Can Protocol Idea –Cans on Alice’s windowsill –Strings lead to Bob’s house –Bob pulls strings, knocks over cans Gotcha –Cans cannot be reused –Bob runs out of cans

58. Art of Multiprocessor Programming58 Interpretation Cannot solve mutual exclusion with interrupts –Sender sets fixed bit in receiver’s space –Receiver resets bit when ready –Requires unbounded number of inturrupt bits

59. Art of Multiprocessor Programming 59 Flag Protocol AB

60. Art of Multiprocessor Programming 60 Alice’s Protocol (sort of) AB

61. Art of Multiprocessor Programming 61 Bob’s Protocol (sort of) AB

62. Art of Multiprocessor Programming62 Alice’s Protocol Raise flag Wait until Bob’s flag is down Unleash pet Lower flag when pet returns

63. Art of Multiprocessor Programming63 Bob’s Protocol Raise flag Wait until Alice’s flag is down Unleash pet Lower flag when pet returns danger!

64. Art of Multiprocessor Programming64 Bob’s Protocol (2 nd try) Raise flag While Alice’s flag is up –Lower flag –Wait for Alice’s flag to go down –Raise flag Unleash pet Lower flag when pet returns

65. Art of Multiprocessor Programming65 Bob’s Protocol Raise flag While Alice’s flag is up –Lower flag –Wait for Alice’s flag to go down –Raise flag Unleash pet Lower flag when pet returns Bob defers to Alice

66. Art of Multiprocessor Programming66 The Flag Principle Raise the flag Look at other’s flag Flag Principle: –If each raises and looks, then –Last to look must see both flags up

67. Art of Multiprocessor Programming67 Proof of Mutual Exclusion Assume both pets in pond –Derive a contradiction –By reasoning backwards Consider the last time Alice and Bob each looked before letting the pets in Without loss of generality assume Alice was the last to look…

68. Art of Multiprocessor Programming 68 Proof time Alice’s last look Alice last raised her flag Bob’s last look QED Alice must have seen Bob’s Flag. A Contradiction Bob last raised flag

69. Art of Multiprocessor Programming69 Proof of No Deadlock If only one pet wants in, it gets in.

70. Art of Multiprocessor Programming70 Proof of No Deadlock If only one pet wants in, it gets in. Deadlock requires both continually trying to get in.

71. Art of Multiprocessor Programming71 Proof of No Deadlock If only one pet wants in, it gets in. Deadlock requires both continually trying to get in. If Bob sees Alice’s flag, he gives her priority (a gentleman…) QED

72. Art of Multiprocessor Programming72 Remarks Protocol is unfair –Bob’s pet might never get in Protocol uses waiting –If Bob is eaten by his pet, Alice’s pet might never get in

73. Art of Multiprocessor Programming73 Moral of Story Mutual Exclusion cannot be solved by –transient communication (cell phones) –interrupts (cans) It can be solved by – one-bit shared variables – that can be read or written

74. Art of Multiprocessor Programming74 The Arbiter Problem (an aside) Pick a point

75. Art of Multiprocessor Programming75 The Fable Continues Alice and Bob fall in love & marry

76. Art of Multiprocessor Programming76 The Fable Continues Alice and Bob fall in love & marry Then they fall out of love & divorce –She gets the pets –He has to feed them

77. Art of Multiprocessor Programming77 The Fable Continues Alice and Bob fall in love & marry Then they fall out of love & divorce –She gets the pets –He has to feed them Leading to a new coordination problem: Producer-Consumer

78. Art of Multiprocessor Programming 78 Bob Puts Food in the Pond A

79. Art of Multiprocessor Programming 79 mmm… Alice releases her pets to Feed B mmm…

80. Art of Multiprocessor Programming80 Producer/Consumer Alice and Bob can’t meet –Each has restraining order on other –So he puts food in the pond –And later, she releases the pets Avoid –Releasing pets when there’s no food –Putting out food if uneaten food remains

81. Art of Multiprocessor Programming81 Producer/Consumer Need a mechanism so that –Bob lets Alice know when food has been put out –Alice lets Bob know when to put out more food

82. Art of Multiprocessor Programming 82 Surprise Solution AB cola

83. Art of Multiprocessor Programming 83 Bob puts food in Pond AB cola

84. Art of Multiprocessor Programming 84 Bob knocks over Can AB cola

85. Art of Multiprocessor Programming 85 Alice Releases Pets AB cola yum… B

86. Art of Multiprocessor Programming 86 Alice Resets Can when Pets are Fed AB cola

87. Art of Multiprocessor Programming 87 Pseudocode while (true) { while (can.isUp()){}; pet.release(); pet.recapture(); can.reset(); } Alice’s code

88. Art of Multiprocessor Programming 88 Pseudocode while (true) { while (can.isUp()){}; pet.release(); pet.recapture(); can.reset(); } Alice’s code while (true) { while (can.isDown()){}; pond.stockWithFood(); can.knockOver(); } Bob’s code

89. Art of Multiprocessor Programming89 Correctness Mutual Exclusion –Pets and Bob never together in pond

90. Art of Multiprocessor Programming90 Correctness Mutual Exclusion –Pets and Bob never together in pond No Starvation if Bob always willing to feed, and pets always famished, then pets eat infinitely often.

91. Art of Multiprocessor Programming91 Correctness Mutual Exclusion –Pets and Bob never together in pond No Starvation if Bob always willing to feed, and pets always famished, then pets eat infinitely often. Producer/Consumer The pets never enter pond unless there is food, and Bob never provides food if there is unconsumed food. safety liveness safety

92. Art of Multiprocessor Programming 92 Could Also Solve Using Flags AB

93. Art of Multiprocessor Programming93 Waiting Both solutions use waiting –while(mumble){} Waiting is problematic –If one participant is delayed –So is everyone else –But delays are common & unpredictable

94. Art of Multiprocessor Programming94 The Fable drags on … Bob and Alice still have issues

95. Art of Multiprocessor Programming95 The Fable drags on … Bob and Alice still have issues So they need to communicate

96. Art of Multiprocessor Programming96 The Fable drags on … Bob and Alice still have issues So they need to communicate So they agree to use billboards …

97. Art of Multiprocessor Programming 97 E 1 D 2 C 3 Billboards are Large B 3 A 1 Letter Tiles From Scrabble™ box

98. Art of Multiprocessor Programming 98 E 1 D 2 C 3 Write One Letter at a Time … B 3 A 1 W 4 A 1 S 1 H 4

99. Art of Multiprocessor Programming 99 To post a message W 4 A 1 S 1 H 4 A 1 C 3 R 1 T 1 H 4 E 1 whe w

100. Art of Multiprocessor Programming 100 S 1 Let’s send another message S 1 E 1 L 1 L 1 L 1 V 4 L 1 A 1 M 3 A 1 A 1 P 3

101. Art of Multiprocessor Programming 101 Uh-Oh A 1 C 3 R 1 T 1 H 4 E 1 S 1 E 1 L 1 L 1 L 1 OK

102. Art of Multiprocessor Programming102 Readers/Writers Devise a protocol so that –Writer writes one letter at a time –Reader reads one letter at a time –Reader sees Old message or new message No mixed messages

103. Art of Multiprocessor Programming103 Readers/Writers (continued) Easy with mutual exclusion But mutual exclusion requires waiting –One waits for the other –Everyone executes sequentially Remarkably –We can solve R/W without mutual exclusion

104. Art of Multiprocessor Programming104 Why do we care? We want as much of the code as possible to execute concurrently (in parallel) A larger sequential part implies reduced performance Amdahl’s law: this relation is not linear…

105. Art of Multiprocessor Programming105 Amdahl’s Law Speedup= …of computation given n CPUs instead of 1

106. Art of Multiprocessor Programming106 Amdahl’s Law Speedup=

107. Art of Multiprocessor Programming107 Amdahl’s Law Speedup= Parallel fraction

108. Art of Multiprocessor Programming108 Amdahl’s Law Speedup= Parallel fraction Sequential fraction

109. Art of Multiprocessor Programming109 Amdahl’s Law Speedup= Parallel fraction Number of processors Sequential fraction

110. Art of Multiprocessor Programming 110 Example Ten processors 60% concurrent, 40% sequential How close to 10-fold speedup?

111. Art of Multiprocessor Programming 111 Example Ten processors 60% concurrent, 40% sequential How close to 10-fold speedup? Speedup=2.17=

112. Art of Multiprocessor Programming 112 Example Ten processors 80% concurrent, 20% sequential How close to 10-fold speedup?

113. Art of Multiprocessor Programming 113 Example Ten processors 80% concurrent, 20% sequential How close to 10-fold speedup? Speedup=3.57=

114. Art of Multiprocessor Programming 114 Example Ten processors 90% concurrent, 10% sequential How close to 10-fold speedup?

115. Art of Multiprocessor Programming 115 Example Ten processors 90% concurrent, 10% sequential How close to 10-fold speedup? Speedup=5.26=

116. Art of Multiprocessor Programming 116 Example Ten processors 99% concurrent, 01% sequential How close to 10-fold speedup?

117. Art of Multiprocessor Programming 117 Example Ten processors 99% concurrent, 01% sequential How close to 10-fold speedup? Speedup=9.17=

118. Art of Multiprocessor Programming118 The Moral Making good use of our multiple processors (cores) means Finding ways to effectively parallelize our code –Minimize sequential parts –Reduce idle time in which threads wait without

119. Art of Multiprocessor Programming119 Multicore Programming This is what this course is about… –The % that is not easy to make concurrent yet may have a large impact on overall speedup Next step: –Lets take A more serious look at mutual exclusion

120. Art of Multiprocessor Programming 120 This work is licensed under a Creative Commons Attribution- ShareAlike 2.5 License.Creative Commons Attribution- ShareAlike 2.5 License You are free: –to Share — to copy, distribute and transmit the work –to Remix — to adapt the work Under the following conditions: –Attribution. You must attribute the work to “The Art of Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work). –Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to –http://creativecommons.org/licenses/by-sa/3.0/. Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights.