1. Introduction Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Modified by Rajeev Alur for CIS 640 at Penn, Spring 2009

2. Art of Multiprocessor Programming2 Moore’s Law Clock speed flattening sharply Transistor count still rising

3. Art of Multiprocessor Programming3 Still on some of your desktops: The Uniprocesor memory cpu

4. Art of Multiprocessor Programming4 In the Enterprise: The Shared Memory Multiprocessor (SMP) cache Bus shared memory cache

5. Art of Multiprocessor Programming5 Your New Desktop: The Multicore Processor (CMP) cache Bus shared memory cache All on the same chip Sun T2000 Niagara

6. Art of Multiprocessor Programming6 Multicores Are Here “Intel's Intel ups ante with 4-core chip. New microprocessor, due this year, will be faster, use less electricity...” [San Fran Chronicle] “AMD will launch a dual-core version of its Opteron server processor at an event in New York on April 21.” [PC World] “Sun’s Niagara…will have eight cores, each core capable of running 4 threads in parallel, for 32 concurrently running threads. ….” [The Inquirer]

7. Art of Multiprocessor Programming7 Why do we care? Time no longer cures software bloat –The “free ride” is over When you double your program’s path length –You can’t just wait 6 months –Your software must somehow exploit twice as much concurrency

8. Art of Multiprocessor Programming8 Traditional Scaling Process User code Traditional Uniprocessor Speedup 1.8x 7x 3.6x Time: Moore’s law

9. Art of Multiprocessor Programming9 Multicore Scaling Process User code Multicore Speedup 1.8x7x3.6x Unfortunately, not so simple…

10. Art of Multiprocessor Programming10 Real-World Scaling Process 1.8x 2x 2.9x User code Multicore Speedup Parallelization and Synchronization require great care…

11. Art of Multiprocessor Programming11 Multicore Programming: Course Overview Fundamentals –Models, algorithms, impossibility Real-World programming –Architectures –Techniques Topics not in Textbook –Memory models and system-level concurrency libraries –High-level programming abstractions

12. A Zoo of Terms Concurrent Parallel Distributed Multicore What do they all mean? How do they differ?

13. Concurrent Computing Programs designed as a collection of interacting threads/processes –Logical/programming abstraction –May be implemented on single processor by interleaving or on multiple processors or on distributed computers –Coordination/synchronization mechanism in a model of concurrency may be realized in many ways in an implementation

14. Parallel Computing Computations that execute simultaneously to solve a common problem (more efficiently) –Parallel algorithms: Which problems can have speed-up given multiple execution units? –Parallelism can be at many levels (e.g. bit-level, instruction-level, data path) –Grid computing: Branch of parallel computing where problems are solved on clusters of computers (interacting by message passing) –Multicore computing: Branch of parallel computing focusing on multiple execution units on same chip (interacting by shared memory)

15. Distributed Computing Involves multiple agents/programs (possibly with different computational tasks) with multiple computational resources (computers, multiprocessors, network) –Many examples of contemporary software (e.g. web services) are distributed systems –Heterogeneous nature, and range of time scales (web access vs local access), make design/programming more challenging

16. Art of Multiprocessor Programming 16 Sequential Computation memory object thread

17. Art of Multiprocessor Programming 17 Concurrent Computation memory object threads

18. Art of Multiprocessor Programming18 Asynchrony Sudden unpredictable delays –Cache misses (short) –Page faults (long) –Scheduling quantum used up (really long)

19. Art of Multiprocessor Programming19 Model Summary Multiple threads –Sometimes called processes Single shared memory Objects live in memory Unpredictable asynchronous delays

20. Art of Multiprocessor Programming20 Road Map Textbook focuses on principles first, then practice –Start with idealized models –Look at simplistic problems –Emphasize correctness over pragmatism –“Correctness may be theoretical, but incorrectness has practical impact” In course, interleaving of chapters from the two parts

21. Art of Multiprocessor Programming21 Concurrency Jargon Hardware –Processors Software –Threads, processes Sometimes OK to confuse them, sometimes not.

22. Art of Multiprocessor Programming22 Parallel Primality Testing Challenge –Print primes from 1 to 10 10 Given –Ten-processor multiprocessor –One thread per processor Goal –Get ten-fold speedup (or close)

23. Art of Multiprocessor Programming23 Load Balancing Split the work evenly Each thread tests range of 10 9 … … 10 910 2·10 9 1 P0P0 P1P1 P9P9

24. Art of Multiprocessor Programming 24 Procedure for Thread i void primePrint { int i = ThreadID.get(); // IDs in {0..9} for (j = i*10 9 +1, j<(i+1)*10 9 ; j++) { if (isPrime(j)) print(j); }

25. Art of Multiprocessor Programming25 Issues Higher ranges have fewer primes Yet larger numbers harder to test Thread workloads –Uneven –Hard to predict

26. Art of Multiprocessor Programming26 Issues Higher ranges have fewer primes Yet larger numbers harder to test Thread workloads –Uneven –Hard to predict Need dynamic load balancing rejected

27. Art of Multiprocessor Programming 27 17 18 19 Shared Counter each thread takes a number

28. Art of Multiprocessor Programming 28 Procedure for Thread i int counter = new Counter(1); void primePrint { long j = 0; while (j < 10 10 ) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); }

29. Art of Multiprocessor Programming 29 Counter counter = new Counter(1); void primePrint { long j = 0; while (j < 10 10 ) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); } Procedure for Thread i Shared counter object

30. Art of Multiprocessor Programming30 Where Things Reside cache Bus cache 1 shared counter shared memory void primePrint { int i = ThreadID.get(); // IDs in {0..9} for (j = i*10 9 +1, j<(i+1)*10 9 ; j++) { if (isPrime(j)) print(j); } code Local variables

31. Art of Multiprocessor Programming 31 Procedure for Thread i Counter counter = new Counter(1); void primePrint { long j = 0; while (j < 10 10 ) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); } Stop when every value taken

32. Art of Multiprocessor Programming 32 Counter counter = new Counter(1); void primePrint { long j = 0; while (j < 10 10 ) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); } Procedure for Thread i Increment & return each new value

33. Art of Multiprocessor Programming 33 Counter Implementation public class Counter { private long value; public long getAndIncrement() { return value++; }

34. Art of Multiprocessor Programming 34 Counter Implementation public class Counter { private long value; public long getAndIncrement() { return value++; } OK for single thread, not for concurrent threads

35. Art of Multiprocessor Programming 35 What It Means public class Counter { private long value; public long getAndIncrement() { return value++; }

36. Art of Multiprocessor Programming 36 What It Means public class Counter { private long value; public long getAndIncrement() { return value++; } temp = value; value = temp + 1; return temp;

37. Art of Multiprocessor Programming 37 time Not so good… Value… 1 read 1 read 1 write 2 read 2 write 3 write 2 232

38. Art of Multiprocessor Programming 38 Is this problem inherent? If we could only glue reads and writes… read write read write

39. Art of Multiprocessor Programming 39 Challenge public class Counter { private long value; public long getAndIncrement() { temp = value; value = temp + 1; return temp; }

40. Art of Multiprocessor Programming 40 Challenge public class Counter { private long value; public long getAndIncrement() { temp = value; value = temp + 1; return temp; } Make these steps atomic (indivisible)

41. Art of Multiprocessor Programming 41 Hardware Solution public class Counter { private long value; public long getAndIncrement() { temp = value; value = temp + 1; return temp; } ReadModifyWrite() instruction

42. Art of Multiprocessor Programming 42 An Aside: Java™ public class Counter { private long value; public long getAndIncrement() { synchronized { temp = value; value = temp + 1; } return temp; }

43. Art of Multiprocessor Programming 43 An Aside: Java™ public class Counter { private long value; public long getAndIncrement() { synchronized { temp = value; value = temp + 1; } return temp; } Synchronized block

44. Art of Multiprocessor Programming 44 An Aside: Java™ public class Counter { private long value; public long getAndIncrement() { synchronized { temp = value; value = temp + 1; } return temp; } Mutual Exclusion

45. Art of Multiprocessor Programming45 Why do we care? We want as much of the code as possible to execute concurrently (in parallel) A larger sequential part implies reduced performance Amdahl’s law: this relation is not linear…

46. Art of Multiprocessor Programming46 Amdahl’s Law Speedup= …of computation given n CPUs instead of 1

47. Art of Multiprocessor Programming47 Amdahl’s Law Speedup=

48. Art of Multiprocessor Programming48 Amdahl’s Law Speedup= Parallel fraction

49. Art of Multiprocessor Programming49 Amdahl’s Law Speedup= Parallel fraction Sequential fraction

50. Art of Multiprocessor Programming50 Amdahl’s Law Speedup= Parallel fraction Number of processors Sequential fraction

51. Art of Multiprocessor Programming 51 Example Ten processors 60% concurrent, 40% sequential How close to 10-fold speedup?

52. Art of Multiprocessor Programming 52 Example Ten processors 60% concurrent, 40% sequential How close to 10-fold speedup? Speedup=2.17=

53. Art of Multiprocessor Programming 53 Example Ten processors 80% concurrent, 20% sequential How close to 10-fold speedup?

54. Art of Multiprocessor Programming 54 Example Ten processors 80% concurrent, 20% sequential How close to 10-fold speedup? Speedup=3.57=

55. Art of Multiprocessor Programming 55 Example Ten processors 90% concurrent, 10% sequential How close to 10-fold speedup?

56. Art of Multiprocessor Programming 56 Example Ten processors 90% concurrent, 10% sequential How close to 10-fold speedup? Speedup=5.26=

57. Art of Multiprocessor Programming 57 Example Ten processors 99% concurrent, 01% sequential How close to 10-fold speedup?

58. Art of Multiprocessor Programming 58 Example Ten processors 99% concurrent, 01% sequential How close to 10-fold speedup? Speedup=9.17=

59. Art of Multiprocessor Programming59 The Moral Making good use of our multiple processors (cores) means Finding ways to effectively parallelize our code –Minimize sequential parts –Reduce idle time in which threads wait

60. Art of Multiprocessor Programming60 Multicore Programming This is what this course is about… –The % that is not easy to make concurrent yet may have a large impact on overall speedup