1. The Future of Distributed Computing Renaissance or Reformation? Maurice Herlihy Brown University

2. PODC 20082 Le Quatorze Juillet SAN FRANCISCO, May 7. 2004 - Intel said on Friday that it was scrapping its development of two microprocessors, a move that is a shift in the company's business strategy…. New York Times

3. PODC 20083 Moore’s Law (hat tip: Simon Peyton-Jones) Clock speed flattening sharply Transistor count still rising

4. PODC 20084 Art of Multiprocessor Programming4 Still on some of your desktops: The Uniprocesor memory cpu

5. PODC 20085 Art of Multiprocessor Programming5 In the Enterprise: The Shared Memory Multiprocessor (SMP) cache Bus shared memory cache

6. PODC 20086 Art of Multiprocessor Programming6 Your New Desktop: The Multicore Processor (CMP) cache Bus shared memory cache All on the same chip Sun T2000 Niagara

7. PODC 20087 Multicores are Here “Learn how the multi-core processor architecture plays a central role in Intel's platform approach. ….” “AMD is leading the industry to multi- core technology for the x86 based computing market …” “Sun's multicore strategy centers around multi-threaded software.... “

8. PODC 20088 Why should we care? First time ever, –PODC research relevant to Real World™ First time ever, –Real World™ relevant to PODC Plato vs Aristotle

9. PODC 20089 Renaissance? World (re)discovers PODC community achievements This has already happened (sort-of) World learns of PODC results

10. PODC 200810 Reformation? Can we respond to the Real World’s challenges? Are we working on problems that matter? Can we recognize what’s going to be important? Bonfire of the Vanities

11. PODC 200811 In Classic Antiquity Time cured software bloat Double your path length? –Wait 6 months, until –Processor speed catches up

12. PODC 200812 Multiprocessor companies failed in 80s Outstripped by sequential processors Field respected, but not taken seriously Parallelism Didn’t Matter

13. PODC 200813 The Old Order Lies in Ruins Six months means more cores, same clock speed Must exploit more paralellism No one really knows how to do this

14. PODC 200814 What Keeps Microsoft and Intel awake at Night? If more cores does not deliver more value … Then why upgrade? ?

15. PODC 200815 Washing Machine Science? Computers could become like washing machines You don’t trade it in every 2 years for a cooler model You keep it until it breaks.

16. PODC 200816 No Cores Please, we’re Theorists! Computer Science is driven by Moore’s law Each year we can do things we couldn’t do last year Means funding, students, excitement !

17. PODC 200817 With Sudden Relevance Comes Great Responsibility Many challenges involve –concurrent algorithms –Data structures –formal models – complexity & lower bounds, –…–… Stuff we’re good at.

18. PODC 200818 Disclaimer What follows are my Opinions (mine, mine, mine!) –And prejudices Targeted to people –New in the field No offence intended –In most cases.

19. PODC 200819 Concurrent Programming Today

20. PODC 200820 Coarse-Grained Locking Easily made correct … But not scalable.

21. PODC 200821 Fine-Grained Locking Here comes trouble …

22. PODC 200822 Locks are not Robust If a thread holding a lock is delayed … No one else can make progress

23. PODC 200823 Locking Relies on Conventions Relation between –Lock bit and object bits –Exists only in programmer’s mind /* * When a locked buffer is visible to the I/O layer * BH_Launder is set. This means before unlocking * we must clear BH_Launder,mb() on alpha and then * clear BH_Lock, so no reader can see BH_Launder set * on an unlocked buffer and then risk to deadlock. */ Actual comment from Linux Kernel (hat tip: Bradley Kuszmaul)

24. PODC 200824 Sadistic Homework enq(x) deq(y) FIFO queue No interference if ends “far enough” apart

25. PODC 200825 Sadistic Homework enq(x) deq(y) FIFO queue Interference OK if ends “close enough” together

26. PODC 200826 You Try It … One lock? –Too Conservative Locks at each end? –Deadlock, too complicated, etc Publishable result? –Once, maybe still?

27. PODC 200827 Locks do not compose add(T 1, item) delete(T 1, item) add(T 2, item) item Move from T 1 to T 2 Must lock T 2 before deleting from T 1 lock T2 lock T1 item Exposing lock internals breaks abstraction Hash Table Must lock T 1 before adding item

28. PODC 200828 The Transactional Manifesto What we do now is inadequate to meet the multicore challenge Research Agenda –Replace locking with a transactional API –Design languages to support this model –Implement the run-time to be fast enough

29. PODC 200829© 2006 Herlihy & Shavit29 Public void enq(item x) { Qnode q = new Qnode(x); q.next = this.tail; this.tail.next = q; } Sadistic Homework Revisited (1) Write sequential Code

30. PODC 200830© 2006 Herlihy & Shavit30 Public void LeftEnq(item x) { atomic { Qnode q = new Qnode(x); q.next = this.tail; this.tail.next = q; } Sadistic Homework Revisited (1)

31. PODC 200831© 2006 Herlihy & Shavit31 Public void LeftEnq(item x) { atomic { Qnode q = new Qnode(x); q.next = this.tail; this.tail.next = q; } Sadistic Homework Revisited (1) Enclose in atomic block

32. PODC 200832© 2006 Herlihy & Shavit32 Warning Not always this simple –Conditional waits –Enhanced concurrency –Complex patterns But often it is –Works for sadistic homework

33. PODC 200833© 2006 Herlihy & Shavit33 Public void Transfer(Queue q1, q2) { atomic { T x = q1.deq(); q2.enq(x); } Composition (1) Trivial or what?

34. PODC 200834 Not All Skittles and Beer Algorithmic choices –Lower bounds –Better algorithms Language design Semantic issues –Like memory models –Atomicity checking

35. PODC 200835 Contention Management & Scheduling How to resolve conflicts? Who moves forward and who rolls back? Lots of empirical work but formal work in infancy Judgment of Solomon

36. PODC 200836 I/O & System Calls? Some I/O revocable –Provide transaction- safe libraries –Undoable file system/DB calls Some not –Opening cash drawer –Firing missile

37. PODC 200837 Privatization Transaction makes object inaccessible Works on it without synchronization Works with locks … But not necessarily with transactions … Need algorithms and models!

38. PODC 200838 Strong vs Weak Isolation How do transactional & non-transactional threads synchronize? Similar to memory- model theory? Efficient algorithms?

39. PODC 200839 Single Global Lock Semantics? Transactions act as if it acquires SGL Good: –Intuitively appealing Bad: –What about aborted transactions? –Expensive? Need better models

40. PODC 200840 Progress, Performance Metrics and Lower Bounds Wait-free –Everyone makes progress Lock-free –Someone makes progress Obstruction-free –Solo threads make progress

41. PODC 200841 Obstruction-Free? Experience suggests simpler, more efficient and easier to reason about But no real formal justification Progress conditions imperfectly understood

42. PODC 200842 Formal Models of Performance Asynchrony

43. PODC 200843 Formal Models of Performance Asynchrony Multi-level Memory

44. PODC 200844 Formal Models of Performance Asynchrony Multi-level Memory Contention

45. PODC 200845 Formal Models of Performance Asynchrony Multi-level Memory Contention Memory Models

46. PODC 200846 Formal Models of Performance Asynchrony Multi-level Memory Contention Memory Models Reads, writes, CAS, TM and other stuff we may devise …

47. PODC 200847 Formal Verification Concurrent algorithms are hard Need routine verification of real algorithms Model checking? Theorem proving? Probably both

48. PODC 200848 PODC Victories Byzantine agreement

49. PODC 200849 PODC Victories Byzantine agreement Paxos, group communication

50. PODC 200850 PODC Victories Byzantine agreement Paxos, group communication Replication algorithms Photoshop™ replication algorithm

51. PODC 200851 PODC Victories Byzantine agreement Paxos, group communication Replication Lock-free & wait- free algorithms

52. PODC 200852 PODC Victories Byzantine agreement Paxos, group communication Replication Lock-free & wait-free algorithms Formalizing what needs to to be formalized!

53. PODC 200853 An Insurmountable Opportunity! (hat tip: Walt Kelley) Multicore forces us to rethink almost everything

54. PODC 200854 An Insurmountable Opportunity! (hat tip: Walt Kelley) Multicore forces us to rethink almost everything The fate of CS as a vibrant field depends on our success

55. PODC 200855 An Insurmountable Opportunity! (hat tip: Walt Kelley) Multicore forces us to rethink almost everything The fate of CS as a vibrant field depends on our success PODC community has unique insights & advantages

56. PODC 200856 An Insurmountable Opportunity! (hat tip: Walt Kelley) Multicore forces us to rethink almost everything The fate of CS as a vibrant field depends on our success PODC community has unique insights & advantages Are we equal to the task?

57. PODC 200857 This work is licensed under a Creative Commons Attribution- ShareAlike 2.5 License.Creative Commons Attribution- ShareAlike 2.5 License