1. Art of Multiprocessor Programming 1 Transactional Memory Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

2. Shared Data Structures 75% Unshared 25% Shared cc cc cc cc Coarse Grained c c c c c c c c cc cc cc cc Fine Grained c c c c c c c c The reason we get only 2.9 speedup 75% Unshared 25% Shared Fine grained parallelism has huge performance benefit

3. A FIFO Queue bcd TailHead a Enqueue(d)Dequeue() => a

4. A Concurrent FIFO Queue Object lock bcd TailHead a P: Dequeue() => a Q: Enqueue(d) Simple Code, easy to prove correct Contention and sequential bottleneck

5. Fine Grain Locks bcd TailHead a P: Dequeue() => a Q: Enqueue(d) Finer Granularity, More Complex Code Verification nightmare: worry about deadlock, livelock…

6. Fine Grain Locks bcd TailHead a P: Dequeue() => a Q: Enqueue(b) b TailHead a Worry how to acquire multiple locks Complex boundary cases: empty queue, last item

7. Art of Multiprocessor Programming 77 Moreover: Locking Relies on Conventions Relation between –Lock bit and object bits –Exists only in programmer’s mind /* * When a locked buffer is visible to the I/O layer * BH_Launder is set. This means before unlocking * we must clear BH_Launder,mb() on alpha and then * clear BH_Lock, so no reader can see BH_Launder set * on an unlocked buffer and then risk to deadlock. */ Actual comment from Linux Kernel (hat tip: Bradley Kuszmaul)

8. Lock-Free (JDK 1.5+) bcd TailHead a P: Dequeue() => a Q: Enqueue(d) Even Finer Granularity, Even More Complex Code Worry about starvation, subtle bugs, hardness to modify…

9. Composing Objects Complex: Move data atomically between structures More than twice the worry… bcd TailHead a P: Dequeue(Q1,a) cda TailHead b Enqueue(Q2,a)

10. Transactional Memory bcd TailHead a P: Dequeue() => a Q: Enqueue(d) Don’t worry about deadlock, livelock, subtle bugs, etc… Great Performance, Simple Code

11. Promise of Transactional Memory bcd TailHead a P: Dequeue() => a Q: Enqueue(d) b TailHead a TM deals with boundary cases under the hood Don’t worry which locks need to cover which variables when…

12. Composing Objects Will be easy to modify multiple structures atomically Provide Composability… bcd TailHead a P: Dequeue(Q1,a) cda TailHead b Enqueue(Q2,a)

13. Art of Multiprocessor Programming 13 The Transactional Manifesto Current practice inadequate –to meet the multicore challenge Research Agenda –Replace locking with a transactional API –Design languages to support this model –Implement the run-time to be fast enough

14. Art of Multiprocessor Programming 14 Transactions Atomic –Commit: takes effect –Abort: effects rolled back Usually retried Serizalizable –Appear to happen in one-at-a-time order

15. Art of Multiprocessor Programming 15 atomic { x.remove(3); y.add(3); } atomic { y = null; } Atomic Blocks

16. Art of Multiprocessor Programming 16 atomic { x.remove(3); y.add(3); } atomic { y = null; } Atomic Blocks No data race

17. Art of Multiprocessor Programming 17 Public void LeftEnq(item x) { Qnode q = new Qnode(x); q.left = this.left; this.left.right = q; this.left = q; } Designing a FIFO Queue Write sequential Code

18. Art of Multiprocessor Programming 18 Public void LeftEnq(item x) { atomic { Qnode q = new Qnode(x); q.left = this.left; this.left.right = q; this.left = q; } Designing a FIFO Queue

19. Art of Multiprocessor Programming 19 Public void LeftEnq(item x) { atomic { Qnode q = new Qnode(x); q.left = this.left; this.left.right = q; this.left = q; } Designing a FIFO Queue Enclose in atomic block

20. Art of Multiprocessor Programming 20 Warning Not always this simple –Conditional waits –Enhanced concurrency –Complex patterns But often it is

21. Art of Multiprocessor Programming 21 Public void Transfer(Queue q1, q2) { atomic { T x = q1.deq(); q2.enq(x); } Composition Trivial or what?

22. Art of Multiprocessor Programming 22 Public T LeftDeq() { atomic { if (this.left == null) retry; … } Roll Back Roll back transaction and restart when something changes

23. Art of Multiprocessor Programming 23 OrElse Composition atomic { x = q1.deq(); } orElse { x = q2.deq(); } Run 1 st method. If it retries … Run 2 nd method. If it retries … Entire statement retries

24. Art of Multiprocessor Programming 24 Transactional Memory Software transactional memory (STM) Hardware transactional memory (HTM) Hybrid transactional memory (HyTM, try in hardware and default to software if unsuccessful) 24

25. Art of Multiprocessor Programming 25 Hardware versus Software Do we need hardware at all? –Analogies: Virtual memory: yes! Garbage collection: no! –Probably do need HW for performance Do we need software? –Policy issues don’t make sense for hardware

26. Transactional Consistency Memory Transactions are collections of reads and writes executed atomically Tranactions should maintain internal and external consistency –External: with respect to the interleavings of other transactions. –Internal: the transaction itself should operate on a consistent state.

27. Art of Multiprocessor Programming 27 External Consistency Application Memory x y 4 2 8 4 Invariant x = 2y Transaction A: Write x Write y Transaction B: Read x Read y Compute z = 1/(x-y)

28. Art of Multiprocessor Programming 28 Simple Lock-Based STM STMs come in different forms –Lock-based –Lock-free Here we will describe a simple lock- based STM

29. Art of Multiprocessor Programming 29 Synchronization Transaction keeps –Read set: locations & values read –Write set: locations & values to be written Deferred update –Changes installed at commit Lazy conflict detection –Conflicts detected at commit

30. Art of Multiprocessor Programming 30 STM: Transactional Locking Map Array of Versioned- Write-Locks Application Memory V#

31. Art of Multiprocessor Programming 31 Reading an Object Check not locked Put V#s & value in RS Mem Locks V#

32. Art of Multiprocessor Programming 32 To Write an Object Add V# and new value to WS Mem Locks V#

33. Art of Multiprocessor Programming 33 To Commit Acquire W locks Check V#s unchanged In RS only Install new values Increment V#s Release … Mem Locks V# X Y V#+1

34. Problem: Internal Inconsistency A Zombie is a currently active transaction that is destined to abort because it saw an inconsistent state If Zombies that see inconsistent states are allowed to have irreversible impact on execution state then errors can occur Eventual abort does not save us

35. Art of Multiprocessor Programming 35 Internal Consistency Application Memory x y 4 2 8 4 Invariant x \neq 0 && x = 2y Transaction A: Write x (kills B) Write y Transaction B: Read x = 4 Transaction B: (zombie) Read y = 4 Compute z = 1/(x-y) DIV by 0 ERROR

36. Art of Multiprocessor Programming 36 Solution: The “Global Clock” Have one shared global clock Incremented by (small subset of) writing transactions Read by all transactions Used to validate that state worked on is always consistent

37. Art of Multiprocessor Programming 37 Read-Only Transactions Copy V Clock to RV Read lock,V# Read mem Check unlocked (1) Recheck V# unchanged (2) (1)+(2)  v# and mem content consistent Check V# < RV Mem Locks 12 32 56 19 17 100 Shared Version Clock Private Read Version (RV) 100 Reads from a snapshot of memory. No read set!

38. Art of Multiprocessor Programming 38 Regular Transactions – during trans. prev. to commit Copy V Clock to RV On read/write, check: Unlocked (acquire) V# < RV Add to R/W set Mem Locks 69 Shared Version Clock 69 12 32 56 19 17 Private Read Version (RV)

39. Art of Multiprocessor Programming 39 Regular Transactions- Commit Acquire locks (write set only) WV = Fetch&Inc(V Clock) For all read set check unlock and revalidate V# < RV Update write set Set write write set V#s to WV Release locks Mem Locks 100 Shared Version Clock 69 101 x y 12 32 56 19 17 100 Private Read Version (RV)

40. Art of Multiprocessor Programming 40 When two transactions have their read and write sets intersected, but both succeed to read before the write of the other transaction occurs, then there is no way to serialize them (see example below by Nir Shavit). Hence the need to revalidate that the read set *after* locking the write set. Also, upon commit of transaction A, *after* A already took the locks on the write set, and after or while the read set revalidated, another transaction cannot succeed to read from A's write set before A writes it (because it is locked). Detailed example: Take two transactions T1 and T2. Lets say that there are 2 memory locations initialized to 0. Lets say that both transactions read both locations, and T1 writes 1 to location 1 if it saw all 0's and T2 writes 1 to location 2 if it saw all 0's. Now if they both do not revalidate the read locations this means that T1 does not revalidate location 2 after acquiring the lock and T2 does not revalidate location 1 after grabbing the lock. So if they both run, both read both locations, both see all 0's in a snapshot, then both grab locks on their respective write locations, revalidate their own write locations, and write the 1 value with a timestamp greater by 1. Since they only revalidated their write locations after locking, neither saw that the other thread changed the location they only read to a 1 with a larger timestamp. Now we have a memory with two 1's in it even though there is no such serializable execution. Seeing a snapshot before grabbing the locks in the commit is thus not sufficient and the algorithm must have the transactions each revalidate the read set locations after acquiring the locks. Some explanations to lock-based implementation of regular transactions: why do we need to revalidate reads?

41. Art of Multiprocessor Programming 41 Hardware Transactional Memory Exploit Cache coherence Already almost does it –Invalidation –Consistency checking Speculative execution –Branch prediction = optimistic synch!

42. Art of Multiprocessor Programming 42 HW Transactional Memory Interconnect caches memory read active T

43. Art of Multiprocessor Programming 43 Transactional Memory caches memory read active TT

44. Art of Multiprocessor Programming 44 Transactional Memory caches memory active TT committed

45. Art of Multiprocessor Programming 45 Transactional Memory caches memory write active committed TD

46. Art of Multiprocessor Programming 46 Rewind caches memory active TT write aborted D

47. Art of Multiprocessor Programming 47 Transaction Commit At commit point –If no cache conflicts, we win. Mark transactional entries –Read-only: valid –Modified: dirty (eventually written back) That’s all, folks! –Except for a few details …

48. Art of Multiprocessor Programming 48 Not all Skittles and Beer Limits to –Transactional cache size –Scheduling quantum Transaction cannot commit if it is –Too big –Too slow –Actual limits platform-dependent

49. Art of Multiprocessor Programming 49 TM Design Issues Implementation choices Language design issues Semantic issues

50. Art of Multiprocessor Programming 50 Granularity Object –managed languages, Java, C#, … –Easy to control interactions between transactional & non-trans threads Word –C, C++, … –Hard to control interactions between transactional & non-trans threads

51. Art of Multiprocessor Programming 51 Direct/Deferred Update Deferred –modify private copies & install on commit –Commit requires work –Consistency easier Direct –Modify in place, roll back on abort –Makes commit efficient –Consistency harder

52. Art of Multiprocessor Programming 52 Conflict Detection Eager –Detect before conflict arises –“Contention manager” module resolves Lazy –Detect on commit/abort Mixed –Eager write/write, lazy read/write …

53. Art of Multiprocessor Programming 53 Conflict Detection Eager detection may abort transaction that could have committed. Lazy detection discards more computation.

54. Art of Multiprocessor Programming 54 Contention Management & Scheduling How to resolve conflicts? Who moves forward and who rolls back?

55. Art of Multiprocessor Programming 55 Contention Manager Strategies Exponential backoff Priority to –Oldest? –Most work? –Non-waiting? None Dominates Judgment of Solomon

56. Art of Multiprocessor Programming 56 I/O & System Calls? Some I/O revocable –Provide transaction- safe libraries –Undoable file system/DB calls Some not –Opening cash drawer –Firing missile

57. Art of Multiprocessor Programming 57 I/O & System Calls One solution: make transaction irrevocable –If transaction tries I/O, switch to irrevocable mode. There can be only one … –Requires serial execution No explicit aborts –In irrevocable transactions

58. Art of Multiprocessor Programming 58 Exceptions int i = 0; try { atomic { i++; node = new Node(); } } catch (Exception e) { print(i); } int i = 0; try { atomic { i++; node = new Node(); } } catch (Exception e) { print(i); }

59. Art of Multiprocessor Programming 59 Exceptions int i = 0; try { atomic { i++; node = new Node(); } } catch (Exception e) { print(i); } int i = 0; try { atomic { i++; node = new Node(); } } catch (Exception e) { print(i); } Throws OutOfMemoryException!

60. Art of Multiprocessor Programming 60 Exceptions int i = 0; try { atomic { i++; node = new Node(); } } catch (Exception e) { print(i); } int i = 0; try { atomic { i++; node = new Node(); } } catch (Exception e) { print(i); } Throws OutOfMemoryException! What is printed?

61. Art of Multiprocessor Programming 61 Unhandled Exceptions Aborts transaction –Preserves invariants –Safer Commits transaction –Like locking semantics –What if exception object refers to values modified in transaction?

62. Art of Multiprocessor Programming 62 Nested Transactions atomic void foo() { bar(); } atomic void bar() { … } atomic void foo() { bar(); } atomic void bar() { … }

63. Art of Multiprocessor Programming 63 Nested Transactions Needed for modularity –Who knew that cosine() contained a transaction? Flat nesting –If child aborts, so does parent First-class nesting –If child aborts, partial rollback of child only

64. Art of Multiprocessor Programming 64 Open Nested Transactions Normally, child commit –Visible only to parent In open nested transactions –Commit visible to all –Escape mechanism –Dangerous, but useful What escape mechanisms are needed?

65. Art of Multiprocessor Programming 65 Strong vs Weak Isolation How do transactional & non-transactional threads synchronize? Interaction with memory-model? Efficient algorithms?