Linked Lists: Locking, Lock- Free, and Beyond … Based on the companion slides for The Art of Multiprocessor Programming (Maurice Herlihy & Nir Shavit)

Art of Multiprocessor Programming© Herlihy-Shavit 20072 Coarse-Grained Synchronization Each method locks the object –Easy to reason about In simple cases –Standard Java model Synchronized blocks and methods So, are we done?

Art of Multiprocessor Programming© Herlihy-Shavit 20073 Coarse-Grained Synchronization Sequential bottleneck –Threads stand in line Adding more threads –Does not improve throughput –Struggle to keep it from getting worse

Art of Multiprocessor Programming© Herlihy-Shavit 20074 This Lecture Introduce four patterns –Bag of tricks … –Methods that work more than once … For highly-concurrent objects Goal: –Concurrent access –More threads, more throughput

Art of Multiprocessor Programming© Herlihy-Shavit 20075 First: Fine-Grained Synchronization Instead of using a single lock.. Split object into –Independently-synchronized components Methods conflict when they access –The same component … –At the same time

Art of Multiprocessor Programming© Herlihy-Shavit 20076 Second: Optimistic Synchronization Search without locking … If you find it, lock and check … –OK: we are done –Oops: start over Evaluation –Usually cheaper than locking –Mistakes are expensive

Art of Multiprocessor Programming© Herlihy-Shavit 20077 Third: Lazy Synchronization Postpone hard work Removing components is tricky –Logical removal Mark component to be deleted –Physical removal Do what needs to be done

Art of Multiprocessor Programming© Herlihy-Shavit 20078 Fourth: Lock-Free Synchronization Dont use locks at all –Use compareAndSet() & relatives … Advantages –Robust against asynchrony Disadvantages –Complex –Sometimes high overhead

Art of Multiprocessor Programming© Herlihy-Shavit 20079 Linked List Illustrate these patterns … Using a list-based Set –Common application –Building block for other apps

Art of Multiprocessor Programming© Herlihy-Shavit 200710 Set Interface Unordered collection of items No duplicates Methods –add(x) put x in set –remove(x) take x out of set –contains(x) tests if x in set

Art of Multiprocessor Programming© Herlihy-Shavit 200711 Set Interface public interface Set { public boolean add(T x); public boolean remove(T x); public boolean contains(T x); }

Art of Multiprocessor Programming© Herlihy-Shavit 200712 Set Interface public interface Set { public boolean add(T x); public boolean remove(T x); public boolean contains(T x); } Add item to set

Art of Multiprocessor Programming© Herlihy-Shavit 200713 Set Interface public interface Set { public boolean add(T x); public boolean remove(T x); public boolean contains(Tt x); } Remove item from set

Art of Multiprocessor Programming© Herlihy-Shavit 200714 Set Interface public interface Set { public boolean add(T x); public boolean remove(T x); public boolean contains(T x); } Is item in set?

Art of Multiprocessor Programming© Herlihy-Shavit 200715 List Node public class Node { public T item; public int key; public Node next; }

Art of Multiprocessor Programming© Herlihy-Shavit 200716 List Node public class Node { public T item; public int key; public Node next; } item of interest

Art of Multiprocessor Programming© Herlihy-Shavit 200717 List Node public class Node { public T item; public int key; public Node next; } Usually hash code

Art of Multiprocessor Programming© Herlihy-Shavit 200718 List Node public class Node { public T item; public int key; public Node next; } Reference to next node

Art of Multiprocessor Programming© Herlihy-Shavit 200719 The List-Based Set abc Sorted with Sentinel nodes (min & max possible keys) - +

Art of Multiprocessor Programming© Herlihy-Shavit 200720 Reasoning about Concurrent Objects Invariant –Property that always holds Established by –True when object is created –Truth preserved by each method Each step of each method

Art of Multiprocessor Programming© Herlihy-Shavit 200721 Specifically … Invariants preserved by –add() –remove() –contains() Most steps are trivial –Usually one step tricky –Often linearization point

Art of Multiprocessor Programming© Herlihy-Shavit 200722 Representation Invariants Sentinel nodes never added/removed –tail reachable from head Elements sorted by key (hash value) No duplicates

Art of Multiprocessor Programming© Herlihy-Shavit 200723 Interference Invariants make sense only if methods considered are the only modifiers Language encapsulation helps –List nodes not visible outside class –This guarantees freedom from interference

Art of Multiprocessor Programming© Herlihy-Shavit 200724 Abstract Data Types Concrete representation Abstract Type –{a, b} ab

Art of Multiprocessor Programming© Herlihy-Shavit 200725 Abstract Data Types Meaning of representation given by abstraction map –S( ) = {a,b} a b

Art of Multiprocessor Programming© Herlihy-Shavit 200726 Abstraction Map S(head) = { x | there exists a such that a reachable from head and a.item = x }

Art of Multiprocessor Programming© Herlihy-Shavit 200727 Sequential List Based Set a c d a b c Add() Remove()

Art of Multiprocessor Programming© Herlihy-Shavit 200728 Sequential List Based Set a c d b a b c Add() Remove()

Art of Multiprocessor Programming© Herlihy-Shavit 200729 Sequential List Based Set a c d b a b c Add() Remove()

Art of Multiprocessor Programming© Herlihy-Shavit 200730 Course Grained Locking a b d

Art of Multiprocessor Programming© Herlihy-Shavit 200731 Course Grained Locking a b d c

Art of Multiprocessor Programming© Herlihy-Shavit 200732 honk! Course Grained Locking a b d c Simple but hotspot + bottleneck honk!

Art of Multiprocessor Programming© Herlihy-Shavit 200733 Coarse-Grained Locking Easy, same as synchronized methods –One lock to rule them all … Simple, clearly correct –Deserves respect! Works poorly with contention –Queue locks help –But bottleneck still an issue

Art of Multiprocessor Programming© Herlihy-Shavit 200734 Fine-grained Locking Requires careful thought Split object into pieces –Each piece has own lock –Methods that work on disjoint pieces need not exclude each other Allows pipelined list traversal

Art of Multiprocessor Programming© Herlihy-Shavit 200735 Hand-over-Hand locking abc

Art of Multiprocessor Programming© Herlihy-Shavit 200740 Removing a Node abcd remove(b)

Art of Multiprocessor Programming© Herlihy-Shavit 200745 Removing a Node acd remove(b)

Art of Multiprocessor Programming© Herlihy-Shavit 200746 Removing a Node abcd remove(c) remove(b)

Art of Multiprocessor Programming© Herlihy-Shavit 200747 Removing a Node abcd remove(b) remove(c)

Art of Multiprocessor Programming© Herlihy-Shavit 200754 Uh, Oh acd remove(b) remove(c)

Art of Multiprocessor Programming© Herlihy-Shavit 200755 Uh, Oh acd Bad news remove(b) remove(c)

Art of Multiprocessor Programming© Herlihy-Shavit 200756 Problem To delete node b –Swing node as next field to c Problem is, –Someone could delete c concurrently ba cbac

Art of Multiprocessor Programming© Herlihy-Shavit 200757 Insight If a node is locked –No one can delete nodes successor If a thread locks node to be deleted and its predecessor, then it works

Art of Multiprocessor Programming© Herlihy-Shavit 200758 Hand-Over-Hand Again abcd remove(b)

Art of Multiprocessor Programming© Herlihy-Shavit 200761 Hand-Over-Hand Again abcd remove(b) Found it!

Art of Multiprocessor Programming© Herlihy-Shavit 200762 Hand-Over-Hand Again abcd remove(b) Found it!

Art of Multiprocessor Programming© Herlihy-Shavit 200763 Hand-Over-Hand Again acd remove(b)

Art of Multiprocessor Programming© Herlihy-Shavit 200774 Removing a Node abd remove(b)

Art of Multiprocessor Programming© Herlihy-Shavit 200777 Removing a Node ad remove(b) remove(c)

Art of Multiprocessor Programming© Herlihy-Shavit 200778 Removing a Node ad

Art of Multiprocessor Programming© Herlihy-Shavit 200779 Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }}

Art of Multiprocessor Programming© Herlihy-Shavit 200780 Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }} Key used to order node

Art of Multiprocessor Programming© Herlihy-Shavit 200781 Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { currNode.unlock(); predNode.unlock(); }} Predecessor and current nodes

Art of Multiprocessor Programming© Herlihy-Shavit 200782 Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }} Make sure locks released

Art of Multiprocessor Programming© Herlihy-Shavit 200783 Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }} Everything else

Art of Multiprocessor Programming© Herlihy-Shavit 200784 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … }

Art of Multiprocessor Programming© Herlihy-Shavit 200785 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } Lock pred == head

Art of Multiprocessor Programming© Herlihy-Shavit 200786 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } Lock current

Art of Multiprocessor Programming© Herlihy-Shavit 200787 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } Traversing list

Art of Multiprocessor Programming© Herlihy-Shavit 200788 Remove: traversing list while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;

Art of Multiprocessor Programming© Herlihy-Shavit 200789 Remove: traversing list while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Search key range

Art of Multiprocessor Programming© Herlihy-Shavit 200790 Remove: traversing list while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; At start of each loop: curr and pred locked

Art of Multiprocessor Programming© Herlihy-Shavit 200791 Remove: traversing list while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; If item found, remove node

Art of Multiprocessor Programming© Herlihy-Shavit 200792 Remove: traversing list while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Unlock predecessor

Art of Multiprocessor Programming© Herlihy-Shavit 200793 Remove: traversing list while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Only one node locked!

Art of Multiprocessor Programming© Herlihy-Shavit 200794 Remove: traversing list while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; demote current

Art of Multiprocessor Programming© Herlihy-Shavit 200795 Remove: traversing list while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = currNode; curr = curr.next; curr.lock(); } return false; Find and lock new current

Art of Multiprocessor Programming© Herlihy-Shavit 200796 Remove: traversing list while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = currNode; curr = curr.next; curr.lock(); } return false; Lock invariant restored

Art of Multiprocessor Programming© Herlihy-Shavit 200797 Remove: traversing list while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Otherwise, not present

Art of Multiprocessor Programming© Herlihy-Shavit 200798 while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Why remove() is linearizable pred reachable from head curr is pred.next So curr.item is in the set

Art of Multiprocessor Programming© Herlihy-Shavit 200799 while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Why remove() is linearizable pred reachable from head curr is pred.next pred.key < key key < curr.key So item is not in the set

Art of Multiprocessor Programming© Herlihy-Shavit 2007100 while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Why remove() is linearizable Linearization point

Art of Multiprocessor Programming© Herlihy-Shavit 2007101 Progress To avoid deadlock, locks must be taken by all methods in the same order Starvation freedom (if locks are starvation free)

Art of Multiprocessor Programming© Herlihy-Shavit 2007102 Adding Nodes To add a node: –Traverse list using hand-by-hand locking –Lock predecessor –Lock successor Neither can be deleted –… and thus item can be inserted inbetween –(Is successor lock actually required?)

Art of Multiprocessor Programming© Herlihy-Shavit 2007103 Same Abstraction Map S(head) = –{ x | there exists a such that a reachable from head and a.item = x –}

Art of Multiprocessor Programming© Herlihy-Shavit 2007104 Representation Invariant Easy to check that –tail always reachable from head –Nodes sorted –No duplicates

Art of Multiprocessor Programming© Herlihy-Shavit 2007105 Drawbacks Better than coarse-grained lock –Threads can traverse in parallel Still not ideal –Long chain of acquire/release locks –Inefficient –Threads working at the beginning of the list may block other threads

Art of Multiprocessor Programming© Herlihy-Shavit 2007106 Optimistic Synchronization Find nodes without locking Lock nodes Check that everything is OK

Art of Multiprocessor Programming© Herlihy-Shavit 2007107 Optimistic: Traverse without Locking b d e a add(c) Aha!

Art of Multiprocessor Programming© Herlihy-Shavit 2007108 Optimistic: Lock and Load b d e a add(c)

Art of Multiprocessor Programming© Herlihy-Shavit 2007109 What Can Possibly Go Wrong? b d e a add(c) Aha!

Art of Multiprocessor Programming© Herlihy-Shavit 2007110 What Can Possibly Go Wrong? b d e a add(c)

Art of Multiprocessor Programming© Herlihy-Shavit 2007111 What Can Possibly Go Wrong? b d e a add(c) remove(b )

Art of Multiprocessor Programming© Herlihy-Shavit 2007112 What Can Possibly Go Wrong? b d e a add(c)

Art of Multiprocessor Programming© Herlihy-Shavit 2007113 What Can Possibly Go Wrong? b d e a add(c) c would be added between b and d, but b is no more reachable (deleted)!

Art of Multiprocessor Programming© Herlihy-Shavit 2007114 Validate (1) b d e a add(c) Yes, b still reachable from head

Art of Multiprocessor Programming© Herlihy-Shavit 2007115 What Else Can Go Wrong? b d e a add(c) Aha!

Art of Multiprocessor Programming© Herlihy-Shavit 2007118 What Else Can Go Wrong? b d e a add(c) b add(b) invalidated by add( c )!

Art of Multiprocessor Programming© Herlihy-Shavit 2007119 Optimistic: Validate(2) b d e a add(c) Yes, b still points to d

Art of Multiprocessor Programming© Herlihy-Shavit 2007121 Same Abstraction Map S(head) = –{ x | there exists a such that a reachable from head and a.item = x –}

Art of Multiprocessor Programming© Herlihy-Shavit 2007122 Invariants Careful: we may traverse deleted nodes But we establish properties by –Validation –After we lock target nodes

Art of Multiprocessor Programming© Herlihy-Shavit 2007123 Correctness If –Nodes b and c both locked –Node b still accessible –Node c still successor to b Then –Neither will be deleted –OK to add and return true

Art of Multiprocessor Programming© Herlihy-Shavit 2007128 Correctness If –Nodes b and d both locked –Node b still accessible –Node d still successor to b Then –No thread can remove b, d, e –No thread can add a node between b and d or between d and e –OK to remove node or return false

Art of Multiprocessor Programming© Herlihy-Shavit 2007129 Validation private boolean validate(Node pred, Node curry) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; }

Art of Multiprocessor Programming© Herlihy-Shavit 2007130 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Predecessor & current nodes

Art of Multiprocessor Programming© Herlihy-Shavit 2007131 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Begin at the beginning

Art of Multiprocessor Programming© Herlihy-Shavit 2007132 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Search range of keys

Art of Multiprocessor Programming© Herlihy-Shavit 2007133 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Predecessor reachable

Art of Multiprocessor Programming© Herlihy-Shavit 2007134 private boolean validate(Node pred, Node curry) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Is current node next?

Art of Multiprocessor Programming© Herlihy-Shavit 2007135 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Otherwise move on

Art of Multiprocessor Programming© Herlihy-Shavit 2007136 private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Predecessor not reachable

Art of Multiprocessor Programming© Herlihy-Shavit 2007137 Remove: searching public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … }

Art of Multiprocessor Programming© Herlihy-Shavit 2007138 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … } Remove: searching Search key

Art of Multiprocessor Programming© Herlihy-Shavit 2007139 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … } Remove: searching Retry on synchronization conflict

Art of Multiprocessor Programming© Herlihy-Shavit 2007140 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … } Remove: searching Examine predecessor and current nodes

Art of Multiprocessor Programming© Herlihy-Shavit 2007141 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … } Remove: searching Search by key

Art of Multiprocessor Programming© Herlihy-Shavit 2007142 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … } Remove: searching Stop if we find item

Art of Multiprocessor Programming© Herlihy-Shavit 2007143 public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … } Remove: searching Move along

Art of Multiprocessor Programming© Herlihy-Shavit 2007144 On Exit from Loop If item is present –curr holds item –pred just before curr If item is absent –curr has first higher key –pred just before curr Assuming no synchronization problems

Art of Multiprocessor Programming© Herlihy-Shavit 2007145 Remove: details on … try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else return false; }} finally { pred.unlock(); curr.unlock(); }

Art of Multiprocessor Programming© Herlihy-Shavit 2007146 try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else return false; }} finally { pred.unlock(); curr.unlock(); } Remove: details on … Always unlock

Art of Multiprocessor Programming© Herlihy-Shavit 2007147 try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else return false; }} finally { pred.unlock(); curr.unlock(); } Remove: details on … Lock both nodes

Art of Multiprocessor Programming© Herlihy-Shavit 2007148 try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else return false; }} finally { pred.unlock(); curr.unlock(); } Remove: details on … Check for synchronization conflicts

Art of Multiprocessor Programming© Herlihy-Shavit 2007149 try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else return false; }} finally { pred.unlock(); curr.unlock(); } Remove: details on … target found, remove node

Art of Multiprocessor Programming© Herlihy-Shavit 2007150 try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else return false; }} finally { pred.unlock(); curr.unlock(); } Remove: details on … target not found

Art of Multiprocessor Programming© Herlihy-Shavit 2007151 Optimistic List Limited hot-spots –Targets of add(), remove(), contains() –No contention on traversals Moreover –Traversals are wait-free –Food for thought …

Art of Multiprocessor Programming© Herlihy-Shavit 2007152 So Far, So Good Much less lock acquisition/release –Performance –Concurrency Problems –Need to traverse list twice –contains() method acquires locks Most common method call

Art of Multiprocessor Programming© Herlihy-Shavit 2007153 Evaluation Optimistic is effective if –cost of scanning twice without locks less than –cost of scanning once with locks Drawback –contains() acquires locks –90% of calls in many apps

Linked Lists: Locking, Lock- Free, and Beyond … Based on the companion slides for The Art of Multiprocessor Programming (Maurice Herlihy & Nir Shavit)

Similar presentations

Presentation on theme: "Linked Lists: Locking, Lock- Free, and Beyond … Based on the companion slides for The Art of Multiprocessor Programming (Maurice Herlihy & Nir Shavit)"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Linked Lists: Locking, Lock- Free, and Beyond … Based on the companion slides for The Art of Multiprocessor Programming (Maurice Herlihy & Nir Shavit)

Similar presentations

Presentation on theme: "Linked Lists: Locking, Lock- Free, and Beyond … Based on the companion slides for The Art of Multiprocessor Programming (Maurice Herlihy & Nir Shavit)"— Presentation transcript:

Similar presentations

About project

Feedback