1. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Concurrent Skip Lists

2. Art of Multiprocessor Programming© Herlihy Shavit 2007 Set Object Interface Collection of elements No duplicates Methods –add() a new element –remove() an element –contains() if element is present

3. Art of Multiprocessor Programming© Herlihy Shavit 2007 Many are Cold but Few are Frozen Typically high % of contains() calls Many fewer add() calls And even fewer remove() calls –90% contains() –9% add() –1% remove() Folklore? –Yes but probably mostly true

4. Art of Multiprocessor Programming© Herlihy Shavit 2007 Concurrent Sets Balanced Trees? –Red-Black trees, AVL trees, … Problem: no one does this well … … because rebalancing after add() or remove() is a global operation

5. Art of Multiprocessor Programming© Herlihy Shavit 2007 Skip Lists 2 2 5 5 8 8 7 7 9 9 0 0 Probabilistic Data Structure No global rebalancing Logarithmic-time search

6. Art of Multiprocessor Programming© Herlihy Shavit 2007 Skip List Property 9 9 0 0 Each layer is sublist of lower-levels

7. Art of Multiprocessor Programming© Herlihy Shavit 2007 Skip List Property 7 7 9 9 0 0 Each layer is sublist of lower-levels

8. Art of Multiprocessor Programming© Herlihy Shavit 2007 Skip List Property 5 5 7 7 9 9 0 0 Each layer is sublist of lower-levels

9. Art of Multiprocessor Programming© Herlihy Shavit 2007 Skip List Property 5 5 8 8 7 7 9 9 0 0 Each layer is sublist of lower-levels

10. Art of Multiprocessor Programming© Herlihy Shavit 2007 Skip List Property 2 2 5 5 8 8 7 7 9 9 0 0 Each layer is sublist of lower-levels Lowest level is entire list

11. Art of Multiprocessor Programming© Herlihy Shavit 2007 Skip List Property 2 2 5 5 8 8 7 7 9 9 0 0 Each layer is sublist of lower-levels Not easy to preserve in concurrent implementations …

12. Art of Multiprocessor Programming© Herlihy Shavit 2007 Search 2 2 5 5 8 8 7 7 9 9 0 0 contains(8) Too far

13. Art of Multiprocessor Programming© Herlihy Shavit 2007 Search 2 2 5 5 8 8 7 7 9 9 0 0 contains(8) OK

14. Art of Multiprocessor Programming© Herlihy Shavit 2007 Search 2 2 5 5 8 8 7 7 9 9 0 0 contains(8) Too far

15. Art of Multiprocessor Programming© Herlihy Shavit 2007 Search 2 2 5 5 8 8 7 7 9 9 0 0 contains(8) Too far

16. Art of Multiprocessor Programming© Herlihy Shavit 2007 Search 2 2 5 5 8 8 7 7 9 9 0 0 contains(8) Yes!

17. Art of Multiprocessor Programming© Herlihy Shavit 2007 7 7 Search 8 0 0 2 2 5 5 9 9 contains(8)

18. Art of Multiprocessor Programming© Herlihy Shavit 2007 7 7 Logarithmic 8 0 0 2 2 5 5 9 9 contains(8) Log N

19. Art of Multiprocessor Programming© Herlihy Shavit 2007 Why Logarthimic 2 2 5 5 8 8 7 7 9 9 0 0 Property: Each pointer at layer i jumps over roughly 2 i nodes Pick node heights randomly so property guaranteed probabilistically 2i2i 2i2i

20. Art of Multiprocessor Programming© Herlihy Shavit 2007 Find() -- Sequential int find(T x, Node [] preds, Node [] succs) { … }

21. Art of Multiprocessor Programming© Herlihy Shavit 2007 Find() -- Sequential int find(T x, Node [] preds, Node [] succs) { … } object height (-1 if not there)

22. Art of Multiprocessor Programming© Herlihy Shavit 2007 Find() -- Sequential int find(T x, Node [] preds, Node [] succs) { … } Object sought object height (-1 if not there)

23. Art of Multiprocessor Programming© Herlihy Shavit 2007 Find() -- Sequential int find(T x, Node [] preds, Node [] succs) { … } object sought return predecessors Object height (-1 if not there)

24. Art of Multiprocessor Programming© Herlihy Shavit 2007 Find() -- Sequential int find(T x, Node [] preds, Node [] succs) { … } object sought return predecessors return successors object height (-1 if not there)

25. Art of Multiprocessor Programming© Herlihy Shavit 2007 Successful Search 2 2 5 5 8 8 7 7 9 9 0 0 find(7, …) prev 0 1 2 3 4

26. Art of Multiprocessor Programming© Herlihy Shavit 2007 Successful Search 2 2 5 5 8 8 7 7 9 9 0 0 find(7, …) curr 0 1 2 3 4 prev 0 1 2 3 4

27. Art of Multiprocessor Programming© Herlihy Shavit 2007 Unsuccessful Search 2 2 5 5 8 8 7 7 9 9 0 0 find(6, …) prev 0 1 2 3 4

28. Art of Multiprocessor Programming© Herlihy Shavit 2007 Unsuccessful Search 2 2 5 5 8 8 7 7 9 9 0 0 find(6, …) curr 0 1 2 3 4 prev 0 1 2 3 4

29. Art of Multiprocessor Programming© Herlihy Shavit 2007 Lazy Skip List Mix blocking and non-blocking techniques: –Use optimistic-lazy locking for add() and remove() –Wait-free contains() Remember: typically lots of contains() calls but few add() and remove()

30. Art of Multiprocessor Programming© Herlihy Shavit 2007 Review: Lazy List Remove aa b c d

31. Art of Multiprocessor Programming© Herlihy Shavit 2007 Review: Lazy List Remove aa b c d Present in list

32. Art of Multiprocessor Programming© Herlihy Shavit 2007 Review: Lazy List Remove aa b c d Logically deleted

33. Art of Multiprocessor Programming© Herlihy Shavit 2007 Review: Lazy List Remove aa b c d Physically deleted

34. Art of Multiprocessor Programming© Herlihy Shavit 2007 Lazy Skip Lists 2 2 5 5 8 8 7 7 Use a mark bit for logical deletion 9 9 0 0 0 0 0 0 0 0

35. Art of Multiprocessor Programming© Herlihy Shavit 2007 add(6) Create node of (random) height 4 2 2 5 5 8 8 7 7 9 9 0 0 0 0 6 6

36. Art of Multiprocessor Programming© Herlihy Shavit 2007 add(6) find() predecessors 6 2 2 5 5 8 8 7 7 9 9 0 0 0 0 0 6 6

37. Art of Multiprocessor Programming© Herlihy Shavit 2007 add(6) find() predecessors Lock them 6 2 2 5 5 8 8 7 7 9 9 0 0 0 0 0 0 6 6

38. Art of Multiprocessor Programming© Herlihy Shavit 2007 add(6) find() predecessors Lock them Validate 6 2 2 5 5 8 8 7 7 9 9 0 0 0 0 0 0 6 6 Optimistic approach

39. Art of Multiprocessor Programming© Herlihy Shavit 2007 add(6) 8 8 7 7 9 9 2 2 5 5 0 0 find() predecessors Lock them Validate Splice 0 6 6 0 0 0

40. Art of Multiprocessor Programming© Herlihy Shavit 2007 add(6) find() predecessors Lock them Validate Splice Unlock 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0

41. Art of Multiprocessor Programming© Herlihy Shavit 2007 remove(6) 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 0

42. Art of Multiprocessor Programming© Herlihy Shavit 2007 remove(6) find() predecessors 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 0

43. Art of Multiprocessor Programming© Herlihy Shavit 2007 remove(6) find() predecessors Lock victim 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 0

44. Art of Multiprocessor Programming© Herlihy Shavit 2007 8 8 7 7 9 9 2 2 5 5 0 0 6 6 remove(6) find() predecessors Lock victim Set mark (if not already set) 0 0 0 0 0 Logical remove…

45. Art of Multiprocessor Programming© Herlihy Shavit 2007 8 8 7 7 9 9 2 2 5 5 0 0 6 6 remove(6) find() predecessors Lock victim Set mark (if not already set) Lock predecessors (ascending order) & validate 0 0 1 0 0 0

46. Art of Multiprocessor Programming© Herlihy Shavit 2007 remove(6) find() predecessors Lock victim Set mark (if not already set) Lock predecessors (ascending order) & validate Physically remove 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 1 0 0 0

47. Art of Multiprocessor Programming© Herlihy Shavit 2007 remove(6) find() predecessors Lock victim Set mark (if not already set) Lock predecessors (ascending order) & validate Physically remove 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 1 0 0 0

48. Art of Multiprocessor Programming© Herlihy Shavit 2007 remove(6) find() predecessors Lock victim Set mark (if not already set) Lock predecessors (ascending order) & validate Physically remove 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 1 0 0 0

49. Art of Multiprocessor Programming© Herlihy Shavit 2007 remove(6) find() predecessors Lock victim Set mark (if not already set) Lock predecessors (ascending order) & validate Physically remove 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 1 0 0 0

50. Art of Multiprocessor Programming© Herlihy Shavit 2007 remove(6) find() predecessors Lock victim Set mark (if not already set) Lock predecessors (ascending order) & validate Physically remove 8 8 7 7 9 9 2 2 5 5 0 0 0 0 0 0 0

51. Art of Multiprocessor Programming© Herlihy Shavit 2007 contains(8) Find() & not marked 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 0

52. Art of Multiprocessor Programming© Herlihy Shavit 2007 contains(8) 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 0 Node 6 removed while traversed

53. Art of Multiprocessor Programming© Herlihy Shavit 2007 contains(8) 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 Node removed while being traversed

54. Art of Multiprocessor Programming© Herlihy Shavit 2007 contains(8) 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 Prove: an unmarked node (like 8) remains reachable

55. Art of Multiprocessor Programming© Herlihy Shavit 2007 8 8 7 7 9 9 2 2 5 5 0 0 6 6 remove(6): Linearization Successful remove happens when bit is set 0 0 0 0 0 Logical remove…

56. Art of Multiprocessor Programming© Herlihy Shavit 2007 Add: Linearization 8 8 7 7 9 9 2 2 5 5 0 0 Successful add() at point when fully linked Add fullyLinked bit to indicate this Bit tested by contains() 0 0 6 6 0 0 0

57. Art of Multiprocessor Programming© Herlihy Shavit 2007 contains(7): Linearization 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 When fully-linked unmarked node found Pause while fullyLinked bit unset

58. Art of Multiprocessor Programming© Herlihy Shavit 2007 contains(7): Linearization 8 8 6 6 9 9 2 2 5 5 0 0 7 7 0 0 1 0 0 When do we linearize unsuccessful Search? 1 So far OK…

59. Art of Multiprocessor Programming© Herlihy Shavit 2007 contains(7): Linearization 8 8 6 6 9 9 2 2 5 5 0 0 7 7 0 0 0 0 When do we linearize unsuccessful Search? 7 7 But what if a new 7 added concurrently?

60. Art of Multiprocessor Programming© Herlihy Shavit 2007 contains(7): Linearization 8 8 6 6 9 9 2 2 5 5 0 0 7 7 0 0 0 0 When do we linearize unsuccessful Search? 1 7 7 Prove: at some point 7 was not in the skip list

61. Art of Multiprocessor Programming© Herlihy Shavit 2007 A Simple Experiment Each thread runs 1 million iterations, each either: –add() –remove() –contains() Item and method chosen in random from some distribution

62. Art of Multiprocessor Programming© Herlihy Shavit 2007 Lazy Skip List: Performance Multiprogramming

63. Art of Multiprocessor Programming© Herlihy Shavit 2007 Lazy Skip List: Performance Multiprogramming Higher contention Throughput

64. Art of Multiprocessor Programming© Herlihy Shavit 2007 Lazy Skip List: Performance Multiprogramming Unrealistic Contention

65. Art of Multiprocessor Programming© Herlihy Shavit 2007 Lock-Free Variant No locking –Cannot preserve “skiplist invariant” –Each list sublist of next Instead –Bottom-level list is the ground truth –Everything else is optimization

66. Art of Multiprocessor Programming© Herlihy Shavit 2007 Lock-Free Skip List 2 2 5 5 8 8 7 7 9 9 0 0 No locking –Cannot preserve “skiplist invariant” –Each list no longer sublist of child

67. Art of Multiprocessor Programming© Herlihy Shavit 2007 Lock-Free Skip List 2 2 5 5 8 8 7 7 9 9 0 0 Bottom-level list is actual list –Ground truth Find() works much as before

68. Art of Multiprocessor Programming© Herlihy Shavit 2007 Lock-Free Skip List 2 2 5 5 8 8 7 7 9 9 0 0 Each level is individual LockFreeList Shortcuts only No need for FullyLinked field

69. Art of Multiprocessor Programming© Herlihy Shavit 2007 Add(7) 2 2 5 5 8 8 7 7 9 9 0 0 Add from bottom level –Bottom level add is linearization point –Other levels optimization

70. Art of Multiprocessor Programming© Herlihy Shavit 2007 5 5 Remove(7) 2 2 8 8 7 7 9 9 0 0 Logically remove from each list –Starting at top Linearization point is lowest remove

71. Art of Multiprocessor Programming© Herlihy Shavit 2007 Summary Lazy Skip List –Optimistic fine-grained Locking –Performs as well as the lock-free solution in “common” cases –But Simpler Lock-Free Skip List –Combines tricks from earlier lectures

72. Art of Multiprocessor Programming© Herlihy Shavit 2007 This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License.Creative Commons Attribution-ShareAlike 2.5 License You are free: –to Share — to copy, distribute and transmit the work –to Remix — to adapt the work Under the following conditions: –Attribution. You must attribute the work to “The Art of Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work). –Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to –http://creativecommons.org/licenses/by-sa/3.0/. Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights.