11/2/20111 The Ordering Requirements of Relativistic and Reader-Writer Locking Approaches to Shared Data Access Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney

11/2/20112 Outline The Problem The RWL Solution The RP Solution Other problems and their solutions Multiple Writers Performance

11/2/20113 The Problem ABC AC

11/2/20114 The Problem obtain refdrop ref unlinkreclaim obtain refdrop ref Reader 1 Reader 2 Writer

11/2/20115 The Problem void init(stuff *node) { node->a = COMPUTE_A; node->b = COMPUTE_B; node->initd = TRUE; } while (!node->initd) {} if (node->a) drop_nuke(); if (node->b) drop_napalm();

11/2/20116 The Problem void init(stuff *node) { int value; value = some_computation; node->a = COMPUTE_A; node->b = COMPUTE_B; node->initd = value; }

11/2/20117 The Problem void init(stuff *node) { node->a = COMPUTE_A; node->b = COMPUTE_B; node->initd = TRUE; } while (!node->initd) {} if (node->a) drop_nuke(); if (node->b) drop_napalm();

11/2/20118 The Problem Compiler reordering CPU reordering Memory System reordering

11/2/20119 How are these dependencies maintained? obtain refdrop ref unlinkreclaim obtain refdrop ref Reader 1 Reader 2 Writer

11/2/ How does → work? Thread 1 A: a=1; Thread 2 B: if (a) A → B else B → A

11/2/ How does  work? Thread 1 A: a=1; mb(); C: c=1; Thread 2 while (!a) B: A  B Thread 3 if (c) D: A  D

11/2/ With Reader-Writer Locks obtain refdrop ref unlinkreclaim read-lockread-unlock write-lockwrite-unlock

11/2/ With Reader-Writer Locks obtain refdrop ref unlinkreclaim read-lockread-unlock write-lock write-unlock

11/2/ With Reader-Writer Locks obtain refdrop ref unlinkreclaim read-lock read-unlock write-lockwrite-unlock

11/2/ Locking primitives must Impose the semantics of the lock Prevent compiler, CPU, or Memory system from reordering operations across them

11/2/ With Relativistic Programming obtain refdrop ref start-waitend-wait start-readend-read unlinkreclaim

11/2/ With Relativistic Programming obtain refdrop ref start-waitend-wait start-readend-read unlinkreclaim

11/2/ RP primitives must Impose the semantics of wait-for-readers Prevent compiler, CPU, or Memory system from reordering operations across them

11/2/ Outline The Problem The RWL Solution The RP Solution Other problems and their solutions Insert Move Down Move Up General Case Multiple Writers Performance

11/2/ Insert CDE CE

11/2/ Insert obtain refderef initlink obtain refderef Reader 1 Reader 2 Writer

11/2/ How does  work? Thread 1 A: a=1; mb(); C: c=1; Thread 2 while (!a) B: A  B Thread 3 if (c) D: A  D

11/2/ Insert Use rp-publish to perform link operation

11/2/ Move Down A B C D F G E H A B C D F GE H

11/2/ Move Down A B C D F G E H A B C D F GE H F’ A B C D GE H 1.init F’ 2.link F’ 3.unlink F

11/2/ Move Down init(F’) link(F’)unlink(F) reclaim(F) deref(H)deref(F)deref(D)deref(E) A B C D F G E H A B C D F G E H F’ A B C D GE H

11/2/ Move Down init(F’) link(F’)unlink(F) reclaim(F) deref(H)deref(F)deref(D)deref(F’) A B C D F G E H A B C D F G E H F’ A B C D GE H deref(E)

11/2/ Move Down init(F’) link(F’)unlink(F) reclaim(F) deref(H)deref(D)deref(F’) A B C D F G E H A B C D F G E H F’ A B C D GE H deref(E)

11/2/ RBTree Delete with Swap (Move Up) D A B E C F null D A B E C F C’ AE D F

11/2/ Move Up init(C’) link(C’)unlink(C) reclaim(C) deref(F)deref(C’) D A B E C F null D A B E C F C’ AE D F

11/2/ Move Up init(C’) link(C’)unlink(C) reclaim(C) deref(F)deref(B)deref(E) D A B E C F deref(C) null D A B E C F C’ AE D F

11/2/ The General Case Mutable vs. Immutable data

11/2/ The General Case for Writers Copy nodes to update immutable data or to make a collection of updates appear atomic Use rp-publish to update mutable data Use wait-for-readers when updates are in traversal order ABC

11/2/ The General Case for Readers Use rp-read for reading mutable data Only dereference mutable data once if (node->next->key == key) { return node->next->value; }

11/2/ Outline The Problem The RWL Solution The RP Solution Other problems and their solutions Multiple Writers Performance

11/2/ Multiple Writers W R2 R1

11/2/ Multiple Writers W1 Reader W2

11/2/ The phone company Can’t wait to get my new phone Can’t wait to ditch this phone Can’t wait to ditch this phone Where’s my phone book?

11/2/ Multiple Writers W1 Reader W2 wait-for-readers W1 Reader W2

11/2/ RP vs RWL delays W1 Reader W2 RP W1 Reader W2 reader pref W1 Reader W2 writer pref W1 Reader W2 TORP

11/2/ Trigger Events RP RWLR RWLW

11/2/ Outline The Problem The RWL Solution The RP Solution Other problems and their solutions Multiple Writers Performance

11/2/ How does  work? Thread 1 A: a=1; mb(); C: c=1; Thread 2 while (!a) B: A  B Thread 3 if (c) D: A  D

11/2/ Reader-Writer Locks read-unlock write-lockwrite-unlock read-lock read-unlockread-lock

11/2/ Reader-Writer Locks Reader while (writing) {} reading=1; Writer while (reading) {} writing=1;

11/2/ Relativistic Programming start-waitend-wait start-readend-read start-readend-read

11/2/ Relativistic Programming start-read(i) { reader[i]=1; } end-read(i) { reader[i]=0; } start-wait() {} end-wait() { for (i) { while (reader[i]) {} }

11/2/ Relativistic Programming start-read(i) { reader[i]=Epoch; } end-read(i) { reader[i]=0; } start-wait() { Epoch++; my_epoch = Epoch; } end-wait() { for (i) { while (reader[i] && reader[i] < my_epoch) {} }

11/2/ Performance Benefits to RP Less expensive read-side primitives Less serialization / more concurrency Less waiting

11/2/ Benchmarked Methods nolockNo synchronization rpRelativistic Programming torpTotally Ordered RP rwlrReader-Writer Lock Read preference rwlwReader-Writer Lock Write preference

11/2/ Read Performance (size=1)

11/2/ Read Performance (size=1000)

11/2/ Update Scalability

11/2/ Update Scalability part 2

11/2/ Update Scalability (part 3)

11/2/ Conclusions Correctness can be preserved by limiting allowable orderings RP read primitives are less expensive than RWL primitives RP allows more concurrency and less waiting

11/2/ Conclusions RP can preserve both correctness and scalability or reads