Distributed Algorithms (22903) Lecturer: Danny Hendler Lock-free stack algorithms

2 Wait-freedom vs. lock-freedom Wait-freedom – each process completes its operation in a finite number of its own steps Lock-freedom – some process completes its operation after a finite number of steps is taken

33 The compare&swap operation Comare&swap(b,old,new) atomically v read from b if (v = old) { b new return success } else return failure; MIPS PowerPC DECAlpha Motorola 680x0 IBM 370 Sun SPARC 80X86

4 Treiber’s stack algorithm Push(int v, Stack S) 1.n := new NODE ;create node for new stack item 2.n.val := v ;write item value 3.do forever ;repeat until success 4. node top := S.top 5. n.next := top ;next points to current (LIFO order) 6. if compare&swap(S, top, n) ; try to add new item 7. return ; return if succeeded 8.od Top val next val next … val next

5 Treiber’s stack algorithm (cont’d) Pop(Stack S) 1.do forever 2. top := S.top 3. if top = null 4. return empty 5. if compare&swap(S, top, top.next) 6. return-val=top.val 7. free top 8. return return-val 9.od Top val next val next … val next

6 Treiber’s stack algorithm (cont’d) It is easily seen that the alg is linearizable and lock-free A disadvantage of the algorithms is that… It has a sequential bottleneck.

An elimination backoff stack algorithm (Hendler, Shavit and Yerushalmi, 2004) Key idea: Key idea: pairs of push/pop operations may collide and eliminate each other without accessing a central stack. Top val next val next … val next Central stack collision array

8 Collision scenarios Collision array push pop push Top val next val next … val next Central stack pop push

9 Elimination: challenges Collision array push pop push Top val next val next … val next Central stack pop push Prevent elimination chains: e.g., A collides with B, which collides with C… Prevent race conditions: e.g., A collides with B, which is already gone…

Data structures Each stack operation is represented by a ThreadInfo structure struct ThreadInfo { id ;the identifier of the thread performing the operation op ;a PUSH/POP opcode cell ;a cell structure spin ; duration to spin } Struct cell { ;a representation of stack item as in Treiber pnext ;pointer to the next cell pdata ;stack item } Location array p1p2 p3 pn Thread Info p4 collision array p1 p7

11 Elimination-backoff stack code void StackOp(ThreadInfo * p) { 1.if (TryPerformStackOP(p) == FALSE) ;if operation not applied to central stack 2. LesOp(p) ;try to eliminate operation by colliding with an opposite-type operation 3.return void LesOP(ThreadInfo * p) 1.while (1) 2. location[mypid]=p ;announce arrival 3. pos=GetPosition(p) ;get a random position at the collision array 4. him=collision[pos] ;read current value of that position 5. while (!compare&swap(&collision[pos],him,mypid);try to write own ID 6. him=collision[pos] ;continue till success 7. if (him != empty) ;if read an ID of another thread 8. q=location[him] ;read a pointer to the other thread’s info 9. if (q!=NULL && q->id=him && q->op != p->op) ;if may collide 10. if (compare&swap(&location[mypid],p,NULL) ;try to prevent unwanted collisions 11. if (TryCollision(p,q)==true) ;if collided successfully 12. return ;return code is already at ThreadInfo structure 13. else goto stack ;try to apply operation to central stack 14. else FinishCollision(p), return ;extract information and finish 15. delay (p->spin) ;Wait for other thread to collide with me 16. if (!compare&swap(&location[mypid],p,NULL) ;if someone collided with me 17. FinishCollision(p), return;Extract information and finish stack: if (TryPerformStackOp(p)==TRUE) return ;try to apply operation to central stack

12 Elimination-backoff stack code (cont’d) void TryCollision(ThreadInfo* p, ThreadInfo *q) 1.if (p->op==PUSH) 2. if (compare&swap(&location[him],q,p)) ;give my record to other thread 3. return TRUE 4. else 5. return FALSE 6.else 7. if (compare&swap(&location[him],q,NULL)) 8. p->cell=q->cell ;get pointer to PUSH operation’s cell 9. return TRUE 10. else 11. return FALSE void FinishCollision(ThreadInfo* p) 1.if (p->op==POP) 2. p->pcell=location[mypid]->pcell 3. location[mypid]=NULL

13 Elimination-backoff stack code (cont’d) Why is this implementation linearizable? Can a record be recycled once popped from stack?

© Herlihy-Shavit Recycling: Simple Solution Each thread has a free list of unused queue nodes Allocate node: pop from list Free node: push onto list Use CAS for atomicity Deal with underflow somehow …

© Herlihy-Shavit Why Recycling is Hard Free pool headtail Want to rediret head from grey to red zzz…

© Herlihy-Shavit Why Recycling is Hard Free pool zzz headtail

© Herlihy-Shavit Why Recycling is Hard Free pool Yawn! headtail

© Herlihy-Shavit Why Recycling is Hard Free pool CAS headtail OK, here I go!

© Herlihy-Shavit Final State Free pool What went wrong? headtail

© Herlihy-Shavit The Dreaded ABA Problem Head pointer has value A Thread reads value A headtail

© Herlihy-Shavit Dreaded ABA continued zzz Head pointer has value B Node A freed headtail

© Herlihy-Shavit Dreaded ABA continued Yawn! Head pointer has value A again Node A recycled & reinitialized headtail

© Herlihy-Shavit Dreaded ABA continued CAS succeeds because pointer matches even though pointer’s meaning has changed CAS headtail

© Herlihy-Shavit The Dreaded ABA Problem Is a result of CAS() semantics (Sun, Intel, AMD) Does not arise with Load- Locked/Store-Conditional (IBM)

© Herlihy-Shavit Dreaded ABA – A Solution Tag each pointer with a counter Unique over lifetime of node Pointer size vs word size issues Overflow? –Don’t worry be happy? –Bounded tags Other solutions exist (e.g. hazard pointers)