1. Challenges in Non-Blocking Synchronization Håkan Sundell, Ph.D. Guest seminar at Department of Computer Science, University of Tromsö, Norway, 8 Dec 2005

2. 8 Dec 2005Håkan Sundell 2 Outline Shared Data Structures and Synchronization Non-Blocking Synchronization Problems Results Open Applications and Performance Future Conclusions

3. 8 Dec 2005Håkan Sundell 3 Shared Memory CPU Cache Cache bus Memory... - Uniform Memory Access (UMA) - Non-Uniform Memory Access (NUMA)

4. 8 Dec 2005Håkan Sundell 4 Synchronization Shared data structures needs synchronization! Accesses and updates must be coordinated to establish consistency.

5. 8 Dec 2005Håkan Sundell 5 Mutual Exclusion Access to shared data will be atomic because of lock Reduced Parallelism by definition Blocking, Danger of priority inversion and deadlocks. Solutions exists, but with high overhead, especially for multi-processor systems P1 P2 P3

6. 8 Dec 2005Håkan Sundell 6 Hardware Synchronization Primitives Weak Atomic Read/Write Stronger Atomic Test-And-Set (TAS), Fetch-And-Add (FAA), Swap Universal Atomic Compare-And-Swap (CAS) Atomic Load-Linked/Store-Conditionally Read Write Read M=f(M,…)

7. 8 Dec 2005Håkan Sundell 7 Universal and Conditional Synchronization primitive Compare-And-Swap (CAS) bool CAS(int *p, int old, int new) { atomic { if(*p == old) { *p=new; return true; } else return false; } }

8. 8 Dec 2005Håkan Sundell 8 Non-blocking Synchronization Perform operation/changes using atomic primitives Lock-Free Synchronization Optimistic approach Retries until succeeding Wait-Free Synchronization Always finishes in a finite number of its own steps Coordination with all participants

9. 8 Dec 2005Håkan Sundell 9 Non-Blocking Synchronization: Example Lock-Free Stack (i.e. Free-list): Create a linked-list of the free nodes, allocate/reclaim using CAS How to make sure that the next-pointer of the first item is not changed before CAS? HeadMem 1Mem 2Mem i … Used 1 Reclaim Allocate …

10. 8 Dec 2005Håkan Sundell 10 Non-Blocking Synchronization: Problems Algorithmic design Operations on shared data structures usually involve updates of several shared variables Modern shared memory systems only support atomic primitives on single memory words! Want parallelism – Avoid bottlenecks All sub-operations also have to be lock-free/wait-free

11. 8 Dec 2005Håkan Sundell 11 Non-Blocking Synchronization: Problems Correctness Linearizability. In order for an implementation to be linearizable, for every concurrent execution, there should exist an equal sequential execution that respects the partial order of the operations in the concurrent execution Proofs Series of intuitive lemmas Formal proofs using semi-automatic proof-engines requiring hundreds of invariants and several man- years of work.

12. 8 Dec 2005Håkan Sundell 12 Non-Blocking Synchronization: Problems Implementation Modern shared memory systems do not offer sequential or equivalent level of consistency by default Out-of-order execution Need to specify required read/write read/write relative order for each memory access as needed Extra instructions inserted Degrades out-of-order execution, i.e. significantly degrades speed!

13. 8 Dec 2005Håkan Sundell 13 Non-Blocking Synchronization: Problems Practical? Efficient? Algorithmic Design Correctness Implementation

14. 8 Dec 2005Håkan Sundell 14 Non-Blocking Synchronization: Details Memory Management Memory allocation Memory reclamation (garbage collection) Atomic primitives Common shared data structures Stack Queue Deque Priority Queue Dictionary Hash Table Linked Lists

15. 8 Dec 2005Håkan Sundell 15 Lock-Free Memory Management Memory Allocation Valois 1995, fixed block-size, fixed purpose Michael 2004, Gidenstam et al. 2004, any size, any purpose Garbage Collection Valois 1995, (Detlefs et al. 2001); reference counting Michael 2002, (Herlihy et al. 2002); hazard pointers Gidenstam, Papatriantafilou, Sundell and Tsigas 2005, ”Efficient and Reliable Memory Reclamation Based on Reference Counting”

16. 8 Dec 2005Håkan Sundell 16 Wait-Free Memory Management Hesselink and Groote 2001. Limited to shared tokens. Sundell 2005. ”Wait-Free Reference Counting and Memory Management” Memory Allocation – fixed block-size, fixed purpose Garbage Collection – reference counting

17. 8 Dec 2005Håkan Sundell 17 Software Synchronization Primitives Atomic Read/Write. Several results published WF/LF. Multi-variable Read/Write, i.e. Snapshot. Several results published WF/LF. LL/SC. Several results published WF/LF. Multi-word Compare-And-Swap (CASN) i.e. transactions Several results published LF.

18. 8 Dec 2005Håkan Sundell 18 Shared Data Structures Lock-Free Stack Valois 1995, Michael 2002 Queue Valois 1995, Tsigas and Zhang 2001, Michael 2002 and much more Deque Michael 2003, Sundell and Tsigas 2004

19. 8 Dec 2005Håkan Sundell 19 Shared Data Structures Lock-Free Priority Queue Sundell and Tsigas 2003 Dictionary (Harris et al. 2001), Sundell and Tsigas 2003 Hash Tables Michael 2002 and much more Linked Lists Singly-Linked: Valois 1995, (Harris et al. 2001) Doubly-Linked: Sundell and Tsigas 2004

20. 8 Dec 2005Håkan Sundell 20 Non-Blocking Synchronization: Open Wait-Free Memory Management. Improvement Atomic primitives. Wait-Free multi-word compare-and-swap Lock-Free Data Structures. Tree Improvement, disjoint-access-parallelism Wait-Free Data Structures. Stack, queue, priority queue, dictionary, hash table, linked lists, trees

21. 8 Dec 2005Håkan Sundell 21 Applications and Performance Oceansimulates eddy currents in an ocean basin. Radiositycomputes the equilibrium distribution of light in a scene using the radiosity method. Volrendrenders 3D volume data into an image using a ray- casting method. WaterEvaluates forces and potentials that occur over time between water molecules. Spark98a collection of sparse matrix kernels. Each kernel performs a sequence of sparse matrix vector product operations using matrices that are derived from a family of three-dimensional finite element earthquake applications. Tsigas and Zhang 2001

22. 8 Dec 2005Håkan Sundell 22 Tsigas and Zhang 2001 58P 32P 24P

23. 8 Dec 2005Håkan Sundell 23 Applications Professional Lock-Free Queues are used in music applications Lock-Free Dictionary by Sundell-Tsigas used by several american financial services companies Operating systems kernels Software libraries Sundell and Tsigas 2002. ”NOBLE: A Non- Blocking Inter-Process Communication Library” JAVA – J2SE 5.0

24. 8 Dec 2005Håkan Sundell 24 Future? Distributed Shared Memory Atomic synchronization primitives? Relaxed Memory Models? Linearizability versus Weak memory consistency + forced consistency? CPU Cache bus Memory... I/OMemoryI/OMemoryI/O

25. 8 Dec 2005Håkan Sundell 25 Conclusions Non-blocking synchronization Can be Practical and Efficient Large scope of lock-free shared data structures available Used in practice Future work Wait-free dynamic data structures. Exploit modern processor architectures Distributed architectures

26. 8 Dec 2005Håkan Sundell 26 Questions? Contact Information: Address: Håkan Sundell Computing Science Chalmers University of Technology Email: phs@cs.chalmers.se Web: http://www.cs.chalmers.se/~phs