1. Kernel Synchronization David Ferry, Chris Gill CSE 522S - Advanced Operating Systems Washington University in St. Louis St. Louis, MO 63143 1

2. Linux is a Preemptive Kernel The Linux kernel executes concurrently with itself: Multiple processors (CONFIG_SMP) Interrupts Kernel Threads Many data structures can be accessed concurrently from different contexts, which may cause concurrency bugs. CSE 522S – Advanced Operating Systems2

3. Single Variable Race Condition Thread 1: load X X = X + 1 store X Thread 2: load X X = X + 1 store X CSE 522S – Advanced Operating Systems3 Suppose two concurrent processes are accessing a variable X with initial value of 1, what is the final value of X?:

4. Data Structure Race Condition Suppose two threads are accessing a list: CSE 522S – Advanced Operating Systems4 Node A Next Node B Next Node C Next Deleter: Node* = A->Next A->Next = B->Next Kfree(Node); Reader: Node* = A->Next Value = Node->val

5. Shared Data Synchronization Accessing shared data requires appropriate synchronization Protect data, not code! A region of code that accesses shared data is called a critical section CSE 522S – Advanced Operating Systems5

6. Basic Kernel Synchronization The Linux kernel supports most of the same synchronization primitives as userspace. 1.Atomic variables 2.Spin locks (non-sleepable locks) 3.Mutexes (sleepable locks) 4.Reader-writer locks 5.Semaphores CSE 522S – Advanced Operating Systems6

7. Locking Example CSE 522S – Advanced Operating Systems7 Node A Next Node B Next Node C Next Deleter: lock( list_lock ); Node* = A->Next; A->Next = B->Next; Kfree(Node); Unlock(list_lock); Reader: lock( list_lock ); Node* = A->Next; Value = Node->val; unlock( list_lock );

8. When to Use Different Types of Locks Spin locks - very short lock durations and/or very low overhead requirements Mutexes - Long lock duration and/or process needs to sleep while holding lock Atomics - When shared data fits inside a single word of memory Other locks - special cases CSE 522S – Advanced Operating Systems8

9. Sleeping With Locks Sleeping with a lock is generally a bad idea unless absolutely necessary Delays other processes Deadlock risk: is it guaranteed to eventually wake up and release the lock? A process cannot sleep holding a spinlock. Restructure code to back out of lock Use a mutex instead CSE 522S – Advanced Operating Systems9

10. Deadlock Deadlock occurs when a process can never progress while waiting for a lock, e.g.: CSE 522S – Advanced Operating Systems10 Lock A Lock B Thread 1: lock( A ); lock( B ); //critical section unlock( A ); unlock( B ); Thread 2: lock( B ); lock( A ); //critical section unlock( B ); unlock( A );

11. Deadlock Deadlock occurs when a process can never progress while waiting for a lock, e.g.: CSE 522S – Advanced Operating Systems11 Lock A Lock B Thread 1: lock( A ); lock( B ); //critical section unlock( A ); unlock( B ); Thread 2: lock( B ); lock( A ); //critical section unlock( B ); unlock( A );

12. Deadlock Deadlock occurs when a process can never progress while waiting for a lock, e.g.: CSE 522S – Advanced Operating Systems12 Lock A Lock B Thread 1: lock( A ); lock( B ); //critical section unlock( A ); unlock( B ); Thread 2: lock( B ); lock( A ); //critical section unlock( B ); unlock( A );

13. Deadlock Deadlock occurs when a process can never progress while waiting for a lock, e.g.: CSE 522S – Advanced Operating Systems13 Lock A Lock B Thread 1: lock( A ); lock( B ); //critical section unlock( A ); unlock( B ); Thread 2: lock( B ); lock( A ); //critical section unlock( B ); unlock( A ); Deadlock!

14. Lock Ordering Always acquire and release locks in the same order! (Does not solve all deadlocks, but it’s a good place to start.) CSE 522S – Advanced Operating Systems14 Lock A Lock B Thread 1: lock( A ); lock( B ); //critical section unlock( B ); unlock( A ); Thread 2: lock( A ); lock( B ); //critical section unlock( B ); unlock( A );

15. Lock Ordering Always acquire and release locks in the same order! (Does not solve all deadlocks, but it’s a good place to start.) CSE 522S – Advanced Operating Systems15 Lock A Lock B Thread 1: lock( A ); lock( B ); //critical section unlock( B ); unlock( A ); Thread 2: lock( A ); lock( B ); //critical section unlock( B ); unlock( A );

16. Lock Ordering Always acquire and release locks in the same order! (Does not solve all deadlocks, but it’s a good place to start.) CSE 522S – Advanced Operating Systems16 Lock A Lock B Thread 1: lock( A ); lock( B ); //critical section unlock( B ); unlock( A ); Thread 2: lock( A ); lock( B ); //critical section unlock( B ); unlock( A );

17. Lock Ordering Always acquire and release locks in the same order! (Does not solve all deadlocks, but it’s a good place to start.) CSE 522S – Advanced Operating Systems17 Lock A Lock B Thread 1: lock( A ); lock( B ); //critical section unlock( B ); unlock( A ); Thread 2: lock( A ); lock( B ); //critical section unlock( B ); unlock( A );

18. Lock Ordering Always acquire and release locks in the same order! (Does not solve all deadlocks, but it’s a good place to start.) CSE 522S – Advanced Operating Systems18 Lock A Lock B Thread 1: lock( A ); lock( B ); //critical section unlock( B ); unlock( A ); Thread 2: lock( A ); lock( B ); //critical section unlock( B ); unlock( A );

19. The BKL is No Longer Used The Big Kernel Lock (BKL) was the first lock introduced to the kernel Locks the entire kernel Provides correctness, but very slow New uses are prohibited Gradually replaced with finer-grained locking schemes CSE 522S – Advanced Operating Systems19