Kernel Synchronization II David Ferry, Chris Gill CSE 422S - Operating Systems Organization Washington University in St. Louis St. Louis, MO 63143

Basic Kernel Synchronization Watch out for non-recursive locks! If you already hold a (spin) lock don’t reacquire it! Atomic variables Spin locks (non-sleepable locks) Mutexes (sleepable locks) Reader-writer locks Semaphores CSE 422S – Operating Systems Organization

When to Use Different Types of Locks Spin locks - very short lock durations and/or very low overhead requirements Mutexes - Long lock duration and/or process needs to sleep while holding lock Atomics - When shared data fits inside a single word of memory, or a lock-free synchronization protocol can be implemented Read-Copy-Update (RCU) - Modern locking approach that may improve performance CSE 422S – Operating Systems Organization

Sleeping while holding Locks Sleeping while holding a lock should be avoided (unless absolutely necessary) Delays other processes Deadlock risk: is it guaranteed to wake up (at least eventually) and release the lock? A process cannot sleep holding a spinlock Change code to back out of lock, then sleep Or, use a mutex instead of a spin lock CSE 422S – Operating Systems Organization

The BKL is No Longer Used The Big Kernel Lock (BKL) was the first lock introduced to the kernel Locked the entire kernel Provides correctness, but very slow New uses are prohibited Gradually replaced with finer-grained locking schemes CSE 422S – Operating Systems Organization

Synchronization Design Challenge A common kernel problem: Multiple threads share a data structure. Some are reading Some are writing Reads and writes should not interfere! Shared data CSE 422S – Operating Systems Organization

Synchronization Design Tradeoffs All synchronization methods need to balance the needs of readers and writers: Reads tend to be more common Lots of reads Few writes Balanced Reads and writes Lots of writes Few reads Synchronization can prevent concurrency… Reader/writer locks Mutual exclusion Or it can allow concurrency at the expense of overheads: Lock free / wait free algorithms Transactional memory CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization RCU Philosophy Under RCU: Concurrent reads are synchronization-free (which means scalability!) Writers must guarantee that all readers only ever see a consistent view of memory Similar to a publish-subscribe model Let’s look at the API… CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization RCU Writer API Even if pointer write is atomic: struct foo *ptr = NULL; p = kmalloc(...); p->A = 1; p->B = 2; p->C = 3; ptr = p; Overall code is not safe! Compiler may re-order lines 3-6 CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization RCU Writer API RCU encapsulates memory fences struct foo *ptr = NULL; p = kmalloc(...); p->A = 1; p->B = 2; p->C = 3; rcu_assign_ptr(ptr,p); rcu_assign_ptr method publishes P CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization RCU Reader API Consider reading a data structure: p = ptr; if (p != NULL) do_something(p->A, p->B, p->C); This is also not safe! The values of A, B, and C could change between reads! CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization RCU Reader API Safely reading the data structure: rcu_read_lock(); p = rcu_dereference(ptr); if (p != NULL) do_something(p->A, p->B, p->C); rcu_read_unlock(); rcu_dereference() can be thought of as subscribing to a specific, valid version of ptr lock/unlock defines RCU critical section CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization RCU Encapsulation Note, RCU semantics can be encapsulated for specific data structures: rcu_list_add() rcu_for_each_read() But not: rcu_for_each_write() RCU does not allow for concurrent writes! CSE 422S – Operating Systems Organization