The read-copy-update mechanism for supporting real-time applications on shared-memory multiprocessor systems with Linux Guniguntala et al.

Slides:



Advertisements
Similar presentations
1 Chapter 5 Concurrency: Mutual Exclusion and Synchronization Principals of Concurrency Mutual Exclusion: Hardware Support Semaphores Readers/Writers Problem.
Advertisements

Concurrency: Mutual Exclusion and Synchronization Chapter 5.
Ch. 7 Process Synchronization (1/2) I Background F Producer - Consumer process :  Compiler, Assembler, Loader, · · · · · · F Bounded buffer.
CS510 – Advanced Operating Systems 1 The Synergy Between Non-blocking Synchronization and Operating System Structure By Michael Greenwald and David Cheriton.
Chapter 6: Process Synchronization
Background Concurrent access to shared data can lead to inconsistencies Maintaining data consistency among cooperating processes is critical What is wrong.
Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition, Chapter 6: Process Synchronization.
Garbage Collecting the World Bernard Lang Christian Queinnec Jose Piquer Presented by Yu-Jin Chia See also: pp text.
Wait-Free Reference Counting and Memory Management Håkan Sundell, Ph.D.
CS510 Concurrent Systems Jonathan Walpole. What is RCU, Fundamentally? Paul McKenney and Jonathan Walpole.
RCU in the Linux Kernel: One Decade Later by: Paul E. Mckenney, Silas Boyd-Wickizer, Jonathan Walpole Slides by David Kennedy (and sources)
Read-Copy Update P. E. McKenney, J. Appavoo, A. Kleen, O. Krieger, R. Russell, D. Saram, M. Soni Ottawa Linux Symposium 2001 Presented by Bogdan Simion.
Multi-Object Synchronization. Main Points Problems with synchronizing multiple objects Definition of deadlock – Circular waiting for resources Conditions.
The Performance of Spin Lock Alternatives for Shared-Memory Microprocessors Thomas E. Anderson Presented by David Woodard.
Review: Process Management Objective: –Enable fair multi-user, multiprocess computing on limited physical resources –Security and efficiency Process: running.
CS510 Advanced OS Seminar Class 10 A Methodology for Implementing Highly Concurrent Data Objects by Maurice Herlihy.
5.6.2 Thread Synchronization with Semaphores Semaphores can be used to notify other threads that events have occurred –Producer-consumer relationship Producer.
CS510 Concurrent Systems Class 2 A Lock-Free Multiprocessor OS Kernel.
What is RCU, fundamentally? Sri Ramkrishna. Intro RCU stands for Read Copy Update  A synchronization method that allows reads to occur concurrently with.
CS510 Concurrent Systems Class 13 Software Transactional Memory Should Not be Obstruction-Free.
Practical Concerns for Scalable Synchronization Jonathan Walpole (PSU) Paul McKenney (IBM) Tom Hart (University of Toronto)
Instructor: Umar KalimNUST Institute of Information Technology Operating Systems Process Synchronization.
Operating Systems CSE 411 CPU Management Oct Lecture 13 Instructor: Bhuvan Urgaonkar.
Chapter 6 Concurrency: Deadlock and Starvation Operating Systems: Internals and Design Principles, 6/E William Stallings Dave Bremer Otago Polytechnic,
Object Oriented Analysis & Design SDL Threads. Contents 2  Processes  Thread Concepts  Creating threads  Critical sections  Synchronizing threads.
Solution to Dining Philosophers. Each philosopher I invokes the operations pickup() and putdown() in the following sequence: dp.pickup(i) EAT dp.putdown(i)
CS510 Concurrent Systems Jonathan Walpole. A Lock-Free Multiprocessor OS Kernel.
Cosc 4740 Chapter 6, Part 3 Process Synchronization.
Concurrency, Mutual Exclusion and Synchronization.
Lecture 3 Process Concepts. What is a Process? A process is the dynamic execution context of an executing program. Several processes may run concurrently,
1 Scheduling The part of the OS that makes the choice of which process to run next is called the scheduler and the algorithm it uses is called the scheduling.
Cpr E 308 Spring 2004 Real-time Scheduling Provide time guarantees Upper bound on response times –Programmer’s job! –Every level of the system Soft versus.
Operating Systems CSE 411 Multi-processor Operating Systems Multi-processor Operating Systems Dec Lecture 30 Instructor: Bhuvan Urgaonkar.
Maged M.Michael Michael L.Scott Department of Computer Science Univeristy of Rochester Presented by: Jun Miao.
Scheduling Lecture 6. What is Scheduling? An O/S often has many pending tasks. –Threads, async callbacks, device input. The order may matter. –Policy,
Kernel Locking Techniques by Robert Love presented by Scott Price.
Operating Systems CSE 411 CPU Management Sept Lecture 10 Instructor: Bhuvan Urgaonkar.
Chapter 6: Process Synchronization. 6.2 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Module 6: Process Synchronization Background The.
CS533 Concepts of Operating Systems Jonathan Walpole.
CS510 Concurrent Systems Jonathan Walpole. A Methodology for Implementing Highly Concurrent Data Objects.
© Janice Regan, CMPT 300, May CMPT 300 Introduction to Operating Systems Operating Systems Processes and Threads.
Operating Systems CSE 411 CPU Management Dec Lecture Instructor: Bhuvan Urgaonkar.
CS510 Concurrent Systems Jonathan Walpole. RCU Usage in Linux.
CSE 153 Design of Operating Systems Winter 2015 Midterm Review.
1 Read-Copy Update Paul E. McKenney Linux Technology Center IBM Beaverton Jonathan Appavoo Department.
Time Management.  Time management is concerned with OS facilities and services which measure real time.  These services include:  Keeping track of.
Hazard Pointers: Safe Memory Reclamation for Lock-Free Objects MAGED M. MICHAEL PRESENTED BY NURIT MOSCOVICI ADVANCED TOPICS IN CONCURRENT PROGRAMMING,
Read-Log-Update A Lightweight Synchronization Mechanism for Concurrent Programming Alexander Matveev (MIT) Nir Shavit (MIT and TAU) Pascal Felber (UNINE)
1 Critical Section Problem CIS 450 Winter 2003 Professor Jinhua Guo.
Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Chapter 6: Process Synchronization.
Multi-Object Synchronization. Multi-Object Programs What happens when we try to synchronize across multiple objects in a large program? – Each object.
CE Operating Systems Lecture 8 Process Scheduling continued and an introduction to process synchronisation.
Read-Copy-Update Synchronization in the Linux Kernel 1 David Ferry, Chris Gill CSE 522S - Advanced Operating Systems Washington University in St. Louis.
Mutual Exclusion -- Addendum. Mutual Exclusion in Critical Sections.
CS510 Concurrent Systems “What is RCU, Fundamentally?” By Paul McKenney and Jonathan Walpole Daniel Mansour (Slides adapted from Professor Walpole’s)
6.1 Silberschatz, Galvin and Gagne ©2005 Operating System Principles 6.5 Semaphore Less complicated than the hardware-based solutions Semaphore S – integer.
Background on the need for Synchronization
Kernel Synchronization II
Practical Concerns for Scalable Synchronization
Concurrency: Mutual Exclusion and Process Synchronization
Conditions for Deadlock
Software Transactional Memory Should Not be Obstruction-Free
Kernel Synchronization II
CSE 451: Operating Systems Autumn 2003 Lecture 7 Synchronization
CSE 451: Operating Systems Autumn 2005 Lecture 7 Synchronization
CSE 451: Operating Systems Winter 2003 Lecture 7 Synchronization
CSE 153 Design of Operating Systems Winter 2019
Chapter 3: Process Management
MV-RLU: Scaling Read-Log-Update with Multi-Versioning
Honnappa Nagarahalli Arm
Presentation transcript:

The read-copy-update mechanism for supporting real-time applications on shared-memory multiprocessor systems with Linux Guniguntala et al.

 Publish-Subscribe ◦ insertion ◦ reader-writer synchronization  Wait for pre-existing readers to complete ◦ deletion ◦ change – wait for readers – free ◦ safe memory reclamation  Maintain multiple versions of update objects ◦ for readers

RCU Semantics: A First Attempt McKenney & Walpole Starting List New Node Copy B to B’ and Modify Move A.Next to B’ B still visible, but not for new readers Readers complete, remove B

 Reader-Writer ◦ rcu_assign_pointer() ◦ rcu_dereference() ◦ Memory barriers embedded in API  Writer-Collection ◦ rcu_synchronize() blocks caller until safe to collect ◦ call_rcu() is asychronous call for collection  Reader-Collection (?) p->a = 1; p->b = 2; p->c = 3; rcu_assign_pointer(gp, p);

 General issues in non-blocking & swap-free ◦ When is it safe to free memory? ◦ Memory reclamation tracking can be relatively costly ◦ Expensive atomic operations / memory barriers required  Non-blocking queue ◦ Atomic operation expense  CAS (15-25 clock cycles on P4) ◦ Retry on contention  Non-blocking synchronization ◦ Atomic operation expense  store_conditional ◦ Data structure copy expense

 With interactions between reader, writer and collector, when is it time to reclaim memory? ◦ Writer identifies what to collect and trigger collection to occur (synchronously or asynch) ◦ Readers (indirectly) indicate when to collect by no longer referencing the freed object

 One solution for collector: ◦ Track copies of global pointer into thread-local memory  Each thread maintains a list of it’s currently active pointers ◦ Collector checks the thread-local list prior to memory reclamation  Sounds a lot like the hazard pointer !

 Hazard Pointer Disadvantages: ◦ Required manual identification of hazard references ◦ Expensive on the read path  Requires two memory barriers on the read path  Copy of the global pointer to local reference  Entry of hazard pointer into the list  Every read thread incurs this extra overhead as the cost for correct memory reclamation. Expensive for many-reader situations

 RCU -> Collection based on ‘quiescent state’ ◦ Threads prevent the occurrence of quiescent state while their local memory is alive ◦ Collector indirectly observes state of all threads to infer when safe to reclaim memory ◦ The definition chosen for ‘quiescent state’ will significantly impact performance  Best choice: Infer by operations that occur anyway

 Reader-Collection ◦ rcu_read_lock() ◦ rcu_read_unlock() ◦ read-side critical section  Non-preemptible kernel ◦ Programming convention is to avoid yielding in the read- side critical section ◦ Memory reclamation on voluntary context switch ◦ rcu_read_lock/unlock calls do nothing in non- preemptible kernel rcu_read_lock(); retval = rcu_dereference(gbl_foo)->a; rcu_read_unlock(); return retval;

 ‘Simple case’: Non-preemptible kernel ◦ All threads use read-side critical section with no voluntary yield  no context switch within a read-side critical section ◦ Collector observes all CPU to determine when all threads have undergone a context switch  Indicates a pass into a quiescent state  All previous read-side critical sections are now guaranteed to have exited  Any new threads no longer have visibility to removed object ◦ Safe–conservative-imprecise–degrades real-time  Detection of quiescent state occurs after last reader use  Collector waits for all readers to finish even if not all readers were accessing the memory to be reclaimed  Delay real-time response due to refusal to yield within read-side critical

 Read-side critical section ◦ Readers can now be preempted in their read-side critical ◦ Disable preemption on entry and re-enable on exit  Memory freed using synchronize_sched() ◦ Counts scheduler preemptions  Benefits and trade-offs ◦ Allows use of RCU with preemptible kernel ◦ Read-side critical section won’t be preempted by RT events, negative consequences for RT responsiveness ◦ Additional read-side work to disable/enable preemption

 Global counter ◦ Atomic increment in rcu_read_lock() ◦ Atomic decrement in rcu_read_unlock()  Quiescent state defined as global counter=0  Not practical ◦ As CPU count increases, counter may never reach 0

 Use two-element array as counter ◦ Atomically increment/decrement as matched pair of ‘current’ and ‘last’ counter ◦ Grace period starts – swap sense of ‘current’ and ‘last’, proceed to only decrement the ‘last’ counter ◦ Counter eventually reaches 0, marking end of grace period  High overhead due to memory contention / cache misses

 2xN arrays, N=thread count (2 per thread)  Global index  Updated with rcu_read_lock() and rcu_read_unlock()  Requires a grace- period detection state machine

 Improves read-side performance ◦ Avoids cache-miss ◦ Does not require (expensive) atomic instructions ◦ Does not require (expensive) memory barriers  Requires state-machine for grace period detection

 Indefinite delays in read-side critical sections ◦ Extends grace period ◦ Exhausts memory since no collection can occur ◦ Writers cannot allocate memory ◦ Need to prevent low-priority threads from being indefinitely preempted  Priority boost would work – but relatively expensive and not required for every reader  Solution is to defer priority boosting ◦ Preempted read-side critical threads added to list ◦ List serves as an ‘aging’ tracker

 Issue List

 Global definition of grace period ◦ Single delayed thread in read-side critical section can stall memory reclamation for everyone ◦ Stall occurs even though reader’s data is unrelated to memory trying to be reclaimed  RCU Control Block ◦ Reader/updater invocations share defined control blocks ◦ Readers won’t block reclamation for unrelated control blocks idx = srcu_read_lock(&scb) /* read-side critical */ srcu_read_unlock(&scb, idx) /* collection */ synchronize_srcu(&scb)

Fast concurrent reads Relatively slow writers Preemption & RT support requires increased read-side work