Scheduler Activations: Effective Kernel Support for the User-Level Management of Parallelism Thomas E. Anderson, Brian N. Bershad, Edward D. Lazowska, and Henry M. Levy. Presented by Shiyan Tao

Scheduler Activations Threads Vehicle for concurrency in many approaches to parallel programming. User-Level Threads Kernel-Level Threads 2019/4/11 Scheduler Activations

User-Level Threads - pros high performance No kernel intervention. Managed by a Runtime library routine linked to each application. inexpensive operation. Thread scheduling happens in user space. flexible No kernel modification. Different application can be linked with different user-level thread libraries. 2019/4/11 Scheduler Activations

User-Level Threads – cons But… Poor Integration The thread package views each process as a virtual processor. These virtual processors are multiplexed across a fixed number of processors by the underlying kernel. If one thread blocks on a system call, such as I/O or page faults, the whole process is blocked even if there are runnable threads. Kernel support is needed. 2019/4/11 Scheduler Activations

Scheduler Activations Kernel-Level Threads high functionality. Kernel directly schedules each application’s threads onto physical processors. Avoid system integration problems. Kernel schedules thread from same or other address space. poor performance Kernel threads use system calls to perform operations.Even switching between threads in the same address space requires a full context switch. lack of flexibility Kernel thread system must support the need of every application that runs it. 2019/4/11 Scheduler Activations

User-level Threads on top of Kernel-Level threads? “Kernel threads are the wrong abstraction for supporting user-level thread management.” The kernel is unaware of user-level threads. Kernel threads block, resume, and are preempted without notification to user level . Kernel threads are scheduled obliviously with respect to the user-level thread state. 2019/4/11 Scheduler Activations

Scheduler Activations Integration Issues A kernel thread could be preempted while its user-level thread is holding a spin-lock. --Spin-wait then. A kernel thread could be preempted to allow another kernel thread to run that happens to be idling in its user-level threads. A kernel thread running high-priority user-level threads may be preempted in favor of a kernel thread that happen to be running low-priority user-level threads. When a user-level thread block, its kernel thread blocks too. can run out of kernel threads even when there are runnable threads and processors are idle. 2019/4/11 Scheduler Activations

Scheduler Activations Goal In the common case when kernel intervention is not needed. do as well as user-level. In the infrequent case when kernel must be involved: No processor idles when thread ready. No higher-priority thread waits while a lower-priority thread runs. During a thread trap (page fault), the processor on which the thread was running can be used to run another thread from the same or a different address space. Allow application-specific customization. 2019/4/11 Scheduler Activations

Approach: Scheduler Activation Structure to provide communication between the kernel-level scheduler and the user-level thread system. Serve as a vessel for running user-level threads, in exactly the same way that a kernel thread does; Vector control from the kernel to the user-level thread system on kernel event. It’s essentially an upcall which invokes a handler function in the user-level thread library. One scheduler activation per processor assigned to address space. 2019/4/11 Scheduler Activations

Scheduler Activations The kernel provides each process with its own virtual multiprocessor. But kernel may change the number of processors during the execution. Kernel allocates processors to address spaces. User level threads system has complete control over scheduling-which thread run on which allocated processor. Kernel notifies user-level thread system whenever it changes the number of processors assigned to it or whenever a user-level thread blocks or wakes up in the kernel. User-level thread system notifies kernel when application needs more or fewer processors. The kernel uses this information to allocate processors among address space. 2019/4/11 Scheduler Activations

Scheduler Activations, cont’d Once a scheduler activation’s user-level thread is stopped by the kernel, that activation is done. A new activation will be created to notify the user-level thread system that it has been stopped. The user-level system removes the state of that thread from the old activation, tell the kernel that the old activation can be reused, and finally decide which thread to run on the processor. V.S. Traditional kernel threads Kernel never notifies the user-level of the event. 2019/4/11 Scheduler Activations

Example of the use of scheduler activation T1: The application receives two processors to run on from the kernel. It starts running threads on them. T2: One of the user-level threads blocks in the kernel. Kernel uses a fresh scheduler activation to notify the user-level system of this. T3: The blocking-event completes. Kernel preempts second processor to do the upcall. Upcall notifies user level of upcall and of event completion. T4: Upcall takes the thread from the ready list and begins running it. The first two scheduler activations have now been discarded and replaced. 2019/4/11 Scheduler Activations

Processor Reallocations Goal: No idle processor in presence of runnable threads. Processor allocation based on available parallelism in each address space. Kernel notified when: User-level has more runnable threads than processors User-level has more processors than runnable threads Kernel uses notifications as hints for its actual processor allocation. 2019/4/11 Scheduler Activations

Scheduler Activations Critical Sections When an upcall informs the user-level thread system that a thread has been preempted which happens to be executing in its critical section. make a copy of each low-level critical section. At the end of the copy, we place code to yield the processor back to the upcall. If a preemption occurs, continue the thread at the corresponding place in the copy of the critical section. The copy relinquishes control back to the original upcall at the end of the critical section. 2019/4/11 Scheduler Activations

Scheduler Activations Implementation Modified Topaz, the native OS for DEC SRC Firefly multiprocessor workstation, to implement scheduler activations. Modified FastThreads, a user-level thread system originally designed to run on top of Topaz kernel threads, to handle the user-level thread system. User-level applications are free to choose any thread scheduling policy they like. Discarded scheduler activations can be collected and returned to the kernel for reuse, avoiding the overhead of recreating them. 2019/4/11 Scheduler Activations

Scheduler Activations Performance with I/O 2019/4/11 Scheduler Activations

Scheduler Activations Performance w/o I/O 2019/4/11 Scheduler Activations

Scheduler Activations Summary User-level threads divide up the processor without the kernel’s knowledge. Fast and flexible but degrade when I/O and other kernel activities get in the way. Kernel level threads Slow and expensive to use. Managing threads at the user-level is needed to achieve high performance. But kernel threads or processes do not support this well. Scheduler activations provide an interface between the kernel and the user-level thread package. Kernel is responsible for processor allocation and notifying the user-level of events that affect it. User-level is responsible for thread scheduling and notifies the kernel of events that affect processor allocation decisions. 2019/4/11 Scheduler Activations