Scheduler Activations: Effective Kernel Support for the User-level Management of Parallelism Thomas E. Anderson, Brian N. Bershad, Edward D. Lazowska, and Henry M. Levy Presenter: Yi Qiao

Outline Introduction User-level threads: advantages and limitations Effective kernel support for user-level management of parallelism Implementation Performance Summary

Introduction Effectiveness of parallel computing –Largely depends on the performance and cost of primitives used to express and control the parallelism within programs Shared memory between multiple processes –Better for uniprocessor environment Use of threads –Separate the notion of a sequential execution stream from other aspects such as address spaces and I/O descriptors –A significant performance advantage over traditional processes

Problem with Threads User-level threads –Execute within the context of traditional processes Thread management requires no kernel intervention Flexible, easily customized without kernel modification Each process – virtual processor –Multiprogramming, I/O, page faults can lead to poor performance or incorrect behavior of user-level threads Kernel-level threads –Avoids system integration problem Directly mapped onto physical processor –Too heavyweight An order of magnitude worse than the best performance user-level threads

Goal of the Work A kernel interface and a user-level thread package that combine the functionality of kernel threads and the performance and flexibility of user-level threads –When no kernel intervention needed, same performance as best user-level thread –When kernel needs to be involved, mimic a kernel thread management system No idle processors No high priority thread waits for low-priority ones Trap of a thread won’t block others –Simple and easy application-specific cutomization Challenge –Necessary control and scheduling information is distributed between the kernel and application address space

Approach Each application provided with a virtual multiprocessor, and control which of its threads run on these processors The OS kernel control the allocation of processors among address spaces Kernel notifies the address space scheduler of relevant kernel event –Scheduler activation Vectors control to the thread scheduler on a kernel event Thread system notified kernel of user-level thread events that affect processor allocation decisions –Thread scheduler Execute user-level threads Make requests to the kernel

User-level Threads: Performance Advantages and Functionality Limitations Inherent cost in kernel threads management –Accessing thread management operations Kernel trap, parameter copy and checking –Cost of generality A single underlying implementation used by all applications User-level threads improve both performance and flexibility

User-level Threads: Performance Advantages and Functionality Limitations (Cont.) Poor integration of user-level threads on kernel interface –Kernel threads are wrong abstraction of supporting user-level systems Kernel threads block, resume and preempted without notification to user level Kernel threads are scheduled obliviously to user-level thread state Cause problems both for uniprogrammed systems and multiprogrammed systems –I/O –Page faults

Effective Kernel Support for the User- level Management of Parallelism A new kernel interface + user-level thread system –Functionality of kernel threads –Performance and flexibility of user-level threads –Each user-level thread system is provided with its own virtual multiprocessor, the abstraction of dedicated physical machine Kernel allocates processors to address spaces – complete control Each user-level thread system has complete control over which threads to run on allocated processors Kernel vectors events to appropriate thread scheduler –# of processors change, I/O, page fault User-level thread system notified kernel when needed –Only a subset of user-level operations which may affect processor allocation Application programmer does the same thing as if programming with kernel threads –Programmers provided with a normal Topaz thread interface

Explicit Vectoring of Kernel Events to the User-level Thread Scheduler Scheduler Activation –Each vectored event causes the user-level thread system to reconsider its scheduling decision –Three roles Serves as a vessel (execution context) for running user- level threads Notify the user-level thread of a kernel event Saving processor context of the activation’s current user- level thread when the thread is stopped by the kernel (I/O or processor preemption) –Similar data structure as a traditional kernel thread

Scheduler Activation (Cont.) Distinction between scheduler activations and kernel threads –Once an activation’s user-level thread is stopped by the kernel, the thread is never directly resumed by the kernel –Maintains the invariant that there are always as many running scheduler activations as processors assigned to the address space Events are vectored where a scheduling decision needs to be made

Example: I/O Request/Completion T1: Two processors allocated by kernel, two upcalls T2: Thread 1 blocks in the kernel, another upcall T3: I/O completes, preempts one processor and do the upcall T4: The upcall takes a thread off the ready list and run it Same mechanism to reallocate a processor from one address space to another

Scheduler Activations (Cont.) Reallocate a processor from one address space to another (multiprogramming) –Stop the old activation, use the processor to do an upcall into the new address space with a new activation –Need a second processor in old address space for an upcall here, notifying stop of two use-level threads Some minor points –If threads have priorities, an additional preemption may be needed –Application is free to build any other concurrency model on top of scheduler activations –Sometimes a user-level thread blocked in the kernel may need to execute further in kernel mode when the I/O completes

Notifying the Kernel of User-level Events Only a small subset of user-level events that affect the kernel processor allocation decision need to be notified –Transition to the state where the address space has more runnable threads than processors –Transition to the state where the address space has more processors than runnable threads How to keep applications honest?

Critical Sections Block or preempt a user-level thread in a critical section –Poor performance –Deadlock Solution –Prevention Requires kernel to yield control over processor allocation to the user-level –Recovery The thread system checks if the thread was executing in a critical section –If so, continue temporarily via a user-level context switch –Then another context switch and relinquished control back to the original upcall

Implementation Modifying Topaz –Change the Topaz thread management routines to implement scheduler activations –Explicit allocation of processors to address spaces Modifying FastThreads –Process upcalls and provide Topaz with information related to processor allocation decisions A few handred lines of code added to FastThreads, 1200 lines to Topaz

Implementation (Cont.) Processor Allocation Policy –Processors divided evenly among highest priority address spaces –Then are divided evenly among the remainder –Time-sliced only if the available processors is not an integer multiple of the number of address spaces that want them –Possible to for an address space to use kernel threads instead of scheduler activations Binary compatibility with existing Topaz applications Thread Scheduling Policy –Application can choose any scheduling policy Default: per-processor ready lists following FIFO

Implementation (Cont.) Performance Enhancements –Critical Sections – need to check whether the preempted user-level thread has a lock Thread set a flag when entering a critical section and clear it when finish –Overhead + latency Making a copy of every low-level critical section with post-processing of complier-generated code, and continues the preempted thread at the copy of the critical section –No overhead on lock latency in the common case –Management of scheduler activations Caching discarded scheduler activations for later reuse

Performance Goal: Combining the functionality of kernel threads with the performance and flexibility of user-level threads Evaluation questions –What is the cost of user-level thread operations? Fork, block –What is the cost of communication between kernel and the user level? –What is the overall effect on the performance of applications?

Performance (Cont.) Thread Performance –Cost of user-level thread operations close to those of the FastThreads package Preserve the order of magnitude advantage over kernel threads Upcall Performance –Help determine the “break-even” point to outperform kernel threads –Two user-level threads signal and wait through the kernel 2.4 milliseconds, five times worse than Topaz threads –Built as a quick modification to existing Topaz thread system –Written in Modula-2+, much slower than assembler Production scheduler activation could be faster

Application Performance Compare Topaz kernel threads, FastThreads, and FastThreads on top of scheduler activations –Application An O(N logN) solution to the N-body problem Can be either compute or I/O bound –Memory used by the application can be controlled All tests run on a six processor CVAX Firefly

Application Performance Case 1- Application makes minimal use of kernel services –Enough memory, negligible I/O and no other applications Run as fast as original FastThreads –1 processor, all perform worse than sequential implementation –More processors, kernel threads prevent good performance –Slight divergence of FastThreads and new FastThreads for 4 or 5 processors

Application Performance Case 2 – Kernel involvement required for I/O purposes –New FastThreads performs best When less and less memory available, all three systems degrade fast –Old FastThreads is the worst one »When a user-level thread blocks, the kernel thread also blocks –New FastThreads and Topaz threads can overlap I/O with useful computation

Application Performance Case 3 – Multiprogramming environment –Two copies of the N-body application on the six processors Speedup of new FastThread is within 5% of uniprogramming environment with 3 processors Old FastThread and Topaz perform much worse –Old FastThread - Physical processors idling waiting for a lock to be released while the lock holder is descheduled –Topaz – common thread operations are more expensive Limitation of the experiments –Limited number of processors makes it impossible for large parallel applications or higher multiprogramming levels

Conclusion Scheduler Activation – a kernel interface that combines with user-level thread package –Achieves the performance of user-level threads (in the common case) with the functionality of kernel threads (correct behavior for infrequent case) –Responsibility division Kernel –Processor allocation –Kernel event notification Application address space –Thread scheduling –Subset of user-level events affecting processor allocation decisions –Any user-level concurrency model can be supported