The Performance of Micro-Kernel- Based Systems H. Haertig, M. Hohmuth, J. Liedtke, S. Schoenberg, J. Wolter Presentation by: Tim Hamilton

The L4 Micro-Kernel Contains only 13 system calls. 7 of them are for IPC. Uses only 12k of memory Based on two basic concepts: threads and address spaces IPC is one of the most fundamental mechanisms. Moved Paging out of the kernel into user-space.

Address Spaces L4 uses "flexpages“: logical memory ranging from one physical page up to a complete address space. Flexpages can be granted, mapped, and unmapped An invoker can only map and unmap pages that have been mapped into its own address space

Address Spaces (cont.) Address spaces are built and maintained by user-level processes: Pagers. Threads can associate different pagers with themselves. If a page fault occurs, the L4 kernel sends it to the pager associated with the faulting thread.

I/O as Address Space I/O ports treated as address space As address space, can be mapped and unmapped Hardware interrupts are handled by user- level processes. The L4 kernel will send a message via IPC.

Linux to L4 Some portions of Linux are architecture dependant: Interrupts Low-level device drivers (DMA) Methods for interaction with use processes. Switching between kernel contexts Copyin/copyout between kernel and user space Mapping/unmapping for address space System calls

The Linux Server L4Linux will use a single-server approach. A single Linux server will run on top of L4, multiplexing a single thread for system calls and page faults. The Linux server maps physical memory into its address space, and acts as the pager for any user processes it creates. The Server cannot directly access the hardware page tables, and must maintain logical pages in its own address space.

Interrupt Handling All interrupt handlers are mapped to messages. The Linux server contains threads that do nothing but wait for interrupt messages. Interrupt threads have a higher priority than the main thread.

User Processes Each different user process is implemented as a different L4 task: Has it’s own address space and threads. The Linux Server is the pager for these processes. Any fault by the user-level processes is sent by RPC from the L4 kernel to the Server.

System Calls Three system call interfaces: 1. A modified version of libc.so that uses L4 primitives. 2. A modified version of libc.a 3. A user-level exception handler (trampoline) calls the corresponding routine in the modified shared library. The first two options are the fastest. The third is maintained for compatibility.

Signalling Each user-level process has an additional thread for signal handling. Main server thread cannot directly manipulate user process stack, so it sends a message for the signal handling thread, telling the user thread to save it’s state and enter Linux

Scheduling All thread scheduling is down by the L4 kernel The Linux server’s schedule() routine is only used for multiplexing it’s single thread. After each system call, if no other system call is pending, it simply resumes the user process thread and sleeps.

Small Memory Spaces In order to reduce TLB conflicts, L4Linux has a special library to customize code and data for communicating with the Linux Server The emulation library and signal thread are mapped close to the application, instead of default high-memory area.

Performance What is the penalty of using L4Linux? Compare L4Linux to native Linux Does the performance of the underlying micro-kernel matter? Compare L4Linux to MkLinux Does co-location improve performance? Compare L4Linux to an in-kernel version of MkLinux

Microbenchmarks Measures specific system calls to determine basic performance.

Macrobenchmarks First benchmark was to recompile the Linux Server. Next use a commercial test suite to simulate a system under full load.

Performance Analysis L4Linux is, on average 8.3% slower than native Linux. Only 6.8% slower at maximum load. MkLinux: 49% average, 60% at maximum. Co-located MkLinux: 29% average, 37% at maximum.

Extensibility Performance A micro-kernel must provide more than just the features of the OS running on top of it. Specialization – improved implementation of Os functionality Extensibility – permits implementation of new services that cannot be easily added to a conventional OS.

Pipes and RPC First five (1) use the standard pipe mechanism of the Linux kernel. (2) Is asynchronous and uses only L4 IPC primitives. Emulates POSIX standard pipes, without signalling. Added thread for buffering and cross-address-space communication. (3) Is synchronous and uses (4) Maps pages into the receiver’s address space.

Virtual Memory Operations The “Fault” operation is an example of extensibility – measures the time to resolve a page fault by a user-defined pager in a separate address space. “Trap” – Latency between a write operation to a protected page, and the invocation of related exception handler. “Appel1” – Time to access a random protected page. The fault handler unprotects the page, protects some other page, and resumes. “Appel2” – Time to access a random protected page where the fault handler only unprotects the page and resumes.

Conclusion Using the L4 micro-kernel imposes a 5-10% slowdown to native Linux. Much faster than previous micro-kernels. Further optimizations such as co-locating the Linux Server, and providing extensibility could improve L4Linux even further.

Currently in L4 Original L4/x86 microkernel was not portable. Other versions include L4/MIPS, L4/Alpha Rewritten in C++ by University of Karlsruhe, Germany: highly-portable, 64-bit support, called Pistachio. Pistachio now used for nearly all research. Gnu HURD project using Pistachio.