Operating System Architectures Kernel Design

Kernel Architectures MicroKernels Extensible Operating Systems

Microkernels Original concept: due to Per Brinch-Hansen RC 4000 Multiprogramming System, 1969 Poor performance, due partly to the IPC overhead. Nevertheless, the concept remained popular.

Basic Idea OS kernel is very small – supports process management, not much else. Other OS functions provided at user level by trusted servers. Windows, later versions of Mach, are examples of systems that use modified microkernels. L4 is a publicly available, pure microkernel

Major Advantages - Claim Modularity Flexibility (easy to replace modules) Safety (each server is protected by the OS from other servers) Correctness (easier to verify a small kernel)

Major Disadvantages of Early Microkernels Slow – based on “cross-domain” information transfers. Server-to-OS, OS-to-server communication involves mode switches. Much faster to communicate between two modules that are both in OS Other problems: cost of creating new processes whenever a server is activated, …

Can MicroKernels Perform Well? Jochen Liedtke argued that microkernels can give the same level of performance as a traditional OS, and conducted experiments using his μ-kernel (L4) to prove his claim. Liedtke argued that 1st generation μ-kernel’s just weren’t small enough.

L4 Highly tuned, small, very efficient Kernel implements threads, address spaces, very efficient thread communication, and simple scheduling. Other operating systems can be configured to run as servers on top of L4

Extensible Operating Systems L4 and other microkernels popularized small kernels and the idea of OS flexibility Extensible operating systems build on this idea

Hierarchy of Communication Mechanisms (Fastest to Slowest) Procedure calls within same process Switch between user-level threads System calls (mode switch) Switch between kernel-level threads Shared memory between processes Context Switch (process switch) IPC between processes (message passing, on the same or different machines)

Extensible Operating Systems Specializing General Purpose Operating Systems

Extensible Operating Systems Motivation Exokernel SPIN

Motivation for New OS Designs Desire to make operating systems more responsive to applications’ needs; e.g., Do your own scheduling Choose your own page replacement algorithm Similar to micro-kernel motivation

Traditional General Purpose OS Provides various abstractions of system resources (virtual address spaces, processes, files, etc.) Provides general-purpose resource management algorithms Able to handle most applications, but handles no applications perfectly because there’s no easy way to specialize the OS to individual needs.

Extensibility Microkernels provide one kind of extensibility, SPIN and exokernel architectures take somewhat different approaches. Exokernel: a very small kernel allocates raw resources to applications. Resource management is done by untrusted user level code, in the form of library operating systems. SPIN provides a basic set of services; applications modify/extend the services by linking code into the kernel (extension model).

“Extensibility, Safety and Performance in the SPIN Operating System” Brian N. Bershad, et. al.

SPIN Motivation “SPIN is motivated by the need to support applications that present demands poorly matched by an operating system’s implementation or interface.” Implementation: what the system does – its functionality Interface: how an application accesses the system functionality.

Implementation and Interfaces OS implementation decisions (file system design, page replacement algorithm, etc.) may not be a good fit for some applications Interface design limits how we can use the functionality of the operating system.

SPIN’s Goal To build an operating system that provides extensibility, safety, and performance. Extensibility: the ability of an application to change/extend basic system services, Safety: but no application can interfere with any other application Performance: and it can’t be slow - guarantee low overhead communication between OS and application

Goals Extensibility comes from letting app.’s access low-level kernel primitives and compose them into larger abstractions. Modify/replace The kernel provides fine-grained interface to system services. Something that would be a large module in a traditional OS (e.g. the paging system) is broken down into smaller modules – each with its own interface. (See figure 3 in the paper.)

Safety Safety is provided by a trusted compiler. Written in Modula-3, the compiler is trusted to generate “safe” code. It allows one module to access another only through the defined interface. It provides “type safety”, which among other things always checks for array indexing errors and prevents pointers from referencing objects of the wrong type

Performance Applications modify the system by writing extensions, code that “changes the way in which a system provides service”. Extensions are linked directly to kernel code when they are needed. Because they run in kernel space, communication between OS code and extension code is fast – no system calls

Fundamental Techniques The success of SPIN is based in large part on the following techniques, implemented by the language and its runtime environment: Co-location Enforced modularity Logical protection domains Dynamic call binding

Co-location Refers to the linking of extension code and system code, which results in low overhead communication This process is done dynamically – an extension is linked in only when the application needs it.

Enforced Modularity Extensions are written as Modula-3 modules with well-defined interfaces enforced by the compiler. Access to memory or privileged instructions only through explicitly granted interfaces. Compiler-enforced modularity ensures low cost isolation between modules.

Logical Protection Domains Protection domain: an abstract notion to provide access control on resources, both hardware and software. A protection domain specifies (by name) the resources a process may access. SPIN domains contain code & exported interfaces to system resources/core services

Protection Domains An extension exists within a specific domain and can reference anything in the domain A dynamic linker in the kernel links code in separate domains with proper permission Once this is done, domains can share resources at memory speed.

Protection Domains & Capabilities Capability: “an unforgeable reference to a resource” – think of it as a “key” When a process is granted access to a resource it receives a capability SPIN implements capabilities as pointers the compiler can prevent them from being forged, or dereferenced incorrectly. this is done at compile-time, which is fast

Dynamic Call Binding Events activate the execution of an extension (think of interrupt processing or exception processing) Applications are allowed to define handlers for the event. Procedures that process the event. Extensions are registered with a central dispatcher and can be executed with the overhead of a procedure call.

SPIN versus Other Systems The authors claim that microkernels are also extensible, but that they are too slow (“kernel crossings”). So – less incentive to provide fine-grained interaction with the kernel. They also claim that compared to other systems that allow insertions into the kernel SPIN is more flexible and safer.

Summary SPIN enables extensibility without compromising safety or performance SPIN provides a set of core services for managing system resources Extensions allow applications to modify or replace the core services Extension are supported by co-location, enforced modularity, logical protection domains, and dynamic call binding.

Dawson R. Engler, M. Frans Kaashoek, and James O’Toole Jr; 1995. Exokernels ”Exokernel: An Operating System Architecture for Application-Level Resource Management” By Dawson R. Engler, M. Frans Kaashoek, and James O’Toole Jr; 1995.

Exokernel “Exokernel” is not the name of a particular operating system – it designates a type of operating system architecture, much like a microkernel is an OS architecture. An exokernel-based system consists of a very small kernel (the exokernel) and one or more library operating systems. The paper describes a proof-of-concept exokernel called Aegis and ExOS, a library operating system.

Exokernel Motivation Motivated by the belief that the OS should not provide high-level abstractions upon which applications are built Instead, the OS should manage physical resources and allow applications to develop abstractions which match the requirements of the application Ex: The authors cite measurements showing that application-controlled file caching can reduce run time by up to 45%.

Premise & Goal The basic premise of exokernel architectures is “the lower the level of a primitive, the more efficiently it can be implemented”. Its goal is to separate resource protection from resource management. The goal is achieved by providing kernel interfaces at the lowest possible level; if possible, directly to the hardware.

Goal Statement In another paper, Engler et.al. state that “The exokernel’s only function is to allocate, deallocate, and multiplex physical resources in a secure way. Resource access is controlled through 64-bit capabilities. The resources include physical memory (in pages), the CPU (in time-slices), the TLB, context-identifiers, and disk memory (divided into blocks)”.

Basic Principles Resource protection is the function of the kernel, and is applied to raw resources (e.g. a section of physical memory) rather than to an abstraction (e.g., a virtual address space). Resource management can be left to the application (untrusted) The exokernel will forward exceptions (e.g., a page fault) to the application for processing. Compare to SPIN event handling

Claim: OS use of high-level abstractions hides info that applications could use to manage their own resources. For example: Database systems may be forced to build random-access files on top of the OS file system instead of being able to interact directly with the disk using raw I/O statements Claim: a large, unmodifiable OS can’t easily incorporate new research developments.

Library Operating Systems Untrusted - Built on top of an exokernel Can be tailored to a set of applications with similar requirements. Run in user space – fewer system calls to the exokernel => enhanced performance. Applications can interact with a library OS or interface directly to the exokernel. Possible to provide a standard interface (e.g. POSIX) for a familiar look.

Design Principles The interface between the exokernel and library OS’s is defined by the exokernel according to these principles Securely expose hardware Expose allocation Expose names Expose revocation In this context “expose” means to make visible to the application or library OS that interfaces to the kernel

Design Principle 1: Expose Hardware The kernel aims to export very low-level hardware resources such as privileged instructions, as well as more abstract resources such as interrupts. The library OS must be free to use the resources as it desires. Exported resources are accessed via system calls that verify the caller’s permission to use the resource.

Design Principle 2: Expose Allocation The library OS should be allowed to ask for specific physical resources. For example access time to files that are always used together may be improved by storing them close together on the disk. If the library OS can request specific disk blocks, it may be able to get better performance.

Design Principle 3: Expose Names Physical names are things like page numbers and disk block addresses. It is more efficient to have the physical address so no translation is needed. Names of system data structures which contain info such as disk arm positions or TLB contents should also be made available to the application level.

Design Principle 4: Expose Revocation Provide details under which resources are revoked, so library OS can manage its resources efficiently. When a page fault occurs the library OS (not the kernel) will decide which page to replace. Since it knows the actual physical names of pages, it can do this easily.

Secure Bindings “…decouple authorization from the actual use of a resource.” Secure bindings are implemented in such a way that the kernel can perform access- checking quickly when the access is performed. Binding takes place when the resource is allocated Access checking is thus quick – e.g., a capability

Summary Exokernels are responsible for resource allocation/protection, user applications are responsible for resource management. Exokernel primitives can be implemented efficiently because they are simple Low-level multiplexing of hardware is fast. Library OSs enable applications to create appropriate abstractions. Secure bindings provide protections

End-to-end Argument The end-to-end argument* argues that (high-level) applications know their own needs best. Services provided by low-level system components (the OS, for example) are often expensive and redundant (they will need to be done again, correctly, at a higher-level) Exokernel authors use this argument to support their claim that OS should give applications almost total control over high-level abstractions. SPIN has a similar philosophy. “End-To-End Arguments In System Design”, by Jerome H. Saltzer, David P. Reed, David D. Clark, ACM Transactions on Computer Systems, Nov., 1984.

Questions Think about similarities and differences between SPIN and Exokernel architectures. Basic philosophy Motivation Approach