1. CS 443 Advanced OS Fabián E. Bustamante, Spring 2005 The Performance of µ-Kernel-Based Systems H. Härtig, M. Hohmuth, J. Liedtke, S. Schönberg and J. Wolter Dresden U. and IBM TJ Watson Appears in SOSP 1997 Presented by: Fabián E. Bustamante

2. 2 Customizability Research in OS Goal –Provide flexible mechanisms and policies to clients –Different context – different goals for customization (perf., power consumption, conflicting client needs, …) Design issues –Performance – while sometimes introduced to improve perf., customizability mechanisms have a cost –Spread – throughout the OS or just part of it (such as the scheduling policy) –Granularity – all aspects of a policy or in some restricted manner (such as a menu of options scheduling policies) –Integrity – clearly, level and complexity of what needs to be enforced is related to both granularity and spread –Purpose – special- or general-purpose OS? –Paradigm – a traditional, posix-like OS or something new (such as Scout data-paths)

3. 3 Taxonomy A high-level taxonomy* – criteria –Initiator of adaptation Human admin or OS designer Application OS itself –Time of adaptation Design, build or installation (static) Boot or run-time (dynamic) *Denys, Piessens and Matthijs, “A Survey of Customizability in Operating Systems Research”, ACM Computing Surveys, 34(4):450-468, Dec. 2002. HumanApplicationOS StaticOSKit, Choices, Scout, Pebble components (portability through conditional compilation) DynamicProduction OSsExokernels, L4, SPIN, Vino Synthetic

4. 4 Historical Context Kernel –The mandatory part of an OS –Can use all features of a processor Most early OS were monolithic –Complete OS was packed into a single kernel (scheduling, file system, memory management,…) Microkernel approach –Minimize kernel, implementing servers on top –Ideally, only ASs, IPC & basic scheduling is left inside

5. 5 Software Technology Advantages Different APIs, FS, even different OS strategies can coexists Flexibility and extensibility Server malfunction is isolated, even device drivers Clean µk i/f enforces a more modular system structure Smaller kernel is easier to maintain and less prone to error Smaller Trusted Computing Base (hw, µk, disk drive and maybe the FS)

6. 6 First Generation & Evaluation Criteria Conceptual breakthrough to µks –Mach’s external pager – k manages physical and virtual memory; forwards page faults to user-level task –Handling hardware interrupts as IPCs & include I/O ports into AS – k captures the interrupt but doesn’t handle it (interrupt handling and device I/O done outside k) Appealing yes, but is it useful? Are the concept flexible & costs low enough? –Critical mechanism for efficiency - IPC –Stabilized in Mach 3 – a Mach RPC 10x Conventional Unix Conclusion – the layer of abstraction provided by µks is either –too low - look at extensible-kernel ideas, figure out how to protect kernels from misbehaved extensions (SPIN, Vino, etc) or –too high – exterminate all abstractions; OS’ two jobs – protection (safely multiplex physical resources) and provide an abstract machine - forget the latter (exokernel)

7. 7 Observation Problem is measurements –The system you measure – legacy problem with 1 st generation µks –How you measure it and what conclusions you draw - is it the approach, the concepts implemented by a particular µk or the implementation of the µk? Evaluation criteria then –Conservative criterion – applications must not be degraded by µk - compatibility performance –Progressive criterion - µk must efficiently support new types of apps w/ good performance - extensibility performance Approach –A monolithic Unix kernel, Linux, was adapted to run on top of L4 (as a user-level single server) –Performance was compared to MkLinux, a third-party port of Linux to a Mach 3.0 derived µk (first generation) –Different application, generic and specific were implemented and evaluated –Performance portability checked w/ an Alpha-port of L4

8. 8 L4 Essentials Based on 2 basic concepts: –Threads – An activity executing inside an AS –AS – a mapping which associates each virtual page to a physical page frame Recursive construction of ASs –By magic there’s a one AS σ 0 –µk provides 3 operations Grant – owner of an AS can grant any page to another AS Map – owner can map any page into another AS, now both can access it Flush/demap/… - onwer can flush any of its pages w/o warning All AS are constructed and maintained by user-level servers – pagers –Threads can dynamically associate individual pagers with themselves I/O ports are treated as parts of ASs Hardware interrupts are handled as IPC Pentium-specific – small-AS (<512MB) can be physically shared & protected by Pentium’s segment mechanism

9. 9 Linux on Top of L4 No-tuned port of Linux 2.0 Goal – full binary compatibility Double page tables (inside the kernel for security, and in the Linux server for portability) Initial space (physical memory) Linux Server User process

10. 10 Source-code Lines Lines of codeLinux/x86L4Linux Architecture2,5006,500 (rewritten) Dependent (reuse)2,000 Part4,5008,500 Linux kernel105,000 Drivers232,700

11. 11 Compatibility Performance What’s the penalty of using L4Linux instead of native Linux –Run benchmarks on both in the same hardware Does the performance of the underlying µk matter? –Compare L4Linux to MkLinux, both in user-level and as an extension SystemTimeCycles Linux1.68microsec223 L4Linux3.95microsec526 L4Linux (trampoline)5.66microsec753 MkLinux in-kernel15.41microsec2050 MkLinux user110.60microsec14710 Microbenchmark – getpid system call costs

12. 12 More Microbenchmark Results

13. 13 Macrobenchmarks Compilation time for the Linux server

14. 14 More Macrobenchmarks Uses the shared libc.so so it avoids overhead of trampoline

15. 15 More Macrobenchmarks For native Linux – maximum load of 130 jobs per minute

16. 16 Extensibility Performance E.g. Application-level Pipes – shows that the pipe performance can be increased drastically when using a user-level pipe implementing using µk facilities –Linux – standard pipe provided by Linux, running native –L4Linux – same on L4Linux w/ shared library –L4 … app-level – runs on bare L4 –Restricted – synchronous L4 RPC (blocking IPC w/o buffering) SystemLatencyBandwidth Linux29microsec41MB/s L4Linux46microsec40MB/s L4Linux, app-level Unix pipe 22microsec48-70MB/s L4Linux, app-level restricted pipe 5microsec65-105MB/s

17. 17 Extensibility Performance Cache partitioning –Run a 64x64-matrix multiplication –Interrupted by a synthetic load that maximizes cache conflicts –Uninterrupted – 10.9ms –Interrupted every 100 microsec, worst-case 96.1 ms –Using cache-partitioning, enabled by user-level pagers, allocating 3 secondary-cache pages (of 64) – worst-case 24.9ms

18. 18 Conclusion L4Linux ~ native Linux – 5-10% penalty Collocation on its own is not sufficient to fix performance problems What about L4 w/ collocation? Pipes and some VM operations used as examples of improve Unix-compatible functionality Real-time memory management example shows the advantage to coexisting systems based on other paradigms

19. 19 General Conclusions Applying the performance criterion to such complex systems is complex – naïve, uninterpreted measurements are sometimes misleading –Early µk measurement suggested reducing the IPC cost, but the real problem was the structure and implementation of kernels Although steady evolution is a powerful methodology, sometimes a radical approach is needed –Most problems of 1 st generation µk were caused by their step-by-step development