Solving the Starting Problem Device Drivers as Self-Describing Artifacts Michael Spear(Rochester) Tom Roeder(Cornell) Galen Hunt(Microsoft) Orion Hodson(Microsoft) Steven Levi(Microsoft)

2 June 2015Solving the Starting Problem2 The Driver Conspiracy Theory Kernel developers write some device drivers therefore driver developers should be smart –Imperative driver code implicitly declares all the resources it will use We hide this information during compilation Then we lament its absence when things fail –Drivers eschew safe programming languages to gain performance –Driver code is written rarely, and it is tedious But, drivers cause too many system failures

2 June 2015Solving the Starting Problem3 What Makes a Driver Dependable? It is bug free (or its bugs are isolated) –If drivers can contain bugs, they must be isolated from the kernel and applications It is configured correctly –The kernel must know exactly which resources the driver uses –The system must be able to verify driver configuration without execution

2 June 2015Solving the Starting Problem4 Outline Singularity Overview Self-Describing Drivers Benefits Experience Related Work Conclusions

2 June 2015Solving the Starting Problem5 Singularity Overview Dependability is a design-time consideration –Fix and verify system behavior as early as possible in code lifecycle Application isolation is not just a run-time issue Design Time Post Mortem Compile Time Install Time Load Time Run Time errors are less expensive when they are found early

2 June 2015Solving the Starting Problem6 Singularity Implementation Microkernel OS written in C# / Sing# –Strong type safety and memory safety –Compiled to native code / no JIT Emphasis on increasing static verification –Verify early, verify often

2 June 2015Solving the Starting Problem7 Applications Applications are composed of one or more processes –Process protection via language safety, not hardware –All inter-process communication (IPC) through channels Application configuration is declarative –Application manifests describe dependencies, resources, and code –System manifest describes configuration of applications and system policies Declarative configuration enables off-line (static) analysis –You shouldn’t have to run an application to determine if it is configured correctly

2 June 2015Solving the Starting Problem8 Drivers Singularity drivers are a distinct category of application Drivers do more than regular applications –Access hardware –Abstract hardware-specific functionality Drivers do less than regular applications –Require limited functionality –Follow simple design patterns Drivers do not run in the kernel –In practice, driver developers are not kernel developers –so, we don’t trust them!

2 June 2015Solving the Starting Problem9 Outline Singularity Overview Self-Describing Drivers Benefits Experience Related Work Conclusions

2 June 2015Solving the Starting Problem10 Drivers in Singularity Drivers are processes –Isolated through language protection, not hardware –Communicate with applications through channels OS enforces strict limits on drivers –Access to limited set of libraries –No file system or namespace access Only drivers can access objects in driver runtime –I/O ports, interrupt vectors, DMA channels, I/O memory regions

2 June 2015Solving the Starting Problem11 Driver Installation Programmer delivers bundle of MSIL (bytecode) files Driver verified against system policy Driver is statically linked to trusted libraries Whole-program analysis/optimization Native instruction code generation Driver MSIL Driver MSIL Driver MSIL System Policy Driver MSIL Trusted Libraries Native Executable Bartok Compiler Verification

2 June 2015Solving the Starting Problem12 Self-Describing = Code + Metadata MSIL format affords convenient and verifiable representation of declarative metadata –Resources are annotated where they are declared –No separate configuration files are needed –Metadata is strongly typed

2 June 2015Solving the Starting Problem13 Driver Configuration Object [DriverCategory] [Signature("/pci/03/00/5333/8811")] class S3Trio64Config : DriverCategoryDeclaration { [IoMemoryRange(0, Length = 0x400000)] IoMemoryRange frameBuffer; [IoFixedMemoryRange(Base = 0xb8000, Length = 0x8000)] IoMemoryRange textBuffer;... [IoFixedPortRange(Base = 0x3c0, Length = 0x20)] IoPortRange control; [ServiceEndpoint(typeof(VideoDeviceContract.Exp))] TRef video;... } requires PCI Device requires 4MB frame buffer (declared in PCI config) requires system console buffer exports channel for clients to access video device requires VGA I/O ports

2 June 2015Solving the Starting Problem14 The Metadata Lifecycle Annotations are: –Type-checked at compile time –Verified at install time Require annotations on object declarations from driver runtime Metadata in driver must match physical hardware –Extracted to form the driver manifest used at driver activation –Declarative and general to any use of the driver For example, all PCI drivers declare the required count / type of I/O objects, minimum length of range The burden on the programmer is minimal, but the kernel is empowered to manage resources proactively

2 June 2015Solving the Starting Problem15 Outline Singularity Overview Self-Describing Drivers Benefits Experience Related Work Conclusions

2 June 2015Solving the Starting Problem16 Eliminates /etc/init.d Booting and activating drivers is a dynamic process –At install time, driver metadata is merged into a system-wide repository –At boot time, the repository and a simple resolution algorithm are sufficient to determine a valid boot order –Process is resistant to failures / device-not-found errors –Same mechanism used for hot-pluggable drivers and applications

2 June 2015Solving the Starting Problem17 Automates Resource Allocation Driver resource needs are known in advance –Conflicts are prevented at install time –Unsafe resource remapping is not possible –Resources provided automatically to the driver Reasoning about drivers can be done off-line –Snapshot of manifests, copy of system policy, and a list of target physical devices are all that is needed

2 June 2015Solving the Starting Problem18 Simplifies Programming All resource requirements are declared and annotated in the driver configuration object The kernel prepares all resources before activating the driver The code to instantiate the driver configuration object is automatically generated at install time

2 June 2015Solving the Starting Problem19 Protects Hardware Access The kernel only starts the driver code after –The declared resources are allocated –The resources are transferred to the driver –The trusted I/O objects are initialized Methods on trusted I/O objects –Validate bounds and type of access –Directly access hardware using privileged instructions

2 June 2015Solving the Starting Problem20 Outline Singularity Overview Self-Describing Drivers Benefits Experience Related Work Conclusions

2 June 2015Solving the Starting Problem21 9 Months, No Problems Metadata is sufficiently expressive for ~20 drivers we’ve written –7 device types: audio, video, ide, network, keyboard, timers, interrupt controllers –3 buses: pci, pnp, lpc –Memory mapped and programmed I/O devices

2 June 2015Solving the Starting Problem22 Safety We Can Afford Performance on par with Windows, Linux, FreeBSD

2 June 2015Solving the Starting Problem23 Related Work Static detection of bugs in driver code –Meta Compilation [Engler et al. OSDI ‘00], Slam [Ball et al. POPL ‘02], ESP [Das et al. PLDI ‘02] Runtime containment of bugs in driver code –SPIN [Bershad et al. SOSP ‘95], Nooks [Swift et al. OSDI ‘04] Xen [Barham et al. SOSP ‘03, Fraser et al. OASIS ‘04] L4 [Härtig et al. SOSP ’97], Minix3 [Herder et al. ‘06] Static detection of driver/device communication bugs –DEVIL [Mérillon et al. OSDI ‘00], HAIL [Sun et al. EMSOFT ‘05] Declarative configuration in other domains –Vesta [Heydon et al. ‘01], [DeTreville HOTOS ‘05]

2 June 2015Solving the Starting Problem24 Contributions Metadata allows flexible inference of total boot order Declaring / verifying I/O requirements affords better isolation –Singularity isolates drivers at design time –Drivers carry their resource requirements at all times –OS never starts a driver with unfilled resource needs Configuration can be checked off-line through manifests

Discussion We thank the rest of the Singularity Team: Mark Aiken, Paul Barham, Michael Carbin, Manuel Fähndrich, Chris Hawblitzel, James Larus, Nick Murphy, Bjarne Steensgaard, David Tarditi, and Brian Zill