IT 344: Operating Systems Winter 2010 Module 3 Operating System Components and Structure Chia-Chi Teng ccteng@byu.edu 265G CTB 1

“Those who disregard the lessons of history are destined to repeat them.” - George Santayana

Keep up with the reading schedule Quiz1 - Tue Jan 26 (in class & timed) on processes and threads Project #1 demo 4/22/2017

OS structure The OS sits between application programs and the hardware it mediates access and abstracts away ugliness programs request services via exceptions (traps or faults) devices request attention via interrupts P2 P3 P4 P1 dispatch exception OS interrupt D1 start i/o D4 D2 D3 4/22/2017

Application Interface (API) User Apps Firefox Photoshop Acrobat Java Portable Application Interface (API) Operating System File Systems Memory Manager Process Manager Network Support Device Drivers Interrupt Handlers Boot & Init Hardware Abstraction Layer Hardware (CPU, devices) 4/22/2017

Secondary Storage Management Command Interpreter Information Services Error Handling Accounting System File System Protection System Memory Management Secondary Storage Management Process Management I/O System 4/22/2017

Major OS components processes (& threads) memory I/O secondary storage file systems protection accounting shells (command interpreter, or OS UI) GUI networking 4/22/2017

OS Services (What things does the OS do?) Services that (more-or-less) map onto components Program execution (process management) How do you execute concurrent sequences of instructions? I/O operations Standardized interfaces to extremely diverse devices File system manipulation How do you read/write/preserve files? Looming concern: How do you even find files??? Communications Networking protocols/Interface with CyberSpace? Cross-cutting capabilities Error detection & recovery Resource allocation Accounting Protection

Process management An OS executes many kinds of activities: users’ programs batch jobs or scripts system programs print spoolers, name servers, file servers, network daemons, … Each of these activities is encapsulated in a process a process includes the execution context PC, registers, VM, OS resources (e.g., open files), etc… plus the program itself (code and data) the OS’s process module manages these processes creation, destruction, scheduling, … 4/22/2017

Program/processor/process Note that a program is totally passive just bytes on a disk that encode instructions to be run A process is an instance of a program being executed by a (real or virtual) processor at any instant, there may be many processes running copies of the same program (e.g., an editor); each process is separate and (usually) independent Linux: ps -auwwx to list all processes process A process B code stack PC registers code stack PC registers page tables resources page tables resources 4/22/2017

States of a user process running dispatch interrupt ready exception interrupt Examples: Disk IO Plug in USB drive blocked 4/22/2017

Process operations The OS provides the following kinds operations on processes (i.e., the process abstraction interface): create a process delete a process suspend a process resume a process clone a process inter-process communication (IPC) inter-process synchronization create/delete a child process (subprocess) 4/22/2017

Memory management The primary memory (or RAM) is the directly accessed storage for the CPU programs must be stored in memory to execute memory access is fast (e.g., 60 ns to load/store) but memory doesn’t survive power failures OS must: allocate memory space for programs (explicitly and implicitly) deallocate space when needed by rest of system maintain mappings from physical to virtual memory through page tables decide how much memory to allocate to each process a policy decision decide when to remove a process from memory also policy 4/22/2017

I/O A big chunk of the OS kernel deals with I/O hundreds of thousands of lines in Windows The OS provides a standard interface between programs (user or system) and devices file system (disk), sockets (network), frame buffer (video) Device drivers are the routines that interact with specific device types encapsulates device-specific knowledge e.g., how to initialize a device, how to request I/O, how to handle interrupts or errors examples: SCSI device drivers, Ethernet card drivers, video card drivers, sound card drivers, … Note: Windows has > 35,000 device drivers! 4/22/2017

Secondary storage Secondary storage (disk, tape) is persistent memory often magnetic media, survives power failures (hopefully) Routines that interact with disks are typically at a very low level in the OS used by many components (file system, VM, …) handle scheduling of disk operations, head movement, error handling, and often management of space on disks Usually independent of file system although there may be cooperation file system knowledge of device details can help optimize performance e.g., place related files close together on disk 4/22/2017

File systems Secondary storage devices are crude and awkward e.g., “write 4096 byte block to sector 12” File system: a convenient abstraction defines logical objects like files and directories hides details about where on disk files live as well as operations on objects like read and write read/write (logical) byte ranges instead of (physical) blocks A file is the basic unit of long-term storage file = named collection of persistent information A directory is just a special kind of file directory = named file that contains names of other files and metadata about those files (e.g., file size) Note: Sequential byte stream is only one possibility! 4/22/2017

File systems Secondary storage devices are crude and awkward e.g., “write 4096 byte block to sector 12” File system: a convenient abstraction defines logical objects like files and directories hides details about where on disk files live as well as operations on objects like read and write read/write (logical) byte ranges instead of (physical) blocks A file is the basic unit of long-term storage file = named collection of persistent information A directory is just a special kind of file directory = named file that contains names of other files and metadata about those files (e.g., file size) 4/22/2017

File system operations The file system interface defines standard operations: file (or directory) creation and deletion manipulation of files and directories (read, write, extend, rename, protect, …) copy lock File systems also provide higher level services accounting and quotas backup (must be incremental and online!) (sometimes) indexing or search (sometimes) file versioning 4/22/2017

Protection Protection is a general mechanism used throughout the OS all resources needed to be protected memory processes files devices CPU time … protection mechanisms help to detect and contain unintentional errors, as well as preventing malicious destruction 4/22/2017

Command interpreter (shell) A particular program that handles the interpretation of users’ commands and helps to manage processes user input may be from keyboard (command-line interface), from script files, or from the mouse (GUIs) allows users to launch and control new programs On some systems, command interpreter may be a standard part of the OS (e.g., MS DOS, Apple II) On others, it’s just non-privileged code that provides an interface to the user e.g., bash/csh/tcsh/zsh on UNIX On others, there may be no command language e.g., “classic” MacOS (9 and earlier) 4/22/2017

Accounting GUI … Networking … etc. Keeps track of resource usage both to enforce quotas “you’re over your disk space limit” or to produce bills timeshared computers like mainframes or super computers hosted services GUI … Networking … etc. 4/22/2017

Secondary Storage Management OS structure It’s not always clear how to stitch OS modules together: Command Interpreter Information Services Error Handling Accounting System File System Protection System Memory Management Secondary Storage Management Process Management I/O System 4/22/2017

OS structure An OS consists of all of these components, plus: many other components system programs (privileged and non-privileged) e.g., bootstrap code, the init program, … Major issue: how do we organize all this? what are all of the code modules, and where do they exist? how do they cooperate? Massive software engineering and design problem design a large, complex program that: performs well, is reliable, is extensible, is backwards compatible, … 4/22/2017

4/22/2017

Operating Systems Structure What is the organizational principle? Simple Only one or two levels of code Layered Lower levels independent of upper levels Microkernel OS built from many user-level processes Modular Core kernel with Dynamically loadable modules

Simple Structure MS-DOS – written to provide the most functionality in the least space Not divided into modules Interfaces and levels of functionality not well separated

UNIX: Also “Simple” Structure UNIX – limited by hardware functionality Original UNIX operating system consists of two separable parts: Systems programs The kernel Consists of everything below the system-call interface and above the physical hardware Provides the file system, CPU scheduling, memory management, and other operating-system functions; Many interacting functions for one level

Early structure: Monolithic Traditionally, OS’s (like UNIX) were built as a monolithic entity: user programs everything OS hardware 4/22/2017

UNIX System Structure User Mode Kernel Mode Hardware Applications Standard Libs

Monolithic design Major advantage: Disadvantages: cost of module interactions is low (function call) Disadvantages: hard to understand hard to modify unreliable (no isolation between system modules) hard to maintain What is the alternative? find a way to organize the OS in order to simplify its design and implementation 4/22/2017

Layered Structure Operating system is divided many layers (levels) Each built on top of lower layers Bottom layer (layer 0) is hardware Highest layer (layer N) is the user interface Each layer uses functions (operations) and services of only lower-level layers Advantage: modularity  Easier debugging/Maintenance Not always possible: Does process scheduler lie above or below virtual memory layer? Need to reschedule processor while waiting for paging May need to page in information about tasks Important: Machine-dependent vs independent layers Easier migration between platforms Easier evolution of hardware platform Good idea for you as well!

Layered Operating System

Layering The traditional approach is layering implement OS as a set of layers each layer presents an enhanced ‘virtual machine’ to the layer above The first description of this approach was Dijkstra’s THE system Layer 5: Job Managers Execute users’ programs Layer 4: Device Managers Handle devices and provide buffering Layer 3: Console Manager Implements virtual consoles Layer 2: Page Manager Implements virtual memories for each process Layer 1: Kernel Implements a virtual processor for each process Layer 0: Hardware Each layer can be tested and verified independently 4/22/2017

Problems with layering Imposes hierarchical structure but real systems are more complex: file system requires VM services (buffers) VM would like to use files for its backing store strict layering isn’t flexible enough Poor performance each layer crossing has overhead associated with it Disjunction between model and reality systems modeled as layers, but not really built that way 4/22/2017

Hardware Abstraction Layer An example of layering in modern operating systems Goal: separates hardware-specific routines from the “core” OS Provides portability Improves readability Core OS (file system, scheduler, system calls) Hardware Abstraction Layer (device drivers, assembly routines) 4/22/2017

Microkernels Goal: Popular in the late 80’s, early 90’s minimize what goes in kernel organize rest of OS as user-level processes Popular in the late 80’s, early 90’s recent resurgence of popularity. Why? This results in: better reliability (isolation between components) ease of extension, porting and customization poor performance (user/kernel boundary crossings) First microkernel system was Hydra (CMU, 1970) Follow-ons: Mach (CMU), Chorus (French UNIX-like OS), OS X (Apple), in some ways NT (Microsoft) 4/22/2017

Microkernel Structure Moves as much from the kernel into “user” space Small core OS running at kernel level OS Services built from many independent user-level processes Communication between modules with message passing Benefits: Easier to extend a microkernel Easier to port OS to new architectures More reliable (less code is running in kernel mode) Fault Isolation (parts of kernel protected from other parts) More secure Detriments: Performance overhead severe for naïve implementation

Microkernel structure illustrated user mode firefox powerpoint user processes apache networking file system system processes paging thread mgmt scheduling kernel communication processor control microkernel low-level VM protection hardware 4/22/2017

Modules-based Structure Most modern operating systems implement modules Uses object-oriented approach Each core component is separate Each talks to the others over known interfaces Each is loadable as needed within the kernel Overall, similar to layers but with more flexible

Administrivia HW1 past due HW2 due next Tuesday Next week: process and thread Keep up with the reading schedule Project 1 – start now! Demo your product during Lab hours in week #6 Lab feedback 4/22/2017

IT 344 In this class we will learn: Philosophy what are the major components of most OS’s? how are the components structured? what are the most important (common?) interfaces? what policies are typically used in an OS? what algorithms are used to implement policies? Philosophy you may not ever build an OS but as a computer engineer you need to understand the foundations most importantly, operating systems exemplify the sorts of engineering design tradeoffs that you’ll need to make throughout your careers – compromises among and within cost, performance, functionality, complexity, schedule … 4/22/2017