Advanced Operating Systems - Spring 2009 Lecture 19 – Monday March 30, 2009 Dan C. Marinescu Office: HEC 439 B. Office hours: M, Wd 3 – 4:30 PM. TA: Chen Yu Office: HEC 354. Office hours: M, Wd 1.00 – 3:00 PM. 1

Last, Current, Next Lecture Last time: Caching Introduction to I/O Subsystem Today I/O subsystem File System Implementation Next time: File System Interface and Mass Storage Structure 2

Communication among asynchronous sub-systems: polling versus interrupts Polling  periodically checking the status of an I/O device Interrupt  deliver data or status information when status information immediately. Intel Pentium Vector Table 3

Interrupts: used for I/O and for exceptions CPU Interrupt-request line  triggered by I/O device Interrupt handler receives interrupts To mask an interrupt  ignore or delay some interrupts Interrupt vector to dispatch interrupt to correct handler Based on priority Some non-maskable 4

Direct Memory Access (DMA) 5 DMA  Bypasses CPU to transfer data directly between I/O device and memory; it allows subsystems within the computer to access system memory for reading and/or writing independently of CPU: disk controller, graphics cards, network cards, sound cards, GPUs (graphics processors), also used for intra-chip data transfer in multi-core processors,. Avoids programmed I/O for large data movement Requires DMA controller

DMA Transfer 6

Device drivers and I/O system calls Multitude of I/O devices Character-stream or block Sequential or random- access Sharable or dedicated Speed of operation Read-write, read only, or write only Device-driver layer hides differences among I/O controllers from kernel: I/O system calls encapsulate device behaviors in generic classes 7

Block and Character Devices Block devices (e.g., disk drives, tapes) Commands e.g., read, write, seek Raw I/O or file-system access Memory-mapped file access possible Character devices (e.g., keyboards, mice, serial ports) Commands e.g., get, put Libraries allow line editing 8

Network Devices and Timers Network devices Own interface different from bloc or character devices Unix and Windows NT/9x/2000 include socket interface Separates network protocol from network operation Includes select functionality Approaches vary widely (pipes, FIFOs, streams, queues, mailboxes) Timers Provide current time, elapsed time, timer Programmable interval timer  for timings, periodic interrupts ioctl (on UNIX) covers odd aspects of I/O such as clocks and timers 9

Blocking and Nonblocking I/O Blocking  process suspended until I/O completed Easy to use and understand Insufficient for some needs Nonblocking  I/O call returns control to the process immediately User interface, data copy (buffered I/O) Implemented via multi-threading Returns quickly with count of bytes read or written Asynchronous  process runs while I/O executes I/O subsystem signals process when I/O completed 10

Synchronous/Asynchronous I/O Synchronous Asynchronous 11

Kernel I/O Subsystem Scheduling Some I/O request ordering using per-device queue Some OSs try fairness Buffering – store data in memory while transferring to I/O device. To cope with device speed mismatch or transfer size mismatch To maintain “copy semantics” 12

Sun Enterprise 6000 Device-Transfer Rates 13

Kernel I/O Subsystem and Error Handling Caching  fast memory holding copy of data Always just a copy Key to performance Spooling  holds output for a device that can serve only one request at a time (e.g., printer). Device reservation  provides exclusive access to a device System calls for allocation and de-allocation Possibility of deadlock Error handling: OS can recover from disk read, device unavailable, transient write failures When I/O request fails error code. System error logs hold problem reports 14

I/O Protection I/O instructions are priviledged Users make system calls 15

Kernel Data Structures Kernel keeps state info for I/O components, including open file tables, network connections, device control blocs Complex data structures to track buffers, memory allocation, “dirty” blocks Some use object-oriented methods and message passing to implement I/O 16

UNIX I/O Kernel Structure 17

Hardware Operations Operation for reading a file: Determine device holding file Translate name to device representation Physically read data from disk into buffer Make data available to the process Return control to process 18

STREAMS in Unix STREAM  a full-duplex communication channel between a user-level process and a device in Unix System V and beyond A STREAM consists of: - STREAM head interfaces with the user process - driver end interfaces with the device - zero or more STREAM modules between them. Each module contains a read queue and a write queue Message passing is used to communicate between queues 19

STREAMS 20

I/O  major factor in system performance: Execute device driver, kernel I/O code Context switches Data copying Network traffic stressful 21

Improving Performance Reduce number of context switches Reduce data copying Reduce interrupts by using large transfers, smart controllers, polling Use DMA Balance CPU, memory, bus, and I/O performance for highest throughput 22

File System Basic Concepts Allocation Methods Free-Space Management Efficiency and Performance Fault tolerance and recovery NFS 23

Basic concepts File  Collection of related information residing on secondary storage. File control block  data structure summarizing information about a file 24

Open and Read operations 25

Virtual File Systems (VFS) Support an object-oriented implementation of a file systems. The VFS API used for different types of file systems. 26

Directory Implementation Directory  provides information about files on a system or on a physical device. Linear list of file names with pointer to the data blocks. simple to program time-consuming to execute Hash Table – linear list with hash data structure. decreases directory search time collisions – situations where two file names hash to the same location fixed size 27

Physical layout of a file An allocation method refers to how disk blocks are allocated for files: Contiguous allocation  Each file occupies a set of contiguous blocks on the disk Simple – only starting location (block #) and length (number of blocks) required Random access Wasteful of space (dynamic storage-allocation problem) Files cannot grow Linked allocation  Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk. Simple – need only starting address Free-space management system – no waste of space No random access Mapping Indexed allocation  Brings all pointers together into the index block. pointer block = 28

Contiguous Allocation 29 Mapping from logical to physical Block to be accessed = ! + starting address Displacement into block = R LA/512 Q R

Extent-Based Systems Extent  a contiguous block of disks. Extent-based file systems  allocate disk blocks in extents. A file consists of one or more extents. Many newer file systems (I.e. Veritas File System) use a modified contiguous allocation scheme 30

Linked Allocation Block to be accessed is the Qth block in the linked chain of blocks representing the file. Displacement into block = R + 1 File-allocation table (FAT) – disk-space allocation used by MS-DOS and OS/2. LA/511 Q R Maping 31

Indexed Allocation- bring all pointers together index table 32

Indexed Allocation Need index table Random access Dynamic access without external fragmentation, but have overhead of index block. Mapping from logical to physical in a file of maximum size of 256K words and block size of 512 words. We need only 1 block for index table. LA/512 Q R Q = displacement into index table R = displacement into block 33

Indexed Allocation – Mapping Mapping from logical to physical in a file of unbounded length (block size of 512 words). Linked scheme – Link blocks of index table (no limit on size). LA / (512 x 511) Q1Q1 R1R1 Q 1 = block of index table R 1 is used as follows: R 1 / 512 Q2Q2 R2R2 Q 2 = displacement into block of index table R 2 displacement into block of file: 34

Indexed Allocation – Mapping (Cont.) Two-level index (maximum file size is ) LA / (512 x 512) Q1Q1 R1R1 Q 1 = displacement into outer-index R 1 is used as follows: R 1 / 512 Q2Q2 R2R2 Q 2 = displacement into block of index table R 2 displacement into block of file: 35

Indexed Allocation – Mapping (Cont.)  outer-index index table file 36

Combined Allocation Scheme: UNIX (4K bytes per block) 37

Free-Space Management Bit vector (n blocks) … 012n-1 bit[i] =  0  block[i] free 1  block[i] occupied Block number calculation (number of bits per word) * (number of 0-value words) + offset of first 1 bit 38

Free-Space Management (Cont.) Bit map requires extra space Example: block size = 2 12 bytes disk size = 2 30 bytes (1 gigabyte) n = 2 30 /2 12 = 2 18 bits (or 32K bytes) Easy to get contiguous files Linked list (free list) Cannot get contiguous space easily No waste of space Grouping Counting 39

Free-Space Management (Cont.) Need to protect: Pointer to free list Bit map Must be kept on disk Copy in memory and disk may differ Cannot allow for block[i] to have a situation where bit[i] = 1 in memory and bit[i] = 0 on disk Solution: Set bit[i] = 1 in disk Allocate block[i] Set bit[i] = 1 in memory 40

Efficiency and Performance Efficiency dependent on: disk allocation and directory algorithms types of data kept in file’s directory entry Performance disk cache – separate section of main memory for frequently used blocks free-behind and read-ahead – techniques to optimize sequential access improve PC performance by dedicating section of memory as virtual disk, or RAM disk 41

Caching for regular I/O operations and memory mapped I/O A unified buffer cache  uses the same page cache to cache both memory-mapped pages and ordinary file system I/O 42 A page cache  caches pages rather than disk blocks using virtual memory techniques: used by memory-mapped I/O Routine I/O through the file system uses the buffer (disk) cache I/O without a unified buffer cache I/O with a unified buffer cache

Fault -tolerance Recovery Consistency checking – compares data in directory structure with data blocks on disk, and tries to fix inconsistencies Backup/Restore  Use system programs to back up data from disk to another storage device Log Structured File Systems All transactions are written to a log A transaction is considered committed once it is written to the log However, the file system may not yet be updated The transactions in the log are asynchronously written to the file system When the file system is modified, the transaction is removed from the log If the file system crashes, all remaining transactions in the log must still be performed 43