1. Oregon Health and Science University
CSE 513 Introduction to Operating Systems Class 8 - Input/Output File Systems
Jonathan Walpole
Dept. of Comp. Sci. and Eng.
Oregon Health and Science University

2. I/O devices Device (mechanical hardware)
Device controller (electrical hardware)
Device driver (software)
Separte I/O memory space
Memory mapped i/o
Hybrid
Memory mapped takes part of memory for moving I/O around. Has nothing to do with DMA.

3. Devices and their controllers
Monitor
Bus
Components of a simple personal computer

4. How to communicate with a device?
Hardware supports I/O ports or memory mapped I/O for accessing device controller registers and buffers

5. Wide performance range for I/O

6. Performance challenges: I/O hardware
How to prevent slow devices from slowing down memory
How to identify I/O addresses without interfering with memory performance

7. Single vs. Dual Memory Bus Architecture
(a) A single-bus architecture
(b) A dual-bus memory architecture

8. Hardware view of Pentium
Structure of a large Pentium system

9. Performance challenges: I/O software
How to prevent CPU throughput from being limited by I/O device speed (for slow devices)
How to prevent I/O throughput from being limited by CPU speed (for fast devices)
How to achieve good utilization of CPU and I/O devices
How to meet the real-time requirements of devices

10. Steps in printing a string
Programmed I/O
Steps in printing a string

11. Programmed I/O Polling/busy-waiting approach
copy_from_user(buffer,p,count);
for(i=0;i<count;i++){
while (*p_stat_reg != READY);
*p_data_reg = p[i]; }
return();

12. Interrupt-driven I/O Asynchronous approach
give device data, do something else!
resume when device interrupts
if (count==0){
unblock_user();
} else {
*p_data_reg = p[i];
count--;
i++; }
ack_interrupt();
return_from_interrupt();
copy_from_user(buffer,p,count);
enable_interrupts();
while (*p_stat_reg != READY);
*p_data_reg=p[0];
scheduler();

13. Interrupt driven I/O
(b)

14. Hardware Support for Interrupts
How interrupts happens. Connections between devices and interrupt controller actually use interrupt lines on the bus rather than dedicated wires

15. DMA Offload all work to a DMA controller
avoids using the CPU to do the transfer
reduces number of interrupts
DMA controller is like a co-processor doing programmed I/O
copy_from_user(buffer,p,count);
set_up_DMA_controller();
scheduler();
ack_interrupt();
unblock_user();
return_from_interrupt();

16. DMA

17. Software engineering-related challenges
How to remove the complexities of I/O handling from application programs
standard I/O APIs (libraries and system calls)
generic categories across different device types
How to support a wide range of device types on a wide range of operating systems
standard interfaces for device drivers
standard/published interfaces for access to kernel facilities

18. Layers of the I/O Software System
I/O Software Layers
Layers of the I/O Software System

19. Interrupt Handlers Interrupt handlers are best hidden
have driver starting an I/O operation block until interrupt notifies of completion
Interrupt procedure does its task
then unblocks driver that started it
Steps must be performed in software after interrupt completed
Save regs not already saved by interrupt hardware
Set up context for interrupt service procedure

20. Interrupt Handlers Set up stack for interrupt service procedure
Ack interrupt controller, reenable interrupts
Copy registers from where saved
Run service procedure
Set up MMU context for process to run next
Load new process' registers
Start running the new process

21. Device Drivers
Communications between drivers and device controllers goes over the bus

22. I/O Software: Device Drivers
Device drivers “connect” devices with the operating system
Typically a nasty assembly-level job
Must deal with hardware changes
Must deal with O.S. changes
Hide as many device-specific details as possible
Device drivers are typically given kernel privileges for efficiency
Can bring down O.S.!
How to provide efficiency and safety???

23. Device-Independent I/O Software Functions
Providing a deice-independent block size
Allocating and releasing dedicate devices
Error reporting
Buffering
Uniform interfacing for device drivers
Functions of the device-independent I/O software

24. Device-Independent I/O Software Interface
(a) Without a standard driver interface
(b) With a standard driver interface

25. Device-Independent I/O Software Buffering
(a) Unbuffered input
(b) Buffering in user space
(c) Buffering in the kernel followed by copying to user space
(d) Double buffering in the kernel

26. Copying Overhead in Network I/O
Networking may involve many copies

27. User-Space I/O Software
Layers of the I/O system and the main functions of each layer

28. Devices as files Before mounting, files on floppy are inaccessible
After mounting floppy on b,
files on floppy are part of file hierarchy

29. Disk Geometry
Disk head, platters, surfaces

30. Physical vs. logical disk geometry
Constant Angular Velocity vs. Constant Linear Velocity

31. Disks
Disk parameters for the original IBM PC floppy disk and a Western Digital WD hard disk

32. CD-ROMs

33. Logical data layout on a CD-ROM
CD-ROM data layout
Logical data layout on a CD-ROM

34. CD-R Structure Cross section of a CD-R disk and laser not to scale
Silver CD-ROM has similar structure
without dye layer
with pitted aluminum layer instead of gold

35. Double sided dual layer DVD

36. Plastic technology CDs DVDs Approximately 650 Mbytes of data
Approximately 74 minutes of audio
DVDs
Many types of formats
DVD-R, DVD-ROM, DVD-Video
Single layer vs. multi-layer
Single sided vs. double sided
Authoring vs. non-authoring

37. Disk Formatting
A disk sector

38. Disk formatting with cylinder skew

39. Disk formatting with interleaving
No interleaving
Single interleaving
Double interleaving

40. Disk scheduling algorithms
Time required to read or write a disk block determined by 3 factors
Seek time
Rotational delay
Actual transfer time
Seek time dominates
Error checking is done by controllers

41. Disk scheduling algorithms
First-come first serve
Shortest seek time first
Scan  back and forth to ends of disk
C-Scan  only one direction
Look  back and forth to last request
C-Look  only one direction

42. Disk scheduling algorithms

43. Error handling complicates scheduling
A disk track with a bad sector
Substituting a spare for the bad sector
Shifting all the sectors to bypass the bad one

44. RAID levels 0 to 2

45. RAID levels 3 to 5

46. Part B File Systems

47. Long-term Information Storage
Must store large amounts of data
Information stored must survive the termination of the process using it
Multiple processes must be able to access the information concurrently

48. File naming and file extensions
Typical file extensions.

49. File Structure Three kinds of file structure byte sequence
record sequence
tree

50. File Types
(a) An executable file (b) An archive

51. File access Sequential access Random access
read all bytes/records from the beginning
cannot jump around, could rewind or back up
convenient when medium was mag tape
Random access
bytes/records read in any order
essential for data base systems
read can be …
move file marker (seek), then read or …
read and then move file marker

52. File attributes

53. File operations Create Append Delete Seek Open Get attributes Close
Read
Write
Append
Seek
Get attributes
Set Attributes
Rename

54. Exmple Program Using File System Calls (1/2)

55. Example Program Using File System Calls (2/2)

56. Memory-mapped files
(a) Segmented process before mapping files into its address space
(b) Process after mapping
existing file abc into one segment
creating new segment for xyz

57. Directories - single level
A single level directory system
contains 4 files
owned by 3 different people, A, B, and C

58. Directories - two-level
Letters indicate owners of the directories and files

59. Hierarchical directory systems

60. Path names in Unix

61. Directory operations Create Readdir Delete Rename Opendir Link
Closedir
Readdir
Rename
Link
Unlink

62. File system implementation on disk
A possible file system disk layout

63. Implementing files - contiguous allocation
(a) Contiguous allocation of disk space for 7 files
(b) State of the disk after files D and E have been removed

64. Implementing files - linked allocation
Storing a file as a linked list of disk blocks

65. Implementing Files - FAT
Linked list allocation using a file allocation table (FAT) in RAM

66. Implementing files - index nodes
An example i-node

67. Implementing directories
(a) A simple directory
fixed size entries
disk addresses and attributes in directory entry
(b) Directory in which each entry just refers to an i-node

68. Implementing directories
Two ways of handling long file names in directory
(a) In-line, (b) In a heap

69. Shared files - hard links
File system containing a shared file

70. Problems with shared files
(a) Situation prior to linking
(b) After the link is created
(c)After the original owner removes the file

71. Disk space management and performance
Block size
Dark line (left hand scale) gives data rate of a disk
Dotted line (right hand scale) gives disk space efficiency
All files 2KB

72. Disk space management (a) Storing the free list on a linked list
(b) A bit map

73. Disk Space Management (3)
(a) Almost-full block of pointers to free disk blocks in RAM
- three blocks of pointers on disk
(b) Result of freeing a 3-block file
(c) Alternative strategy for handling 3 free blocks
- shaded entries are pointers to free disk blocks

74. Disk space management - quotas
Quotas for keeping track of each user’s disk use

75. Maintaining File System Consistency
Crashes can cause file system to have corrupted data
First: bitmap vs. linked storage maps
fsck / scandisk
Block-level  ensure that all blocks on disk are accounted for
traverse inodes counting allocated blocks
compare result to free list
File-level  ensure that all files are accounted for
traverse directory system counting files

76. File system consistency
File system states
(a) consistent
(b) missing block
(c) duplicate block in free list
(d) duplicate data block

77. Maintaining File System Consistency
File level consistency
Have directories with i-node #’s of files
Have link status in the i-nodes themselves
Compare!

78. Performance issues Buffer cache
A buffer in between the application and the FS
Hold buffer of accessed data
Allows data to be reused by other apps
Can implement prefetch strategies to minimize latency
Serves a buffer for writing out application data
Delayed writes allow the writes to happen when convenient
Trade-off between consistency and performance
Hash table indexed to minimize the searching

79. File system performance - buffer cache
The buffer cache data structures

80. File system performance - data placement
I-nodes placed at the start of the disk
Disk divided into cylinder groups
each with its own blocks and i-nodes

81. Log-Structured File Systems
With CPUs faster, memory larger
disk caches can also be larger
increasing number of read requests can come from cache
thus, most disk accesses will be writes
LFS Strategy structures entire disk as a log
have all writes initially buffered in memory
periodically write these to the end of the disk log
long contiguous writes to disk
requires large contiguous free space!
data not updated in place
when file opened, locate i-node, then find blocks

82. Example: The Unix file system
File systems structure
The file system in UNIX is an integral part of the operating system. Files are used for:
Terminal handling /dev/tty***
Pipes “ls | more”
Directories
Sockets (for networking)
The design of the UNIX file system allows the user to write code that has a uniform interface. …
Boot block
superblock
i-nodes
data block
data block

83. Unix file system structures
Superblock - tells the UNIX O.S. what the format of the disk is, the # of blocks, the number of i-nodes, etc...
I-nodes - are the metadata data structures for files and directories
Files - holds information such as owner, permissions, date of modification, where the data is
Directories are just a special case of files that have the pairs <file-name, i-node #> stored in the file

84. The Unix name space Files Directories
Represented with an I-node and disk blocks that hold the data
File names are not stored with the i-node (they’re in the directories that refer to them)
This is why there is a link count in the inode
Directories
Directories are also files
Entries are disblocks with <filename, i-node #> pairs
Special directory marking in i-node

85. Unix directory entries
A UNIX V7 directory entry

86. The Unix name space / inode #22 / inode #24 / inode #53 . 22 .. 22
usr 24
var 35
home 26
txt 23
src 38
conf 53
## config.. ##
Frame rate 33
Frame size ..

87. Pathname translation in Unix
The steps in looking up /usr/ast/mbox

88. Unix I-node structure UNIX files consist of i-node and data blocks
UNIX uses an indexed allocation scheme with
10 direct pointers to blocks
1 indirect pointer to blocks
1 double indirect pointer to blocks
1 triple indirect pointer to blocks

89. Unix I-node structrure
A UNIX i-node

90. Maximum file size in Unix
Several parameters determine how many (and of what size) files can be represented
No. of bits in a disk address
No. of bits in a virtual memory reference
Disk block size
Example:
10 direct, 1 indirect, 1 double indirect, 1 triple indirect
No. bits in disk address --> 16-bits
No. of bits in virtual memory reference --> 32-bits
Disk block size --> 1024 kbytes
What is the maximum file size in this system?

91. The “mount” command in Unix
Commands
“mount” allows file systems to be added to the names space
“umount” takes a file systems out of the name space
Mount points
Mount points can be any directory in the name space
Once a file system is mounted, the entire subtree at the mount point is no longer accessible (until the file system is unmounted)

92. Some UNIX file system features
mounting - allows other file systems (disks), whether local or remote to be “mounted” into a uniform file system name space

93. Associating files with processes
To provide uniform access to data, UNIX has a level of indirection that is used for opening files

94. File system vs. virtual memory
Open
File
Table
Disk FS
Process
Swap
Paging

95. A file-centric view of IPC
All input and output in UNIX are handled through the open file table structure
This is how UNIX can provide a single uniform interface for local or remote data access
Pipes in UNIX are nothing more than
A writing process that has its “stdout” linked to a “pipe” file (instead of a /dev/tty*** file)
A reading process that has its “stdin” linked to a “pipe” file (instead of a /dev/tty*** file)