1. Chapter 12: I/O Systems

2. Chapter 12: I/O Systems Overview I/O Hardware
Application I/O Interface
Kernel I/O Subsystem
Transforming I/O Requests to Hardware Operations
STREAMS
Performance

3. Objectives
Explore the structure of an operating system’s I/O subsystem
Discuss the principles of I/O hardware and its complexity
Provide details of the performance aspects of I/O hardware and software

4. Overview
I/O management is a major component of operating system design and operation
Important aspect of computer operation
I/O devices vary greatly
Various methods to control them
Performance management
New types of devices frequent
Ports, busses, device controllers connect to various devices
Device drivers encapsulate device details
Present uniform device-access interface to I/O subsystem

5. Network Devices
Because the performance and addressing characteristics of network I/O differ significantly from those of disk I/O, most operating systems provide a network I/O interface
Linux, Unix, Windows and many others include socket interface
Separates network protocol from network operation
Includes select() functionality
A call to select() returns information about which sockets have a packet waiting to be received and which sockets have room to accept a packet to be sent
Approaches vary widely for interprocess communication and network communication
UNIX provides half-duplex pipes, full-duplex FIFOs, full-duplex STREAMS, message queues, and sockets

6. Clocks and Timers Provide current time, elapsed time, timer
Normal resolution about 1/60 second
Some systems provide higher-resolution timers
Programmable interval timer used for timings, periodic interrupts
ioctl() (on UNIX) covers odd aspects of I/O such as clocks and timers
High-performance event timer (HPET) runs at rates in the 10-megahertz range.
It has several comparators that can be set to trigger once or repeatedly when the value they hold matches that of the HPET.
The trigger generates an interrupt, and the operating system's clock management routines determine what the timer was for and what action to take.
The precision of triggers is limited by the resolution of the timer, together with the overhead of maintaining virtual clocks.

7. Nonblocking and Asynchronous I/O
Blocking - process suspended until I/O completed
Easy to use and understand
Insufficient for some needs
Nonblocking - I/O call returns as much as available
User interface, that receives keyboard and mouse input while processing and displaying data on the screen.
video application that reads frames from a file on disk while simultaneously decompressing and displaying the output on the display
Implemented via multi-threading
Returns quickly with count of bytes read or written
select() to find if data ready then read() or write() to transfer
Asynchronous call – returns immediately, without waiting for the I/O to complete
process runs while I/O executes
I/O subsystem signals process when I/O completed
Examples: Secondary storage device and network I/O

8. Two I/O Methods
Synchronous
Asynchronous

9. Vectored I/O
Vectored I/O allows one system call to perform multiple I/O operations
For example, Unix readv() accepts a vector of multiple buffers to read into or write from
This scatter-gather method better than multiple individual I/O calls
Decreases context switching and system call overhead
Some versions provide atomicity assuring that all the I/O is done without interruption (and avoiding corruption of data if other threads are also performing I/O involving those buffers).
When possible, programmers make use of scatter–gather I/O features to increase throughput and decrease system overhead.

10. Kernel I/O Subsystem Kernels provide many services related to I/O.
Scheduling, buffering, caching, spooling, device reservation, and error handling—are provided by the kernel's I/O subsystem and build on the hardware and device-driver infrastructure.
The I/O subsystem is also responsible for protecting itself from errant processes and malicious users
I/O Scheduling
The I/O scheduler rearranges the order of the queue of I/O requests for a device to improve the overall system efficiency and the average response time experienced by applications.
OS may try fairness, or it may give priority service for delay-sensitive requests, e.g. virtual memory may get priority over application
Some implement Quality Of Service (i.e. IPQOS)
When a kernel supports asynchronous I/O, it keeps track of many I/O requests through device-status table.
Each table entry indicates the device's type, address, and state (not functioning, idle, or busy).

11. Device-status Table

12. Kernel I/O Subsystem
Buffering - store data in memory while transferring between two devices or between a device and an application
#1: To cope with device speed mismatch between the producer and consumer of a data stream
File received via Internet for storage on an SSD, the network speed may be a thousand times slower than the drive.
Double buffering – two buffers to decouple the producer of data from the consumer, thus relaxing timing requirements between them
In the above example, after the network fills the first buffer, the drive write is requested.
The network then starts to fill the second buffer while the first buffer is written to storage.
#2: To cope with device transfer size mismatch, e.g. fragmentation and reassembly of messages into small network packets

13. Common PC and data-center I/O device and interface speeds

14. Kernel I/O Subsystem Buffering (cont’d)
#3: To support “copy semantics” for application I/O
The version of the data written to disk is guaranteed to be the version at the time of the application system call, independent of any subsequent changes in the application's buffer
write() system call copies the application data into a kernel buffer before returning control to the application.
The disk write is performed from the kernel buffer, so that subsequent changes to the application buffer have no effect
The same effect can be achieved more efficiently by clever use of virtual memory mapping and copy-on-write page protection

15. Kernel I/O Subsystem
Caching – region of fast memory holding copy of data
Access to the cached copy is more efficient than access to the original, e.g. instructions of currently running process
A buffer may hold the only existing copy of a data item, whereas a cache holds a copy of an item that resides elsewhere.
Sometimes a region of memory can be used for both caching and buffering
Spooling – buffer that holds output for a device, e.g. printer
Each application's output is spooled to a separate secondary storage file.
The spooling system copies the queued spool files to the printer one at a time.
Device reservation - provides exclusive access to a device
Some OS’s (VMS) provide system calls for allocation and de-allocation of an idle device
Other OS’s enforce a limit of one open file handle to such a device
Up to the application to avoid deadlock

16. Kernel I/O Subsystem Error Handling
OS can recover from disk read, device unavailable, transient write failures
a read(), write() retry, and a network send() error results in a resend()
Some systems more advanced – Solaris FMA, AIX
Track error frequencies, stop using device with increasing frequency of retry-able errors
Most return an error number or code when I/O request fails
System error logs hold problem reports

17. Kernel I/O Subsystem I/O Protection
User process may accidentally or purposefully attempt to disrupt normal operation via illegal I/O instructions
All I/O instructions defined to be privileged
I/O must be performed via system calls
Memory-mapped and I/O port memory locations must be protected from user access by the memory-protection system
A kernel cannot simply deny all user access.
Most graphics games and video editing and playback software need direct access to memory-mapped graphics controller memory to speed the performance of the graphics
The kernel can provide a locking mechanism to allow a section of graphics memory (representing a window on screen) to be allocated to one process at a time.

18. Use of a System Call to Perform I/O

19. Kernel I/O Subsystem Kernel Data Structures
Kernel keeps state info for I/O components through in-kernel data structures, including open file tables, network connections, character-device communication
Many, many complex data structures to track buffers, memory allocation, “dirty” blocks
UNIX provides file-system access to a variety of entities, such as user files, raw devices, and the address spaces of processes.
Some use object-oriented methods and message passing to implement I/O
Windows uses message passing
An I/O request is converted into a message that is sent through the kernel to the I/O manager and then to the device driver, each of which may change the message contents.
The message-passing approach can add overhead, by comparison with procedural techniques that use shared data structures,
but it simplifies the structure and design of the I/O system and adds flexibility

20. UNIX I/O Kernel Structure

21. Kernel I/O Subsystem Power Management
Not strictly domain of I/O, but much is I/O related
Computers and devices use electricity, generate heat, frequently require cooling
OSes can help manage and improve use
Cloud computing environments move virtual machines between servers
Can end up evacuating whole systems and shutting them down
Mobile computing has power management as first class OS aspect

22. Power Management (Cont.)
For example, Android implements
Component-level power management
Understands relationship between components
Build device tree representing physical device topology
System bus -> I/O subsystem -> {flash, USB storage}
Device driver tracks state of device, whether in use
Unused component – turn it off
All devices in tree branch unused – turn off branch
Wakelocks – like other locks but prevent sleep of device when lock is held
Power collapse – put a device into very deep sleep
Marginal power use
Only awake enough to respond to external stimuli (button press, incoming call)

23. I/O Requests to Hardware Operations
Consider reading a file from disk for a process:
Determine device holding file
Translate name to device representation
Physically read data from disk into buffer
Make data available to requesting process
Return control to process
How is the connection made from the file name to the disk controller?
In MS DOS file name, preceding colon, a string identifies the hardware device, e.g. C: is mapped to a specific port address through a device table.
Because of the colon separator, the device name space is separate from the file-system name space.
In UNIX, the device name space is incorporated in the regular file-system name space
Names can be used to access the devices themselves or to access the files stored on the devices.

24. I/O Requests to Hardware Operations
UNIX has a mount table that associates prefixes of path names with specific device names.
This device name has the form of a name in the file-system name space, a <major, minor> device number.
The major device number identifies a device driver that should be called to handle I/O to this device.
The minor device number is passed to the device driver to index into a device table.
The corresponding device-table entry gives the port address or the memory-mapped address of the device controller.

25. Life Cycle of An I/O Request

26. STREAMS
STREAM – a full-duplex communication channel between a user-level process and a device driver in Unix System V and beyond
A STREAM consists of:
STREAM head interfaces with the user process
driver end interfaces with the device
zero or more STREAM modules between them
Each module contains a pair of queues – a read queue and a write queue
Message passing is used to communicate between queues
Flow control option to indicate available or busy
Asynchronous internally, synchronous where user process communicates with stream head
Most UNIX variants support STREAMS, and it is the preferred method for writing protocols and device drivers.
System V UNIX and Solaris implement the socket mechanism using STREAMS.

27. The STREAMS Structure

28. Performance I/O a major factor in system performance:
Demands CPU to execute device driver, kernel I/O code and to schedule processes fairly and efficiently as they block and unblock.
Context switches due to interrupts stress the CPU and its hardware caches
Data copying
I/O loads down the memory bus during data copies between controllers and physical memory and again during copies between kernel buffers and application data space
Interrupt handling is a relatively expensive task.
Each interrupt causes the system to perform a state change, to execute the interrupt handler, and then to restore state.
Network traffic can also cause a high context-switch rate
E.g. a remote login from one machine to another (next slide)

29. Intercomputer Communications

30. Intercomputer Communications
Front-end processors for terminal I/O to reduce the interrupt burden on the main CPU.
A terminal concentrator can multiplex the traffic from hundreds of remote terminals into one port on a large computer.
An I/O channel is a dedicated, special-purpose CPU found in mainframes and in other high-end systems.
The channels keep the data flowing smoothly, while the main CPU remains free to process the data.

31. Improving Performance
Reduce number of context switches
Reduce data copying in memory while passing between device and application
Reduce interrupts by using large transfers, smart controllers, polling (if busy waiting can be minimized)
Increase concurrency by using DMA-knowledgeable controllers or channels to offload simple data copying from the CPU
Move processing primitives into hardware, to allow their operation in device controllers to be concurrent with CPU and bus operation
Balance CPU, memory, bus, and I/O performance for highest throughput
Move user-mode processes / daemons to kernel threads

32. Device-Functionality Progression
Where should the I/O functionality be implemented—in the device hardware, in the device driver, or in application software?
Initially at application level
Flexible, bugs unlikely to cause system crashes
Inefficient – context switches, kernel functionality not exploited
Reimplement in kernel
In hardware, either in the device or in the controller

33. I/O performance of storage (and network latency)
I/O devices have been increasing in speed.
Nonvolatile memory devices are growing in popularity and in the variety of devices available.
The speed of NVM devices varies from high to extraordinary, with next-generation devices nearing the speed of DRAM.
Increasing pressure on I/O subsystems and operating system algorithms to take advantage of the read/write speeds now available.

34. End of Chapter 12