1. The Buffer Cache
Jeff Chase
Duke University

2. The kernel system call layer: file API fault entry: VM page faults
syscall trap/return
fault/return
system call layer: file API
fault entry: VM page faults
memory management: block/page cache
policy
I/O completions
interrupt/return
timer ticks

3. DeFiler interfaces: overview
create, destroy, read, write a dfile
list dfiles
DFS
read(), write()
startFetch(), startPush()
waitValid(), waitClean()
DBuffer dbuf = getBlock(blockID)
releaseBlock(dbuf)
DBufferCache
DBuffer
ioComplete()
startRequest(dbuf, r/w)
VirtualDisk

4. Memory Allocation
How should an OS allocate its memory resources among contending demands?
Virtual address spaces: fork, exec, sbrk, page fault.
The kernel controls how many machine memory frames back the pages of each virtual address space.
The kernel can take memory away from a VAS at any time.
The kernel always gets control if a VAS (or rather a thread running within a VAS) asks for more.
The kernel controls how much machine memory to use as a cache for data blocks whose home is on slow storage.
Policy choices: which pages or blocks to keep in memory? And which ones to evict from memory to make room for others?

5. Memory/storage hierarchy
small and fast
(ns)
registers
caches
L1/L2
Terms to know
cache index/directory
cache line/entry, associativity
cache hit/miss, hit ratio
spatial locality of reference
temporal locality of reference
eviction / replacement
write-through / writeback
dirty/clean
off-core L3
off-chip
main memory (RAM)
big and slow
(ms)
off-module
disk, other storage, network RAM
In general, each layer is a cache over the layer below.
inclusion property
Technology trends  rapid change
The triangle is expanding vertically  bigger gaps, more levels

7. virtual address spaces backing storage volumes
Memory as a cache
Processes access external storage objects through file APIs and VM abstraction. The OS kernel manages caching of pages/blocks in main memory.
virtual address spaces
data
data
files and filesystems,
databases,
other storage objects
disk and other storage
network RAM
memory
(frames)
backing storage volumes
(pages and blocks)
page/block read/write accesses

8. Ritchie and Thompson The UNIX Time-Sharing System, 1974
The Buffer Cache
Memory
File
cache
Proc
Ritchie and Thompson The UNIX Time-Sharing System, 1974

9. Editing Ritchie/Thompson
The system maintains a buffer cache (block cache, file cache) to reduce the number of I/O operations.
Suppose a process makes a system call to access a single byte of a file. UNIX determines the affected disk block, and finds the block if it is resident in the cache. If it is not resident, UNIX allocates a cache buffer and reads the block into the buffer from the disk.
Then, if the op is a write, it replaces the affected byte in the buffer. A buffer with modified data is marked dirty: an entry is made in a list of blocks to be written. The write call may then return. The actual write may not be completed until a later time.
If the op is a read, it picks the requested byte out of the buffer and returns it, leaving the block in the cache.
Memory
File
cache
Proc

10. The DeFiler buffer cache
File abstraction implemented in upper DFS layer.
All knowledge of how files are laid out on disk is at this layer.
Access underlying disk volume through buffer cache API.
Obtain buffers (dbufs), write/read to/from buffers, orchestrate I/O.
read(), write()
startFetch(), startPush()
waitValid(), waitClean()
DBuffer dbuf = getBlock(blockID)
releaseBlock(dbuf)
DBufferCache
DBuffer
Device I/O interface
Asynchronous I/O to/from buffers
block read and write
Blocks numbered by blockIDs

11. Page/block cache internals
HASH(blockID)
Each frame/buffer of memory is described by a meta-object (header).
Resident pages or blocks are accessible through through a global hash table.
An ordered list of eviction candidates winds through the hash chains.
Some frames/buffers are free (no valid data). These are on a free list.

12. DBufferCache internals
HASH(blockID)
Any given block (blockID) is either resident or not. If resident, then it has exactly one copy (dbuf) in the cache. If it is resident then getBlock finds the dbuf (cache hit).
This requires some kind of cache index, e.g., a hash table.
DBuffer dbuf = getBlock(blockID)
DBufferCache
DBuffer
I/O cache buffers
Each is byte[blocksize]
Buffer headers
DBuffer dbuf
There is a one-to-one correspondence of dbufs to buffers.

13. DBufferCache internals
HASH(blockID)
If the requested block is not resident, then getBlock allocates a dbuf for the block and places the correct block contents in its buffer (cache miss). If there are no free dbufs in the cache, then we must evict some other block from the cache and reuse its dbuf.
DBuffer dbuf = getBlock(blockID)
DBufferCache
DBuffer
I/O cache buffers
Each is byte[blocksize]
Buffer headers
DBuffer dbuf
There is a one-to-one correspondence of dbufs to buffers.

14. Page/block cache internals
HASH(blockID)
List(s) of free buffers (bufs) or eviction candidates. These dbufs might be listed in the cache directory if they contain useful data, or not, if they are truly free.
To replace a dbuf
Remove from free/eviction list.
Remove from cache directory.
Change dbuf blockID and status.
Enter in directory w/ new blockID.
Re-register on eviction list.
Beware of concurrent accesses.
cache directory

15. Dbuffer (dbuf) states DFS DBuffer read(…) write(...)
startFetch(), startPush()
waitValid(), waitClean()
A DBuffer dbuf returned by getBlock is always associated with exactly one block in the disk volume. But it might or might not be “in sync” with the underlying disk contents.
DBuffer
A dbuf is valid iff it has the “correct” copy of the data. A dbuf is dirty iff it is valid and has an update (a write) that has not yet been written to disk. A valid dbuf is clean if it is not dirty.
Your DeFiler should return only valid data to a client. That may require you to zero the dbuf or fetch data from the disk. Your DeFiler should ensure that all dirty data is eventually pushed to disk.

16. Asynchronous I/O on dbufs
Start I/O on a dbuf by posting it to a producer/consumer queue for service by a device thread.
startFetch(), startPush()
waitValid(), waitClean()
startFetch(), startPush()
Client threads may wait on the dbuf for asynchronous I/O to complete.
DBuffer
waitValid(), waitClean()
ioComplete()
startRequest(dbuf, r/w)
Thread upcalls dbuf ioComplete when I/O operation is done.
Device I/O interface
Async I/O on dbufs
device threads
VirtualDisk

17. startRequest(dbuf, r/w);
More dbuf states
Do not evict a dbuf that is in active use (busy)!
A dbuf is pinned if I/O is in progress, i.e., a disk request has started but not yet completed.
A dbuf is held if DFS obtained a reference to the dbuf from getBlock but has not yet released the dbuf.
DFS
dbuf = getBlock(blockID)
releaseBlock(dbuf)
DBufferCache
DBuffer
startRequest(dbuf, r/w);
ioComplete()
VirtualDisk

18. File system layer (DFS)
create, destroy, read, write a dfile
list dfiles
Allocate blocks to files and file metadata.
Allocate DFileIDs to files.
Track which blockIDs and DFileIDs are free and which are in use.
Maintain a block map “inode” for each file, as metadata stored on disk.
“inode”
read(), write()
startFetch(), startPush()
waitValid(), waitClean()
DBuffer dbuf = getBlock(blockID)
releaseBlock(dbuf)
sync()
DBufferCache
DBuffer

19. A Filesystem On Disk Data 11100010 00101101 10111101 10011010 00110001
sector 0
allocation
bitmap file
sector 1
directory
file
wind: 18
snow: 62
rain: 32
hail: 48
once upo
n a time
/n in a l
and far
far away
, lived th
Data

20. A Filesystem On Disk Metadata 11100010 00101101 10111101 10011010
sector 0
allocation
bitmap file
sector 1
directory
file
wind: 18
snow: 62
rain: 32
hail: 48
once upo
n a time
/n in a l
and far
far away
, lived th
Metadata

21. Managing files DBufferCache DBuffer
create, destroy, read, write a dfile
list dfiles
Each file has a size: it is the first byte offset in the file that has never been written. Never return data past a file’s size.
Fetch blocks for data and metadata (or zero new ones fresh), read and write in place, and push dirty blocks back to the disk.
Serialize DFS read/write on each inode.
“inode”
read(), write()
startFetch(), startPush()
waitValid(), waitClean()
DBuffer dbuf = getBlock(blockID)
releaseBlock(dbuf)
sync()
DBufferCache
DBuffer

22. Representing a File On Disk
file attributes e.g., size
once upo
n a time
/nin a l
logical
block 0
block map
Index by logical block number
maps to a blockID
and far
far away
,/nlived t
logical
block 1
blockID
access blocks through the block cache with getBlock, startFetch, waitValid, read,
releaseBlock.
he wise and sage
wizard.
logical
block 2
“inode”

23. Filesystem layout on disk
inode 0
bitmap file
inode 1
root directory
fixed locations on disk
inode 1
root directory
allocation
bitmap file
blocks
wind: 18
snow: 62
rain: 32
hail: 48
once upo
n a time
/n in a l
file blocks
and far
far away
, lived th
inode
This is a toy example (Nachos).

24. Filesystem layout on diskX
inode 0
bitmap file
inode 1
root directory X
DeFiler must be able to find all valid inodes on disk. X X
rain: 32
hail: 48
once upo
n a time
/n in a l
Your DeFiler volume is small. You can keep the free block/inode maps in memory. You don’t need metadata structures on disk for that. But you have to scan the disk to rebuild the in-memory structures on initialization.
file blocks
DeFiler has no directories.
You just need to keep track of which DFileIDs are currently valid, and return a list.
and far
far away
, lived th
inode

25. Disk layout: the easy way
DeFiler must be able to find all valid inodes on disk.
Given a list of valid inodes,
you can determine which inodes and blocks are free and which are in use.
once upo
n a time
/n in a l
file blocks
and far
far away
, lived th
inode