1. Buffer Cache

2. UNIX Kernel Architecture
libraries
User level
Kernel level
User programs
hardware
Hardware level
trap
Sytem call interface
File subsytem
Buffer cache
Chracter block
Device drivers
Hardware control
Inter-process
communication
scheduler
Memory
management
Process
control
sybsystem

3. Buffer Cache
When a process wants to access data from a file, the kernel brings the data into main memory, alters it and then request to save in the file system
Example: copy cp one.c two.c
To increase the response time and throughput, the kernel minimizes the frequency of disk access by keeping a pool of internal data buffer called buffer cache.

4. Buffer Cache
Buffer cache contains the data in recently used disk blocks
When reading data from disk, the kernel attempts to read from buffer cache.
If data is already in the buffer cache, the kernel does not need to read from disk
If data is not in the buffer cache, the kernel reads the data from disk and cache it

5. Buffer Headers A buffer consists of two parts
a memory array
buffer header
disk block : buffer = 1 : 1
Figure 3.1 Buffer Header
device num
block num
status
ptr to next buf on hash queue
ptr to previous buf on hash queue
ptr to next buf on free list
ptr to previous buf on free list
ptr to data area

6. Buffer Headers device num block num status
logical file system number
block num
block number of the data on disk
status
The buffer is currently locked.
The buffer contains valid data.
delayed-write
The kernel is currently reading or writing the contents of the disk.
A process is currently waiting for the buffer to become free.
kernel identifies the buffer content by examing device num and block num.

7. Buffer Headers struct buffer_head{ /*First cache line: */
struct buffer_head *b_next; /*Hash queue list*/
unsigned long b_blocknr; /*block number*/
unsigned long b_size; /*block size*/
kdev_t b_dev; /*device(B_FREE = free)*/
kdev_t b_rdev; /*Read device*/
unsigned long b_rsector; /*Real Buffer location on disk*/
struct buffer_head *b_this_page; /*circular list of buffers in one page*/
unsigned long b_state; /*buffer state bitmap(see above)*/
struct buffer_head *b_next_free;
unsigned int b_count; /*users using this block*/
char *b_data; /*pointer to data block(1024 bytes)*/
unsigned int b_list; /*List that this buffer appears*/
unsigned long b_flushtime; /*Time when this(dirty) buffer should be written*/
struct wait_queue *b_wait;
struct buffer_head **b_pprev; /*doubly linked list of hash-queue*/
struct buffer_head *b_prev_free; /* double linked list of buffers*/
struct buffer_head *b_reqnext; /*request queue*/
/*I/O completion*/
void (*b_end_io)(struct buffer_head *bh, int uptodate);
void *b_dev_id; };

8. Buffer Headers /* buffer head state bits*/
#define BH_Uptodate 0 /*1 if the buffer contains valid data*/
#define BH_Dirty 1 /*1 if the buffer is dirty*/
#define BH_Lock 2 /*1 if the buffer is locked*/
#define BH_Req 3 /*0 if the buffer has been invalidated*/
#define BH_Protected 6 /*1 if the buffer is protected*/

9. Structures of the buffer pool
Buffer pool according to LRU
The kernel maintains a free list of buffer
doubly linked list
take a buffer from the head of the free list.
When returning a buffer, attaches the buffer to the tail.
free list
head
buf 1
buf 2
buf n
Forward ptrs
Back ptrs
Figure 3.2 Free list of Buffers

10. Structures of the buffer pool
When the kernel accesses a disk block
separate queue (doublely linked circular list)
hashed as a function of the device and block num
Every disk block exists on one and only on hash queue and only once on the queue 4 5 17 10 50 98 99 35 3 28 64 97
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
Hash queue headers
Figure 3.3 Buffers on the Hash Queues

11. Scenarios for retrieval of a buffer
Determine the logical device num and block num
The algorithms for reading and writing disk blocks use the algorithm getblk
The kernel finds the block on its hash queue
The buffer is free.
The buffer is currently busy.
The kernel cannot find the block on the hash queue
The kernel allocates a buffer from the free list.
In attempting to allocate a buffer from the free list, finds a buffer on the free list that has been marked “delayed write”.
The free list of buffers is empty.

12. Retrieval of a Buffer:1st Scenario (a)
The kernel finds the block on the hash queue and its buffer is free 4 5 17 10 50 98 99 35 3 28 64 97
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
Hash queue headers
freelist header
Search for block 4

13. Retrieval of a Buffer:1st Scenario (b)4 5 17 10 50 98 99 35 3 28 64 97
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
freelist header
Remove block 4 from free list

14. Retrieval of a Buffer: 2nd Scenario (a)
The kernel cannot find the block on the hash queue, so it allocates a buffer from free list 4 5 17 10 50 98 99 35 3 28 64 97
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
Hash queue headers
freelist header
Search for block 18: Not in cache

15. Retrieval of a Buffer: 2nd Scenario (b)
Hash queue headers 18 4 5 17 10 50 98 99 35 28 64 97
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
freelist header
Remove 1st block from free list: Assign to 18

16. Retrieval of a Buffer: 3rd Scenario (a)
The kernel cannot find the block on the hash queue, and finds delayed write buffers on hash queue 4 5 17 10 50 98 99 35 3 28 64 97
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
Hash queue headers
freelist header
delay
delay
Search for block 18, Delayed write blocks on free list

17. Retrieval of a Buffer: 3rd Scenario (b)5 17 10 50 98 99 35 3 28 64 97
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
Hash queue headers
freelist header 18
writing
(b) Writing Blocks 3, 5, Reassign 4 to 18
Figure 3.8

18. Retrieval of a Buffer: 4th Scenario
The kernel cannot find the buffer on the hash queue, and the free list is empty 28
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
Hash queue headers
freelist header 4 64 5 97 17 10 50 98 99 35 3
Search for block 18, free list empty

19. Retrieval of a Buffer: 5th Scenario
Kernel finds the buffer on hash queue, but it is currently busy 4 5 17 10 50 98 99 35 3 28 64 97
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
Hash queue headers
freelist header
busy
Search for block 99, block busy

20. Algorithm: GetBlock GetBlock (file_system_no,block_no)
while (buffer not found)
if (buffer in hash queue)
if (buffer busy)
sleep (event buffer becomes free)
continue
mark buffer busy
remove buffer from free list
return buffer
else
if (there is no buffer on free list)
sleep (event any buffer becomes free)
if (buffer marked as delayed write)
asyschronous white buffer to disk
remove buffer from hash queue
put buffer onto hash queue

21. Algorithm: ReleaseBlock
ReleaseBlock (locked buffer)
wakeup all process event, waiting for any buffer to become free
wakeup all process event, waiting for this buffer to become free
raise processor execution level to block interrupt
if (buffer content valid and buffer not old)
enqueue buffer at the end of free list
else
enqueue buffer at the beginning of free list
lower processor execution level to allow interrupt
unlock (buffer)

22. Reading and Writing disk blocks
To read block ahead
The kernel checks if the block is in the cache or not.
If the block in not in the cache, it invokes the disk driver to read the block.
The the process goes to sleep awaiting the event that the I/O is complete.
The disk controller interrupts the processor when the I/O is complete
The disk interrupt handler awakens the sleeping processes
The content of disk blocks are now in the buffer
When the process no longer need the buffer, it releases it so that other processes can access it

23. Reading and Writing disk blocks
To write a disk block
The kernel informs the disk driver that it has a buffer whose contents should be output.
The disk driver schedules the block for I/O.
If the write is synchronous, the calling process goes the sleep awaiting I/O completion and releases the buffer when awakens.
If the write is asynchronous, the kernel starts the disk write. The kernel release the buffer when the I/O completes.

24. INTERNAL REPRESENTATION OF FILES

25. contents Inodes Structure of a regular file
Conversion of a path name to an inode
Super block
Inode assignment to a new file
Delete Inode

26. File System Algorithms
namei
alloc free ialloc ifree
iget iput
buffer allocation algorithms
getblk brelse bread bwrite
Lower Level File system Algorithms

27. Inode
contains the information necessary for a process to access a file
exits in a static form on disk and the kernel reads them into an in-core inode

28. Inodes - file owner identifier consists of - file type
- file access permissions
- file access times
- number of links to the file
- table of contents for the disk address of data in a file
- file size

29. Inodes in-core copy of the inode contains
- status of the in-core inode
- logical device number of file system
- inode number
- pointers to other in-core inodes
- reference count

30. iget (inode_no) //getIncoreInode
while (not done)
if (inode in inode cache)
if (inode locked)
sleep(event inode becomes unlocked)
continue
if (inode on inode free list)
remove from free list
return locked inode
if (no inode on free list) return error
remove new inode from free list
set inode number
remove inode from old hash queue and place on new one
read inode from disk
set reference count 1
return locked indoe

31. iput (inode_no) //releaseIncoreInode
lock inode if not locked
decrement inode refernece count
if (refernce count==0)
if (inode link==0)
free disk block
set file type to 0
free inode
if (file accessed or inode changed or file changed)
update disk inode
put inode on free list
Release inode lock

32. Structure of a regular file
direct0
direct1
direct2
direct3
direct4
direct5
direct6
direct7
direct8
direct9
single indirect
double indirect
triple indirect
Inode
Data Blocks

33. Structure of a regular file
Suppose System V UNIX
Assume that a logical on the file system holds 1K bytes and that a block number is addressable by a 32 bit integer, then a block can hold up to 256 block numbers
10 direct blocks with 1K bytes each=10K bytes
1 indirect block with 256 direct blocks= 1K*256=256K bytes
1 double indirect block with 256 indirect blocks=
256K*256=64M bytes
1 triple indirect block with 256 double indirect blocks=
64M*256=16G bytes

34. Path conversion to an inode
if (path name starts with root)
working inode= root inode
else
working inode= current directory inode
while (there is more path name)
read next component from input
read directory content
if (component matches an entry in directory)
get inode number for matched component
release working inode
working inode=inode of matched component
return no inode
return (working inode)

35. Conversion of a path name to an inode
algorithm namei
- If path name starts from root, then the kernel assigns root inode(iget) to working inode
- Otherwise, the kernel assigns current directory inode to working inode
- While there is more path name, the kernel reads next path name component from input, and verifies that working inode is of directory, access permissions OK
• If working inode is of root and component is ‘..’, then the kernel checks whether there is more path name or not
• Otherwise the kernel reads directory by repeated use of bmap,bread,brelse

36. Super block File System consists of - the size of the file system
- the number of free blocks in the file system
- a list of free blocks available on the file system
- the index of the next free block in the free block list
- the size of the inode list
- the number of free inodes in the file system
- a list of free inodes in the file system
- the index of the next free inode in the free inode list
- lock fields for the free block and free inode lists
- a flag indicating that the super block has been modified
boot block super block inode list data blocks

37. Inode assignment to a new file
free inodes empty
array1
Super Block Free Inode List
index
free inodes empty
array2
Assigning Free Inode from Middle of List

38. Inode assignment to a new file
Assigning Free Inode – Super Block List Empty
empty
array1
Super Block Free Inode List
index
free inodes
array2
remembered inode
index

39. Inode assignment to a new file
free inodes
remembered inode
Original Super Block List of Free Inodes
index
Free Inode 499
free inodes
remembered inode
index
Free Inode 601
free inodes
remembered inode
index

40. Inode assignment to a new file
Locked inode illoc()
while (not done)
If (super block locked)
Sleep (event super block becomes free)
Continue
If (inode list in super block empty)
Lock super block
Get remember inode for free inode search
Search until super block full or no more free inode
Unlock super block and wake up (event super block free)\
If no free inode found on disk return error
Set remmbered inode for next free inode search
Get inode number from super block inode list
Get inode
Write inode to disk
Decrement free inode count
Return inode

41. Freeing inode ifree(inode_no) Increment free inode count
If super block locked return
If (inode list full) //at super block
if (inode number <remembered inode)
Set remembered inode as input inode
Else
Store inode number in inode list
return

42. Allocation of disk blocks
…………………………..
……………………
109
…………………… 214
211
……………………
310
linked list of free disk block number

43. Allocation of disk blocks
algorithm alloc
- The kernel wants to allocate a block from a file system
it allocates the next available block in the super block list
- Once allocated , the block cannot be reallocated until it becomes free
- If the allocated block is the last block , the kernel treats it as a pointer to a block that contains a list of free blocks
• The kernel locks super block, reads block jut taken from
free list, copies block numbers in block into super
block, releases block buffer,and unlocks super block
- Otherwise,
• The kernels gets buffer for block removed from super
block list , zero buffer contents, decrements total count
of free blocks, and marks super block modified

44. Allocation of disk blocks
super block list
109 …………………………………………………………
109
…………………………….. 112
original configuration
…………………………………………………..
109
………………………………. 112
After freeing block number 949

45. Allocation of disk blocks
109 ………………………………………………………..
109
………………………………. 112
After assigning block number(949)
……………………………… 112
211
………………………………. 243
After assigning block number(109)
replenish super block free list