File System Implementation Chapter 12

File system Organization Application programs Application programs Logical file system Logical file system manages directory information manages directory information manages file control blocks manages file control blocks File-organization module File-organization module Knows about logical blocks and physical blocks Knows about logical blocks and physical blocks Basic file system Basic file system issue generic commands to the appropriate device driver issue generic commands to the appropriate device driver I/O control I/O control Device drivers Device drivers Interrupt handlers Interrupt handlers devices devices

On disk structures Boot control block Boot control block information needed by the system to boot the OS information needed by the system to boot the OS in the partition boot sector in the partition boot sector Partition control block Partition control block partition details partition details # of blocks in partition # of blocks in partition size of blocks size of blocks free-block count free-block count free-block pointers free-block pointers free FCB count free FCB count FCB pointers FCB pointers Unix – superblock Unix – superblock NTFS – Master File Table NTFS – Master File Table Directory Structure Directory Structure FCB FCB Unix – inode Unix – inode NTFS – stored in Master file table NTFS – stored in Master file table Relational structure Relational structure

In-memory structures In-memory partition table containing info about each mounted partition In-memory partition table containing info about each mounted partition In-memory directory structure In-memory directory structure directory “cache” directory “cache” system-wide open-file table system-wide open-file table FCB of each open file FCB of each open file per-process open file table per-process open file table pointer to the system-wide open-file table pointer to the system-wide open-file table

Creating a new file Application program calls the logical file system Application program calls the logical file system scan directory structure scan directory structure read appropriate directory in memory read appropriate directory in memory allocate a new FCB allocate a new FCB update directory structure update directory structure write directory structure back to disk write directory structure back to disk

Typical FCB File Permissions File owner, group, ACL File Size File data blocks File dates (create access write)

Opening a File User program issues an open call User program issues an open call Directory structure searched for filename Directory structure searched for filename If file not already opened – FCB copied into system-wide open-file table If file not already opened – FCB copied into system-wide open-file table Includes a count of number of processes that possess the file Includes a count of number of processes that possess the file Entry made in the per-process open-file table Entry made in the per-process open-file table File operations made via this entry File operations made via this entry File name not necessarily in memory (why?) File name not necessarily in memory (why?)

Closing a file Per-process table entry is removed Per-process table entry is removed System-wide entry count is reduced System-wide entry count is reduced When all processes that have opened the file close it, it is written back to disk When all processes that have opened the file close it, it is written back to disk

Directory Implementation Linear List Linear List file pointers file pointers name  data blocks name  data blocks create file create file search directory for duplicate names search directory for duplicate names add new entry at end of the directory add new entry at end of the directory delete file delete file search for named file search for named file release space allocated to it release space allocated to it To reuse a directory entry To reuse a directory entry mark the entry as unused mark the entry as unused attach it to the list of free directory entries attach it to the list of free directory entries copy last directory entry to freed location (shorter) copy last directory entry to freed location (shorter)

Directory Implementation Linear List Linear List Disadvantages: Disadvantages: linear search linear search Sorted list Sorted list faster search faster search more complex insert and delete more complex insert and delete B-tree B-tree fast for both fast for both

Directory Implementation Hash table Hash table Advantage: Speed Advantage: Speed Disadvantage: Disadvantage: ?? ??

Allocation Methods Contiguous Allocation Contiguous Allocation Each file occupies a set of contiguous blocks on the disk Each file occupies a set of contiguous blocks on the disk Fast search time Fast search time Block location can be computed from relative block number Block location can be computed from relative block number Directory structures need only contain the first block number for the file Directory structures need only contain the first block number for the file Easy access for read and write Easy access for read and write

Allocation Methods Contiguous Allocation Contiguous Allocation Dynamic storage allocation problem Dynamic storage allocation problem first fit or best fit algorithms first fit or best fit algorithms Suffers from external fragmentation Suffers from external fragmentation How much space is needed for a file? How much space is needed for a file? Solved by using extents Solved by using extents another chunk of disk space linked to the end of the file another chunk of disk space linked to the end of the file Fixed or variable size? Fixed or variable size?

Allocation Methods Linked Allocation Linked Allocation Each file is a linked list of disk blocks Each file is a linked list of disk blocks Directory entry is pointer to first link Directory entry is pointer to first link Advantages Advantages No external fragmentation No external fragmentation Size of file need not be declared when file is created Size of file need not be declared when file is created File can grow as long as free blocks are available File can grow as long as free blocks are available Disadvantages Disadvantages Effective only for sequential access Effective only for sequential access space required for pointers space required for pointers can be mitigated by using clusters can be mitigated by using clusters reliability reliability what happens if a link is lost? what happens if a link is lost?

Allocation Methods FAT FAT file allocation table file allocation table keeps the block table in one place on disk keeps the block table in one place on disk One entry for each disk block, indexed by block number One entry for each disk block, indexed by block number Entry for a block contains the block number for the next block in the file Entry for a block contains the block number for the next block in the file Fat needs to be in memory to avoid large number of disk seeks Fat needs to be in memory to avoid large number of disk seeks

Allocation Methods Linked allocation solves Linked allocation solves External Fragmentation External Fragmentation Size-declaration problems Size-declaration problems But has problems with But has problems with Efficient access (with no FAT) Efficient access (with no FAT)

Indexed Allocation jeep19 Directory Entry Block 19

Allocation Methods Indexed allocation Indexed allocation All pointers are in an index block All pointers are in an index block Each file has its own index block Each file has its own index block Directory contains address of index block Directory contains address of index block To read ith block, use pointer in ith block entry to find the block To read ith block, use pointer in ith block entry to find the block

File Allocation Table test… end of file FAT Directory Entry no. of disk blocks

Indexed Allocation Supports direct access without suffering from external fragmentation Supports direct access without suffering from external fragmentation Suffers from wasted space Suffers from wasted space pointer overhead is greater than with linked allocation pointer overhead is greater than with linked allocation all nil index entries are wasted. all nil index entries are wasted. What size should index block be? What size should index block be? one block may be either too big or too small. one block may be either too big or too small.

Index Block Variants Linked Scheme Linked Scheme An index block is normally one disk block An index block is normally one disk block For large files the last word in the file is a link to an additional index block For large files the last word in the file is a link to an additional index block Multilevel index Multilevel index First index block contains pointers to second level index blocks First index block contains pointers to second level index blocks with 4096-byte blocks this approach would support 4gb files with 4096-byte blocks this approach would support 4gb files Combined approach Combined approach First 15 or so pointers as direct index blocks First 15 or so pointers as direct index blocks next pointer is to a single indirect index next pointer is to a single indirect index following pointer is to double indirect following pointer is to double indirect last pointer is to triple indirect last pointer is to triple indirect

Unix inode mode owners(2) timestamps(3) size block count single indirect double indirect triple indirect direct blocks data

Indexed Allocation Disadvantages Disadvantages Since blocks are not localized on disk, disk access can be slower because of excessive seeks. Since blocks are not localized on disk, disk access can be slower because of excessive seeks.

Allocation Performance Which allocation method to use depends on how files will be used in the system Which allocation method to use depends on how files will be used in the system Systems with mostly sequential access should use a different system than systems with mostly random access Systems with mostly sequential access should use a different system than systems with mostly random access For both types of access, contiguous allocation requires only one access per block For both types of access, contiguous allocation requires only one access per block Linked allocation requires up to i disk reads for the ith block Linked allocation requires up to i disk reads for the ith block

Performance Some OS’s Some OS’s support direct-access files by contiguous allocation support direct-access files by contiguous allocation Maximum length must be declared on creation Maximum length must be declared on creation support sequential access by linked allocation support sequential access by linked allocation sequential access must be declared on creation sequential access must be declared on creation OS must support both allocation systems OS must support both allocation systems

Allocation Performance Other OS’s Other OS’s Combine contiguous allocation with indexed allocation Combine contiguous allocation with indexed allocation contiguous allocation for small files (3-4 blocks) contiguous allocation for small files (3-4 blocks) switch to indexed allocation if file grows large switch to indexed allocation if file grows large

Allocation Performance Study of disk performance can lead to optimizations Study of disk performance can lead to optimizations Sun changed cluster size to 56k in order to improve performance Sun changed cluster size to 56k in order to improve performance also implemented read ahead and free behind also implemented read ahead and free behind went from 50% of CPU for 1.5 Mb per second went from 50% of CPU for 1.5 Mb per second to a substantial increase in throughput with less CPU to a substantial increase in throughput with less CPU

Free Space management Bit vector Bit vector each unallocated block is set to 1 in a bit vector each unallocated block is set to 1 in a bit vector advantage is efficiency in finding first free block or n consecutive free blocks advantage is efficiency in finding first free block or n consecutive free blocks disadvantage is that the bit vector can be large: 1.3GB disk needs 332kb for vector disadvantage is that the bit vector can be large: 1.3GB disk needs 332kb for vector Linked List Linked List free blocks make a linked list free blocks make a linked list inefficient. why? inefficient. why? Fat Fat automatically has free blocks automatically has free blocks

Free Space Management Grouping Grouping store address of first n free blocks in first free block store address of first n free blocks in first free block nth pointer actually pointer to next group nth pointer actually pointer to next group Counting Counting like grouping but also store a count of number of contiguous free blocks like grouping but also store a count of number of contiguous free blocks overall list is shorter overall list is shorter

Efficiency Unix pre-allocates inodes across partitions in order to improve performance Unix pre-allocates inodes across partitions in order to improve performance Unix also tries to keep a file’s data near its inodes Unix also tries to keep a file’s data near its inodes BSD Unix varies the cluster size as a file grows in order to reduce internal fragmentation BSD Unix varies the cluster size as a file grows in order to reduce internal fragmentation Many OS’s have supported various sized pointers in order to increase the addressable disk space Many OS’s have supported various sized pointers in order to increase the addressable disk space

Performance Disk Cache Disk Cache Page Cache Page Cache Unified virtual memory Unified virtual memory both disk cache and virtual memory use the same virtual memory space both disk cache and virtual memory use the same virtual memory space Unified buffer cache Unified buffer cache Both memory-mapped IO and regular system calls use memory-mapped addresses Both memory-mapped IO and regular system calls use memory-mapped addresses The regular disk buffer is not used in this case The regular disk buffer is not used in this case