1. Chapter 11: File System Implementation

2. 11.2 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Outline n File-System Structure n File-System Implementation n Directory Implementation n Allocation Methods n Free-Space Management n Efficiency and Performance n Recovery n Log-Structured File Systems n NFS n Example: WAFL File System

3. 11.3 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Objectives n To describe the details of implementing local file systems and directory structures n To describe the implementation of remote file systems n To discuss block allocation and free-block algorithms and trade-offs

4. 11.4 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 File-System Structure n File structure l Logical storage unit l Collection of related information n File system organized into layers n File system resides on secondary storage (disks) l Provides efficient and convenient access to disk by allowing data to be stored, located retrieved easily File control block – storage structure consisting of information about a file n Device driver controls the physical device

5. 11.5 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Layered File System

6. 11.6 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 File-System Implementation n Boot control block contains info needed by system to boot OS from that volume n Volume control block ( super block ) contains volume details n Directory structure organizes the files n Per-file File Control Block (FCB) contains many details about the file ( inode )

7. 11.7 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 A Typical File Control Block

8. 11.8 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 In-Memory File System Structures n The following figure illustrates the necessary file system structures provided by the OS n Figure 11.3(a) refers to opening a file n Figure 11.3(b) refers to reading a file

9. 11.9 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 In-Memory File System Structures

10. 11.10 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Virtual File Systems n Virtual File Systems (VFS) provide an object- oriented way of implementing file systems n VFS allows the same system call interface (the API) to be used for different types of file systems n The API is to the VFS interface, rather than any specific type of file system

11. 11.11 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Schematic View of Virtual File System

12. 11.12 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Directory Implementation n Linear list of file names with pointer to the data blocks l simple to program l time-consuming to execute Hash Table – linear list with hash data structure l decreases directory search time collisions – situations where two file names hash to the same location l fixed size

13. 11.13 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Allocation Methods n An allocation method refers to how disk blocks are allocated for files: l Contiguous allocation l Linked allocation l Indexed allocation

14. 11.14 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Contiguous Allocation n Each file occupies a set of contiguous blocks on the disk n Advantages Simple – only starting location (block #) and length (number of blocks) are required l Random access n Disadvantages l Wasteful of space (dynamic storage-allocation problem) l Files cannot grow

15. 11.15 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Contiguous Allocation n Mapping from logical to physical LA/512 Q R Block to be accessed = Q + starting address Displacement into block = R

16. 11.16 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Contiguous Allocation of Disk Space

17. 11.17 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Extent-Based Systems n Many newer file systems (i.e. Veritas File System) use a modified contiguous allocation scheme n Extent-based file systems allocate disk blocks in extents n An extent is a contiguous block of disks l Extents are allocated for file allocation l A file consists of one or more extents

18. 11.18 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Linked Allocation n Each file is a linked list of disk blocks l blocks may be scattered anywhere on the disk pointer block =

19. 11.19 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Linked Allocation (Cont.) Simple – need only starting address Free-space management system – no waste of space n No random access n Mapping Block to be accessed = Qth block in the linked chain of blocks representing the file Displacement into block = R + 1 nEx: File-allocation table (FAT) – disk-space allocation used by MS-DOS and OS/2 LA/511 Q R

20. 11.20 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Linked Allocation

21. 11.21 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 File-Allocation Table

22. 11.22 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Indexed Allocation n Brings all pointers together into the index block n Logical view index table

23. 11.23 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Example of Indexed Allocation

24. 11.24 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Indexed Allocation (Cont.) n Need index table n Random access n Dynamic access without external fragmentation, but have overhead of index block n Mapping from logical to physical in a file of maximum size of 256K words and block size of 512 words l We need only 1 block for index table LA/512 Q R Q = displacement into index table R = displacement into block

25. 11.25 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Indexed Allocation – Mapping (Cont.) n Mapping from logical to physical in a file of unbounded length (block size of 512 words) Linked scheme – Link blocks of index table (no limit on size) LA / (512 x 511) Q1Q1 R1R1 Q 1 = block of index table R 1 is used as follows: R 1 / 512 Q2Q2 R2R2 Q 2 = displacement into block of index table R 2 = displacement into block of file

26. 11.26 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Indexed Allocation – Mapping (Cont.) n Two-level index (maximum file size is 512 3 ) LA / (512 x 512) Q1Q1 R1R1 Q 1 = displacement into outer-index R 1 is used as follows: R 1 / 512 Q2Q2 R2R2 Q 2 = displacement into block of index table R 2 = displacement into block of file

27. 11.27 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Indexed Allocation – Mapping (Cont.)  outer-index index table file

28. 11.28 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Combined Scheme: UNIX inode

29. 11.29 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Free-Space Management n Bit vector ( n blocks) … 012n-1 bit[ i ] = 0  block[ i ] free 1  block[ i ] occupied Calculation of the first free block number (number of bits per word) * (number of 0-value words) + offset of first 1 bit 

30. 11.30 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Free-Space Management (Cont.) n Bit map requires extra space l Example: block size = 2 12 bytes disk size = 2 30 bytes (1 gigabyte) n = 2 30 /2 12 = 2 18 bits (or 32K bytes) l Easy to get contiguous files n Linked list (free list) l Cannot get contiguous space easily l No waste of space n Grouping l Allocate blocks in groups n Counting l First block & length

31. 11.31 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Free-Space Management (Cont.) n Need to protect: l Pointer to free list l Bit map  Must be kept on disk  Copy in memory and disk may differ  Cannot allow for block[ i ] to have a situation where bit[ i ] = 1 in memory and bit[ i ] = 0 on disk l Solution:  Set bit[ i ] = 1 in disk  Allocate block[ i ]  Set bit[ i ] = 1 in memory

32. 11.32 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Linked Free Space List on Disk

33. 11.33 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Efficiency and Performance n Efficiency dependent on: l disk allocation and directory algorithms types of data kept in file ’ s directory entry n Performance disk cache – separate section of main memory for frequently used blocks free-behind and read-ahead – techniques to optimize sequential access l improve PC performance by dedicating section of memory as virtual disk, or RAM disk

34. 11.34 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Page Cache n A page cache caches pages rather than disk blocks using virtual memory techniques n Memory-mapped I/O uses a page cache n Routine I/O through the file system uses the buffer (disk) cache

35. 11.35 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 I/O Without a Unified Buffer Cache

36. 11.36 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Unified Buffer Cache n A unified buffer cache uses the same page cache to cache both memory-mapped pages and ordinary file system I/O

37. 11.37 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 I/O Using a Unified Buffer Cache

38. 11.38 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Recovery Consistency checking – compares data in directory structure with data blocks on disk, and tries to fix inconsistencies n Use system programs to back up data from disk to another storage device (floppy disk, magnetic tape, other magnetic disk, optical) n Recover lost file or disk by restoring data from backup

39. 11.39 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Log Structured File Systems n Log structured (or journaling) file systems record each update to the file system as a transaction n All transactions are written to a log l A transaction is considered committed once it is written to the log l However, the file system may not yet be updated n The transactions in the log are asynchronously written to the file system l When the file system is modified, the transaction is removed from the log n If the file system crashes, all remaining transactions in the log must still be performed

40. 11.40 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 The Sun Network File System (NFS) n An implementation and a specification of a software system for accessing remote files across LANs (or WANs) n The implementation is part of the Solaris and SunOS running on Sun workstations using an unreliable datagram protocol (UDP/IP protocol) and Ethernet

41. 11.41 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 NFS (Cont.) n Interconnected workstations viewed as a set of independent machines with independent file systems, which allows sharing among these file systems in a transparent manner l A remote directory is mounted over a local directory  The mounted directory looks like an integral subtree of the local file system l Specification of the remote directory (hostname) for the mount operation is nontransparent  Files in the remote directory can then be accessed in a transparent manner l Potentially any file system (or directory within a file system), can be mounted remotely on top of any local directory

42. 11.42 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 NFS (Cont.) n NFS is designed to operate in a heterogeneous environment of different machines, OS, and network architectures n NFS specifications independent of these media l This independence is achieved through the use of RPC primitives built on top of an External Data Representation (XDR) protocol used between two implementation-independent interface n The NFS specification distinguishes between the services provided by a mount mechanism and the actual remote-file-access services

43. 11.43 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Three Independent File Systems

44. 11.44 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Mounting in NFS Mounts Cascading mounts

45. 11.45 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 NFS Mount Protocol n Establishes initial logical connection between server and client n Mount operation includes name of remote directory to be mounted and name of server machine storing it l Mount request is mapped to corresponding RPC and forwarded to mount server running on server machine Export list – specifies local file systems that server exports for mounting, along with names of machines that are permitted to mount them

46. 11.46 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Following a mount request that conforms to its export list, the server returns a file handle — a key for further accesses File handle – a file-system identifier, and an inode number to identify the mounted directory within the exported file system The mount operation changes only the user ’ s view and does not affect the server side

47. 11.47 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 NFS Protocol n Provides a set of RPCs for remote file operations: l searching for a file within a directory l reading a set of directory entries l manipulating links and directories l accessing file attributes l reading and writing files n NFS servers are stateless ; each request has to provide a full set of arguments l NFS V4 is stateful Modified data must be committed to the server ’ s disk before results are returned to the client (lose advantages of caching) n The NFS protocol does not provide concurrency- control mechanisms

48. 11.48 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Three Major Layers of NFS Architecture n UNIX file-system interface (based on the open, read, write, and close calls, and file descriptors ) Virtual File System (VFS) layer – distinguishes local files from remote ones, and local files are further distinguished according to their file- system types l The VFS activates file-system-specific operations to handle local requests according to their file-system types l Calls the NFS protocol procedures for remote requests NFS service layer – bottom layer of the architecture l Implements the NFS protocol

49. 11.49 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Schematic View of NFS Architecture

50. 11.50 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 NFS Path-Name Translation n Performed by breaking the path into component names and performing a separate NFS lookup call for every pair of component name and directory vnode To make lookup faster, a directory name lookup cache on the client ’ s side holds the vnodes for remote directory names

51. 11.51 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 NFS Remote Operations n Nearly one-to-one correspondence between regular UNIX system calls and the NFS protocol RPCs (except opening and closing files) n NFS adheres to the remote-service paradigm, but employs buffering and caching techniques for the sake of performance File-blocks cache – when a file is opened, the kernel checks with the remote server whether to fetch or revalidate the cached attributes File-attribute cache – the attribute cache is updated whenever new attributes arrive from the server n Clients do not free delayed-write blocks until the server confirms that the data have been written to disk

52. 11.52 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Example: WAFL File System Used on Network Appliance “ Filers ” – distributed file system appliances “ Write-anywhere file layout ” n Serves up NFS, CIFS, http, ftp n Random I/O optimized, write optimized l NVRAM for write caching n Similar to Berkeley Fast File System, with extensive modifications

53. 11.53 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 The WAFL File Layout

54. 11.54 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Snapshots in WAFL

55. End of Chapter 11