1. Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition, Chapter 11: File System Implementation

2. 11.2 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Outline n File-System Structure n File-System Implementation n Directory Implementation n Allocation Methods n Free-Space Management n Efficiency and Performance n Recovery n NFS n Example: WAFL File System

3. 11.3 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Objectives n To describe the details of implementing local file systems and directory structures n To describe the implementation of remote file systems n To discuss block allocation and free-block algorithms and trade-offs

4. 11.4 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition File-System Structure n File structure l Logical storage unit l Collection of related information l File system organized into layers n File system resides on secondary storage (disks) l Provides efficient and convenient access to disk by allowing data to be stored, located retrieved easily n File control block – storage structure consisting of information about a file n Device driver controls the physical device

5. 11.5 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Layered File System

6. 11.6 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition File-System Implementation n Boot control block contains info needed by system to boot OS from that volume n Volume control block contains volume details n Directory structure organizes the files n Per-file File Control Block (FCB) contains many details about the file

7. 11.7 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition A Typical File Control Block

8. 11.8 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition In-Memory File System Structures n The following figure illustrates the necessary file system structures provided by the OS n Figure 11-3(a) refers to opening a file n Figure 11-3(b) refers to reading a file

9. 11.9 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition In-Memory File System Structures

10. 11.10 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Virtual File Systems n Virtual File Systems (VFS) provide an object-oriented way of implementing file systems n VFS allows the same system call interface (the API) to be used for different types of file systems n The API is to the VFS interface, rather than any specific type of file system

11. 11.11 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Schematic View of Virtual File System

12. 11.12 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Directory Implementation n Linear list of file names with pointer to the data blocks l simple to program l time-consuming to execute n Hash Table – linear list with hash data structure l decreases directory search time l collisions – situations where two file names hash to the same location l fixed size

13. 11.13 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Allocation Methods n An allocation method refers to how disk blocks are allocated for files: n Contiguous allocation n Linked allocation n Indexed allocation

14. 11.14 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Contiguous Allocation n Each file occupies a set of contiguous blocks on the disk n Advantages l Simple – only starting location (block #) and length (number of blocks) are required l Random access n Disadvantages l Wasteful of space (dynamic storage-allocation problem) l Files cannot grow

15. 11.15 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Contiguous Allocation n Mapping from logical to physical LA/512 Q R Block to be accessed = Q + starting address Displacement into block = R

16. 11.16 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Contiguous Allocation of Disk Space

17. 11.17 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Extent-Based Systems n Many newer file systems (i.e. Veritas File System or VxFS in HP-UX) use a modified contiguous allocation scheme n Extent-based file systems allocate disk blocks in extents n An extent is a contiguous block of disks l Extents are allocated for file allocation l A file consists of one or more extents

18. 11.18 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Linked Allocation n Each file is a linked list of disk blocks l blocks may be scattered anywhere on the disk pointer block =

19. 11.19 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Linked Allocation (Cont.) n Simple – need only starting address n Free-space management system – no waste of space n No random access n Mapping Block to be accessed is the Qth block in the linked chain of blocks representing the file Displacement into block = R + 1 File-allocation table (FAT) – disk-space allocation used by MS-DOS and OS/2 LA/511 Q R

20. 11.20 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Linked Allocation

21. 11.21 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition File-Allocation Table

22. 11.22 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Indexed Allocation n Brings all pointers together into the index block n Logical view index table

23. 11.23 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Example of Indexed Allocation

24. 11.24 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Indexed Allocation (Cont.) n Need index table n Random access n Dynamic access without external fragmentation, but have overhead of index block n Mapping from logical to physical in a file of maximum size of 256K words and block size of 512 words l We need only 1 block for index table l Q = displacement into index table l R = displacement into block LA/512 Q R

25. 11.25 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Indexed Allocation – Mapping (Cont.) n Mapping from logical to physical in a file of unbounded length (block size of 512 words) n Linked scheme – Link blocks of index table (no limit on size) LA / (512 x 511) Q1Q1 R1R1 Q 1 = block of index table R 1 is used as follows: R 1 / 512 Q2Q2 R2R2 Q 2 = displacement into block of index table R 2 displacement into block of file:

26. 11.26 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Indexed Allocation – Mapping (Cont.) n Two-level index (maximum file size is 512 3 ) LA / (512 x 512) Q1Q1 R1R1 Q 1 = displacement into outer-index R 1 is used as follows: R 1 / 512 Q2Q2 R2R2 Q 2 = displacement into block of index table R 2 displacement into block of file:

27. 11.27 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Indexed Allocation – Mapping (Cont.)  outer-index index table file

28. 11.28 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Combined Scheme: UNIX UFS (4K bytes per block)

29. 11.29 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Free-Space Management n Bit vector ( n blocks) … 012n-1 bit[i] =  0  block[i] free 1  block[i] occupied Block number calculation (number of bits per word) * (number of 0-value words) + offset of first 1 bit

30. 11.30 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Free-Space Management (Cont.) n Bit map requires extra space l Example: block size = 2 12 bytes disk size = 2 30 bytes (1 gigabyte) n = 2 30 /2 12 = 2 18 bits (or 32K bytes) n Easy to get contiguous files n Linked list (free list) l Cannot get contiguous space easily l No waste of space n Grouping n Counting

31. 11.31 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Free-Space Management (Cont.) n Need to protect: l Pointer to free list l Bit map  Must be kept on disk  Copy in memory and disk may differ  Cannot allow for block[ i ] to have a situation where bit[ i ] = 1 in memory and bit[ i ] = 0 on disk l Solution:  Set bit[ i ] = 1 in disk  Allocate block[ i ]  Set bit[ i ] = 1 in memory

32. 11.32 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Linked Free Space List on Disk

33. 11.33 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Efficiency and Performance n Efficiency dependent on: l disk allocation and directory algorithms l types of data kept in file’s directory entry n Performance l disk cache – separate section of main memory for frequently used blocks l free-behind and read-ahead – techniques to optimize sequential access l improve PC performance by dedicating section of memory as virtual disk, or RAM disk

34. 11.34 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Page Cache n A page cache caches pages rather than disk blocks using virtual memory techniques n Memory-mapped I/O uses a page cache n Routine I/O through the file system uses the buffer (disk) cache n This leads to the following figure

35. 11.35 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition I/O Without a Unified Buffer Cache

36. 11.36 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Unified Buffer Cache n A unified buffer cache uses the same page cache to cache both memory-mapped pages and ordinary file system I/O

37. 11.37 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition I/O Using a Unified Buffer Cache

38. 11.38 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Recovery n Consistency checking – compares data in directory structure with data blocks on disk, and tries to fix inconsistencies n Use system programs to back up data from disk to another storage device (magnetic tape, other magnetic disk, optical) n Recover lost file or disk by restoring data from backup

39. 11.39 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Log Structured File Systems n Log structured (or journaling) file systems record each update to the file system as a transaction n All transactions are written to a log l A transaction is considered committed once it is written to the log l However, the file system may not yet be updated n The transactions in the log are asynchronously written to the file system l When the file system is modified, the transaction is removed from the log n If the file system crashes, all remaining transactions in the log must still be performed

40. 11.40 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition The Sun Network File System (NFS) n An implementation and a specification of a software system for accessing remote files across LANs (or WANs) n The implementation is part of the Solaris and SunOS operating systems running on Sun workstations using an unreliable datagram protocol (UDP/IP) and Ethernet

41. 11.41 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition NFS (Cont.) n Interconnected workstations viewed as a set of independent machines with independent file systems, which allows sharing among these file systems in a transparent manner l A remote directory is mounted over a local file system directory  The mounted directory looks like an integral subtree of the local file system, replacing the subtree descending from the local directory l Specification of the remote directory for the mount operation is nontransparent; the host name of the remote directory has to be provided  Files in the remote directory can then be accessed in a transparent manner l Subject to access-rights accreditation, potentially any file system (or directory within a file system), can be mounted remotely on top of any local directory

42. 11.42 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition NFS (Cont.) n NFS is designed to operate in a heterogeneous environment of different machines, OS, and network architectures; the NFS specifications independent of these media n This independence is achieved through the use of RPC primitives built on top of an External Data Representation (XDR) protocol used between two implementation-independent interfaces n The NFS specification distinguishes between the services provided by a mount mechanism and the actual remote- file-access services

43. 11.43 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Three Independent File Systems

44. 11.44 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Mounting in NFS Mounts Cascading mounts

45. 11.45 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition NFS Mount Protocol n Establishes initial logical connection between server and client l Mount operation includes name of remote directory to be mounted and name of server machine storing it l Mount request is mapped to corresponding RPC and forwarded to mount server running on server machine l Export list – specifies local file systems that server exports for mounting, along with names of machines that are permitted to mount them

46. 11.46 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition n Following a mount request that conforms to its export list, the server returns a file handle—a key for further accesses l File handle – a file-system identifier, and an inode number to identify the mounted directory within the exported file system n The mount operation changes only the user’s view and does not affect the server side

47. 11.47 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition NFS Protocol n Provides a set of remote procedure calls for file operations: l searching for a file within a directory l reading a set of directory entries l manipulating links and directories l accessing file attributes l reading and writing files n NFS servers are stateless ; each request has to provide a full set of arguments l NFS V4 is very different, stateful n Modified data must be committed to the server’s disk before results are returned to the client (lose advantages of caching) n The NFS protocol does not provide concurrency-control mechanisms

48. 11.48 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Three Major Layers of NFS Architecture n UNIX file-system interface (based on the open, read, write, and close calls, and file descriptors ) n Virtual File System (VFS) layer – distinguishes local files from remote ones, and local files are further distinguished according to their file-system types l The VFS activates file-system-specific operations to handle local requests according to their file-system types l Calls the NFS protocol procedures for remote requests n NFS service layer – bottom layer of the architecture l Implements the NFS protocol

49. 11.49 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Schematic View of NFS Architecture

50. 11.50 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition NFS Path-Name Translation n Performed by breaking the path into component names and performing a separate NFS lookup call for every pair of component name and directory vnode n To make lookup faster, a directory name lookup cache on the client’s side holds the vnodes for remote directory names

51. 11.51 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition NFS Remote Operations n Nearly one-to-one correspondence between regular UNIX system calls and the NFS protocol RPCs (except opening and closing files) n NFS adheres to the remote-service paradigm, but employs buffering and caching techniques for the sake of performance n File-blocks cache – when a file is opened, the kernel checks with the remote server whether to fetch or revalidate the cached attributes l Cached file blocks are used only if the corresponding cached attributes are up to date n File-attribute cache – the attribute cache is updated whenever new attributes arrive from the server n Clients do not free delayed-write blocks until the server confirms that the data have been written to disk

52. 11.52 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Example: WAFL File System n Used on Network Appliance “Filers” – distributed file system appliances l “Write-anywhere file layout” n Serves up NFS, CIFS, http, ftp n Random I/O optimized, write optimized l NVRAM for write caching n Similar to Berkeley Fast File System, with extensive modifications

53. 11.53 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition The WAFL File Layout

54. 11.54 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Snapshots in WAFL

55. Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition, End of Chapter 11