1 File Systems Chapter Files 6.2 Directories 6.3 File system implementation 6.4 Example file systems

2 Long-term Information Storage Must store large amounts of data Information stored must survive the termination of the process using it Multiple processes must be able to access the information concurrently

3 File Naming Typical file extensions.

4 File Structure Three kinds of files –byte sequence –record sequence –tree

5 File Types (a) An executable file (b) An archive

6 File Access Sequential access –read all bytes/records from the beginning –cannot jump around, could rewind or back up –convenient when medium was mag tape Random access –bytes/records read in any order –essential for data base systems –read can be … move file marker (seek), then read or … read and then move file marker

7 File Attributes Possible file attributes

8 File Operations 1.Create 2.Delete 3.Open 4.Close 5.Read 6.Write 1.Append 2.Seek 3.Get attributes 4.Set Attributes 5.Rename

9 An Example Program Using File System Calls (1/2)

10 An Example Program Using File System Calls (2/2)

11 Memory-Mapped Files (a) Segmented process before mapping files into its address space (b) Process after mapping existing file abc into one segment creating new segment for xyz

12 Directories Single-Level Directory Systems A single level directory system –contains 4 files –owned by 3 different people, A, B, and C

13 Two-level Directory Systems Letters indicate owners of the directories and files

14 Hierarchical Directory Systems A hierarchical directory system

15 A UNIX directory tree Path Names

16 Directory Operations 1.Create 2.Delete 3.Opendir 4.Closedir 1.Readdir 2.Rename 3.Link 4.Unlink

17 File System Implementation A possible file system layout

18 Implementing Files (1) (a) Contiguous allocation of disk space for 7 files (b) State of the disk after files D and E have been removed

19 Implementing Files (2) Storing a file as a linked list of disk blocks

20 Implementing Files (3) Linked list allocation using a file allocation table in RAM

21 Implementing Files (4) An example i-node

22 Implementing Directories (1) (a) A simple directory fixed size entries disk addresses and attributes in directory entry (b) Directory in which each entry just refers to an i-node

23 Implementing Directories (2) Two ways of handling long file names in directory –(a) In-line –(b) In a heap

24 Shared Files (1) File system containing a shared file

25 Shared Files (2) (a) Situation prior to linking (b) After the link is created (c)After the original owner removes the file

26 Disk Space Management (1) Dark line (left hand scale) gives data rate of a disk Dotted line (right hand scale) gives disk space efficiency All files 2KB Block size

27 Disk Space Management (2) (a) Storing the free list on a linked list (b) A bit map

28 Disk Space Management (3) (a) Almost-full block of pointers to free disk blocks in RAM - three blocks of pointers on disk (b) Result of freeing a 3-block file (c) Alternative strategy for handling 3 free blocks - shaded entries are pointers to free disk blocks

29 Disk Space Management (4) Quotas for keeping track of each user’s disk use

30 File System Reliability (1) A file system to be dumped –squares are directories, circles are files –shaded items, modified since last dump –each directory & file labeled by i-node number File that has not changed

31 File System Reliability (2) Bit maps used by the logical dumping algorithm

32 File System Reliability (3) File system states (a) consistent (b) missing block (c) duplicate block in free list (d) duplicate data block

33 File System Performance (1) The block cache data structures

34 File System Performance (2) I-nodes placed at the start of the disk Disk divided into cylinder groups –each with its own blocks and i-nodes

35 Log-Structured File Systems With CPUs faster, memory larger –disk caches can also be larger –increasing number of read requests can come from cache –thus, most disk accesses will be writes LFS Strategy structures entire disk as a log –have all writes initially buffered in memory –periodically write these to the end of the disk log –when file opened, locate i-node, then find blocks

36 Example File Systems CD-ROM File Systems The ISO 9660 directory entry

37 The CP/M File System (1) Memory layout of CP/M

38 The CP/M File System (2) The CP/M directory entry format

39 The MS-DOS File System (1) The MS-DOS directory entry

40 The MS-DOS File System (2) Maximum partition for different block sizes The empty boxes represent forbidden combinations

41 The Windows 98 File System (1) The extended MOS-DOS directory entry used in Windows 98 Bytes

42 The Windows 98 File System (2) An entry for (part of) a long file name in Windows 98 Bytes Checksum

43 The Windows 98 File System (3) An example of how a long name is stored in Windows 98

44 The UNIX V7 File System (1) A UNIX V7 directory entry

45 The UNIX V7 File System (2) A UNIX i-node

46 The UNIX V7 File System (3) The steps in looking up /usr/ast/mbox

47 Inode Table

48 File Read

49 File Write

50 File Statistics

51 File Usage Locality

52 Arrangement of Blocks

53 fsck Super block Free blocks Inode state Inode links Duplicates Bad blocks Directory checks

54 Journaling

55 Block Reuse 1) Add entry to directory foo 2) Write block (b) of foo to disk 3) Delete directory that contains foo 4) Create file foobar (reuse block b) 5) Data write of foobar and crash

56 Log-structured File Systems Memory sizes were growing Gap between random and sequential I/O performance Existing file systems perform poorly Convert random I/O to sequential I/O

57 Write Buffering - Segments

58 Buffer size? T_w = write time T_pos = time to position R_peak MB/sec = rate of transfer R_eff = effective rate of transfer R_eff = F * R_peak D = size of buffer. = (F * R_peak * T_pos)/(1-F)

59 Finding inodes - Imap

60 Handling directories

61 Garbage Collection

62 Segment Summary Block

63 Distributed File Systems Complexity – thousands of machines Failure Performance Security Communication

64 Unreliable Communication Layers -UDP/IP Reliable Communication - TCP/IP

65 Dropped Request

66 Dropped Reply

67 Communication Abstractions Distributed Shared Memory -- failure of one machine results in loss of universal data structure Remote Procedure Call (RPC) -- stub generator -- run time library

68 Functions in the Stub Create a message buffer Pack the needed information into the message buffer Send the message to the destination RPC server Wait for reply Unpack return code and other arguments Return to caller

69 Server actions Unpack the message (unmarshaling) Call into the actual function Package the results Send the reply

70 Run-time Library Naming Type of protocol Use unreliable communication layer Time out machinery

71 Network File System

72 DFS architecture

73 NFSv2 Simple and Fast Server Crash Recovery Stateless protocol File Handle Volume identifier Inode number Generation number

74 Open in NFS

75 Read in NFS

76 Idempotent Operations Causes of failure Message dropped Server has crashed Perform same operation multiple times lookup, read, write mkdir?

77 Three Types of Loss

78 Improving Performance Client side caching Cache file data Write buffering Cache Consistency Problems

79 Handling Cache Consistency GETATTR: Get the attributes of file Eliminates stale cache problem Not a common case Performance issues Attribute Cache Refreshed every three seconds

80 Server-side Write Buffering Not return success until write is on stable storage To handle failures Put writes in battery-backed memory

81 Andrew File System (AFS) Focus: Scalability AFSv1 Whole-file caching Contrast with blocks in NFS Store the file in local disk and cache Use TestAuth to check for consistency

82 Problems with AFSv1 Path-traversal costs are too high Client issues too many TestAuths to the server Addressed in AFSv2

83 AFSv2 Callback Promise from server to inform the client about changes Stateless -> State File handles Cache Consistency Last writer wins No partial winners as given by NFS

84 Crash Recovery Server unable to contact client Client is rebooting Client checks with the server Server crash Inform all clients