1. File Management Introduction to Operating Systems: Module 12

2. File-system implementation u File-system structure u Allocation methods u Free-space management u Directory implementation u Efficiency and performance

3. File-system structure u What are files?  Fundamental abstraction for organizing information on secondary storage  Named collection of information u File system resides mainly on disk  Exceptions exist: /proc, /dev u File system organized into a tree-like structure u File manager administers the collection by:  Storing the information on a device  Mapping the block storage to the logical view  Allocating/de-allocating storage  Providing file directories

4. Records Applications Structured Record Files Record-Stream Translation Stream-Block Translation Byte Stream Files Storage device Information structure u View of information varies based on needs and requirements u Applications need structured perspective  Via programming language primitives  Or provided by OS u Translation to byte-streams  Facilitate storage to media  Bi-directional u Still must organize blocks on storage devices u Where does object serialization fit in?

5. Record-oriented files u Manage & store set of records on list u For example, mail messages are records with  Header, sender, subject, receiver, body  Manipulated by editor, mailer (sender and receiver), browser, etc. u Structured sequential file is a named sequence of logical records indexed by non-negative numbers.  Random access is only possible if records have fixed length  Operations include:  Fileid=open(filename)  close(fileid)  Getrecord(fileid, record)  putrecord(fileid, record)  Seek(fileid, position)

6. File Record-oriented files u A disk contains many files u Files are sets of records u Records are sets of fields u Fields are variable types Disk File Record Field 1 Field 2 Field 3 Field 4 Field 5

7. Stream-Block Translation b0b0 b1b1 b2b2 bibi... Low level files u Byte-Streams Translated to Block Streams u Byte-Stream is a Named Sequence of Bytes of Unsigned Integers u File Pointer Maintains Position within Stream u After open operation, File Pointer is at the beginning

8. Byte stream file interface  Operations  Fileid = open(filename)  Close(fileid)  Read(fileid, buffer, length)  Write(fileid, buffer, length)  Seek(fileid, fileposition) u File manager maps filenames to a collection of physical blocks on storage device  Device drivers to read/write blocks  Write operation allocates unused blocks  Delete operation deallocate blocks  Approach taken part of file management implementation strategy

9. Implementing low level files u Secondary storage device contains:  Volume directory (sometimes a root directory for a file system)  External file descriptor for each file  Contents of files u Manages blocks  Assigns blocks to files (descriptor indexed)  Keeps track of available blocks u Maps file to/from device byte stream

10. An open operation u Multi-step process  Locate the external file descriptor  Extract information needed to read/write file  Authenticate that the process can access the file  Create internal file descriptor in primary memory  Create entry in a “per process” open file status table  Allocate resources (buffers) for file usage u Close operation completes all pending operations, releases I/O buffers, frees locks, updates external file descriptor, deallocates file status table entry

11. fid = open(“fileA”, flags); … read(fid, buffer, len); 0 stdin 1 stdout 2 stderr 3... Descriptor Table File structure inode Opening a UNIX file u Changes to in-memory inode occur when inode copied back to secondary storage either periodically or when file is closed

12. A typical file control block

13. In-memory file system structures

14. Block management u Assigning storage blocks to the file u Fixed sized, k, blocks u File of length m requires N =  m/k  blocks u Byte b i is stored in block  i/k  u File manager has three basic strategies for allocating and managing blocks:  Contiguous allocation  Linked lists  Indexed allocation

15. Contiguous allocation u Each file occupies contiguous blocks on the disk u Simple – only starting location (block #) and length (number of blocks) are required u Random access is possible u Wasteful of space  dynamic storage-allocation problem  Files cannot grow

16. Contiguous allocation of disk space

17. Extent-based systems u Many newer file systems (I.E. Veritas file system) use a modified contiguous allocation scheme u Extent-based file systems allocate blocks in extents  An extent is a contiguous set of disk blocks. A file consists of one or more extents

18. Linked allocation u Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk. u Simple  need only starting address u Free-space management system  no waste of space u No random access u Mapping is O(n) pointer block =

19. Linked allocation

20. File-allocation table u FAT  Modified linked allocation u All the pointers are stored together  The resulting structure might fit in memory u Used for floppies

21. Indexed allocation u Brings all pointers together into the index block. u Logical view. index table

22. Example of indexed allocation

23. Indexed allocation u Need index table u Random access u Dynamic access without external fragmentation, but have overhead of index block. u Mapping from logical to physical in a file of maximum size of 256k words and block size of 512 words.  We need only 1 block for index table u Mapping from logical to physical in a file of unbounded length (block size of 512 words)  Linked scheme – link blocks of index table (no limit on size)

24. Two-level index INODE  outer-index index table file

25. Combined scheme: UNIX (4k/block)

26. Free-space management u Bit vector (n blocks) u Bit map requires extra space. Example: Block size = 2 12 bytes Disk size = 2 30 bytes (1 gigabyte) N = 2 30 /2 12 = 2 18 bits (or 32K bytes) u Easy to get contiguous files u Linked list (free list)  Cannot get contiguous space easily  No waste of space

27. Free-space management u Need to protect:  Pointer to free list  Bit map  Must be kept on disk  Copy in memory and disk may differ  Cannot allow for block[i] to have a situation where bit[i] = 1 in memory and bit[i] = 0 on disk  Solution:  Set bit[i] = 1 in disk  Allocate block[i]  Set bit[i] = 1 in memory

28. Linked free space list on disk

29. Efficiency and performance u Efficiency dependent on:  Disk allocation and directory algorithms  Types of data kept in file’s directory entry u Performance  Disk cache  Separate section of main memory for frequently used blocks  Free-behind and read-ahead  Techniques to optimize sequential access  Improve PC performance by dedicating section of memory as virtual disk, or RAM disk.

30. Various disk-caching locations

31. Managing the byte stream u Implementation of byte stream on top of contiguous set of blocks u Packing and unpacking blocks  Must read-ahead on input (convert secondary storage blocks to byte stream - unpack)  Must write-behind on output (convert byte strings to secondary storage blocks - pack) u Block I/O  Buffer several blocks  Memory mapped files

32. Directories u A set of logically associated files and sub directories u File manager provides set of controls:  Enumerate: return file list and nested directories  Copy: duplicate a file  Rename: change symbolic name of file  Delete: remove file from directory  Release all blocks and file descriptor  Traverse: maintain at least hierarchical structure and ability to navigate u Unix supports graphs (through symbolic links)

33. Directory structures u How should files be organized within directory?  Flat name space: all files appear in a single directory  Hierarchical name space  Directory contains files and subdirectories  Each file/directory appears as an entry in exactly one other directory -- a tree  Popular variant: all directories form a tree, but a file can have multiple parents u Dominance of directory structure in other tools  Email folders in Netscape, Outlook  Organizing designs in Rational Rose

34. Directory implementation u Device directory  A device can contain a collection of files  Easier to manage if there is a single root for all files: the device root directory u File directory  Typical implementations have directories implemented as a file with a special format  Entries are pointers to other files or subdirectories  Symbolic linkage is possible (graphs) u Visibility of directory structure controlled by protection and security mechanism

35. Directory implementation u Linear list of file names with pointer to the data blocks  Simple to program  Time-consuming to execute u Hash table – linear list with hash data structure  Decreases directory search time  Collisions – situations where two file names hash to the same location  Fixed size

36. / binusretcfoo billnutt abc bar blah cdexyz UNIX mount command Mount bar at foo