1. CENG334 Introduction to Operating Systems
13/03/07
CENG334 Introduction to Operating Systems
Filesystems
Topics:
Erol Sahin
Dept of Computer Eng.
Middle East Technical University
Ankara, TURKEY
URL:

2. Virtual File Systems VFS: another layer of abstraction
13/03/07
VFS: another layer of abstraction
Upper interface: for processes implementing POSIX interface
Lower interface: for concrete file systems
VFS translates the POSIX calls to the calls of the filesystems under it.
Developed by Sun to support NFS (Network File System) protocol.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

3. VFS- 2 At boot time, the root filesystem is registered with VFS.
13/03/07
VFS- 2
At boot time, the root filesystem is registered with VFS.
When other filesystems are mounted, they must also register with VFS.
When a filesystem registers, it provides the list of addresses of the functions that the VFS demands, such as reading a block.
After regitratin, when one opens a file:
open(“/usr/include/unistd.h”, O_RDONLY)
VFS creates a v-node and makes a call to the concrete filesystem to return all the information needed.
The created v-node also contains pointers to the table of functions for the concrete filesystem that the file resides.

4. Virtual File Systems (2)
13/03/07
Figure A simplified view of the data structures and code used by the VFS and concrete file system to do a read.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

5. Keeping track of Free Blocks -1
13/03/07
Keeping track of Free Blocks -1
Linked list of disk blocks, with each block filled with free disk block numbers.
With 1KB block size, and 32 bit disk block number, each block can hold the numbers for 255 free blocks. The 256th slot points to the next block holding free disk blocks.
Example:
500 GB disk has approx million blocks.
If the disk is empty: storing the free blocks requires 488 million/255 = 1.9 million blocks.
If the disk is nearly full, then the linked list requires small.
Storage is essential free.. Free blocks are used.

6. Cont’d The free list method can be enhanced..
13/03/07
Cont’d
The free list method can be enhanced..
Instead of keeping track of each free block, we can keep track of consecutive free blocks represented by:
The address of the first free block
The count of free blocks
In the free list method, only one block of pointers need to be stored in the memory. When it runs out, another one can be brought into the memory.

7. Keeping track of Free Blocks -2
13/03/07
Keeping track of Free Blocks -2
One bit for keeping the state of each block.
1 : free blocks
0 : allocated blocks
500 GB disk, 1KB blocks
488 million bits = KB blocks
For an empty disk:
Less space than linked list method, since we use 1-bit instead of 32-bits for each free block

8. Disk Quotas
13/03/07
Figure Quotas are kept track of on a per-user basis in a quota table.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

9. File System Backups (1)
13/03/07
Backups are generally made to handle one of two potential problems:
Recover from disaster.
Recover from stupidity.
Thrash bins
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

10. Backup what?
13/03/07
In order to have storage efficiency, one can choose not to backup:
Executable files (since they can usually be restored from manufacturer CD-ROMs
Temporary files
/tmp
Special files (which correspond to I/O devices)
/dev
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

11. Back-up issues Full backup Incremental backup Compressed backups
backup all the filesystem providing a snapshot of the system at that point which can be fully restored
Typically done weekly or monthly
Incremental backup
Backup only the files that were changed after the most recent backup
Typically, a weekly backup is followed by daily incremental backups
Smaller backup size, and faster
Compressed backups
Reduced storage
Less secure: a single bad byte can screw the whole backup
Backing up and active filesystem is tricky
Since during backup, files and directories are being modified
Makes the system less secure
Each backup tape/disk needs to be as safe as the serve itself..
It doesn’t matter if your backup tapes are lying around, even if you have the most secure computer system..

12. Dumping strategies – physical dump
Algorithm
Starts at block 0 of the disk and copies all the data onto a tape/disk
Pros
Simple to implement in a bug-free way
Fast
Cons
Backups the free blocks as well
Backuping a free disk takes as much storage/time as a full disk
Dumping of “bad blocks” is a concern
Typically disk controllers provide bad block replacement transparently without the OS even knowing about it
If a block goes bad after formatting, then the OS typically creates a “file” consisting of al the bad blocks.

13. Dumping strategies – logical dump
Algorithm
Full backup: Traverses the filesystem as a tree and creates the same filesystem structure on the backup disk/tape.
Partial backup: all the directories on the path to the particular file needs to be saved. For instance, backing up file 9 requires the saving of directories, 1,5,6, and 7.
Squares are directories, circles are files. Shaded items have been modified since last dump. Each directory and file is labeled by its i-node number.

14. Dumping strategies – incremental logical dump
Bitmaps, indexed by i-node number are used.
Phase 1: Examine all files and directories below the starting directory (root in this case), and
mark the modified files.
mark ALL THE DIRECTORIES
Squares are directories, circles are files. Shaded items have been modified since last dump. Each directory and file is labeled by its i-node number.

15. Dumping strategies – incremental logical dump
Phase 2: Recursively walk the tree again, and
UNMARK all the directories that do not have any modified files under them.
Note:
10 and 11 are unmarked
5 and 6 remain marked
Squares are directories, circles are files. Shaded items have been modified since last dump. Each directory and file is labeled by its i-node number.

16. Dumping strategies – incremental logical dump
Phase 3: Scan the i-nodes in numerical order and
Dump all the marked directories
Squares are directories, circles are files. Shaded items have been modified since last dump. Each directory and file is labeled by its i-node number.

17. Dumping strategies – incremental logical dump
Phase 4: dump the marked files.
Squares are directories, circles are files. Shaded items have been modified since last dump. Each directory and file is labeled by its i-node number.

18. Dumping strategies – incremental logical dump
Restoring:
Restore all the directories that were backupped
Restore all the files
Squares are directories, circles are files. Shaded items have been modified since last dump. Each directory and file is labeled by its i-node number.

19. Issues Links: Holes: Special files
If a file is linked to more than one directory, only one copy should be saved.
Holes:
In UNIX, some file, such as core files may contain holes.
These files, write a few bytes, and then seek to a distant file offset and write some more bytes.
These empty blocks that are not written should not be dumped and stored.
Cores typically have may megabytes of empty blocks.
Special files
Such as named pipes (which can appear anywhere in the filesystem) should not be dumped.

20. Dumping strategies – logical dump
Pros
Allows to restore a single file. If directories that lie on the path to the file-to-be-restored were deleted, then they would also be restored.
Cons
Slow and complicated.

21. File system consistency
Consistency check is typically done after a crash..
UNIX: fsck
Windoze: scandisk
Redundant information in the filesystem is used:
Check the blocks
Check the files
Two tables, each containing a counter initialized to 0
Blocks in use: How many times a block is present in a file
Read all the i-nodes using a raw device (not through the filesystem calls)
For each block that is referenced in the inode structure, increment the corresponding block use counter by one.
Free blocks:
Examine the free block list of free block bitmap structure
Each appearance of a free block increments the counter by one

22. File System Consistency
13/03/07
Figure File system states. (a) Consistent. (b) Missing block. (c) Duplicate block in free list. (d) Duplicate data block.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

23. Missing block Harmless but wastes space
13/03/07
Harmless but wastes space
Action: Add the missing block to the free list.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

24. Duplicate block in free list
13/03/07
Can only occur in linked list representation. Bitmap representation does not have this problem.
Action: Rebuild the free list.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

25. Duplicate data block
13/03/07
The worst thing that can happen! A data block appears in two different files..
Action:
Allocate a free block
Copy the contents into the new block.
Change the links such that each copy appears once in each file.
For sure, the contents of one of the files is garbled.
The filesystem is made to be consistent.
The use is informed.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

26. Consistency check for directories
Uses a table of counters per file (rather than per block)
Starts from the root and traverses the tree
For each i-node, it increments the corresponding counter for that file
Remember due to hard links, a file can appear more than once
It then checks the link counts stored in the i-nodes to these values.
If the link count > counter
Even if the file is deleted by from all the directory entries, it will continue to exist.
Solution: correct the link count
If the counter > link count
Although the file is linked from, say, two directories, removal from one would cause the i-node deleted leaving the other one invalid.

27. Caching Reading a 32-bit word from memory takes 10 nsec.
Hard disks can transfer data at: 100MB/sec, that is 40nsec per 32- bit words..
PLUS 5-10 msecs of seek time!
Caching aims to fill in the gap..
Often thousands of blocks are kept in cache.

28. Caching (1)
13/03/07
Hash the device and the disk address and look up the result in a hash table.
All the blocks with the same cache value are chained together through a linked list.
In addition to this, a bidirectional link runs through all the blocks implementing a LRU list.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

29. Caching (2)
13/03/07
Some blocks, such as i-node blocks, are rarely referenced two times within a short interval.
Consider a modified LRU scheme, taking two factors into account:
Is the block likely to be needed again soon?
Is the block essential to the consistency of the file system?
For both questions, blocks can be divided into categories
i-node blokcs
Indirect blocks
Directory blocks
Data blocks
Full
Partially full
Blocks that are essential for filesystem consistency should be written to disk immediately – write-through-caches
UNIX: synch (every 30 seconds)
Windows: In the past: none, recent ones: FlushFileBuffers
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

30. Block read ahead
The system tries to get blocks into the cache, before they are accessed to increase the hit rate.
Sequential access: performance improvement
Random access: performance degradation

31. Reducing Disk Arm Motion
13/03/07
Figure (a) I-nodes placed at the start of the disk. (b) Disk divided into cylinder groups, each with its own blocks and i-nodes.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

32. Figure 4-30. The ISO 9660 directory entry.
The ISO 9660 File System
13/03/07
Figure The ISO 9660 directory entry.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

33. Rock Ridge Extensions Rock Ridge extension fields:
13/03/07
Rock Ridge extension fields:
PX - POSIX attributes.
PN - Major and minor device numbers.
SL - Symbolic link.
NM - Alternative name.
CL - Child location.
PL - Parent location.
RE - Relocation.
TF - Time stamps.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

34. Joliet Extensions Joliet extension fields: Long file names.
13/03/07
Joliet extension fields:
Long file names.
Unicode character set.
Directory nesting deeper than eight levels.
Directory names with extensions
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

35. The MS-DOS File System (1)
13/03/07
Figure The MS-DOS directory entry.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

36. The MS-DOS File System (2)
13/03/07
Figure Maximum partition size for different block sizes. The empty boxes represent forbidden combinations.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

37. The UNIX V7 File System (1)
13/03/07
Figure A UNIX V7 directory entry.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

38. The UNIX V7 File System (2)
13/03/07
Figure A UNIX i-node.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

39. The UNIX V7 File System (3)
13/03/07
Figure The steps in looking up /usr/ast/mbox.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved