Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Silberschatz, Galvin and Gagne ©2007 Chapter 11: File System Implementation.

11.2 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Chapter 11: File System Implementation  File-System Structure  File-System Implementation  Directory Implementation  Allocation Methods  Free-Space Management  Efficiency and Performance  Recovery  Log-Structured File Systems  NFS  Example: WAFL File System

11.3 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Objectives  To describe the details of implementing local file systems and directory structures  To describe the implementation of remote file systems  To discuss block allocation and free-block algorithms and trade-offs

11.4 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 File-System Structure  File structure Logical storage unit Collection of related information  File system resides on secondary storage (disks)  File system organized into layers  File control block – storage structure consisting of information about a file

11.5 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Layered File System  I/O Control: device drivers and interrupt handlers that talk to the disk  Basic file system: Generic block reads and writes e.g., read cylinder 73, track 2, sector 10  File organization: Files and logical blocks Translate logical blocks to physical Manage free space  Logical file system: Metadata information e.g., owner, permissions, size, etc.

11.6 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 A Typical File Control Block  FCB has all meta-information about a file Linux calls these i-nodes

11.7 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Implementing File Operations (1)  Create a file: Find space in the file system, add directory entry.  Open a file: System call specifying name of file. System searches directory structure to find file. System keeps current file position pointer to the location where next write/read occurs System call returns file descriptor (a handle) to user process  Reading a file: System call specifying file descriptor and number of bytes to read (and possibly where in memory to stick contents).

11.8 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Implementing File Operations (2)  Writing in a file: System call specifying file descriptor and information to be written Writes information at location pointed by the files current pointer  Repositioning within a file: System call specifying file descriptor and new location of current pointer (also called a file seek even though does not interact with disk)  Closing a file: System call specifying file descriptor Call removes current file position pointer and file descriptor associated with process and file  Deleting a file: Search directory structure for named file, release associated file space and erase directory entry  Truncating a file: Keep attributes the same, but reset file size to 0, and reclaim file space.

11.9 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Other File Operations  Most FS require an open() system call before using a file.  OS keeps an in-memory table of open files, so when reading a writing is requested, they refer to entries in this table via a file descriptor.  On finishing with a file, a close() system call is necessary. (creating & deleting files typically works on closed files)  What happens when multiple files can open the file at the same time?

11.10 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Multiple Users of a File  OS typically keeps two levels of internal tables:  Per-process table Information about the use of the file by the user (e.g. current file position pointer)  System wide table Gets created by first process which opens the file Location of file on disk Access dates File size Count of how many processes have the file open (used for deletion)

11.12 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Virtual File Systems  Virtual File Systems (VFS) provide an object- oriented way of implementing file systems.  VFS allows the same system call interface (the API) to be used for different types of file systems.  The API is to the VFS interface, rather than any specific type of file system.

11.14 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Allocating and Storing Files  An allocation method refers to how disk blocks are allocated for files:  Contiguous allocation All bytes together, in order  Linked allocation Each block points to the next block  Indexed allocation An index block contains pointers to many other blocks

11.16 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Contiguous Allocation  Pros: Simple: state required per file is start block and size Performance: entire file can be read with one seek  Cons: Files can’t grow Fragmentation: external frag is bigger problem Wastes space  Used in CDROMs, DVDs

11.18 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Extent-Based Systems  Many newer file systems (I.e. Veritas File System) use a modified contiguous allocation scheme  Extent-based file systems allocate disk blocks in extents  An extent is a contiguous block of disks Extents are allocated for file allocation A file consists of one or more extents.

11.19 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Linked List Allocation  Each file is stored as linked list of blocks First word of each block points to next block Rest of disk block is file data

11.20 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Linked List Allocation  Pros: No space lost to external fragmentation Disk only needs to maintain first block of each file  Cons: Random access is costly Overheads of pointers

11.22 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Example: MS-DOS File System  Implement a linked list allocation using a table Called File Allocation Table (FAT) Take pointer away from blocks, store in this table Can cache FAT in-memory

11.24 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 FAT Discussion  Pros: Entire block is available for data Random access is faster since entire FAT is in memory  Cons: Entire FAT should be in memory  For 20 GB disk, 1 KB block size, FAT has 20 million entries  If 4 bytes used per entry  80 MB of main memory required for FS

11.25 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Indexed Allocation  Index block contains pointers to each data block  Pros? Space (max open files * size per I- node)  Cons? what if file expands beyond I-node address space?

11.28 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Unix inodes  If data blocks are 4K … First 48K reachable from the inode Next 4MB available from single-indirect Next 4GB available from double-indirect Next 4TB available through the triple-indirect block  Any block can be found with at most 3 disk accesses

11.29 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Implementing Directories  Directory: map ASCII file name to file attributes & location  When a file is opened, OS uses path name to find dir Directory has information about the file’s disk blocks  Whole file (contiguous), first block (linked-list) or I- node (indexed allocation) Directory also has attributes of each file  2 options: entries have all attributes, or point to file I-node

11.30 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Implementing Directories  What if files have large, variable-length names? Limit file name length, say 255 chars, and use previous scheme  Pros: Simple  Cons: wastes space Directory entry comprises fixed and variable portion  Fixed part starts with entry size, followed by attributes  Variable part has the file name  Pros: saves space  Cons: holes on removal, page fault on file read, word boundaries Directory entries are fixed in length, pointer to file name in heap  Pros: easy removal, no space wasted for word boundaries  Cons: manage heap, page faults on file names

11.32 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Directory Search  Linear search Pro: simple Con: slow  Alternatives: Use a per-directory hash table  Could use hash of file name to store entry for file  Pros: faster lookup  Cons: More complex management Caching: cache the most recent searches  Look in cache before searching FS

11.33 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Shared Files  What if B wants to share a file owned by C? One Solution: copy disk addresses in B’s directory entry Problem: modification by one not reflected in other user’s view

11.34 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Sharing Files: Solutions  2 approaches: Use i-nodes to store file information in directories (hard link)  Cons:  What happens if owner deletes file? Symbolic links: B links to C’s file by creating a file in its directory  The new Link file contains path name of file being linked  Cons: read overhead

11.36 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Hard vs Soft Links Foo.txt2433 Soft.lnk43234 Inode #2433 Inode #43234..and then redirects to Inode #2433 at open() time.. /path/to/Foo.txt

11.37 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Disk Space Management  Files are stored as fixed-size blocks  What is a good block size? If 131,072 bytes/track, rotation 8.33 ms, seek 10 ms To read k bytes block: 10+ 4.165 + (k/131072)*8.33 ms Median file size: 2 KB Block size

11.38 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Managing Free Disk Space  2 approaches to keep track of free disk blocks Linked list and bitmap approach

11.39 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Free-Space Management Tradeoffs  Bit Map: Pro: Easy to get contiguous files Con: Bit map requires extra space  Example: block size = 2 12 bytes disk size = 2 30 bytes (1 gigabyte) n = 2 30 /2 12 = 2 18 bits (or 32K bytes)  Linked list: Pro: no wasted extra space  Use free blocks themselves to store free block list Con: Cannot get contiguous space easily

11.40 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Managing Disk Quotas  Sys admin gives each user max space Open file table has entry to Quota table Soft limit violations result in warnings Hard limit violations result in errors Check limits on login

11.41 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Efficiency and Performance  Efficiency dependent on: disk allocation and directory algorithms types of data kept in file’s directory entry  Performance disk cache – separate section of main memory for frequently used blocks free-behind and read-ahead – techniques to optimize sequential access improve PC performance by dedicating section of memory as virtual disk, or RAM disk

11.42 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Page Cache  A page cache caches pages rather than disk blocks using virtual memory techniques  Memory-mapped I/O uses a page cache  Routine I/O through the file system uses the buffer (disk) cache  This leads to the following figure

11.44 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 I/O Using a Unified Buffer Cache  A unified buffer cache uses the same page cache to cache both memory-mapped pages and ordinary file system I/O

11.45 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Recovery  Consistency checking – compares data in directory structure with data blocks on disk, and tries to fix inconsistencies  Use system programs to back up data from disk to another storage device (floppy disk, magnetic tape, other magnetic disk, optical)  Recover lost file or disk by restoring data from backup

11.46 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Log-Structured File Systems  The trend: CPUs are faster, RAM & caches are bigger So, a lot of reads do not require disk access Most disk accesses are writes  pre-fetching not very useful Worse, most writes are small  10 ms overhead for 50 µs write Example: to create a new file:  i-node of directory needs to be written  Directory block needs to be written  i-node for the file has to be written  Need to write the file Delaying these writes could hamper consistency  Solution: LFS to utilize full disk bandwidth

11.47 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 LFS Basic Idea  Structure the disk a log Periodically, all pending writes buffered in memory are collected in a single segment The entire segment is written contiguously at end of the log  Segment may contain i-nodes, directory entries, data Start of each segment has a summary If segment around 1 MB, then full disk bandwidth can be utilized  Note, i-nodes are now scattered on disk Maintain i-node map (entry i points to i-node i on disk) Part of it is cached, reducing the delay in accessing i-node  This description works great for disks of infinite size

11.48 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 file1 file2 dir1dir2 Unix File System file1file2 dir1 dir2 Log-Structured File System Log inode directory data inode map Blocks written to create two 1-block files: dir1/file1 and dir2/file2, in UFS and LFS UFS vs. LFS

11.49 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 LFS Cleaning  Finite disk space implies that the disk is eventually full Fortunately, some segments have stale information A file overwrite causes i-node to point to new blocks  Old ones still occupy space  Solution: LFS Cleaner thread compacts the log Read segment summary, and see if contents are current  File blocks, i-nodes, etc. If not, the segment is marked free, and cleaner moves forward Else, cleaner writes content into new segment at end of the log The segment is marked as free!  Disk is a circular buffer, writer adds contents to the front, cleaner cleans content from the back

11.50 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Network File System (NFS)  Developed by Sun Microsystems in 1984 Used to join FSes on multiple computers as one logical whole  Used commonly today with UNIX systems  Assumptions Allows arbitrary collection of users to share a file system Clients and servers might be on different LANs Machines can be clients and servers at the same time  Architecture: A server exports one or more of its directories to remote clients Clients access exported directories by mounting them  The contents are then accessed as if they were local  Network Protocol Specification: Independent of OS and hardware

11.52 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Three Major Layers of NFS Architecture  System Call Layer UNIX file-system interface (based on the open, read, write, and close system calls, and file descriptors)  Virtual File System (VFS) layer Maintains table with one entry (v-node) for each open file v-nodes indicate if file is local or remote  If remote, it has hooks (function pointers) to NFS  For local files, FS type and i-node are recorded  NFS service layer – bottom layer of the architecture Implements the NFS protocol

11.54 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 NFS Mount Protocol  Establishes initial logical connection between server and client  Mount operation includes name of remote directory to be mounted and name of server machine storing it Mount request is mapped to corresponding RPC and forwarded to mount server running on server machine Export list – specifies local file systems that server exports for mounting, along with names of machines that are permitted to mount them  If mount request is OK (in server’s export list), server returns a file handle—a key for further accesses File handle – a file-system identifier, and an inode number to identify the mounted directory within the exported file system Subsequent requests use handle provided by server  The mount operation changes only the client’s view and does not affect the server side  Can be done at multiple points: boot time, explicit mounting, automount

11.57 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 NFS Protocol over the Wire  Supports directory and file access via remote procedure calls (RPCs)  All UNIX system calls supported other than open & close  Open and close are intentionally not supported For a read, client sends lookup message to server Server looks up file and returns handle Unlike open, lookup does not copy info in internal system tables Subsequently, read contains file handle, offset and num bytes Each message is self-contained  Pros: server is stateless, i.e. no state about open files Exception is new V4 which introduces state  Cons: Locking is difficult, no concurrency control

11.58 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 How NFS works?  Mount: Sys ad calls mount program with remote dir, local dir Mount program parses for name of NFS server Contacts server asking for file handle for remote dir If directory exists for remote mounting, server returns handle Client kernel constructs v-node for remote dir Asks NFS client code to construct r-node for file handle  Open: Kernel realizes that file is on remotely mounted directory Finds r-node in v-node for the directory NFS client code then opens file, enters r-node for file in VFS, and returns file descriptor for remote node

11.59 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 NFS Caching  Caching is used for performance File blocks cache File attributes cache  Mechanism: Each cache block has a timer (3 sec for data, 30 sec for dir)  Entry is discarded when timer expires On open of cached file, its last modify time on server is checked  If cached copy is old, it is discarded; new one fetched Every 30 sec, cache time expires  All dirty blocks are written back to the server  Clients do not free delayed-write blocks until the server confirms that the data have been written to disk

11.60 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 NFS Path-Name Translation  Performed by breaking the path into component names and performing a separate NFS lookup call for every pair of component name and directory vnode  To make lookup faster, a directory name lookup cache on the client’s side holds the vnodes for remote directory names

11.62 Silberschatz, Galvin and Gagne ©2007 Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Example: WAFL File System  Used on Network Appliance “Filers” – distributed file system appliances  “Write-Anywhere File Layout”  Serves up NFS, CIFS, http, ftp  Random I/O optimized, write optimized NVRAM for write caching  Similar to Berkeley Fast File System, with extensive modifications

Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Silberschatz, Galvin and Gagne ©2007 Chapter 11: File System Implementation.

Similar presentations

Presentation on theme: "Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Silberschatz, Galvin and Gagne ©2007 Chapter 11: File System Implementation."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Silberschatz, Galvin and Gagne ©2007 Chapter 11: File System Implementation.

Similar presentations

Presentation on theme: "Operating System Concepts with Java – 7 th Edition, Nov 15, 2006 Silberschatz, Galvin and Gagne ©2007 Chapter 11: File System Implementation."— Presentation transcript:

Similar presentations

About project

Feedback