UNIX File Systems (Chap 4. in the book “the design of the UNIX OS”) Acknowledgement : Soongsil Univ. Presentation Materials.

Slides:



Advertisements
Similar presentations
Operating Systems File Management.
Advertisements

Concepts about the file system 2. The disk structure 3. Files in disk – The ext2 FS 4. The Virtual File System (c) 2013, Prof. Jordi Garcia.
Free Space and Allocation Issues
File Systems.
Operating system services Program execution I/O operations File-system manipulation Communications Error detection Resource allocation Accounting Protection.
Chapter 10: File-System Interface
File System Interface CSCI 444/544 Operating Systems Fall 2008.
Chapter 11: File System Implementation
Lecture 3 & 4: FILE SYSTEM.
File System Implementation CSCI 444/544 Operating Systems Fall 2008.
Files. System Calls for File System Accessing files –Open, read, write, lseek, close Creating files –Create, mknod.
Introduction to Kernel
1 Operating Systems Chapter 7-File-System File Concept Access Methods Directory Structure Protection File-System Structure Allocation Methods Free-Space.
Ceng Operating Systems
6/24/2015B.RamamurthyPage 1 File System B. Ramamurthy.
1 File Management in Representative Operating Systems.
File System Structure §File structure l Logical storage unit l Collection of related information §File system resides on secondary storage (disks). §File.
1 Friday, July 07, 2006 “Vision without action is a daydream, Action without a vision is a nightmare.” - Japanese Proverb.
7/15/2015B.RamamurthyPage 1 File System B. Ramamurthy.
Secondary Storage Management Hank Levy. 8/7/20152 Secondary Storage • Secondary Storage is usually: –anything outside of “primary memory” –storage that.
1 Project: File System Textbook: pages Lubomir Bic.
1 File Systems Chapter Files 6.2 Directories 6.3 File system implementation 6.4 Example file systems.
File Systems (1). Readings r Silbershatz et al: 10.1,10.2,
Rensselaer Polytechnic Institute CSCI-4210 – Operating Systems David Goldschmidt, Ph.D.
UNIX File Systems (based on Chap 4. in the book “the design of the UNIX OS”) Acknowledgement : Soongsil Univ. Presentation Materials 단국대 최종무 교수님 강의노트.
1Fall 2008, Chapter 11 Disk Hardware Arm can move in and out Read / write head can access a ring of data as the disk rotates Disk consists of one or more.
File System Implementation Chapter 12. File system Organization Application programs Application programs Logical file system Logical file system manages.
Chapter 4. INTERNAL REPRESENTATION OF FILES
File Systems CSCI What is a file? A file is information that is stored on disks or other external media.
10/23/ File System Architecture. 10/23/ / bin unixdev etc user jim mike x y z tty00 tty01 File System architecture.
File Management Chapter 12. File Management File management system is considered part of the operating system Input to applications is by means of a file.
File System Implementation
Chapter 4. INTERNAL REPRESENTATION OF FILES
Files & File system. A Possible File System Layout Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved
Computer Science Lecture 19, page 1 CS677: Distributed OS Last Class: Fault tolerance Reliable communication –One-one communication –One-many communication.
Chapter 11: File System Implementation Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Chapter 11: File System Implementation Chapter.
Disk & File System Management Disk Allocation Free Space Management Directory Structure Naming Disk Scheduling Protection CSE 331 Operating Systems Design.
Chapter 16 File Management The Architecture of Computer Hardware and Systems Software: An Information Technology Approach 3rd Edition, Irv Englander John.
CE Operating Systems Lecture 17 File systems – interface and implementation.
1 Chapter 4. INTERNAL REPRESENTATION OF FILES THE DESIGN OF THE UNIX OPERATING SYSTEM Maurice J. bach Prentice Hall.
UNIX File System (UFS) Chapter Five.
File Systems. 2 What is a file? A repository for data Is long lasting (until explicitly deleted).
© Janice Regan, CMPT 300, May CMPT 300 Introduction to Operating Systems File systems.
Linux File system Implementations
Laface 2007 File system 2.1 Operating System Design Filesystem system calls buffer allocation algorithms getblk brelse bread breada bwrite iget iput bmap.
Foundation of Unix Operating Systems
THE FILE SYSTEM Files long-term storage RAM short-term storage Programs, data, and text are all stored in files, which is stored on.
Lecture 19 Linux/Unix – File System
14.1 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Chapter 10 & 11: File-System Interface and Implementation.
11.1 Silberschatz, Galvin and Gagne ©2005 Operating System Principles 11.5 Free-Space Management Bit vector (n blocks) … 012n-1 bit[i] =  1  block[i]
4P13 Week 9 Talking Points
Operating Systems Files, Directory and File Systems Operating Systems Files, Directory and File Systems.
MINIX Presented by: Clinton Morse, Joseph Paetz, Theresa Sullivan, and Angela Volk.
Computer Science Lecture 19, page 1 CS677: Distributed OS Last Class: Fault tolerance Reliable communication –One-one communication –One-many communication.
Day 28 File System.
File System Implementation
EXT in Detail High-Performance Database Research Center
Introduction to Kernel
Today topics: File System Implementation
Day 27 File System.
File System Implementation
Buffer Cache.
Filesystems.
Chapter 4: System calls for the file system
An overview of the kernel structure
Practical Session 11 File Systems & Midterm 2013
File System B. Ramamurthy B.Ramamurthy 11/27/2018.
UNIX File Systems (Chap 4. in the book “the design of the UNIX OS”)
Internal Representation of Files
Mr. M. D. Jamadar Assistant Professor
Presentation transcript:

UNIX File Systems (Chap 4. in the book “the design of the UNIX OS”) Acknowledgement : Soongsil Univ. Presentation Materials

File System Abstraction used by the kernel to represent and organize the system’s storage resources Properties Hierarchical structure Ability to create and delete files Dynamic growth of files Protection of file data Treatment of peripheral devices as files

UNIX File System Overview

File System Layout

Sample File System

Table of Contents Inodes Structure of a regular file Directories Conversion of a path name to an Inode Super block Inode assignment to a new file Allocation of disk blocks Other file types Summary

Summary Inode is the data structure that describes the attributes of a file, including the layout of its data on disk. Two version of the inode Disk copy : store the inode information when file is not in use In-core copy : record the information about active files. ialloc/ifree : assignment of a disk inode iget/iput : allocation of in-core inodes bmap : locate disk blocks of a file, according to byte offset Directories : files that correlate file name components to inode numbers namei : convert file names to inodes alloc/free : assignment of new disk blocks to a file

Table of Contents Inodes Structure of a Regular File Directories Conversion of a Path Name to an Inode Super Block Inode Assignment to a New File Allocation of Disk Blocks Other File Types Summary

Definition of Inodes Every file has a unique inode Contain the information necessary for a process to access a file Exist in a static form on disk Kernel reads them into an in-core inode to manipulate them.

Contents of Disk Inodes File owner identifier (individual/group owner) File type (regular, directory,..) File access permission (owner,group,other) File access time Number of links to the file (chap5) Table of contents for the disk address of data in a file (byte stream vs discontiguous disk blocks) File size * Inode does not specify the path name that access the file

Sample Disk Inode File owner identifier File type File access permission File access time Number of links to the file Table of contents for the disk address of data in a file File size Owner mjb Group os Type regular file Perms rwxr-xr-x Accessed Oct :45 P.M Modified Oct :3 A.M Inode Oct :30 P.M Size 6030 bytes Disk addresses

Distinction Between Writing Inode and File File change only when writing it. Inode change when changing the file, or when changing its owner, permisson,or link settings. Changing a file implies a change to the inode, But, changing the inode does not imply that the file change.

Contents of The In-core copy of The Inode Fields of the disk inode Status of the in-core inode, indicating whether Inode is locked Process is waiting for the inode to become unlocked Differ from the disk copy as a result of a change to the data in the inode Differ from the disk copy as a result of a change to the file data File is a mount point

Logical device number of the file system Inode number (linear array on disk, disk inode not need this field) Pointers to other in-core inodes Reference count Contents of The In-core copy of The Inode

Accessing Inodes Kernel identifies inodes by their file system and inode number Allocate in-core inodes at the request of higher-level algorithms (in-core inode, by iget algorithm) Kernel maps the device number & inode number into a hash queue Search the queue for the inode …

Block Number & Byte Offset Computing logical disk block number Block number = ((inode number –1) / number of inodes per block) + start block inode list Computing byte offset of the inode in the block ((inode number –1) mod (number of inodes per block)) * size of disk inode

Inode Lock and Reference Count Kernel manipulates them independently Inode lock Set during execution of a system call to prevent other processes from accessing the inode while it is in use. Kernel releases the lock at the conclusion of the system call Inode is never locked across system calls. Reference count Kernel increase/decrease when reference is active/inactive Prevent the kernel from reallocating an active in-core inode

Table of Contents Inodes Structure of a Regular File Directories Conversion of a Path Name to an Inode Super Block Inode Assignment to a New File Allocation of Disk Blocks Other File Types Summary

Direct and Indirect Blocks in Inode direct0 direct1 direct2 direct3 direct4 direct5 direct6 direct7 direct8 direct9 single indirect double indirect triple indirect InodeData Blocks

Byte Capacity of a File System V UNIX. Assume that Run with 13 entries 1 logical block : 1K bytes Block number address : a 32 bit (4byte) integer 1 block can hold up to 256 block number (1024byte / 4byte) 10 direct blocks with 1K bytes each=10K bytes 1 indirect block with 256 direct blocks= 1K*256=256K bytes 1 double indirect block with 256 indirect blocks=256K*256=64M bytes 1 triple indirect block with 256 double indirect blocks=64M*256=16G Size of a file : 4G (2 32 ), if file size field in inode is 32bits

Byte Offset and Block Number Process access data in a file by byte offset. The file starts at logical block 0 and continues to a logical block number corresponding to the file size Kernel accesses the inode and converts the logical file block into the appropriate disk block (bmap algorithm)

Conversion of Byte Offset to Block Number Algorithm bmap /* block map of logical file byte offset to file system block */ Input : inode, byte offset Output: (1)block number in file system, (2)byte offset into block, (3)bytes of I/O in block, (4)read ahead block number calculate logical block number in file from byte offset; calculate start byte in block for I/O;/* output 2 */ calculate number of bytes to copy to user;/* output 3 */ check if read-ahead applicable, mark inode;/* output 4*/ determine level of indirection; while(not at necessary level of indirection) calculate index into inode or indirect block from logical block number in file; get disk block number from inode or indirect block; release buffer from previous disk read, if any (algorithm brelse); if(no more levels of indirection) return (block number); read indirect disk block (algorithm bread); adjust logical block number in file according to level of indirection;

Block Layout of a Sample File and Its inode Single indirect 9156 Double indirect 3333 Data block 367 Data block 075 Byte 9000 in a file -> 8block 808 th byte Byte 350,000 in a file (10K+256K) 816 th byte

Block Entry in the Inode is 0 Logical block entry contain no data. Process never wrote data into the file at that byte offset No disk space is wasted Cause by using the lseek and write system call

Two Extensions to the inode Structure 4.2 BSD file system The more data the kernel can access on the disk in a single operation, the faster file access becomes But it increase block fragmentation Solution : one disk block can contain fragments belonging to several files To store file data in the inode By expanding the inode to occupy an entire disk block The remainder can store the entire file

Table of Contents Inodes Structure of a Regular File Directories Conversion of a Path Name to an Inode Super Block Inode Assignment to a New File Allocation of Disk Blocks Other File Types Summary

Directories A directory is a file Its data is a sequence of entries, each consisting of an inode number and the name of a file contained in the directory Path name is a null terminated character string divided by “/” Each component except the last must be the name of a directory, last component may be a non-directory file

Directory Layout for /etc Byte Offset in Directory Inode Number (2 bytes) File Names init fsck … crash mkfs inittab

Table of Contents Inodes Structure of a Regular File Directories Conversion of a Path Name to an Inode Super Block Inode Assignment to a New File Allocation of Disk Blocks Other File Types Summary

Algorithm for Conversion of a Path Name to an Inode Algorithm namei/* convert path name to inode */ Input : path name Output : locked inode { if(path name starts from root) working inode = root inode (algorithm iget); else working inode = current directory inode (algorithm iget); while(there is more path name){ read next path name component from input; verify that working inode is of directory,access permission OK; if(working inode is of root and component is “..”) continue;/* loop back to while */ read directory (working inode) by repeated use of algorithms bmap,bread and brelse; …

Algorithm for Conversion of a Path Name to an Inode if(component matches an entry in directory (working inode)){ get inode number for matched component; release working inode (algorithm iput); working inode=inode of matched component(algorithm iget); } else/* component not in directory return (no inode); } return (working inode); }

Table of Contents Inodes Structure of a Regular File Directories Conversion of a Path Name to an Inode Super Block Inode Assignment to a New File Allocation of Disk Blocks Other File Types Summary

Super block File System Consists of the size of the file system the number of free blocks in the file system a list of free blocks available on the file system the index of the next free block in the free block list the size of the inode list the number of free inodes in the file system a list of free inodes in the file system the index of the next free inode in the free inode list lock fields for the free block and free inode lists a flag indicating that the super block has been modified boot block super block inode list data blocks

Table of Contents Inodes Structure of a Regular File Directories Conversion of a Path Name to an Inode Super Block Inode Assignment to a New File Allocation of Disk Blocks Other File Types Summary

Inode Assignment to a New File File system contains a linear list of inodes Inode is free : its type field is zero (0) Super block contains an array to cache the numbers of free inodes in the file system (to improve performance)

Algorithm for Assigning New Inodes Algorithm ialloc/* allocate inode */ Input : file system Output : locked inode { while(not done){ if(super block locked) { sleep(event super block becomes free); continue; } if(inode list in super block is empty){ lock super block; get remembered inode for free inode search; search disk for free inodes until super block full, or no more free inodes (bread and brelese); unlock super block; wake up (event super block becomes free); if(no free inodes found on disk) return (no inode); set remembered inode for next free inode search; }

Algorithm for Assigning New Inodes /* there are inodes in super block inode list */ get inode number from super block inode list; get inode (algorithm iget); if(inode not free after all) { write inode to disk; release inode (algorithm iput); continue;/* while loop */ } /* inode is free */ initialize inode; write inode to disk; decrement file system free inode count; return (inode); } // end of while }

Assigning Free Inode from Middle of List free inodes empty array1 Super Block Free Inode List index free inodes 83 empty array2 Super Block Free Inode List index

Assigning Free Inode – Super Block List Empty index 470 empty array1 Super Block Free Inode List index free inodes array2Super Block Free Inode List remembered inode

Algorithm for Freeing Inode Algorithm ifree/* inode free */ Input : file system inode number Output : none { increment file system free inode count; if(super block locked) return; if(inode list full){ if(inode number less than remembered inode for search) set remembered inode for search = input inode number; } else store inode number in inode list; return; }

Placing Free Inode Numbers Into the Super Block free inodes remembered inode Original Super Block List of Free Inodes index Free Inode free inodes remembered inode index Free Inode free inodes remembered inode index

Table of Contents Inodes Structure of a Regular File Directories Conversion of a Path Name to an Inode Super Block Inode Assignment to a New File Allocation of Disk Blocks Other File Types Summary

Linked List of Free Disk Block Numbers ………………………… …………………… …………………… …………………… Super block list

Algorithm for Allocating Disk Block Algorithm alloc /* file system block allocation */ Input : file system number Output : buffer for new block { while(super block locked) sleep (event super block not locked); remove block from super block free list; if(removed last block from free list){ lock super block; read block just taken from free list (algorithm bread); copy block numbers in block into super block; release block buffer (algorithm brelse); unlock super block; wake up processes (event super block not locked); } …

Algorithm for Allocating Disk Block … get buffer form block removed from super block list (algorithm getblk); zero buffer contents; decrement total count of free blocks; mark super block modified; return buffer; }

Requesting and Freeing Disk Blocks 109 ………………………………………………………… …………………………… ………………………………………………… ………………………………. 112 super block list original configuration 109 After freeing block number 949

……………………………… ………………………………. 243 After assigning block number(109) replenish super block free list ……………………………………………………… ……………………………… After assigning block number(949) Requesting and Freeing Disk Blocks

Table of Contents Inodes Structure of a regular file Directories Conversion of a path name to an Inode Super block Inode assignment to a new file Allocation of disk blocks Other file types Summary

Other File Types Pipe fifo(first-in-first-out) Its data is transient: once data is read from a pipe, it cannot be read again Use only direct block (not the indirect block) Special file block device, character device The inode contains the major and minor device number Major number indicates a device type such as terminal or disk Minor number indicates the unit number of the device