CS510 Operating System Foundations

Slides:



Advertisements
Similar presentations
Chapter 6 File Systems 6.1 Files 6.2 Directories
Advertisements

File Management.
Chapter 6 File Systems 6.1 Files 6.2 Directories
Chapter 4 : File Systems What is a file system?
File Systems.
File System Interface CSCI 444/544 Operating Systems Fall 2008.
CS503: Operating Systems Spring 2014 General File Systems
Long-term Information Storage
Operating Systems File Systems CNS 3060.
1 File Systems Chapter Files 6.2 Directories 6.3 File system implementation 6.4 Example file systems.
File Systems Topics –File –Directory –File System Implementation Reference: Chapter 5: File Systems Operating Systems Design and Implementation (Second.
File Systems. 2 Storing Information Applications can store it in the process address space Why is it a bad idea? –Size is limited to size of virtual address.
1 Operating Systems Chapter 7-File-System File Concept Access Methods Directory Structure Protection File-System Structure Allocation Methods Free-Space.
1 File Systems Chapter Files 6.2 Directories 6.3 File system implementation 6.4 Example file systems.
Ceng Operating Systems
Why Do We Need Files? Must store large amounts of data. Information stored must survive the termination of the process using it - that is, be persistent.
6/24/2015B.RamamurthyPage 1 File System B. Ramamurthy.
CS 333 Introduction to Operating Systems Class 17 - File Systems Jonathan Walpole Computer Science Portland State University.
1 Friday, July 07, 2006 “Vision without action is a daydream, Action without a vision is a nightmare.” - Japanese Proverb.
7/15/2015B.RamamurthyPage 1 File System B. Ramamurthy.
1 File Systems Chapter Files 6.2 Directories 6.3 File system implementation 6.4 Example file systems.
Chapter 8 File Management
File Systems (1). Readings r Silbershatz et al: 10.1,10.2,
Rensselaer Polytechnic Institute CSCI-4210 – Operating Systems David Goldschmidt, Ph.D.
ITEC 502 컴퓨터 시스템 및 실습 Chapter 10-1: File Systems Mi-Jung Choi DPNM Lab. Dept. of CSE, POSTECH.
File Systems Long-term Information Storage Store large amounts of information Information must survive the termination of the process using it Multiple.
File System Implementation Chapter 12. File system Organization Application programs Application programs Logical file system Logical file system manages.
Operating System Concepts and Techniques Lecture 17
File Systems CSCI What is a file? A file is information that is stored on disks or other external media.
File Systems (1). Readings r Reading: Disks, disk scheduling (3.7 of textbook; “How Stuff Works”) r Reading: File System Implementation ( of textbook)
CSC 322 Operating Systems Concepts Lecture - 20: by Ahmed Mumtaz Mustehsan Special Thanks To: Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall,
CS333 Intro to Operating Systems Jonathan Walpole.
Module 4.0: File Systems File is a contiguous logical address space.
Some basic concepts and information on file systems Portions taken and modified from books by ANDREW S. TANENBAUM.
Disk & File System Management Disk Allocation Free Space Management Directory Structure Naming Disk Scheduling Protection CSE 331 Operating Systems Design.
Chapter 16 File Management The Architecture of Computer Hardware and Systems Software: An Information Technology Approach 3rd Edition, Irv Englander John.
CS 346 – Chapter 11 File system –Files –Access –Directories –Mounting –Sharing –Protection.
CE Operating Systems Lecture 17 File systems – interface and implementation.
Why Do We Need Files? Must store large amounts of data. Information stored must survive the termination of the process using it - that is, be persistent.
File Systems. 2 What is a file? A repository for data Is long lasting (until explicitly deleted).
CS 333 Introduction to Operating Systems Class 17 - File Systems Jonathan Walpole Computer Science Portland State University.
Annotated by B. Hirsbrunner File Systems Chapter Files 5.2 Directories 5.3 File System Implementation 5.4 Security 5.5 Protection Mechanism 5.6 Overview.
File Systems cs550 Operating Systems David Monismith.
Chapter 6 File Systems. Essential requirements 1. Store very large amount of information 2. Must survive the termination of processes persistent 3. Concurrent.
Operating Systems 1 K. Salah Module 4.0: File Systems  File is a contiguous logical address space (of related records)  Access Methods  Directory Structure.
操作系统原理 OPERATING SYSTEMS Chapter 4 File Systems 文件系统.
File System Department of Computer Science Southern Illinois University Edwardsville Spring, 2016 Dr. Hiroshi Fujinoki CS 314.
W4118 Operating Systems Instructor: Junfeng Yang.
Lecture : chapter 9 and 10 file system 1. File Concept A file is a collection of related information defined by its creator. Contiguous logical address.
File-System Management
Today topics: File System Implementation
Operating Systems Chapter 6: File Management
MODERN OPERATING SYSTEMS Third Edition ANDREW S
Chapter 11: File System Implementation
File System Structure How do I organize a disk into a file system?
Filesystems.
File Management.
Chapter 11: File System Implementation
File Systems Implementation
CS510 Operating System Foundations
File System B. Ramamurthy B.Ramamurthy 11/27/2018.
Directory Structure A collection of nodes containing information about all files Directory Files F 1 F 2 F 3 F 4 F n Both the directory structure and the.
Chapter 10: File-System Interface
Files Management – The interfacing
Chapter 16 File Management
Chapter 14: File-System Implementation
File system : Disk Space Management
Department of Computer Science
Chapter 6 File Systems 6.1 Files 6.2 Directories
Chapter 5 File Systems -Compiled for MCA, PU
Presentation transcript:

CS510 Operating System Foundations Jonathan Walpole

Disk Space Management How to keep track of allocated vs free blocks What strategy to use for allocating free blocks to a file

Keeping Track of Free Blocks Approach #1: Keep a bitmap 1 bit per disk block Approach #2 Keep a free list

Keeping Track of Free Blocks Approach #1: Keep a bitmap 1 bit per disk block Example: 1 KB block size 16 GB Disk  16M blocks = 224 blocks Bitmap size = 224 bits  2KB blocks 1/8192 space lost to bitmap Approach #2 Keep a free list

Free List of Disk Blocks Linked list of free blocks Each list block on disk holds A bunch of addresses of free blocks Address of next block in the list null

Free List of Disk Blocks Two kinds of blocks: Free Blocks Blocks containing pointers to free blocks Always keep one block of pointers in memory for fast allocation and freeing Ideally this block will be partially full

Comparison: Free List vs Bitmap Goal: keep all the blocks in one file close together

Comparison: Free List vs Bitmap Goal: keep all the blocks in one file close together Free Lists: Free blocks could be anywhere Allocation comes from (almost) random location

Comparison: Free List vs Bitmap Goal: keep all the blocks in one file close together Free Lists: Free blocks could be anywhere Allocation comes from (almost) random location Bitmap: Much easier to find a free block “close to” a given position Bitmap implementation: Easier to keep entire bitmap in memory

Allocation Strategies Determining which blocks make up a file: Contiguous allocation Linked allocation FAT file system Unix I-nodes

Contiguous Allocation After deleting D and F...

Contiguous Allocation After deleting D and F...

Contiguous Allocation Advantages: Simple to implement (Need only starting sector & length of file) Performance is good (... for sequential reading)

Contiguous Allocation Advantages: - Simple to implement (Need only starting sector & length of file) - Performance is good (... for sequential reading) Disadvantages: - After deletions, disk becomes fragmented - Will need periodic compaction (time-consuming) - Will need to manage free lists If new file put at end of disk... no problem If new file is put into a “hole” we must know maximum file size ... at the time it is created!

Contiguous Allocation Good for CD-ROMs All file sizes are known in advance Files are never deleted

Linked List Allocation Each file is a sequence of blocks First word in each block contains the number of the next block

Linked List Allocation Random access into the file is slow!

File Allocation Table (FAT) Keep the link information in a table in memory One entry per block on the disk Each entry contains the address of the “next” block End of file marker (-1) A special value (-2) indicates the block is free

File Allocation Table (FAT) Random access... Searching the linked list is fast because it is all in memory Directory entry needs only one number: The starting block number Disadvantage: Entire table must be in memory all at once! This is a problem for large capacity file systems

File Allocation Table (FAT) Disadvantage: Entire table must be in memory all at once! Example: 200 GB = disk size 1 KB = block size 4 bytes = FAT entry size 800 MB of memory used to store the FAT

I-nodes Each I-node (“index-node”) is a structure containing info about the file Attributes and location of the blocks containing the file Blocks on disk Other attributes I-node

I-nodes Indexed by Disk virtual block Blocks number Other attributes

I-nodes Enough space for 10 pointers plus indirect Disk pointers Blocks Other attributes I-node

The UNIX I-node

File Storage on Disk

File Storage On Disk Sector 0: “Master Boot Record” (MBR) Contains the partition map Rest of disk divided into “partitions” - Partition: sequence of consecutive sectors. Each partition can hold its own file system Unix file system Window file system Apple file system Every partition starts with a “boot block” Contains a small program This “boot program” reads in an OS from the file system in that partition OS Startup - Bios reads MBR , then reads & execs a boot block

An Example Disk Unix File System

What Is a File? Files can be structured or unstructured Unstructured: just a sequence of bytes Structured: a sequence or tree of typed records In Unix-based operating systems a file is an unstructured sequence of bytes similar to memory

File Structure asd Sequence of bytes Sequence of records Tree

File Extensions Even though files are just a sequence of bytes, programs can impose structure on them, by convention Files with a certain standard structure imposed can be identified using an extension to their name Application programs may look for specific file extensions to indicate the file’s type But as far as the operating system is concerned its just a sequence of bytes

Typical File Extensions

Executable Files Executable files are special The OS must understand the format of executable files in order to execute programs The exec system call needs this information Exec puts program and data in process address space

Executable File Format An archive An executable file

File Attributes Various meta-data needs to be associated with files Owner Creation time Access permissions / protection Size etc This meta-data is called the file attributes Maintained in file system data structures for each file Stored in the I-node in Unix file systems

File Access Models Sequential access Read all bytes/records from the beginning Cannot jump around (but could rewind or back up) convenient when medium was magnetic tape Random access Can read bytes (or records) in any order Move position (seek), then read

File-Related System Calls Create a file Delete a file Open Close Read (n bytes from current position) Write (n bytes to current position) Append (n bytes to end of file) Seek (move to new position) Get attributes Set/modify attributes Rename file

File System Call Usage Examples fd = open (name, mode) byte_count = read (fd, buffer, buffer_size) byte_count = write (fd, buffer, num_bytes) close (fd)

File Bytes vs Disk Sectors Files are sequences of bytes - Granularity of file I/O is bytes Disks are arrays of sectors (512 bytes) - Granularity of disk I/O is sectors - Files data must be stored in sectors File systems define a block size - Block size = 2n * sector size - Contiguous sectors are allocated to a block File systems must resolve the granularity mismatch - find, read/write complete blocks that contain the bytes specified in the file read or write calls

Naming Files How do we find a file, or a file’s I-node, given its name? How can we ensure that file names are unique?

Single Level Directories Sometimes called folders Early OSs had single-Level Directory Systems - Sharing amongst users was a problem Appropriate for small, embedded systems Root Directory a b c d

Two-Level Directories Each user has a directory /peter/g Root Directory harry peter todd micah a b c d e c g a d b e

Hierarchical Directories A tree of directories Interior nodes: Directories Leaves: Files / A B C i D j E F k l m n G H o p q

Hierarchical Directories Root Directory A tree of directories Interior nodes: Directories Leaves: Files / A B C User’s Directories i D j E F k l m n G H Sub-directories o p q

Path Names Windows \usr\jon\mailbox Unix /usr/jon/mailbox Absolute Path Name Relative Path Name - “working directory” (or “current directory”) - Each process has its own working directory

A Unix directory tree . is the “current directory” .. is the parent

Typical Directory Operations Create ... a new directory Delete ... a directory Open ... a directory for reading Close Readdir ... return next entry in the directory Rename ... a directory Link ... add this directory as a sub directory in another Unlink ... remove a “hard link”

Implementing Directories List of files File name File Attributes Simple Approach: Put all attributes in the directory

Implementing Directories Simple approach “Kernel.h” attributes “Kernel.c” attributes “Main.c” attributes “Proj7.pdf” attributes “temp” attributes “os” attributes • •

Implementing Directories List of files File name File Attributes Unix Approach: Directory contains I-Node number I-Node contains - File Attributes

Implementing Directories Unix approach i-node i-node “Kernel.h” “Kernel.c” i-node “Main.c” “Proj7.pdf” i-node “temp” i-node “os” • • i-node

Sharing Files / A B C i D j E F k l m G H n o p q One file appears in several directories Tree  DAG / A B C i D j E F k l m G H n o p q

Sharing Files / A B C i D j E F k l m G H n o p q One file appears in several directories Tree  DAG (Directed Acyclic Graph) / A B C i D j E F k l m G H n o p q

Sharing Files / A B C i D j E F k l m G H n o p q One file appears in several directories Tree  DAG (Directed Acyclic Graph) / A B C i D j E F k l m What if the file changes? New disk blocks are used. Better not store this info in the directories!!! G H n o p q

Hard Links and Symbolic Links In Unix: Hard links Both directories point to the same i-node Symbolic links One directory points to the file’s i-node Other directory contains the “path”

Hard Links / A B C i D j E F k l m G H n o p q

Hard Links Assume i-node number of “n” is 45 / A B C i D j E F k l m G p q

Hard Links Assume i-node number of “n” is 45 Directory “D” / “m” 123 • • i D j E F k l Directory “G” “n” 45 m G H “o” 87 • • n o p q

Hard Links Assume i-node number of “n” is 45. Directory “D” / “m” 123 The file may have a different name in each directory /B/D/n1 /C/F/G/n2 Assume i-node number of “n” is 45. Directory “D” / “m” 123 “n” 45 A B C • • i D j E F k l Directory “G” “n” 45 m G H “o” 87 • • n o p q

Symbolic Links Assume i-node number of “n” is 45. / A B C i D j E F k Hard Link i D j E F k l m G H n o Symbolic Link p q

Symbolic Links Assume i-node number of “n” is 45. Directory “D” / “m” 123 “n” 45 A B C • • Hard Link i D j E F k l m G H n o Symbolic Link p q

Symbolic Links Assume i-node number of “n” is 45. Directory “D” / “m” 123 “n” 45 A B C • • Hard Link i D j E F k l Directory “G” “n” /B/D/n m G H “o” 87 • • n o Symbolic Link p q

Symbolic Links Assume i-node number of “n” is 45. Directory “D” / “m” 123 “n” 45 A B C • • i D j E F k l Directory “G” “n” 91 m n G H “o” 87 • • Separate file i-node = 91 “/B/D/n” o Symbolic Link p q

Deleting a File Directory entry is removed from directory All blocks in file are returned to free list What about sharing??? Multiple links to one file (in Unix) Hard Links Put a “reference count” field in each i-node Counts number of directories that point to the file When removing file from directory, decrement count When count goes to zero, reclaim all blocks in the file Symbolic Link Remove the real file... (normal file deletion) Symbolic link becomes “broken”

Example: open,read,close fd = open (filename,mode) Traverse directory tree find i-node Check permissions Set up open file table entry and return fd byte_count = read (fd, buffer, num_bytes) figure out which block(s) to read copy data to user buffer return number of bytes read close (fd) reclaim resources

The UNIX File System

Example: open,write,close byte_count = write (fd, buffer, num_bytes) figure out how many and which block(s) to write Read them from disk into kernel buffer(s) copy data from user buffer send modified blocks back to disk adjust i-node entries return number of bytes written

Spare Slides

Implementing Filenames Short, Fixed Length Names MS-DOS/Windows 8 + 3 “FILE3.BAK” Each directory entry has 11 bytes for the name Unix (original) Max 14 chars

Implementing Filenames Short, Fixed Length Names MS-DOS/Windows 8 + 3 “FILE3.BAK” Each directory entry has 11 bytes for the name Unix (original) Max 14 chars Variable Length Names Unix (today) Max 255 chars Directory structure gets more complex

Variable-Length Filenames

Variable-Length Filenames