File Management Chase Goehring

What is a File? “From a user’s perspective, a file is the smallest allotment of logical secondary storage; that is, data cannot be written to secondary storage unless they are within a file.” - Silberschatz et al. In other words… It’s what everything on your hard drive is stored in.

Properties of Files Name - what the human user can identify the file by Identifier - what the computer can identify the file by Type - what kind of file it is Location - where it is on the storage device Size - number of bytes Protection - read/write permissions, execution, etc. Time, Date - shows when a file was created, last modified, or used

File Operations Create a file - Space allocated on disk, then file entry is placed on directory Write to a file - information is added to a file in the location of the write pointer Read a file - get information from a file from the read pointer’s location Reposition - change the location of the read/write pointer Delete a file - release all file space and erase directory entry Truncate a file - keep properties but erase contents

File Organization Extremely wide variety of file organization implementations Stallings’ top 5: Pile Sequential File Indexed Sequential File Indexed File B-Tree

The Pile Least complicated form of file organization Data collected in the order it arrives Each record is a single “burst” of data Each record must be self-describing (include field name and value, length of field must be delimited or included as a subfield) Records can only be accessed by exhaustive search Works well when stored data varies in size and structure Typically used when data is not easy to organize Unsuitable for most applications

The Sequential File Most common form of file structure Fixed length records w/ fixed length fields Key field stores record identifier Records ordered by key sequence Typically used in batch applications Poor performance for applications that involve queries/ updates of individual records Access requires key match or sequential search

Indexed Sequential File Index added to sequential file structure to support random access Index provides for quick searches Includes Overflow file Multiple levels of indexing possible

Indexed File Multiple indexes, one for each type of searchable field No sequentiality, no single key Record access only through indexes Two indexes: Main, Partial Main index - sequential exhaustive index Partial index - entries to records with field of interest Useful for situations where information is needed quickly Not very good for exhaustive searching

B-Trees B-Tree: Balanced-Tree structure of an index file Sequential file of indexes is not fast enough for large databases All branches are of equal length Structure found in Mac OS X, Windows, and many Linux file systems

Properties of B-Trees B-Trees have a minimum degree d Every node has at most 2d-1 keys and 2d children Every node has at least d-1 keys and d pointers (is at least half full) Root has at least one key and 2 children All leaves are on the same level Nonleaf nodes with k pointers contain k-1 keys

B-Tree insertions

Directories A directory is a file accessible by file management routines Directories store information on other files Basic information - File name, type, organization Address information - Volume, Starting address, size used/allocated Access Control information - permissions Usage information - date created, last modified, etc.

Directory Structures List of entries Entire structure is a sequential file with the filename being the key Bad for systems with many files or multiple users Hierarchical Tree Allows for directories and subdirectories

Structure of Tree Directory

File System Properties Users can create, read, write, delete, modify files Users should have controlled access to other user’s files Each user should be able to move data between files Each user should be able to back up and recover user’s files in case of damage Users should see files by filename instead of identifier

Basic File System Structure Device drivers - communicate with devices, start I/O operations, process I/O completion (considered part of OS) Basic file system - physical i/o, deals with blocks of data exchanged with disk/tape (considered part of OS) Basic I/O supervisor - responsible for all I/O initiation and termination, is part of OS Logical I/O - enables users and apps to access records Access method - provides interface between applications and file systems and devices

FAT 32 File System Introduced in 1996 to Windows 95 FAT = File allocation table: holds start of file cluster and file size Has a backup file allocation table stored after the first one in the volume Max filesize = 4GB Max volume size = 2TB Does not support file permissions Still used today in portable drives, like USB or SDHC cards

NTFS file system NTFS = New technology file system Introduced in July 1993 with Windows NT 3.1 Uses master file table, which is duplicated much like FAT MFT allows for faster access to files than FAT Uses B+ Tree instead of table for directory contents Maximum file size = 16 exabytes (exabyte = 1,000 GB) JOURNALING - changes to volume are saved in a log before they are committed = less corrupted data

HSF+ Introduced January 1998 with Mac OS 8.1 HSF+ = The Mac OS Extended Volume Hard Drive Format Volume header keeps information about volume, has a backup at end of volume Catalog file holds all folder and file hierarchy information, organized as B-Tree Max volume size = 8 exabytes Journaling as of 2002

References “A Short History of the BTree.” A Short History of the BTree | Perforce, Perforce, 28 Sept. 2011, www.perforce.com/blog/short-history-btree. Hoffman, Chris. “Why Do Removable Drives Still Use FAT32 Instead of NTFS?” How-To Geek, How-To Geek, 5 July 2017, www.howtogeek.com/177529/htg-explains-why-are-removable-drives-still-using-fat32-instead-of-ntfs/. “NTFS: FAT File System.” NTFS.com. The FAT File System, www.ntfs.com/fat-systems.htm. Silberschatz, Abraham, et al. Operating System Concepts, 8th Edition. 8th ed., John Wiley & Sons, 2008. Stallings, William. Operating systems: Internals and Design Principles. 8th ed., Pearson, 2015. Tanenbaum, Andrew S. Modern Operating Systems. 3rd ed., Pearson Education, 2009. “Technical Note TN1150 HFS Plus Volume Format.” Technical Note TN1150: HFS Plus Volume Format, dubeiko.com/development/FileSystems/HFSPLUS/tn1150.html.