1. File Management
Lecture 3

2. What is a file?
File is term applied to anything held on secondary storage
Includes programs (source and executable) text files such as word documents, spreadsheet files, database files etc.

3. File naming
MS DOS format is generally up to eight character name followed by a dot and then a three character extension.
Example Letter1.doc , hello.exe , stock.dat
Linux/Unix file naming is more flexible can have something like myfile.example.doc normally .bin files are equivelent to exe files in windows.

4. Objectives of O.S. Hide complexity of how files are saved from user
Problems with physical addressing, perform error checking, i/o tasks to storage.
Develop file management strategies
Explore files and folders
Create, name, copy, move, and delete folders
Name, copy, move, and delete files
Work with compressed files

5. Organizing Files and Folders
A file, or document, is a collection of data that has a name and is stored in a computer
You organize files by storing them in folders
Disks contain folders that hold documents, or files
Floppy disks
Zip disks
Compact Discs (CDs)
Hard Disks
Removable disks are inserted into a drive

6. Organizing Files and Folders

7. Understanding the Need for Organizing Files and Folders
Windows organizes the folders and files in a hierarchy, or file system
Windows stores folders and important files that it needs when you turn on the computer in the root directory
Folders stored within other folders are called subfolders

8. Understanding the Need for Organizing Files and Folders

9. Directories A directory is a logical grouping of files
All modern operating systems have a directory structure
Why security and housekeeping on system
Example on Linux system only root user will have access to sbin directory.

10. Linux/Unix file structure

11. File Management System
Provides a logical view for the user and hides the physical implementation
Where a file is located and how it is stored on disk is role of OS
Manages directory structures and space allocation for each I/O device
Permits manipulation of data within a file
Requests data transfers from I/O device drivers
File security and protection of file integrity

12. File Management and I/O Functions
Separation between the two allows
I/O devices can change while keeping the file system the same
Redirecting of data is simple

13. File Manager Request Handling

14. File Storage
Over time file sizes change this can be a problem for OS. As files reduce get deleted, compressed etc spaces develop on disk – fragmentation.
OS provides fixed size blocks for storing data, called cluster.
Problem is clusters are often not sequential.

15. File Access Methods Sequential Access Random Access Indexed Access
File is read in sequence from beginning to end
Majority of all files
Program source and binary files
Random Access
Assumes file is made up of fixed length logical records
Hashing is a common method used to calculate the location of an internal logical record
Indexed Access
Additional means for accessing and viewing records in a file
Key indexes

16. Physical File Storage Contiguous Non-contiguous Examples Linked
Indexed
Examples
DOS/Windows FAT
UNIX i-nodes
Windows NTFS
Free space management

17. Contiguous Storage Allocation
Assign blocks (all in a row) to hold the file
Access is simple for both sequential and random methods
Disadvantages
Space must be large enough
Have to take into account file growth
May need to be moved if it outgrows its space
Fragmentation of disk
Allocation strategies to minimize fragmentation
First-fit, best-fit
Eventually disk becomes fragmented

18. Contiguous Storage Allocation

19. Linked Allocation Non-contiguous
Each block contains a link to the next physical block
Variant – links in both directions
Advantages
no fragmentation
Adding to a file is easy
Disadvantages
Not usable for random access
Additional disk head searching
Overhead in storing the pointers
Recovery of a defective block is difficult

20. Linked Allocation

21. MS-DOS FAT File Allocation Table (FAT)
Table contains the first block of each file on the disk or disk partition
Successive blocks contain a link to the next block
Requires a tremendous amount of space
File integrity can be easily compromised

22. MS-DOS FAT Linked Allocation and File Allocation Table
FAT table with file name and address of first cluster

23. Indexed Allocation
Index blocks for indexed allocation of linked files shown in MS-DOS FAT example

24. Indexed Allocation Non-contiguous
All link pointers are stored together in a single block called the index block
One index block per file
Advantages
No fragmentation
Can be used for random access
Disadvantage
Slower due to additional access of the index block
Additional disk head searching
Recovery of a defective block is difficult

25. Unix i-nodes Indexed file allocation Index block contains Advantages
File attributes
10 direct blocks
1 single indirect
1 double indirect
1 triple indirect
Advantages
Fast for small blocks
Can accommodate very large files – 100’s of gigabytes

26. Unix i-nodes

27. Windows 2000 - NTFS Dynamically sized volumes
Volumes may be a fraction of a disk or span many disks
Master File Table (MFT) of 1kb records
1st 16 records are attributes of the MFT ie system files used to manage the volume
Each file has an MFT entry

28. NTFS Volume Layout
Mtf table can store actual data or a list of clusters where data for file is stored. Where mtf stores file it must be small ie under 700bytes.

29. Other Secondary Storage Allocation
Tape Allocation
Not practical to reallocate space in the middle of the tape
Files that grow must be re-written
Files are stored contiguously whenever possible
CD-ROM and DVD-ROM Allocation
Block system described in Chapter 10
Eight levels of subdirectories
Directory format similar to MS-DOS although extensions permit longer filenames and deeper subdirectory levels
Files can be stored non-contiguously

30. Directory Structure
Provides a means of organization so that files can be located easily and efficiently
Hide the physical devices from the logical view of the files
Partitions
Independent subsections of a device
Volume
Directory structure for a particular partition
Needs to be mounted to be incorporated into the overall file system structure
Contain file attributes

31. Tree-Structure Directory
Hierarchical with a top-level root directory from which all other directories stem
All directories and files have names
Separator
Used to indicate subdirectories and files located in a directory
/ UNIX
\ DOS, Windows
Pathname
Absolute – full pathname starting from the root directory
Relative – pathname is created starting from the current directory
Search Paths
Directory locations that the operating system uses to locate files

32. Tree-Structure Directory

33. Linux/Unix Tree Structure

34. RAID Redundant Array of Independent Disks
Redundant Array of Inexpensive Disks
6 levels in common use
Not a hierarchy
Set of physical disks viewed as single logical drive by O/S
Data distributed across physical drives
Can use redundant capacity to store parity information

35. RAID 0 No redundancy Data striped across all disks
Round Robin striping
Increase speed
Multiple data requests probably not on same disk
Disks seek in parallel
A set of data is likely to be striped across multiple disks

36. RAID 1 Mirrored Disks Data is striped across disks
2 copies of each stripe on separate disks
Read from either
Write to both
Recovery is simple
Swap faulty disk & re-mirror
No down time
Expensive

37. RAID 2 Disks are synchronized Very small stripes
Often single byte/word
Error correction calculated across corresponding bits on disks
Multiple parity disks store Hamming code error correction in corresponding positions
Lots of redundancy
Expensive
Not used

38. RAID 3
Similar to RAID 2
Only one redundant disk, no matter how large the array
Simple parity bit for each set of corresponding bits
Data on failed drive can be reconstructed from surviving data and parity info
Very high transfer rates

39. RAID 4 Each disk operates independently Good for high I/O request rate
Large stripes
Bit by bit parity calculated across stripes on each disk
Parity stored on parity disk

40. RAID 5 Like RAID 4 Parity striped across all disks
Round robin allocation for parity stripe
Avoids RAID 4 bottleneck at parity disk
Commonly used in network servers
N.B. DOES NOT MEAN 5 DISKS!!!!!

41. RAID 6 Two parity calculations
Stored in separate blocks on different disks
User requirement of N disks needs N+2
High data availability
Three disks need to fail for data loss
Significant write penalty

42. RAID 0, 1, 2

43. RAID 3 & 4

44. RAID 5 & 6