1. INTRODUCTION TO UNIX: The File System
© Ronald J. Leach All Rights Reserved

2. © Ronald J. Leach All Rights Reserved
UNIX files
Files are streams of bytes
No assumption about file organization, database design, size of records, is made by the operating system.
Any other file structure is imposed by applications accessing the file.
© Ronald J. Leach All Rights Reserved

3. © Ronald J. Leach All Rights Reserved
File organization
Files are organized into directories
Tree structure (almost)
Files can are included in file systems
Files are accessed indirectly via i-nodes
© Ronald J. Leach All Rights Reserved

4. Typical UNIX file systems/
/usr
/usr2
/home
/var
/pub
© Ronald J. Leach All Rights Reserved

5. © Ronald J. Leach All Rights Reserved
Types of files
Directory
Ordinary
Special
device drivers
sockets
links
FIFOs
etc.
© Ronald J. Leach All Rights Reserved

6. © Ronald J. Leach All Rights Reserved
File names
Text, often limited to 32, 16, or 8 characters
Usually must start with a letter
Cannot contain a ‘/’ symbol
© Ronald J. Leach All Rights Reserved

7. © Ronald J. Leach All Rights Reserved
Path names
Relative, such as:
file1 . ..
../newdir
Full, such as
/home/rjl/file1
© Ronald J. Leach All Rights Reserved

8. © Ronald J. Leach All Rights Reserved
Disk Organization
UNIX makes few assumptions about organization of physical disks on which file systems reside
This view is consistent with UNIX flexibility
© Ronald J. Leach All Rights Reserved

9. Major sections of a UNIX disk
Bootblock
Superblock
Array of i-nodes
Data in the actual files
© Ronald J. Leach All Rights Reserved

10. © Ronald J. Leach All Rights Reserved
The bootblock
logical block # 0
used when the system is booted initially or rebooted after a system shutdown
never used during normal operation
© Ronald J. Leach All Rights Reserved

11. © Ronald J. Leach All Rights Reserved
The superblock
logical block # 1
keeps track of essential information that is for the file system to work properly:
Number of blocks used to store list of i-nodes
Size of the file system in blocks
Number of free blocks
List of free blocks
© Ronald J. Leach All Rights Reserved

12. Superblock contents, continued
List of free i-nodes
A flag to indicate that the superblock has been modified
Time of the last update of the superblock
A count of the number of free blocks and free i-nodes
File system name
© Ronald J. Leach All Rights Reserved

13. © Ronald J. Leach All Rights Reserved
Array of i-nodes
Usually stored in contiguous disk blocks
The elements of the array are indexed by an int called an i-number
© Ronald J. Leach All Rights Reserved

14. © Ronald J. Leach All Rights Reserved
Contents of an i-node
File type (ordinary, directory, or special)
Number of links to the file
File owner's user ID, group ID
File access permissions (3 sets)
File size in bytes
Time and date of creation, last access, modification
© Ronald J. Leach All Rights Reserved

15. i-node contents, continued
Pointers to logical disk blocks containing file data (or pointers)
There are typically a maximum of 13 such pointers in an i-node
An i-node associated with a file does not contain the file’s name
© Ronald J. Leach All Rights Reserved

16. Accessing files by i-number
Directories are tables with two columns:
names of files
i-numbers
ls prints the names of files in the directory
ls -i prints the names of the files in the directory together with their i-numbers
ncheck -i 10 /dev/sd0a gives a file name to a disk
© Ronald J. Leach All Rights Reserved

17. Accessing files by name
Get request for file "filename”
Parse filename string to determine directory to start search
Use array of i-nodes to find directory
From directory, find the i-number
Find place where (i-node, i-number) pairs are stored
© Ronald J. Leach All Rights Reserved

18. Accessing files by name, cont.
Read i-node and find where on disk the file is stored
Go to place on disk where file is stored
Continue process until file is located on disk, repeating process for each directory file within the filename string
© Ronald J. Leach All Rights Reserved

19. © Ronald J. Leach All Rights Reserved
This process is slow!
Lots of disk access to locate file information
Indirect access!
Most UNIX systems use an “in-core i-node table” to speed up file location
The in-core and disk i-node tables must be synchronized, roughly every 30 seconds
© Ronald J. Leach All Rights Reserved

20. File access is also indirect
Most UNIX file systems must accommodate small, intermediate and large files
Using contiguous collections of bytes for file storage may be very slow for large files
i-nodes contain pointers to disk blocks
© Ronald J. Leach All Rights Reserved

21. © Ronald J. Leach All Rights Reserved
A file access example
System block size = 1024 bytes
Addresses are 32 bit 4 byte integers.
The 13 disk pointers in an i-node are:
10 direct pointers
1 singly indirect pointer
1 doubly indirect pointer
1 triply indirect pointer
© Ronald J. Leach All Rights Reserved

22. A file access example, cont.
The 10 direct pointers can directly access a total of 10 disk blocks
10 * 1,024 or 10,240 bytes
This is adequate for very small files
© Ronald J. Leach All Rights Reserved

23. A file access example, cont.
The singly indirect pointer points to a disk block that contains a maximum of 1,024/4 (=256) four-byte pointers
Each pointer can point to a disk block containing 1,024 bytes of data in a block
Single indirection increases the maximum possible size of a file by 256 * 1024 or 262,144 bytes
© Ronald J. Leach All Rights Reserved

24. A file access example, cont.
In double indirection, there are 256 pointers to disk blocks, each of which is a pointer to 256 disk blocks
Each disk block can contain 1,024 bytes of data
A total of 256 * 256 * 1,024 or 67,108,864 bytes available
© Ronald J. Leach All Rights Reserved

25. A file access example, cont.
For triple indirection
256 pointers to disk blocks
each of which is a pointer to 256 disk blocks
each of which can contain 1024 bytes of data
A total of 256 * 256 * 256 * 1,024 or 17,179,869,184 bytes
Maximum file size = 17,247,258,432 bytes.
© Ronald J. Leach All Rights Reserved

26. Maximum file size is tunable
Can be set by the system administrator
Many systems do not allow large files for performance reasons
A limit of one or two megabytes is common for systems used primarily for teaching
Such systems use fewer than the 13 maximum possible number of pointers to disk blocks in an i-node
© Ronald J. Leach All Rights Reserved

27. Directories are not always trees
One field in an i-node is the number of links
A newly created file always has at least one link - to the directory the file is created in
The file may be linked to other directories by using the ln shell command or within a C program by using the link() system call
© Ronald J. Leach All Rights Reserved

28. Why files might have several links
Applications might be installed with “unusual” location of library, directory files
Not reasonable to move existing files from their current directories
Instead, create a link to the new directory by
ln olddir newdir
This increases the "number of links" by 1
© Ronald J. Leach All Rights Reserved

29. Linked files save file systems room
Files can be moved a new file system, still accessed directly from the original file system by a “symbolic link”
ln -s olddir newdir
Frees up space in one file system, at the expense of the other
“Symbolic link" renames files - does not increment "number of links" field of i-node
© Ronald J. Leach All Rights Reserved

30. Link files for backward compatibility
/usr and /var are often symbolically linked on newer UNIX systems
Important to allow application programs access to directories they’re designed to use
Symbolic links allow application programs to work even if directory organization has changed
© Ronald J. Leach All Rights Reserved

31. © Ronald J. Leach All Rights Reserved
Links, cont.
rm filename
removes one link to the file from the current directory
When the number of links to a file becomes 0, the file can no longer be accessed by the UNIX file system (although the disk blocks containing the file's data are not erased from the disk)
© Ronald J. Leach All Rights Reserved

32. © Ronald J. Leach All Rights Reserved
Links, cont.
Can link, unlink in a program using:
link(char * name); unlink(char *name);
On many UNIX systems, a link is a special file created by the mknod() system call with an appropriate set of arguments
Can also use the system call system() as in
system("rm filename");
system("ls -s "file1 file2);
© Ronald J. Leach All Rights Reserved

33. © Ronald J. Leach All Rights Reserved
Disk Scheduling
Disk reads, writes do not happen instantaneously
UNIX attempts to maximize throughput for a multiple process computer system
Disk write access requests are queued up in a special software data structure called the buffer cache in the UNIX kernel
© Ronald J. Leach All Rights Reserved

34. © Ronald J. Leach All Rights Reserved
The buffer cache
A complex data structure to allow storage of disk request data so that it can be processed by the disk scheduling subsystem at the “appropriate time”
The disk scheduling scheme may be C-SPAN", "Eschenbach scheme," or any commonly used disk scheduling algorithm
© Ronald J. Leach All Rights Reserved

35. Alternate Views of File System Organization
At the highest levels, open a file, read data from or write data, and close the file
At the lowest level, hardware interprets electric and magnetic fields
The system call level is intermediate: processes request the services of the kernel and the level where commands are sent to the disk and memory controllers
© Ronald J. Leach All Rights Reserved

36. © Ronald J. Leach All Rights Reserved

37. © Ronald J. Leach All Rights Reserved
The syscall vector
An array of pointers to lower-level routines run when a system call is executed
namei (parses names)
iput (allocates, accesses space for i-nodes )
iget (acts on existing i-nodes),
alloc (allocates free disk blocks for storage i-nodes), ...
not available directly to programmer
© Ronald J. Leach All Rights Reserved

38. C code with file pointers to echo a file
#include <stdio.h>
main()
{int ch; FILE *fp;
fp = fopen("infile", "r");
while ((ch = getc(fp)) != EOF)
putchar(ch);
fclose(fp); }
© Ronald J. Leach All Rights Reserved

39. C code with file descriptors
#include <stdio.h>
#define BUFFSIZE 1
main()
{ int fd: char buf[BUFFSIZE];
fd = open("infile", 0644);
while (read(fd,buf,BUFFSIZE) == BUFFSIZE)
putchar(buf[0]);
close(fd); }
© Ronald J. Leach All Rights Reserved

40. These are high-level programs
What about lower level routines?
Compile the C programs with the standard -p option turns on the profiler
The profiler takes over execution of the executable file by means of process tracing
The profiler creates mon.out
Use utility prof (gprof) for detailed information about execution time
© Ronald J. Leach All Rights Reserved

41. Program to read directory contents
/* The directory name is specified as a */
/* command-line argument. The size of each */
/* ordinary file in the directory is given in bytes. */
#include <fcntl.h>
#include <sys/types.h>
#include <dirent.h>
#include <sys/stat.h>
#include <stdio.h>
#include <errno.h>
© Ronald J. Leach All Rights Reserved

42. Directory program, cont.
main (int argc, char *argv[] )
{extern int errno; DIR *dir_ptr; struct dirent *temp ;
struct stat *stat_buf = (struct stat *) malloc
(sizeof( struct stat ));
if (argc != 2)
{fputs("Error: not enough arguments", stderr);
exit(1); }
© Ronald J. Leach All Rights Reserved

43. Directory program, cont.
if ( (dir_ptr = opendir(argv[1]) ) == NULL) {
perror(argv[1]);
exit(1); }
© Ronald J. Leach All Rights Reserved

44. Directory program, cont.
for(temp = readdir(dir_ptr); temp != NULL; temp = readdir(dir_ptr) )
{if (stat(temp->d_name, stat_buf) == -1)
perror("stat");
if ((stat_buf->st_mode & S_IFMT) == S_IFREG)
printf("%-14s %d\n", temp->d_name,
stat_buf->st_size);
else printf("DIRECTORY FILE : %-14s\n", temp->d_name); }
closedir(dir_ptr); }/* end program */
© Ronald J. Leach All Rights Reserved

45. Sample output, not formatted
DIRECTORY FILE : .
DIRECTORY FILE : ..
fork.c
awkfile
poll.c
inode_check.sh 274
a.out
© Ronald J. Leach All Rights Reserved

46. © Ronald J. Leach All Rights Reserved
SVR4 file systems
allows file systems of different types to be mounted on the same disk or disks
ufs file system, BSD-type, names as long as 256 bytes, blocksize as large as 8KB.
s5 file system, earlier System V-type, max name 14 bytes, block sizes one-half to 2 KB
© Ronald J. Leach All Rights Reserved

47. © Ronald J. Leach All Rights Reserved
SVR4 file systems, cont.
“process file system,” - another process may have access to the address space of a process simply by knowing the process ID. Names of files are replaced by ints, since these are valid for process IDs
NFS (network file system)
RFS (remote file system)
© Ronald J. Leach All Rights Reserved

48. © Ronald J. Leach All Rights Reserved
Linux file systems
Directory structure is fairly consistent with other UNIX variants
“Fast file system” can be much faster, since fewer indirect access are commonly used
© Ronald J. Leach All Rights Reserved