1. Advanced Programming in the Unix Environment
Ch 4. Files and Directories
System Programming

2. Contents File Attributes File Types
IDs in Unix systems and File Access Permissions
Overview of the Unix File System
Symbolic link
File Times
Reading Directories
Special Device Files
Functions
File access permissions : access, umask, chmod, chown
File system operations: link, unlink, remove, rename
Symbolic links: symlink readlink
Directory Functions : mkdir, rmdir, chdir, getcwd
Sync functions: sync, fsync
System Programming

3. File Attributes Stat structure struct stat {
mode_t st_mode; /* file type & mode (permissions) */
ino_t st_ino; /* i-node number (serial number)*/
dev_t st_dev; /* device number (filesystem)*/
dev_t st_rdev; /* device number for special files */
nlink_t st_nlink; /* number of links */
uid_t st_uid; /* user ID of owner */
gid_t st_gid; /* group ID of owner */
off_t st_size; /* size in bytes, for regular files */
time_t st_atime; /* time of last access */
time_t st_mtime; /* time of last modification */
time_t st_ctime; /* time of last file status change */
long st_blksize; /* best I/O block size */
long st_blocks; /* number of 512-byte blocks allocated */ };
System Programming

4. stat, fstat, and lstat functions
int stat(const char *pathname, struct stat *buf);
int fstat(int fildes, struct stat *buf);
int lstat(const char *pathname, struct stat *buf);
all three return: 0 if OK, -1 on error
stat() returns a structure of information on the named file
fstat() obtains information about the file that is already open on descriptor fildes
lstat() is similar to stat, but returns information about the symbolic link.
buf : a pointer a structure that will be filled by the functions.
ls commands use the functions
System Programming

5. File types Regular file Directory file Character special file
A file that contains the names of other files and pointers to information on these files
Character special file
A type of file used for certain types of devices
Block special file
A type of file typically used for disk devices
FIFO (Named pipe)
A type of file used for interprocess communication
Socket
A type of file used for network communication
Symbolic link
A type of file that points to another file
System Programming

6. File types (cont’d) Encoded in st_mode
macros defined in <sys/stat.h>
int main(int argc, char *argv[]) {
int i; char *ptr;
struct stat buf;
for (i = 1; i < argc; i++) {
printf("%s: ", argv[i]);
if (lstat(argv[i], &buf) < 0) {
err_ret("lstat error");
continue; }
if (S_ISREG(buf.st_mode)) ptr = "regular";
else if (S_ISDIR(buf.st_mode)) ptr = "directory";
else if (S_ISCHR(buf.st_mode)) ptr = "character special";
else if (S_ISBLK(buf.st_mode)) ptr = "block special";
else if (S_ISFIFO(buf.st_mode)) ptr = "fifo";
...
else ptr = "** unknown mode **";
printf("%s\n", ptr);
exit(0);
S_ISLNK()
regular file
S_ISFIFO()
S_ISBLK()
S_ISCHR()
S_ISDIR()
S_ISREG()
S_ISSOCK()
directory file
character specific file
block specific file
pipe or FIFO
symbolic link
socket
File Type
Macro
System Programming

7. IDs in Unix IDs associate with every process
real user and group ID : who we really are
from the password file when we log in (by login)
only superuser process can change it, otherwise never change.
effective user and group ID, supplementary group IDs
determine file access permissions
usually effective user and group ID are real user and group ID
saved set-user-ID, saved set-group-ID : saved by exec functions
contain copies of the effective user and group ID when a program executed
System Programming

8. Process ID vs File Owner
Owner  st_uid
Group owner  st_gid
Usually when process executes
Effective (group) ID equals real (group) ID
Exception
Special flag (in file’s st_mode): set-user-ID & set-group-ID
“when this file is executed, set the effective user ID of the process to be the owner of the file”
passwd(1) program is a set-user-ID program
System Programming

9. Set-User-ID and Set-Group-ID
Encoded in st_mode
Set-user-ID (S_ISUID)
regular file
Set effective user ID on execution to the program file’s owner ID.
ex) passwd(1) runs as the root
-r-s--x--x 1 root root Sep 26 00:52 passwd
directory : not used
Figure 4.6
#define S_IRUSR /* read permission: owner */
#define S_IWUSR /* write permission: owner */
#define S_IXUSR /* execute permission: owner */
#define S_IRGRP /* read permission: group */
#define S_IWGRP /* write permission: group */
#define S_IXGRP /* execute permission: group */
#define S_IROTH /* read permission: other */
#define S_IWOTH /* write permission: other */
#define S_IXOTH /* execute permission: other */
System Programming

10. File Access Permissions
All the file types have permissions (encoded in st_mode)
User/Group/Other – Read/Write/Execute (Figure 4.6)
File’s owner and group owner IDs (specified by st_uid and st_gid)
ex) -rwxr-xr-x 1 sikim sal Sep test
File access rules
Read permission for a file to open the file with O_RDONLY and O_RDWR.
Write permission for a file to open the file with O_WRONLY, O_RDWR and O_TRUNC.
Execution permission for a file to execute the file using exec functions (The file also has to be a regular file).
System Programming

11. File Access Permissions (cont’d)
Directory access rules
Read permission for a directory to obtain a list of all the file names in the directory.
Execution permission for a directory to pass through the directory when it is a component of a pathname to access.
Ex) to access file /usr/dict/words, need execution permissions for /, /usr, /usr/dict
Write and execution permissions for a directory to create a new file in the directory.
Write and execution permissions for a directory to delete a file in the directory (but no need of read or write permission for the file itself! cf: sticky bit).
System Programming

12. File Access Test
Done by kernel each time a process opens, creates or deletes a file.
Depends on two owner IDs associated with a file and two effective IDs with a process.
The conditions to allow the access of a file.
The effective user ID of the process is 0 (superuser).
The effective user ID equals the owner ID of the file and the appropriate user access permission bit is set, such as user-read bit for read access.
The effective group ID or one of the supplementary group IDs of the process equals the group ID of the file and the appropriate group access permission bit is set.
The appropriate other access permission bit is set.
The four steps are tried in sequence.
If the access is denied at any step, no further step is tried.
System Programming

13. File Ownership Ownership of new files and directories
The user ID of a new file (directory) is set to the effective user ID of the process.
The group ID of a new file can be
the effective group ID of the process OR
the group ID of the directory where the file is being created (depending on the system).
System Programming

14. access function
int access(const char *pathname, int mode);
return: 0 if OK, -1 on error
Access test based on the real UID/GID (Not on the effective UID/GID)
mode
Description
R_OK
W_OK
X_OK
F_OK
Test for read permission
Test for write permission
Test for execute permission
Test for existence of file
If (access(argv[1], R_OK) < 0)
err_ret(“access error for %s”,argv[1]);
else printf(“read access OK\n”);
If (open(argv[1], O_RDONLY) < 0)
err_ret(“open error for %s”, argv[1]);
else printf(“open for reading OK\n”);
$whoami
kim
$ ls –l a.out
-rwsrwxr-x 1 uucp Jan 18 08:48 a.out
$ ls -l /etc/uucp/Systems
-rw-r uucp Jul 18 15:05 /etc/uucp/Systems
$ a.out /etc/uucp/Systems
access error for /etc/uucp/Systems: Permission denied
open for reading OK
System Programming

15. umask function File mode creation mask (umask)
mode_t umask(mode_t cmask);
returns: previous file mode creation mask
File mode creation mask (umask)
associated with every process.
Any bits that are on in the file mode creation mask are turned off in the file’s mode of the file being created with open() or creat().
Changing the file mode creation mask of a process doesn’t affect the mask of its parent.
Usually set once, on login by the shell’s start-up file and never changed.
umask() sets the file mode creation mask for the process and returns the previous value.
cmask: the bitwise OR of any of the nine constants from file access permission st_mode mask (Figure 4.4)
System Programming

16. umask function (cont’d)
main(void){
umask(0);
if (creat("foo",S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP |
S_IROTH | S_IWOTH) < 0) err_sys("creat error for foo");
umask(S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH);
if (creat("bar", S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP |
S_IROTH | S_IWOTH) < 0) err_sys("creat error for bar");
exit(0); }
$umask 02
$ a.out
$ ls –l foo bar
-rw stevens 0 Nov 16 16:23 bar
-rw-rw-rw- 1 stevens 0 Nov 16 16:23 foo
To assure the specific access permission of a file created, we must modify the umask value of the running process (ex: umask(0))
System Programming

17. chmod() and fchmod()
int chmod(const char *pathname, mode_t mode);
int fchmod(int filedes, mode_t mode);
Both return: 0 if OK, -1 on error
Change the file access permissions for an existing file
fchmod operates on a file that has already opened.
mode: bitwise OR of the constants shown in Figure 4.6
The effective uid of the process must equal the owner of the file to change the permission bits of the file.
System Programming

18. Sticky Bit (Save-text)
Sticky bit for an executable file
Used on earlier version of Unix.
Program’s text was saved in the swap space for faster loading into memory.
Today’s systems do not need this technique due to virtual memory and faster file systems.
Sticky bit for a directory
A file in the directory can be removed or renamed only if the user has write and execution permission for the directory and either
owns the file
owns the directory
is the superuser
Ex) /tmp – candidate for the sticky bit so that users should not be able to delete or rename files owned by others.
drwxrwxrwt 9 root root Mar 22 16:08 tmp
System Programming

19. chown, fchown, and lchown
int chown(const char *pathname, uid_t owner, git_t group);
int fchown(int filedes, uid_t owner, gid_t group);
int lchown(const char *pathname, uid_t owner, gid_t group);
All three return: 0 if OK, -1 on error
Change the user ID and group ID of a file
lchown() changes the owners of the symbolic link itself, not the file pointed to by the symbolic link.
System Programming

20. File Size File size in stat structure Holes in a file
st_size : file size in bytes.
st_blksize : the preferred block size for I/O for the file.
st_blocks : the actual number of 512-byte blocks allocated.
Holes in a file
Created by seeking past the current end of file and writing some data
Ex) holes in core file
du reports the disk space used by the file is only 139,264 bytes (512x 272)
read() returns 0 for any byte positions in holes
When copying the file, all holes are written out as actual data bytes of 0
$ls –l core
-rw-r—r– 1 stevens Nov 18 12:18 core
$du –s core
272 core
$cat core > core.copy
16592 core.copy
System Programming

21. File Truncation Truncate an existing file to length bytes.
int truncate(const char *pathname, off_t length);
int ftruncate(int filedes, off_t length);
Both return: 0 if OK, -1 on error
Truncate an existing file to length bytes.
If the previous size > length,
the data beyond length is no longer accessible.
If the previous size < length,
the effect is system dependent.
System Programming

22. Overview of the Unix File Systems
File system : consist of a sequence of logical blocks
Device drive
partition
partition
partition
File system
I-list
Directory blocks and data blocks …
Boot block(s)
I-node
I-node
I-node
Super block
Figure 4.7 Device drive, partitions, and a file system
System Programming

23. Overview of the Unix File Systems
Super block
contains
The size of the file system, the size of the I-node list
The number of free blocks/I-nodes in the file system
A list of free blocks/I-nodes in the file system
I-node : internal representation of a file
description of the disk layout of the file data
file information (e.g. owner, access permission, access time, a link count…)
one-to-one mapping with a file (cf. link())
Directory
file name
I-node number (in the same file system, not in the different file system)
Data blocks
System Programming

24. contain ptr’s to data block
I-node
File owner identifier
File type
File Access permissions
File access times
# of links to the file
File size
struct stat
file information
ptr’s to data blocks
contain ptr’s to data block
direct ptr’s
single
indirect ptr
double
indirect prt
triple
Data Blocks
System Programming

25. directory blocks and data blocks
File system in more detail
directory blocks and data blocks
I-list
directory
block
data
second data block
first data block
third data block
I-node
DirB
100
DirA
100
File name = a
I-node
number
File name = b
System Programming

26. directory blocks and data blocks
i-list
I-node
1267
lc = 3
2549
lc= 2 .
2549 ..
1267 .
1267 ..
I-node
number
Fig4.9 sample filesystem after creating the directory testdir (mkdir testdir)
testdir
2549
2549
System Programming

27. link(“/dirA/name1”, “/dirB/name2”);
int link(const char *existingpath, const char *newpath);
return: 0 if OK, -1 on error
Creates a new directory entry, newpath, that references the existing file, existingpath
increment the link count of the I-node for the existing file.
The creation of the new directory entry and the increment must be an atomic operation.
Only a superuser process can create a new link that points to a directory because this could cause loops in the file system.
Hard links created by link() can’t cross file systems.
link(“/dirA/name1”, “/dirB/name2”);
System Programming

28. What really happens with link()
12345
name1
I-node
name
directory entry in /dirA
12345
name2
I-node
name
directory entry in /dirB
Two hard links to the same file
I-node
12345
block
23567
“This is the
text in the
file.”
...
st_link=2
st_link=1
...
23567
...
link(“/dirA/name1”, “/dirB/name2”);
System Programming

29. unlink()
int unlink(const char *pathname);
return: 0 if OK, -1 on error
Removes the directory entry and decrements the link count of the file referenced by pathname
must have write and execute permission in the directory containing the file/directory.
Only when link count reaches 0, the contents of the file can be deleted
if some process has the file open, the file will not be deleted.
What kernel does when a file is closed
checks if the count of the number of processes that have the file open is 0
checks if the link count is 0
if both are yes, the file’s contents are deleted.
System Programming

30. unlink() - continued
A way to assure that a temporary file not left when the program crashes - call unlink immediately after open or create the file.
If pathname is a symbolic link, unlink references the symbolic link, not the file referenced by the link
The superuser can call unlink with directory pathname, but use rmdir() to unlink a directory
unlink a file or directory
For a file, remove() = unlink()
For a directory remove() = rmdir()
int remove(const char *pathname);
return: 0 if OK, -1 on error
System Programming

31. rename() rename a file or directory
int rename(const char *oldname, const char *newname);
return: 0 if OK, -1 on error
rename a file or directory
if oldname is a file and newname exists(and is not a directory), the newname is removed and oldname is renamed to newname.
Need write permission for both directories containing oldname and newname
if oldname is a directory and newname exists, the newname must refer to a directory and that directory must be empty.
oldname is renamed to newname
Renaming a file without changing file system is done by updating directory entries pointing to the file’s i-node without moving the actual contents of the file.
System Programming

32. Symbolic links A symbolic link is an indirect pointer to a file
A function that refer to a file by name may or may not follow a symbolic link; i.e. it may operate on the symbolic link itself or the file referred to by the symbolic link. (See Figure 4.10)
hard link limitations
normally requires the link and the file in the same file system
Only the superuser can create a hard link to a directory
$ ln -s /home/no_file myfile
$ ls -l myfile
lrwxrwxrwx 1 stevens Dec 6 07:27 myfile -> /home/no_file
System Programming

33. Symbolic links ... ... 12345 name1 13579 name2 1 23567 1 15123
I-node
name
directory entry in /dirA
13579
name2
I-node
name
directory entry in /dirB
I-node
12345
... 1
23567
block
23567
“This is the
text in the
file.”
I-node
13579
... 1
15123
block
15123
“/dirA/name1”
symlink(“/dirA/name1”, “/dirB/name2”);
System Programming

34. symlink()
int symlink(const char *actualpath, const char *sympath);
returns: 0 if OK, -1 on error
symlink()
create a new directory entry, sympath, which points to actualpath
actualpath does not have to exist
The permissions of a symbolic link are irrelevant
The ownership is ignored when following the link
The ownership is checked when removal or renaming of the link is requested and the link is in a directory with the sticky bit set.
System Programming

35. readlink() readlink()
int readlink(const char *pathname, char *buf, int bufsize);
returns: number of bytes read if OK, -1 on error
readlink()
open the symbolic link itself and read the name in the link into the buf and close it. (open() function follows a symbolic link)
System Programming

36. File Times (utime function)
Field
Description
Example
ls option
st_atime
last access time of file data
read
-lu
st_mtime
last-modification time of file data
write -l
st_ctime
last-change time of I-node status
chmod, chown
-lc
int utime(const char *pathname, const struct utimbuf *times);
returns: 0 if OK, -1 on error
struct utimbuf { time_t actime; /* access time */
time_t modtime; /* modification time */ }
changes the access and modification time of a file
times = NULL, the access time and the modification time are set to the current time
time  NULL, the access time and the modification time are set to the values in the structure pointed to by times
st_ctime is automatically updated when utime() is called.
The time values are calendar times (see Section 1.10)
System Programming

37. mkdir() and rmdir() mkdir() rmdir() creates a new, empty directory
int mkdir(const char *pathname, mode_t mode);
return: 0 if OK, -1 on error
int rmdir(const char *pathname);
mkdir()
creates a new, empty directory
The entries for dot and dot-dot are automatically created
rmdir()
The directory is freed if the link count of the directory becomes 0 and no other process has the directory open.
System Programming

38. Reading Directories Open and read directory entries
DIR *opendir(const char *pathname)
returns: pointer if OK, NULL on error
struct dirent *readdir(DIR *dp);
returns: pointer if OK, NULL at end of directory or error
void rewinddir(DIR *dp);
int closedir(DIR *dp);
returns: 0 if OK, -1 on error
struct dirent {
/* I-node number */
ino_t d_ino:
/* null terminated filename*/
char d_name[NAME_MAX + 1]; }
Open and read directory entries
Directories can be read by anyone with read access permissions; but only kernel can write to a directory!
opendir() returns the pointer to a DIR structure that is used by other three functions;
It initializes things so that the first readdir reads the first entry in the directory; The ordering of entries is implementation dependent
System Programming

39. chdir(), fchdir() and getcwd()
int chdir(const char *pathname);
int fchdir(int filedes);
Both returns: 0 if OK, NULL on error
char *getcwd(char *buf, size_t size);
Returns: buf if OK, NULL on error
chdir() and fchdir()
change the current working directory
Every process has a current working directory; i.e. it is an attribute of a process
chdir() can not affect a parent process
getcwd()
copies the absolute pathname of the current working directory to the array pointed to by buf, which is of length size.
System Programming

40. Special Device Files
$ ls -l /dev/sd0[ah] /dev/tty[ab]
brw-r root , 0 Jan 31 08:23 /dev/sd0a
brw-r root , 7 Jan 31 08:23 /dev/sd0h
crw-rw-rw- 1 root , 0 Jan 31 08:23 /dev/ttya
crw-rw-rw- 1 root , 1 Jan 31 08:23 /dev/ttyb
Every filesystem is known by its major and minor device number encoded in the primitive system data type dev_t.
Two macros, major() and minor() to access the major and minor device numbers, respectively.
The st_dev for every filename is the device number for the filesystem containing that filename and its i-node.
The st_rdev contains the device number for the actual device, character special files or block special files
System Programming

41. Special Device Files (cont’d)
main(int argc, char *argv[]) {
int i; struct stat buf;
for (i = 1; i < argc; i++) {
printf("%s: ", argv[i]);
if (lstat(argv[i], &buf) < 0) {
err_ret("lstat error");
continue; }
printf("dev = %d/%d", major(buf.st_dev), minor(buf.st_dev) );
if (S_ISCHR(buf.st_mode) || S_ISBLK(buf.st_mode) ) {
printf(" (%s) rdev = %d/%d",
(S_ISCHR(buf.st_mode) ) ? "character" : "block",
major(buf.st_rdev), minor(buf.st_rdev));
printf("\n");
exit(0);
$ a.out / /home/stevens /dev/tty[ab]
/: dev = 7/0
/home/stevens: dev = 7/7
dev/ttya: dev = 7/0 (character) rdev = 12/0
dev/ttyb: dev = 7/0 (character) rdev = 12/1
System Programming

42. Mount/Unmount Mount: connecting a file system to an existing hierarchy
Unmount: disconnect that which was mounted
Example : two partitions on /dev/hda
ntfs filesystem (MS windows): /dev/hda0
ext3 filesystem: /dev/hda1 (mounted on / )
$ mount –t ntfs /dev/hda0 /mnt /
etc
mnt
usr
var
Documents
and Settings
Program Files
WINDOWS
System Programming

43. sync() and fsync() Delayed write
void sync(void);
int fsync(int filedes);
Returns: 0 if OK, -1 on error
Delayed write
writing data to a file normally copies the data into one of buffers in the kernel (buffer cache) and queues it for I/O at some later time
sync() queues all the modified block buffers for writing and returns; does not wait for the actual I/O to take place
sync() is normally called every 30 seconds from a system daemon
fsync() refers only to a single file; waits for the I/O complete before returning; a database application may use it
System Programming