Unix Directories unix etc home pro dev motd snt unix ... slide1 slide2 passwd motd snt unix ... Here is a portion of a Unix directory tree. The ovals represent files, the rectangles represent directories (which are really just special cases of files). slide1 slide2

Directory Representation Component Name Inode Number directory entry . 1 .. 1 unix 117 etc 4 A directory consists of an array of pairs of component name and inode number, where the latter identifies the target file’s inode to the operating system (an inode is data structure maintained by the operating system that represents a file). Note that every directory contains two special entries, “.” and “..”. The former refers to the directory itself, the latter to the directory’s parent (in the case of the slide, the directory is the root directory and has no parent, thus its “..” entry is a special case that refers to the directory itself). home 18 pro 36 dev 93

Standard File Descriptors main( ) { char buf[100]; int n; const char* note = "Write failed\n"; while ((n = read(0, buf, sizeof(buf))) > 0) if (write(1, buf, n) != n) { (void)write(2, note, strlen(note)); exit(EXIT_FAILURE); } return(EXIT_SUCCESS); File descriptor 0: standard input File descriptor 1: standard output File descriptor 2: standard error output The file descriptors 0, 1, and 2 are opened to access your terminal when you log in, and are preserved across forks, unless redirected.

File operations int open(const char *pathname, int flags, mode_t mode); Returns a file descriptor ssize_t read(int fildes, void *buf, size_t count); Use the file descriptor ssize_t write(int fildes, const void *buf, size_t count); Use file descriptor off_t lseek(int fildes, off_t offset, int whence); Delete, read attributes, set attributes, etc

Representing an Open File (1) file descriptor table system file table active inode table 1 2 3 . n–1 ref count f pointer access inode A process’s set of open files is represented by a data structure known as the file descriptor table, which is used to map file descriptors, representing open files, to system file table entries, each representing an open file. Each process has a separate file descriptor table, but there is exactly one system file table for the entire system. Each active file (i.e., a file that is open or otherwise being used) is represented by an inode that is entered in the active inode table. Recently accessed blocks of data from the file are kept in the kernel’s data section in what is known as the buffer cache—an area of memory reserved for buffering data from files. Thus data from these blocks can be accessed again and again without wasting time going to the disk. When a disk block is modified, the modifications are written in the buffer cache and only later are written to disk. This allows the process doing the modification to proceed without having to wait for the data to be written to disk. disk buffer cache

Allocation of File Descriptors Whenever a process requests a new file descriptor, the lowest numbered file descriptor not already associated with an open file is selected. One can depend on always getting the lowest available file descriptor.

Representing an Open File (2) file descriptor table system file table active inode table 1 2 3 fdrw 1 rw . . . n–1 ref count f pointer access inode This slide shows what happens when a file is opened. The lowest-numbered available file descriptor is allocated from the file descriptor table. Next, an entry in the system file table is allocated, and the file descriptor table entry is set to point to the system file table entry. An inode-table entry for the file is allocated (or found if it already exists) and the system file table entry is set to point to it. Additional fields of the system file table entry are initialized, including: a reference count each file-descriptor-table entry pointing to it contribute one to the reference count the entry is freed when the reference count goes to zero the allowed access (i.e., how the file was opened—read-only, read-write, etc.) the file pointer (i.e., the location within the file at which the next transfer will start) after every read and write system call the file pointer is incremented by the number of bytes actually transferred, thus facilitating sequential I/O fdrw = open("x", O_RDWR); disk buffer cache Copyright © 2002 Thomas W. Doeppner. All rights reserved.

Reading From a File #include <sys/types.h> #include <unistd.h> ssize_t read(int fd, void *buffer, size_t n) read up to n bytes of data into buffer returns number of bytes transferred returns 0 on end of file returns –1 on error When will “read” transfer fewer bytes than specified ? The read system call tells the system to read up to n bytes of data into the area of memory pointed to by buffer from the file referred to by fd. It returns the number of bytes actually transferred, or, as usual, –1 if there was an error. Read will transfer fewer bytes than specified if: the number of bytes left in the file was less than n the read request was interrupted by a signal after some of the bytes were transferred the file is a pipe, FIFO, or special device with less than n bytes immediately available for reading It transfers as many bytes as are present up to the given maximum; if it returns a zero, that means the end of the file has been reached.

Writing To a File #include <sys/types.h> #include <unistd.h> ssize_t write(int fd, void *buffer, size_t n) write up to n bytes of data from buffer returns number of bytes transferred returns –1 on error The write system call tells the system to write up to n bytes of data from the area of memory pointed to by buffer into the file referred to by fd. It returns the number of bytes actually transferred, or, as usual, –1 if there was an error. If write transfers fewer bytes than specified, then something caused the transfer to stop prematurely: a signal interrupted the system call this is pretty complicated and we’ll discuss it in more detail later. However, this would happen if the requested size was large enough so that the write was split by the kernel into a number of segments, at least one segment was written (otherwise the call would have returned with an error and errno set to EINTR), and the signal occurred while a subsequent segment was being written a filesize limit was reached an I/O error occurred The response in all these cases should be to attempt to rewrite those bytes that were not transferred; if a signal had interrupted the previous try, then the next try will succeed (unless again interrupted by a signal); if a filesize limit had been reached or an I/O error has occurred, this next write will yield the appropriate error code.

Example main( ) { char buf[BUFSIZE]; int nread; const char* note = "Write failed\n"; while ((nread = read(0, buf, sizeof(buf))) > 0) { int bytes_left = nread; int bpos = 0; while ((n = write(1, &buf[bpos], bytes_left)) < bytes_left) { if (n == –1) { write(2, note, strlen(note)); exit(EXIT_FAILURE); } bytes_left  n; bpos += n; return(EXIT_SUCCESS); Here we copy from the program’s standard input to its standard output, but take advantage of what we’ve learned about the behavior of write. In particular, since it’s not guaranteed that write will transfer all the bytes requested, we must supply code to make sure that all the data does get transferred. Thus, after each call to write (except, for reasons of space on the slide, when we write to file descriptor 2), we check to see how much data was written and call write again, if necessary, to handle the unwritten data.

Random Access #include <sys/types.h> #include <unistd.h> off_t lseek(int fd, off_t offset, int whence) sets the file pointer for fd: if whence is SEEK_SET, the pointer is set to offset bytes; if whence is SEEK_CUR, the pointer is set to its current value plus offset bytes; if whence is SEEK_END, the pointer is set to the size of the file plus offset bytes it returns the (possibly) updated value of the file pointer relative to the beginning of the file. Thus, n = lseek(fd, (off_t)0, SEEK_CUR); returns the current value of the file pointer for fd To effect random access to files (i.e., access files other than sequentially), you first set the file pointer, then perform a read or write. Setting the file pointer is done with the lseek system call.

lseek Example What does this piece of code do? fd = open("textfile", O_RDONLY); fptr = lseek(fd, (off_t)–1, SEEK_END); while (fptr != –1) { read(fd, buf, 1); write(1, buf, 1); fptr = lseek(fd, (off_t)–1, SEEK_CUR); } This example prints the contents of a file backwards. Note what’s happening in the while statement: it continues as long as fptr does not return –1, a value it will return if the call to lseek fails. Our intent is that the last successful call to lseek sets the file pointer to 0. The next call, which attempts to set it to –1, will fail, thus causing lseek to return –1 (recall that failed system calls always return –1).

Representing an Open File (3) file descriptor table system file table active inode table 1 2 3 1 r 10 fdrw 1 rw 20 . . fdr . n–1 ref count f pointer access inode In this slide we see the effect of two opens of the same file within the same process. fdrw = open("x", O_RDWR); fdr = open("x", O_RDONLY); write(fdrw, buf, 20); read(fdr, buf2, 10); disk buffer cache Copyright © 2002 Thomas W. Doeppner. All rights reserved.

Multiple Descriptors; One File How are standard file descriptors set up? suppose 1 and 2 are opened separately while ((n = read(0, buf, sizeof(buf))) > 0) if (write(1, buf, n) != n) { (void)write(2, note, strlen(note)); exit(EXIT_FAILURE); } error message clobbers data bytes! By convention, file descriptors 1 and 2 are used for processes’ normal and diagnostic output. Normally they both refer to the display, and thus diagnostic output is intermingled with normal output. Suppose, however, one wanted to redirect both file descriptors so that all output, normal and diagnostic, was sent to a file. One might open this file twice, once as file descriptor 1 and again as file descriptor 2, thereby creating two system file table entries. As file descriptor 1 receives output, the offset field of its file table entry advances with each write. After 1000 bytes have been written (sequentially), the offset field is set to 1000, representing the current end-of-file. If at this point a diagnostic message is written to file descriptor 2, it will start at the beginning of the file, overwriting the data already there, since file descriptor 2’s file table entry’s offset is still at 0. This outcome is certainly not desirable.

dup System Calls dup returns a new file descriptor referring to the same file as its argument int dup(int fd) dup2 is similar, but it allows you to specify the new file descriptor int dup2(int oldfd, int newfd) We can use one of the dup system calls to solve this problem. dup obeys our rule of always allocating the lowest available file descriptor. However, with dup2, one can specify, via the second argument, which file descriptor is allocated. If the second argument is the file descriptor of an open file, the file is first closed, then associated with the file of the first argument.

dup Example /* redirect stdout and stderr to same file */ /* assumes file descriptor 0 is in use */ close(1); open("file", O_WRONLY|O_CREAT, 0666); close(2); dup(1); /* alternatively, replace last two lines with: */ dup2(1, 2); Here we see how to use dup and dup2 to set file descriptors 1 and 2 to refer to the same system file-table entry. Note the extra argument to open. We’ve given open the O_CREAT flag, which tells the system that if the file does not exist, it should create it. The third argument helps to specify the access permissions assigned to the file if it’s created by this call. We discuss this in detail in a few slides.

Representing an Open File (4) file descriptor table system file table active inode table 1 2 3 fdrw 2 rw 20 . fdrw2 . . n–1 ref count f pointer access inode The dup system call causes two file descriptors to refer to the same file table entry and hence share the offset. fdrw = open("x", O_RDWR); fdrw2 = dup(fdrw); write(fdrw, buf, 20); disk buffer cache Copyright © 2002 Thomas W. Doeppner. All rights reserved.

Representing an Open File (5) file descriptor table system file table active inode table 1 2 3 2 r 10 fdrw 4 rw 20 . fdrw2 . fdr . n–1 ref count f pointer access inode If our process executes a fork system call, creating a child process, the child is given a file-descriptor table that’s a copy of its parent’s. Of course, the reference counts on the system-file-table entries are increased appropriately. fork( ) disk buffer cache Copyright © 2002 Thomas W. Doeppner. All rights reserved.

I/O Redirection % who > file & if (fork( ) == 0) { } char *args[ ] = {"who", 0}; close(1); open("file", O_WRONLY|O_TRUNC, 0666); execv("who", args); printf("you screwed up\n"); exit(1); } This is an example of what a shell might do to handle I/O redirection: it first creates a new process in which to run a command (“who”, in this case). In the new process it closes file descriptor 1 (standard output—to which normal output is written). It then opens “file” (the arguments indicate that “file” is only to be written to, that any prior contents are erased, and that if “file” didn’t already exist, it will be created with read and write permission for all; assuming that file descriptor 0 is not available (it’s assigned to standard input), file descriptor 1 will be assigned to “file”. Assuming that execv succeeds, when “who” runs, its output is written to “file”. Note that the parent process does not wait for its child to terminate; it goes on to execute further commands. (This behavior occurs because we’ve placed an “&” at the end of the command line.) Note the args argument to execv: By convention, the first argument to each command is the name of the command (“who” in this case). To indicate that there are no further arguments, a zero is supplied. Note that we aren’t checking for errors: this is only because doing so would cause the resulting code not to fit in the slide. You should always check for errors.