1. NCHU System & Network Lab Lab 13 File I/O & Standard I/O

2. NCHU System & Network Lab Introduction Each regular file, device, connection socket, directory…etc, are treated as a file by Linux. –We can perform I/O operations on these different types of files. Ex : open, read, write … etc –We will start our discussion of these functions on: Basic file I/O Standard I/O library

3. NCHU System & Network Lab File I/O A set of methods dealing with files provided by Linux. Each opened file is referred to a file descriptor: –fd is a non-negative integer. –creat() and open() functions return a fd to process. UNIX system shells associate file descriptors : –0 : standard input –1 : standard output –2 : standard error

4. NCHU System & Network Lab File I/O (cont.) These are functions introduced later : #include #include int open (const char *pathname, int oflag, mode_t mode); int creat (const char *pathname, mode_t mode); int close (int filedes); off_t lseek (int filedes, off_t offset, int whence); ssize_t read (int filedes, void *buf, size_t nbytes); ssize_t write (int filedes, const void *buf, size_t nbytes);

5. NCHU System & Network Lab open() A file is opened or created by calling open() function. –pathname is the name of the file. –This function has a multitude of options are specified by oflag argument. –mode specifies the access permission of file. #include int open (const char *pathname, int oflag, mode_t mode);

6. NCHU System & Network Lab open (cont.) oflags of open() Flagsdescription O_RDONLYOpen for reading only O_WRONLYOpen for writing only O_RDWROpen for reading and writing O_APPENDAppend to the end of file on each write O_CREATCreate a file if it doesn’t exist. This requires mode argument O_EXCLGenerate an error if O_CREAT is set and the file already exists. O_TRUNCIf the file exists and is opened,truncate its length to 0. O_NONBLOCKSet non-blocking mode on this opened file. O_NOCTTYDo not allocate the device as the controlling terminal. O_RSYNCRead operation on this fd waits until any pending writes finished. O_DSYNCEach write wait for physical I/O to complete except file attributes. O_SYNCEach write wait for physical I/O to complete,include file attributes.

7. NCHU System & Network Lab creat() This create function is equivalent to open (pathname, O_WRONLY|O_CREAT|O_TRUNC, mode) –One deficiency with creat() is that the file is opened only for writing. –A better way is to use open() with O_CREAT instead of creat(). #include int creat (const char *pathname, mode_t mode);

8. NCHU System & Network Lab lseek() Every open file has an associated “current file offset” –A non-negative integer that measures the number of bytes from the beginning of the file. –Read/Write operation increments cur_offset value. –The interpretation of the offset argument depends on the value of whence argument. #include off_t lseek (int filedes, off_t offset, int whence); whence SEEK_SETFrom the beginning SEEK_CURFrom the current file offset SEEK_ENDFrom the end of the file

9. NCHU System & Network Lab read()/write() Data is read from an open file with the read() function. –It will read nbytes bytes into buf from file filedes. Return values –Count number of read bytes, 0 if EOF, -1 on error #include ssize_t read (int filedes, void *buf, size_t nbytes); ssize_t write (int filedes, void *buf, size_t nbytes);

10. NCHU System & Network Lab Consistency Traditional implementations of UNIX system have a buffer cache in the kernel which most disk I/O passes through. –Ex : delayed write –Three functions are provided to ensure consistency of the file system on disk with the data of buffer. sync() fsync() fdatasync() #include int fsync (int fd); int fdatasync (int fd); void sync(void);

11. NCHU System & Network Lab Consistency (cont.) Sync () –It simply schedules all the modified block buffers in RAM to be write into disk, and it returns without waiting for write completed. fsync() –It refers only to a single file, specified by the fd, and waits for the disk writes to complete before returning. fdatasync() –It affects only data portions of a file.

12. NCHU System & Network Lab Example : Basic I/O #include #define BUFFERSIZE 1 int main() { char buf[BUFFERSIZE]; int in,out,readn; in = open("file.in",O_RDONLY); out = open("file.out",O_WRONLY | O_CREAT,S_IRUSR | S_IWUSR); while((readn = read(in,buf,sizeof(buf))) > 0) write(out,buf,readn); exit(0); }

13. NCHU System & Network Lab Standard I/O Library This library is specified by ISO C standard –Each opened file is associated with a stream. –A file is opened with a FILE structure. FILE is a structure that contains all information required by the standard I/O library to manage the stream. –Buffer allocation and optimal I/O.

14. NCHU System & Network Lab Buffering of Standard I/O The goal of the buffering is to use minimum number of read() and write() calls. –Types of buffering : Fully buffered Line buffered –The library performs I/O when a newline character is encountered. Unbuffered

15. NCHU System & Network Lab Buffering of Standard I/O (cont.) We can change the buffering by calling these two functions before any other operations on the stream: –setbuf() turns buffering on or off with a buffer buf of length BUFSIZ or NULL. –setvbuf() specifies exactly which type of buffering. #include void setbuf (FILE *restrict fp, char *restrict buf); int setvbuf (FILE *restrict fp, char *restrict buf, int mode, size_t size); _IOFBFFully buffered _IOLBFLine buffered _IONBFunbuffered

16. NCHU System & Network Lab Opening a Stream fopen() function opens a specified file and returns a *FILE pointer of the stream. Modedescription rOpen for reading wCreate for writing or truncate to 0 length aAppend, for writing at EOF r+Open for reading and writing w+Truncate to 0 length or create for R/W a+Open or create for R/W at EOF #include FILE *fopen (char *restrict pathname, char *restrict type);

17. NCHU System & Network Lab Other Functions for Standard I/O #include Int getc (FILE *fp); int fgetc (FILE *fp); int putc (FILE *fp); int fputc (int c,FILE *fp) char *gets (char *buf); char *fgets (char *buf, int n, FILE *fp); char *puts (char *str); char *fputs (char *str, FILE *fp); int printf (char *format, …); int fprintf (FILE *fp, const char *format, …); int scanf (const char *restrict format, … ); int fscanf (FILE *fp, char *format, …);

18. NCHU System & Network Lab File I/O vs. Standard I/O Standard I/O library ends up calling basic I/O routines. Standard I/O is specified by ISO C standard, and another one is specified by POSIX.1 (Portable Operating System Interface). –POSIX.1 includes ISO C standard library. fopen() deals with FILE structure whereas open() uses a file descriptor integer.

19. NCHU System & Network Lab File I/O vs. Standard I/O (cont.) More flexible buffering than basic I/O that take place with standard library. Device files only can be opened by open().

20. NCHU System & Network Lab Example : Standard I/O

21. NCHU System & Network Lab Lab I/O efficiency –We want to know how does File I/O or standard I/O work to improve I/O efficiency. Create a 2MB file and copy it into a new output file. Use “clock()” to measure the process time of each case. Repeat the step above in different cases and show the result of all : –Basic I/O »write() to output file 1 byte each time »write() to output file 64 bytes each time »open() with O_SYNC set, write() 64 bytes each time »write() followed by fsync(), 64 byes each time –Standard I/O »fopen() with setbuf( unbuffered ) »fopen()

22. NCHU System & Network Lab Lab (cont.) Clock() –Returns the number of clock ticks elapsed since the program was launched. –CLOCKS_PER_SEC represents the number of clocks in a second. #include #include clock_t start,end; start = clock(); delay(2000); end = clock(); elapsetime = (end-start)/(double)CLOCKS_PERSEC ;

23. NCHU System & Network Lab Lab (cont.)

24. NCHU System & Network Lab Reference Advanced Programming in the UNIX Environment 2nd Author : Richard Stevens, Stephen A.Rago, Publisher : Addison-Wesley Beginning Linux Programming Author : Richard Stones, Neil MatthewPublisher : Wrox http://linux.vbird.org/ http://www.jollen.org/blog/ jollen’s Bloghttp://www.jollen.org/blog/ http://www.cplusplus.com/ C++ Resource Networkhttp://www.cplusplus.com/