Kernel File Interface operating systems (or programming I/O in Unix)

Input and Output When programming in C on Unix, there are two very different I/O libraries you can use: The C language libraries: o Buffered o Part of the C language o The basic unit is a FILE* The Kernel I/O calls o Unbuffered o System calls – not part of C o The basic unit is a File Descriptor

operating systems

Standard C I/O operating systems

As in C++, the fundamental notion used in doing I/O is the stream (but it is not an object as it is in C++... it is a data structure) When a file is created or opened in C, the system associates a stream with the file. When a stream is opened, the fopen( ) call returns a pointer to a FILE data structure. The FILE data structure contains all of the information necessary for the I/O library to manage the stream: * a file descriptor * a pointer to the I/O buffer * error flags * etc The original C I/O library was written around 1975 by Dennis Ritchie. Little has changed since then. operating systems

Three streams are predefined and available to a Process. These standard streams are referenced Through the pre-defined FILE pointers stdin, stdout, and stderr. These pointers are defined in Standard Streams operating systems

One of the keys of the C I/O library is that I/O is normally buffered to minimize context switches. Fully Buffered: I/O takes place when a buffer is full. Disk files are normally fully buffered. The buffer is allocated by the I/O library itself by doing a malloc. Line Buffered: I/O takes place when a new line character is encountered. Line buffering is used for terminal I/O. Note that I/O may take place before a new line character is encountered because of the size of the buffer. Buffering I/O operating systems

Unbuffered: No buffering is done. Data is output immediately. operating systems

Most Unix systems default to the following: Standard Error is always un-buffered. All streams referring to a terminal device are line buffered (stdin and stdout). All other streams are fully buffered. operating systems

You can force a stream to be flushed, (all unwritten bytes are passed to the kernel) #include int fflush (FILE *fp); Flushing a Stream operating systems I’ve not seen an issue in Windows, but in Unix, you may not see output when you expect to if you don’t flush the buffers.

fopenfopen #include FILE *fopen (const char *filename, const char *mode); pointer to the FILE structure holding the internal state information about the connection to the associated file. Returns a NULL pointer if open fails. full path to the file to be opened Mode bits “r”open text file for reading “rb”open binary file for reading “w”open text file for writing - truncate “wb”open binary file for writing - truncate “a”open text file for writing-append “ab”open binary file for writing-append “r+”open text file to read & write (file must exist) “rb+”open binary file to read & write - ditto “w+”open text file to read & write – truncate “wb+”open binary file to read & write – truncate “a+”open text file to read & write – append “ab+”open binary file to read & write - append when opened for reading and writing * input cannot immediately follow output without an intervening fflush, fseek, fsetpos, or rewind. * output cannot immediately follow input without an intervening fseek, fsetpos, or rewind. operating systems

Restriction r w a r+ w+ a+ file must already exist * * previous contents are discarded * * stream can be read * * * * stream can be written * * * * * stream can only be written at end * * You cannot set permission when a file is opened with w or a Opening a Stream operating systems

FILE *in; if ((in = fopen(“file1.txt”, “r”)) == NULL) perror(“could not open file1.txt”); Example of using fopen

FILE *freopen (const char *pathname, const char *mode, FILE *fp); Opens a specified file on a specified stream. Closes the file first, if it is already open. Most typically used with stdin, stdout, and stderr to open a file as one of these streams. FILE *fdopen (int filedes, const char *mode); takes a file descriptor as a parameter. Used with pipes and network connections, because these use file descriptors. Associates an I/O stream with the descriptor. Related Calls operating systems

fclosefclose #include int fclose (FILE *stream); returns a zero if the close is successful Otherwise it returns -1 All files are closed when the program terminates normally, but this allows no opportunity to do error recovery if termination is not normal. Therefore, it is recommended that all files be closed explicitly. operating systems

Binary I/O is commonly used to read or write arrays or to read and write structures, because both deal with fixed size blocks of information. Note: Binary files are not necessarily interchangeable across systems! * compilers change how data is packed * binary formats are different on different cpu architectures. Binary I/O operating systems

There are three types of unformatted I/O: * Character at a time * Line at a time * Direct I/O (fread and fwrite for binary data) Unformatted I/O operating systems

#include long ftell (FILE *fp); int fseek (FILE *fp, long offset, int whence); void rewind (FILE *fp); returns the current byte offset or -1L SEEK_SET – from beginning of file SEEK_CUR – from the current position SEEK_END – from the end of the file returns 0 if successful nonzero on error for binary files and text files on GNU systems Stream Positioning operating systems

For portability across POSIX systems use: int fgetpos (FILE *fp, fpos_t *pos); int fsetpos (FILE *fp, const fpos_t *pos); the position is passed in this parameter, a new data type defined by the POSIX standard. The position value in an fsetpos must have been obtained in a previous fgetpos call. returns 0 if successful operating systems

freadfread fread is used to read binary data and text in fixed sized blocks #include size_t fread (void *ptr, size_t size, size_t nblocks, FILE *stream); The number of items read. It could be less than nblocks if there is an error or eof is reached. address of where first byte is to be stored The size of each block or record The number of blocks to read The stream to read from operating systems

Interpreting Binary Data If the data that you are reading has some record structure … struct record_fmt data_buf;... fread(&data_buf, sizeof(char), sizeof(data_buf), file_handle); operating systems

struct record_fmt { int a; float b; char id[8]; char pw[8]; }; operating systems databuf from the file cout << data_buf.id;

fwritefwrite #include size_t fwrite (void *ptr, size_t size, size_t nblocks, FILE *stream); The number of blocks written. If not the same as nblocks, some error has occurred. address of the first byte to write The size of each block or record The number of blocks to write The stream to write to operating systems

#include int fgetc (FILE *stream); fgetc gets the next character in the stream as an unsigned char and returns it as an int. If an eof or an error is encountered, EOF is returned instead. This call is guaranteed to be written as a function. The return value is an unsigned char that has been converted to an int. The constant EOF (usually -1) is returned if there is an error or if the end of the file is encountered. Character at a time Input operating systems

int getc (FILE *stream); int getchar ( void ); highly optimized –best function for reading a single character. Usually implemented as a macro. Equivalent to getc(stdin) Character at a time Input operating systems

In most implementations, each stream maintains * an error flag * an end-of-file flag To distinguish between EOF and an error call one of the following functions: #include int ferror (FILE *fp); int feof (FILE *fp); returns nonzero (true) if error flag is set, otherwise returns 0 returns nonzero (true) if eof flag is set, otherwise returns 0 Clear the flags by calling void clearerr (FILE *fp); operating systems

After reading a character from a stream, it can be pushed back into the stream. #include int ungetc (int c, FILE *fp); the character to push back. Note that it is not required that you push back the same character that you read. You cannot pushback EOF. Implementations are not required to support more than a single character of pushback, so don’t count on it. operating systems

Character Output int fputc (int c, FILE *stream); fputc converts c to an unsigned char and writes it to the stream. EOF is returned if there is an error. int putc (int c, FILE *stream); int putchar( int c ); optimized for single character input assumes stdout is the output stream operating systems

Line at a Time Input #include char *fgets (char *buf, int n, FILE *fp); char *gets (char *fp); gets has been deprecated because it does not allow the size of the buffer to be specified. This allows buffer overflow! reads up through and including the next newline character, but no more than n-1 characters. The buffer is terminated with a null byte. If the line is longer than n-1, a partial line is returned. The buffer is still null terminated. If the input contains a null, you can’t tell. returns buf if successful and NULL on end of file or error. Warning operating systems

String Output #include int fputs (const char *str, FILE *fp); int puts (const char *str); writes a null-terminated string to the stream. It does not write the null terminating character. It does not write a newline character. Returns EOF if the function fails. writes the null terminated string to standard-out, replacing the zero terminating character with a new-line character. If successful, the function returns a non-negative value. If the function fails, it returns EOF. operating systems

#include int main (void) { int c; while ( (c =getc(stdin)) != EOF) if (putc(c, stdout) == EOF) perror("Error writing output"); if(ferror(stdin)) perror("Error reading input"); exit(0); } EOF is ctrl-D I/O Efficiency Char at a time operating systems

#include #define MAXLINE 4096 int main (void) { char buf[MAXLINE]; while (fgets(buf, MAXLINE, stdin) != NULL) if (fputs(buf, stdout) == EOF) perror("Output Error"); if (ferror(stdin)) perror("Input Error"); exit(0); } Line at a time operating systems

for copying a file of 1.5M bytes in 30,000 lines Function user CPU fgets, fputs2.2 seconds getc, putc4.3 seconds fgetc, fputc4.6 seconds loop is executed 30,000 times loop is executed 1.5M times operating systems

Formatted Output int printf (const char *format-spec, print-data … ); int fprintf (FILE *fp, const char *format-spec, print data); int sprintf(char *s, const char *format-spec, print-data…); a format-specification has the following format: %[flags] [width] [.precision] type - left align, default is to right align + prefix value with a sign 0 pad output with zeros prefix positive values with a blank Minimum field width. If width is prefixed with 0, add zeros until minimum width is reached. digits after decimal point. This can truncate data d signed decimal integer i signed decimal integer u unsigned decimal integer o unsigned octal integer x unsigned hex integer f double in fixed point notation e double in exponent notation c single character, an int s a string writes to stdout writes to buffer and appends a null byte at the end. % -this is format-spec operating systems

“%-10.8f” Example Format Specification % - introduces the format specification left justify the output output field is 10 chars wide as a minimum. Padded if fewer characters in the output. Data is never truncated. print 8 digits after the decimal point operating systems

ExampleExample int n = 3; double cost-per-item = 3.25; printf(“Cost of %3d items at $%4.2f each = $%6.2f\n”, n, cost-per-item, n*cost-per-item); first field is 3 characters wide data is right justified 3 second field is 4 characters wide with two characters after decimal point Cost ofitems at $3.25= $ 9.75 third field is 6 characters wide with 2 characters after decimal point right justified operating systems

Formatted Input #include int scanf (const char* format-spec, data fields); int fscanf (FILE *fp, const char *format-spec, data fields); int sscanf (const char *buf, const char *format-spec, data fields); operating systems

scanf reads formatted data from stdin into the data fields given in the argument string. Each argument must be a pointer to a variable that corresponds to a type specifier in the format specification. The format specification can contain: * white space characters. A white space character causes scanf to read in, but not store all consecutive white space characters in the input stream, up to the next non-white space character. * non-white space characters, except % sign. Causes scanf to read but not store a matching non-white space character. If the character does not match, scanf terminates. * format specification, introduced by %. Causes scanf to read in and convert characters in the input into values of the specified type. The resulting value is assigned to the next data field in the arg list.

Temporary Files #include FILE *tmpfile (void); creates a temporary file (type wb+) that is automatically deleted when the file is closed or the program terminates. operating systems

Sample Program Write a simple version of the cat command. It takes an optional parameter, a file name. It copies the file to stdout. - if no file name is given, it copies stdin to stdout operating systems

Preliminaries operating systems #include #define LINELEN 256 void send_to_stdout( FILE*); header files for I/O required to define NULL C programmers use #define to define constants. It works like a macro … the value 256 gets inserted wherever the name LINELEN appears in the code. There is no type checking! function prototype

Main declaration int main (int argc, char* argv[ ]) {... } The number of arguments on the command line Array contains the command line arguments

Body of main int main (int argc, char* argv[ ]) { FILE *fp; if (argc == 1) send_to_stdout ( stdin); Declare a FILE* to hold the file handle If there is just one command line argument it is the command. Copy from stdin.

int main (int argc, char* argv[ ]) { FILE *fp; if (argc == 1) send_to_stdout ( stdin); else if (argc == 2) { if ( (fp = fopen(*++argv, “r”) ) != NULL) { send_to_stdout ( fp ); fclose ( fp ); } If there are two command line arguments, the second one is the file name.

int main (int argc, char* argv[ ]) { FILE *fp; if (argc == 1) send_to_stdout ( stdin); else if (argc == 2) { if (fp = fopen(*++argv, “r”) ) != NULL) { send_to_stdout ( fp ); fclose ( fp ); } else { perror(“could not open the file.”); exit(1); } handle file won’t open error

else { perror(“could not open the file.”); exit(1); } else { perror(“Invalid command – too many arguments”); exit(1); } return 0; } Handle the case where there are too many arguments on the command line.

send_to_stdout function void send_to_stdout(FILE *fp) { char line[LINELEN]; while ( fgets (line, LINELEN, fp) ) { if (fputs ( line, stdout ) == EOF ) { perror(“Write to stdout failed”); exit(1); }