CS162B: Pipes Jacob T. Chan

Pipes  These allow output of one process to be the input of another process  One of the oldest and most basic forms of Inter-Process Communications  We shall discuss this first before going on to semaphores. Both actually deal with IPC, but piping will be discussed so that the basics will be tackled  This is useful when two processes run different programs and there is a need for both of them to communicate  This is simple enough to understand, but it looks harder than it seems  Process management issues

Inter-Process Communication (IPC)  Exchange of data among multiple threads in one or more processes (according to Wikipedia)  Issues include  Read-write to shared data  Synchronization  Memory sharing  Remote Procedure Calls (RPC) aka execution of subroutine in another address space  In other words, it’s how processes communicate with one another (hence, the name)  Processes need to communicate too!  Example: ALL processes depend on the mother of all processes known as the SCHEDULER

Pipes in Unix  In bash, what we do is to run a piped process. Example: ls | wc –l  Running this command will result into the output of the first command being the input of the second command  So, ls will run first. Then whatever its output, it will be the input of the second command wc –l

Invoking Pipe  #include (This will allow the use of libraries like pipe() )  int pipe(int filedes[2])  Creates data pipe  filedes[2] are your file descriptors, with filedes[0] opened for READING ONLY and filedes[1] opened for WRITING ONLY  Note: reading data on filedes[0] will access data written by filedes[1]  Output:  0 if successful  -1 if not successful  The variable errno will indicate what type of error that caused the pipe to fail  Example of pipe posted in Moodle (Piping Basic Example)

Piping Features  When two processes are piped, they are unaware of it  They continue writing to and reading from standard file descriptors  Whatever the output of the first process, the second process will use that REGARDLESS of conditions (in short, second process of pipe is dependent on first one)  Pipes are byte streams!  Process reading from pipes is free to read any size of data blocks  Implication: not everything written by the writing process is read by the reading process  You cannot lseek pipes! Because data is read in FIFO fashion

Piping Features  If the file descriptor to the write end is closed (as a result of EOF), all reads will return 0 or EOF once everything is read  PIPES ARE UNIDIRECTIONAL (one end for reading, one end for writing ONLY)  Each pipe write is atomic  Writing to the same pipe will not be mixed if they write at most X bytes at a time (that’s the function of having a specific file descriptor)  When writing to full pipe (meaning max space for writing is reached), operation will block UNTIL space becomes available on pipe  Reading from empty pipe will be blocked until new data is written to the pipe

Piping Features  Analysis: Producer-Consumer (which will be like your lab)  Producer (writing side) will keep on producing the characters until there is nothing left to write  Consumer will then read what the producer produced until everything is read  In cases where there is nothing to read, the consumer “sleeps” (or gets BLOCKED) until there is something to read  In cases where pipe is full, producer will stop producing in the pipe!  Will wait until consumer consumes something. Only then will producer will write output to the pipe

Pipe Sample Code Snippet (more examples at Moodle) int filedes[2]; if(pipe(filedes) == -1) exit(1); switch(fork()) { case 0: if(close(filedes[1]) == -1) errExit(“close”); break; default: if(close(filedes[0]) == -1) errExit(“close”) break; }

Piping Issues  Rarely do two or more processes read from the same pipe  If both processes read at the same time, there will be RACE CONDITIONS (and there will be no way to guarantee will be reading data over the other)  Any two processes (not just parents and children) that are “related” to one another should communicate!  Running the previous command ls | wc –l should work because they communicate via shell (which is their parent that created the pipe)  If the processes being piped belong to different parents, then the pipe will not work (in most cases, since they do not share similar data)

Piping Issues  Implementing parent-child communication is simple enough, but connecting two programs reading from stdin and writing to stdout might be trickier  This is an issue of shared memory, semaphores, socketing (which will be discussed in the coming weeks)  Redirection of stdout to a pipe and stdin being pointed to the same pipe (which is technically socketing)  In piping, we can use the function dup2()

Pipes: int dup2(int oldfd, int newfd)  This duplicates of file descriptor oldfd and assigns it the new number newfd  If newfd corresponds to open descriptor at time of calling, the file descriptor is CLOSED first (to ensure that no blocking will occur in this time)  This allows read and write descriptors to get their corresponding descriptor numbers  Returns the new file descriptor number; otherwise returns -1 on error  Example posted in Moodle

Pipes: FILE* popen(char* command, char* type)  A simpler way to creating shell-like piping  Pipes are usually used to read from output or write to input in the context of shell commands  This command creates a pipe and forks child process that execs a shell  This creates child process to execute shell command (which is first parameter)  Type is a string that indicates whether the process will write to (noted by “w”) or read from (noted by “r”) the pipe  For using popen, instead of fclose, use pclose(FILE *stream) instead to close the pipe itself  Example posted in Moodle

Pipes: FILE* popen(char* command, char* type)

Pipes  There are many more techniques to piping, and what was shown were just one of them  We only handled the basic ones.  If you want more examples on piping, I posted some on Moodle for your reference  Or, you can consult your man pages in your terminal.

Lab 9: pcpipe.c  Create a program pcpipe.c that run as follows:  Create a pipe in your program  Fork a child (this will be the producer named “prod”)  Fork another child in the PARENT part (this will be the consumer named “cons”)  Make output of prod the same as the read end of pipe using dup2  Make input of cons the same as write end of pipe using dup2  In prod, execve() the producer program, including arguments  In cons, execve() the consumer program, including arguments  Note: popen() is NOT allowed. You have execve()

Lab 9: pcpipe.c  The program runs the same way as executing bash commands. For example:./pcpipe producerprog prodargs … -consumer consumerprog consumerargs… should produce the same command as running its bash version: producerprog prodargs … | consumerprog consumerargs …  Example:./pcpipe ls –l –consumer grep “ ” This program runs similarly as doing the following: ls –l | grep “ ”

Lab 9: pcpipe.c  Other sample input you can use:./pcpipe cat /etc/passwd -consumer cut -d\: -f1,3,6./pcpipe ls -l /home -consumer grep "$LOGNAME“ This should produce the same expected results as its bash version

Lab 9: pcpipe.c  Legal stuff  Don’t forget Certificate of Authorship (with COMPLETE details), as well as your pcpipe.c program.  Filename: CS162B_Lab9_ _ _.tar  Deadline: Next week  Tuesday for Section A  Thursday for Section B