System Calls: pipes CSRU3130 Ellen Zhang

Review: fork() int K; main() { int i; int j; j = 200; K = 300; printf("Before forking: j = %d, K = %d\n", j, K); i = fork(); if (i > 0) { sleep(10); printf("After forking, parent: j = %d, K = %d\n", j, K); } else { j++; K++; printf("After forking, child: j = %d, K = %d\n", j, K); }

… main() { int i; int seekp; int fd; char … main() { int i; int seekp; int fd; char *s1; char s2[1000]; fd = open("tmpfile", O_WRONLY | O_TRUNC | O_CREAT, 0666); s1 = "Before forking\n"; write(fd, s1, strlen(s1)); i = fork(); if (i > 0) { sleep(10); /* Delay the parent by ten seconds */ s1 = "Parent"; } else { s1 = "Child"; seekp = lseek(fd, 0, SEEK_CUR); sprintf(s2, "%s: After forking: Seek pointer = %d\n", s1, seekp); write(fd, s2, strlen(s2)); [zhang@storm Demo]$ ./a.out [zhang@storm Demo]$ more tmpfile Before forking Child: After forking: Seek pointer = 15 Parent: After forking: Seek pointer = 55

Signals Mechanism for OS to notify processes of special events such as Timer expire User type Ctrl-C on the controlling terminal Owner kill the process … Signals cause process to suspend its usual execution to run a signal handling procedure Similar to hardware interrupt

Signals (cont’d) A signal is an interruption of program when you hit CNTL-C, SIGINT signal is sent to your program. When you hit CNTL-\, SIGQUIT signal is sent to your program. When you generate a segmentation violation, that sends the SIGSEGV signal to your program. Command “kill” can be used to send a signal to a process kill -9 [process_id] By default, there are certain actions that take place. When you hit CNTL-C, program usually exits. When you hit CNTL-\ or get a segmentation violation, your program dumps core and then exits (default action for SIGQUIT and SIGSEGV)

Example of signals Signal Value Action Comment SIGHUP 1 Term Hangup detected on controlling terminal or death of controlling process SIGINT 2 Term Interrupt from keyboard SIGQUIT 3 Core Quit from keyboard SIGILL 4 Core Illegal Instruction SIGABRT 6 Core Abort signal from abort(3) SIGFPE 8 Core Floating point exception SIGKILL 9 Term Kill signal SIGSEGV 11 Core Invalid memory reference SIGPIPE 13 Term Broken pipe: write to pipe with no readers

Register signal handler #include < signal.h > void cntl_c_handler(int dummy) { printf("You just typed cntl-c\n"); signal(SIGINT, cntl_c_handler); } main() { int i, j; for (j = 0; j < 40; j++) { for (i = 0; i < 1000000; i++); Example file: signal.c

Signal as IPC } else { /* parent process */ printf("Hello, "); void parentdone(int signum) {} int main() { int pid; if ((pid = fork())== -1) { perror("fork() failed"); exit(1); } if (pid == 0) { /* child process */ signal(SIGUSR1, parentdone); pause(); /* sleep until a signal is received */ printf("World!\n"); fflush(stdout); } else { /* parent process */ printf("Hello, "); fflush(stdout); kill(pid, SIGUSR1); /* send a signal to the child */ wait(NULL); /* wait the child exit */ } exit(0); } // end of main()

Pipe as inter-process communication method In Unix, all inter-process communication must take place with help of operating system Process is considered as primary unit of isolation: one process cannot crash whole system, instead you get segmentation fault Well, not quite, the forkbomb we saw last class could halt whole system ! Be extremely careful! Pipes are a nice clean IPC method One process write to the pipe, one process read from it

Processes stack Heap Data Text(Code) Process memory layout Per-process opened file table stack 1 2 pipefd[0] pipefd[1] fd r/w r 1 w 2 3 rw Operating system Heap Data Text(Code)

Pipe Pipe is a inter-process communication mechanism: Allow parent-child processes to communicate with each other Make pipeline in shell possible (avoid using disk file) Pipes are treated as files You can call read() on the reading end, call write() on writing end, close() it… FIFO: data that are written first into the pipe are read out first Operating system: buffer data to handler faster writer & slower reader… Producer: Write data to writing end Consumer: Read data from reading end

Pipe system call #include <unistd.h> int pipe(int fd[2]); Parameter: an array of two integers. Upon return: the array is filled with two file descriptions: fd[0]: reading end of pipe fd[1]: writing end of pipe Anything that is written on fd[1] can be read from fd[0]. Pipes are treated as files You can use system call read(), write(), close() … on pipes

Simple example #include <stdio.h> main() { int pipefd[2]; int i; char s[1000]; char *s2; if (pipe(pipefd) < 0) { perror("pipe"); exit(1); } s2 = “Hello World!"; write(pipefd[1], s2, strlen(s2)); i = read(pipefd[0], s, 1000); s[i] = '\0'; printf("Read %d bytes from the pipe: '%s'\n", i, s); write() to a reading end of a pipe: asks operating system to hold those bytes in a buffer until some process requests for them by performing a read() on the read end of the pipe.

Visualize pipes stack Heap Data Text(Code) Process memory layout Per-process opened file table stack 1 2 pipefd[0] pipefd[1] fd r/w r 1 w 2 pipefd[0] pipefd[1] Operating system buffer Heap Data Text(Code) Hello World!

Pipe: inter-process communication main() { int pipefd[2]; int pid, i, line; char s[1000]; if (pipe(pipefd) < 0) { perror("pipe"); exit(1); } pid = fork(); if (pid > 0) { // parent process while(fgets(s, 1000, stdin) != NULL) { write(pipefd[1], s, strlen(s)); close(pipefd[1]); else { //child process i = 0; line = 1; while(read(pipefd[0], s+i, 1) == 1) { if (s[i] == '\n') { s[i] = '\0'; printf("%6d %s\n", line, s); line++; i = 0; } i++; } // end of while loop } // end of else{ } //end of main()

Visualize pipes stack stack Heap Data Text(Code) Heap Data Text(Code) Parent Process Child process Per-process opened file table Per-process opened file table stack stack 1 2 pipefd[0] pipefd[1] fd r/w r 1 w 2 pipefd[0] pipefd[1] fd r/w r 1 w 2 pipefd[0] pipefd[1] Heap Data Text(Code) Heap Data Text(Code) Operating system buffer Hello World!

stack stack Heap Data Text(Code) Heap Data Text(Code) Let’s try it out: ctrl-d to finish standard input Why the child process lingers ? OS does not know no one is writing to pipe Parent Process Child process Per-process opened file table Per-process opened file table stack stack 1 2 pipefd[0] pipefd[1] fd r/w r 1 w 2 pipefd[0] pipefd[1] fd r/w r 1 w 2 pipefd[0] pipefd[1] Heap Data Text(Code) Heap Data Text(Code) Operating system buffer Hello World!

while(read(pipefd[0], s+i, 1) == 1) { if (s[i] == '\n') { s[i] = '\0'; main() { int pipefd[2]; int pid; int i, line; char s[1000]; if (pipe(pipefd) < 0) { perror("pipe"); exit(1); } pid = fork(); if (pid > 0) { close(1); close(pipefd[0]); while(fgets(s, 1000, stdin) != NULL) { write(pipefd[1], s, strlen(s)); close(pipefd[1]); else { close(0); close(pipefd[1]); i = 0; line = 1; while(read(pipefd[0], s+i, 1) == 1) { if (s[i] == '\n') { s[i] = '\0'; printf("%6d %s\n", line, s); line++; i = 0; } i++; } //end of while } //end of else } //end of main Sol: close unused ends of the pipe!!

Handling broken pipes When read from a pipe that has no write end, read() returns 0. When write to a pipe that has no read end, a SIGPIPE signal is generated. If signal isn't handled, program exits silently Example: If you execute: UNIX> cat exec1.c | head -5 | tail -1 and you kill the middle process (the head one), the other two will exit automatically

Now: command pipeline In shell, we can build command pipeline How to implement this ? Create a pipe Create two child processes (both children inherit the pipe) Redirect standard output of first child to writing end of pipe Redirect standard input of second child to reading end of pipe First child to run first command, another child to run second command How to redirect standard input/output ?

Redirect standard output To redirect standard output to a different target Open the target (whether a file, or a pipe) which returns a file descriptor, say fd Duplicate the file descriptor, fd, to descriptor 1 (standard output) i.e., let file descriptor 1 pointing to the target Old and new descriptors may be used interchangeably, and they share locks, file position pointers and flags. To duplicate file descriptor: int dup(int oldfd); int dup2(int oldfd, int newfd);   //Prefer to use this one create a copy of file descriptor oldfd, to lowest-numbered unused file descriptor (dup), or makes newfd be copy of oldfd, closing newfd first if necessary.  

Example: pipeline.c if (pipe(pipefd) < 0) { perror("pipe"); exit(1); } //upon successful return, pipefd[0] is reading end, pipefd[1] is writing end pid = fork(); if (pid==0) { ret=dup2 (pipefd[1],1); //close standard output, make 1 pointing to pipefd[1] close (pipefd[1]); //as 1 is pointing to writing end too, we can close pipefd[1] if (ret==-1){ fprintf (stderr,"first child cannot dup\n"); } close(pipefd[0]); //close reading end of pipe execlp("ls","ls",0); //the standard output of ls will be pipefd[1] fprintf (stderr,”failed to execue ls”); } //end of else

Redirect standard input to a file Here is code segment for redirect standard input to a file int fd=open(“file.txt”,O_RDONLY); if (fd<0) … ret=dup2 (fd,0); //close current standard input, make 0 //pointing to the file that fd is pointing to if (ret==-1){ fprintf (stderr,"first child cannot dup\n"); exit(1); } cin >> number >> name; //all subsequent read from standard // input is from the file file.txt

Pipe vs named pipe For two processes to communicate through pipes, they need access to same pipe Only possible if the pipe is created by their common ancestors Named pipe make it possible for two unrelated processes to communicate in a similar way Named-pipe has a name (like other files), process can open the pipe to read or write

Using named pipe Create a named pipe ls –l will show myPipe as mkfifo myPipe ls –l will show myPipe as prw-r--r-- 1 zhang staff 0 2008-04-25 09:30 myPipe In one terminal, type: ls -l > myPipe in another type: cat < myPipe Difference with using file: You can start “cat” command first “ls” process will wait until “cat” pick up the output Remember that “ls” and “cat” program are not aware of the fact that input/output is from/to a named pipe

Named pipes: server #define HALF_DUPLEX "/tmp/halfduplex" #define MAX_BUF_SIZE> 255 int main(int argc, char *argv[]) { int fd, ret_val, count, numread; char buf[MAX_BUF_SIZE]; ret_val = mkfifo(HALF_DUPLEX, 0666); if ((ret_val == -1) && (errno != EEXIST)) { perror("Error creating the named pipe"); exit (1); } fd = open(HALF_DUPLEX, O_RDONLY); numread = read(fd, buf, MAX_BUF_SIZE); buf[numread] = '0'; printf("Half Duplex Server : Read From the pipe : %sn", buf); …. printf("Half Duplex Server : Converted String : %sn", buf);

Named pipe: client #define HALF_DUPLEX "/tmp/halfduplex" #define MAX_BUF_SIZE 255 int main(int argc, char *argv[]) { int fd; if (argc != 2) { printf("Usage : %s <string to be sent to the server>n", argv[0]); exit (1); } fd = open(HALF_DUPLEX, O_WRONLY); write(fd, argv[1], strlen(argv[1]));

Process & Child Process Processes do not share data or stack Parent Process Child process Per-process opened file table Per-process opened file table stack stack 1 2 pipefd[0] pipefd[1] fd r/w r 1 w 2 fd r/w r 1 w 2 Heap Data Text(Code) Heap Data Text(Code) Operating system Inter-Processes Communication: pipe, shared memory, file, signal, …

Multi-thread programming Threads within a process share data and code (less expensive way to achieve parallelism) Each thread has separate stack space Programs need to ensure data integrity: E.g., Multiple threads cannot write a global variable simultaneously Process Per-process opened file table stack 1 2 pipefd[0] pipefd[1] fd r/w r 1 w 2 pthread_create New stack for thread Heap Data Text(Code)

Hints for final project: analysis How to get started ? Reading the requirement ! Start with one simple cast: For example, focus first upon understanding how to support the –i mode (i.e., read command from standard input)… Identify modules: I will need to be able to run a simple command (w/o pipeline) I will need to split pipeline into two commands I will need to see if there is a trailing & in command … Do not start next stage until you are clear what the program needs to do …

Hints for final project: design How to implement the required functionalities? Do I know how to solve one case ? E.g., interactive mode Try write the pseudocode (a plan for the coding) Is the scripting mode (i.e., read command from file) much different ? Try to generalize your solution as much as possible ! Scripting mode and interactive mode Similarity ? Difference ?

Modular Design Identify and write the headers of functions Function body can be done later … Write main function A mixture of code and comments describing the pseudocode will be fine Verifying your design before start coding Walk-through your code, pay attention to: The information flow: argument passing, value returning, …

Finally, coding/testing/debugging Whenever call a library function, make sure the arguments are passed correctly And check the value returned for possible errors !!! Add assertion statement in the code to verify and isolate errors assert (fp!=NULL); assert (n>0); Add printf () or cout statements to check the value of important variables Is the program doing what I assume it is doing ? If not, when does it diverge from my assumption ?