1. [Unix Programming] Interprocess Communication - PIPE Young-Ju, Han Email: yjhan@imtl.skku.ac.kr yjhan@imtl.skku.ac.kr

2. 2007 UNIX Programming Contents  Pipes  Using pipe a single process  Using pipe a multiple processes  Using pipe in the shell

3. 2007 UNIX Programming Overview  different forms of IPC pipes(half duplex) FIFOs(named pipes) stream pipes(full duplex) named stream pipes message queues semaphores shared memory sockets streams on the same host on different host

4. 2007 UNIX Programming Pipes  only form of IPC on all UNIX implementations  The oldest form of UNIX IPC  most commonly used form of IPC  limitations half-duplex  one-way communication channel used only between processes that have a common ancestor (related processes) stream pipes FIFOs & named stream pipes

5. 2007 UNIX Programming Pipes  Pipes connects the standard output of one program to the standard input of another program I/O Redirections and Pipelines Stdout stderr myprog1 file2 stdin file1 myprog2 stdout stdin stderr $ (myprog1 | myprog2 ) file2

6. 2007 UNIX Programming Pipes  Pipes at command level (Example) I/O Redirection and Pipelines Method 1. I/O Redirection Method $ ls /home/kc > tempfile $ wc -l < tempfile 19 $ rm tempfile 2. Pipeline Method $ ls /home/kc | wc -l 19

7. 2007 UNIX Programming Pipes (Example) cd /bin 1. I/O Redirection Method (Failure) $ ls > tempfile tempfile: permission denied 2. Pipeline Method ( Success) $ ls | wc -l 777  pipes at command level Weak Point of I/O Redirection Method 1. Cannot create temporary files if you don’t have permission 2. May have a invalid result because of temporary file 3. Easy to forget to remove temporary file

8. 2007 UNIX Programming pipes  % who | sort

9. 2007 UNIX Programming Pipes   Create pipes pipe() system call   Data transmitting data is written into pipes using the write() system call data is read from a pipe using the read() system call automatic blocking when full or empty automatic blocking when full or empty FIFO basis, so lseek() don’t work   Types of pipes (unnamed) pipes named pipes

10. 2007 UNIX Programming Pipes  Programming with pipes create a pipe #include int pipe (int fd[2]); 0 : ok -1 : error more open than per-user process limit(EMFILE) kernel’s open file table overflow(ENFILE) fd[0] : read only open fd[1] : write only open

11. 2007 UNIX Programming Pipes  ex) pipe on a single process #include #define MSGSIZE 1024 char *msg1= “hello, world #1”; char *msg2= “hello, world #2”; char *msg3= “hello, world #3”; int main(){ char inbuf[MSGSIZE]; int fd[2], j, ret; if (pipe(fd) == -1) {perror(“pipe call”); exit(1);} write(fd[1], msg1, strlen(msg1)); write(fd[1], msg2, strlen(msg2)); write(fd[1], msg3, strlen(msg3)); for (j = 0; j < 3; j++) { ret = read(fd[0], inbuf, MSGSIZE); inbuf[ret]=0; printf(“%s\n”, inbuf); } $a.out hello, world #1 hello, world #2 hello, world #3

12. 2007 UNIX Programming Pipes  ex) pipe on a single process fd[0]fd[1] process fd[0]fd[1] process kernel pipe read in the order in which they were written first in/first out(FIFO) communication channel => Since lseek don’t work on a pipe or

13. 2007 UNIX Programming Pipes  kernel’s pipe buffer size constant PIPE_BUF rule overfill pipeblocked  if a write overfill pipe, then blocked until reading emptyblocked  if a read when empty, then blocked until writing  if a write > pipe, then block after write  if a read > pipe, then return after read pipe_size = fpathconf( p[0], _PC_PIPE_BUF );

14. 2007 UNIX Programming Pipes  rules (when one end of a pipe is closed) If we read from a pipe whose write end has been closed,  after all the data has been read, read return 0 to indicate an end of file  End of file is not generated until there are no more writers for the pipe If we write to a pipe whose read end has been closed, SIGPIPE  the signal SIGPIPE is generated Default action : terminate

15. 2007 UNIX Programming Pipes  ex) pipe from child to parent int main(){ char inbuf[MSGSIZE]; int fd[2], j; pid_t pid; if (pipe(fd) == -1) {perror(“pipe call”); exit(1);} switch(pid = fork()) { case -1 : perror(“fork error”); exit(1); case 0 : write(fd[1], msg1, strlen(msg1)); write(fd[1], msg2, strlen(msg1)); write(fd[1], msg3, strlen(msg1)); break; default : for (j = 0; j < 3; j++) { read(fd[0], inbuf, MSGSIZE); inbuf[ret] = 0; printf(“%s\n”, inbuf); } wait(NULL); } $a.out hello, world #1 hello, world #2 hello, world #3 => 무한 대기 무한대기

16. 2007 UNIX Programming Pipes  ex) pipe from child to parent fd[0]fd[1] parent fd[0]fd[1] child kernel pipe fork write() read()

17. 2007 UNIX Programming Pipes  ex) pipe from child to parent (recommended) switch(pid = fork()) { case -1 : perror(“fork error”); exit(1); case 0 : close(fd[0]); write(fd[1], msg1, MSGSIZE); write(fd[1], msg2, MSGSIZE); write(fd[1], msg3, MSGSIZE); break; default : close(fd[1]); for (j = 0; j < 3; j++) { read(fd[0], inbuf, MSGSIZE); printf(“%s\n”, inbuf); } wait(NULL); }

18. 2007 UNIX Programming Pipes  pipe from child to parent (recommended) parentchild kernel pipe fork fd[0]fd[1] write() read()

19. 2007 UNIX Programming Pipes  Non-blocking reads and writes to use fstat  If st_size > 0, and then read() from pipe  st_size : number of characters currently in the pipe  problem : if several processes are reading the pipe, another process could read from a pipe in the gap between fstat and read to use fcntl ( recommend)  O_NONBLOCK flag errno -1 : EAGAIN #include if( fcntl(fd, F_SETFL, O_NONBLOCK) == -1 ) perror(“fcntl”);

20. 2007 UNIX Programming Pipes  Non-blocking read and write #include #define MSGSIZE 6 int parent (int *); int child (int *); char *msg1 = "hello"; char *msg2 = "bye!!"; void fatal(char* msg) { fprintf(stderr,”%s\n”,msg);exit(1);} int main(){ int pfd[2]; /* 파이프를 개방한다 */ if(pipe (pfd) == -1) fatal ("pipe call"); /* p[0] 의 O_NONBLOCK 플래그를 1 로 설정한다 */ if (fcntl (pfd[0], F_SETFL, O_NONBLOCK) == -1) fatal ("fcntl call"); switch(fork()){ case -1: fatal("fork call"); /* 오류 */ case 0: child(pfd); /* 자식 */ default: parent (pfd); /* 부모 */ } }

21. 2007 UNIX Programming Pipes  Non-blocking read and write int parent (int p[2]) {/* 부모의 코드 */ int nread; char buf[MSGSIZE]; close (p[1]); for(;;) { switch (nread = read(p[0], buf, MSGSIZE)){ case -1: /* 파이프에 아무것도 없는지 검사한다. */ if (errno == EAGAIN){ printf ("(pipe empty)\n"); sleep (1); break; } else { perror("read call"); exit(1);} case 0: /* 파이프가 닫혔음. */ printf ("End of conversation\n"); exit (0); default: printf ("MSG=%s\n", buf); } } }

22. 2007 UNIX Programming Pipes  Non-blocking read and write int child(int p[2]) { int count; close (p[0]); for (count= 0; count < 3; count++) { write (p[1], msg1, MSGSIZE); sleep(3); } /* 마지막 메시지를 보낸다 */ write (p[1], msg2, MSGSIZE); exit (0); } $a.out (pipe empty) MSG=hello (pipe empty) MSG=hello (pipe empty) MSG=hello (pipe empty) MSG=bye!! (pipe empty) End of conversation

23. 2007 UNIX Programming Pipes  Using select to handle multiple pipes Server & client Applications parent Child 1 pipe fork fd1[0] fd1[1] write() read() Child 2 fd2[0]fd3[0] fd2[1] fd3[1] pipe Child 3 fork

24. 2007 UNIX Programming Pipes  Using select to handle multiple pipes nfds : specifies the range of file descriptors to be tested.  0 ~ nfds-1 readfds : the specified file descriptors is ready for reading writefds : FDs is ready for writing errorfds : FDs is ready for error checking Timeout  Timeout = NULL : block forever or until something of interest turns up  timeout.tv_sec = 0 && timeout.tv_usec = 0: returns immediately without blocking  Timeout.tv_sec != 0 || timeout.tv_usec != 0 : return after that number of tv_sec or tv_usec specified if there is no FDs activity #include int select (int nfds, fd_set *readfds, fd_set *writefds, fd_set* errorfds, struct timeval *timeout); #include struct timeval { long tv_sec; /*second */ long tv_usec; /*microseconds */ } 0 : timeout, -1 : error >0 : the number of “interesting” FDs

25. 2007 UNIX Programming Pipes  Macro for manipulating sets of file descriptor (bit mask) #include /* initialize the mask pointed to by fdset */ void FD_ZERO(fd_set *fdset); /* set the bit, fd, in the mask pointed to by fdset */ void FD_SET(int fd, fd_set* fdset); /* is the bit, fd, set in the mask pointed to by fdset */ void FD_ISSET(int fd, fd_set* fdset); /* turn off the bit, fd, in the mask pointed to by fdset */ void FD_CLR(int fd, fd_set *fdset);

26. 2007 UNIX Programming Pipes  Using select to handle multiple regular files #include. int fd1, fd2; fd_set readset; fd1 = open(“file1”, O_RDONLY); fd2 = open(“file2”, O_RDONLY); FD_ZERO(&readset); FD_SET(fd1, &readset); FD_SET(fd2, &readset); switch(select(5,&readset,NULL,NULL,NULL)) { /* logic */ } select(fd2+1,&readset,NULL,NULL,NULL) Not recommended

27. 2007 UNIX Programming pipes  Examples (Using select to handle multiple pipes) #include #define MSGSIZE 6 char *msg1 = "hello"; char *msg2 = "bye!!"; void parent(int a[][]); int child(int b[]); void fatal(char* msg){ fprintf(stderr,"%s\n",msg);exit(1);} int main(){ int pip[3][2]; int i; for (i = 0; i < 3; i++) { if (pipe(pip[i]) == -1) fatal("pipe call"); switch (fork()){ case -1: /* 오류 */ fatal("fork call"); case 0: /* 자식 */ child(pip[i]); } parent(pip); /* 부모 */ exit(0); }

28. 2007 UNIX Programming pipes  Examples (parent process) void parent(int p[3][2]){ char buf[MSGSIZE], ch; fd_set set, master; int i; for (i = 0; i < 3; i++) close(p[i][1]); FD_ZERO(&master); FD_SET(0, &master); for (i = 0; i <3; i++) FD_SET(p[i][0], &master); while(set=master, select (p[2][0]+1, &set, NULL, NULL, NULL) > 0){ if(FD_ISSET(0, &set)){ /* standard input check */ printf ("From standard input..."); read (0, &ch, 1); printf("%c\n", ch); } for (i = 0; i < 3; i++) { if(FD_ISSET(p[i][0], &set)) { if(read(p[i][0], buf, MSGSIZE) > 0) { printf ("Message from child%d\n", i); printf ("MSG=%s\n",buf); } if(waitpid (-1, NULL,WNOHANG) == -1) return; }

29. 2007 UNIX Programming pipes  Examples (child process) int child(int p[2]){ int count; close(p[0]); for (count = 0; count < 2; count++) { write (p[1], msg1, MSGSIZE); sleep (getpid()%4); } write (p[1], msg2, MSGSIZE); exit (0); } $a.out Message from child0 MSG=hello Message from child1 MSG=hello Message from child2 MSG=hello Message from child2 MSG=hello Message from child2 MSG=bye!! d From standard input...d From standard input... Message from child0 MSG=hello.

30. 2007 UNIX Programming Sell pipe implementation  Pipes and the exec system call %ls –l | wc Shell /bin/sh parent child ls child wc pipe fd[1]fd[0] /* stdin 에서 command read */ /* command parsing */ /* 2 개의 command 이고 둘이 pipe 로 연결되어 있다면 */ int fd[2], c1; pipe(fd); c1 = fork (); If(!c1) {/*first child*/ close( fd[0] ); close(1); dup2( fd[1],1 ); close( fd[1]); execlp(/*ls –l first command */); } c2= fork (); If(!c2) {/*second child*/ close( fd[1] ); close(0); dup2( fd[0],0 ); close( fd[0]); execlp(/*wc second command */); } close( fd[0] ); close( fd[1] );

31. 2007 UNIX Programming Sell pipe implementation  dup(), dup2() 사용중인 파일 디스크립터의 복사본을 만듦  dup() 는 새 파일 디스크립터가 할당  dup2() 는 fd2 를 사용 리턴 값  성공하면 복사된 새 파일 디스크립터, 실패하면 -1  dup() 는 할당 가능한 가장 작은 번호를 리턴  dup2() 는 fd2 를 리턴  dup2() 는 복사본을 만들기 전에 기존의 fd2 를 close 함 #include int dup(int fd); int dup2( int fd, int fd2);

32. 2007 UNIX Programming Sell pipe implementation  Another method: The Join program parent Child 2 (com1) wait() write() (stdout) read() (stdin) pipe Child 1 (com2) P[1]P[0]

33. 2007 UNIX Programming Sell pipe implementation  Pseudo-code form of the Join program Process forks, parent waits for child And child continues Child create a pipe Child then forks In child created by second fork (child 2): standard output is coupled to write end of pipe using dup2 excess file descriptors are closed program described by ‘com1’ is exec’ed In child of first fork (child 1): standard input is coupled to read end of pipe using dup2 excess file descriptors are closed program described by ‘com2’ is exec’ed

34. 2007 UNIX Programming Sell pipe implementation  The Join program int join(char* com1[], char* com2[]){ int p[2], status; /* 명령을 수행할 자식을 생성한다. */ switch (fork()){ case -1: /* 오류 */ fatal ("1st fork call in join"); case 0: /* 자식 */ break; default: /* 부모 */ wait(&status); return (status); } /* 루틴의 나머지 부분으로 자식에 의해 수행된다. */ /* 파이프를 만든다. */ if (pipe(p) == -1) fatal ("pipe call in join");

35. 2007 UNIX Programming Sell pipe implementation  The Join program switch (fork()){/* 다른 프로세스를 생성한다. */ case -1: /* 오류 */ fatal ("2nd fork call in join"); case 0: /* 쓰는 프로세스 */ dup2 (p[1],1); /* 표준 출력이 파이프로 가게 한다. */ close (p[0]); /* 화일 기술자를 절약한다. */ close (p[1]); execvp (com1[0], com1); /* execvp 가 복귀하면, 오류가 발생한 것임. */ fatal("1st execvp call in join"); default: /* 읽는 프로세스 */ dup2(p[0], 0); /* 표준 입력이 파이프로부터 오게 한다 */ close (p[0]); close (p[1]); execvp (com2[0], com2); fatal ("2nd execvp call in join"); }}

36. 2007 UNIX Programming Sell pipe implementation  The Join program 앞서 join routine 은 다음 프로그램을 이용하여 호출될 수 있다 : #include main() { char *one[4] = {"ls", "-l", "/usr/lib", NULL}; char *two[3] = {"grep", " ∧ d", NULL}; int ret; ret = join (one, two); printf ("join returned %d\n", ret); exit (0); }