Chapter 5 TCP Client/Server Example

TCP Client-Server Example TCP echo server: main and str_echo TCP echo client: main and str_cli Normal startup and termination POSIX signal handling Handling SIGCHILD, interrupted system calls, and preventing zombies Connection abort before accept returns Crash of server process

SIGPIPE signal Crash, reboot, shutdown of server host Summary of TCP example Data format: passing string or binary

TCP Echo Server and Client To expand this example to other applications, just change what the server does with the client input. Many boundary conditions to handle: signal, interrupted system call, server crash, etc. The first version does not handle them.

TCP Echo Server main() Algorithm A typical fork()-based concurrent server Algorithm outline : Create socket Bind it to a designated port (supposedly to be a well-known port) Allow incoming traffic for any local network interface (wildcard address: INADDR_ANY) Convert it to a listening socket Set up a listening queue Loop around forever : Block in call to accept(), wait for a client to connect Spawn a child to handle each client upon successful connection Close listening socket Execute str_echo() Close connected socket for child. ‹—— parent no wait 5

TCP Echo Server: main function #include "unp.h" int main(int argc, char **argv) { int listenfd, connfd; pid_t childpid; socklen_t clilen; struct sockaddr_in cliaddr, servaddr; listenfd = Socket(AF_INET, SOCK_STREAM, 0); bzero(&servaddr, sizeof(servaddr)); servaddr.sin_family = AF_INET; servaddr.sin_addr.s_addr = htonl(INADDR_ANY); servaddr.sin_port = htons(SERV_PORT); Bind(listenfd, (SA *) &servaddr, sizeof(servaddr)); tcpcliserv/tcpserv01.c 9877

Listen(listenfd, LISTENQ); for ( ; ; ) { clilen = sizeof(cliaddr); connfd = Accept(listenfd, (SA *) &cliaddr, &clilen); if ( (childpid = Fork()) == 0) { /* child process */ Close(listenfd); /* close listening socket */ str_echo(connfd); /* process the request */ exit(0); } Close(connfd); /* parent closes connected socket*/ }

str_echo() Algorithm It provides very simple service for each client It reads data from a client and echoes it back to the client Algorithm outline : Read a buffer from the connected socket If n (number of characters read) > 0, Echo back to the client (writen(): p. 89), read again Else if n < 0 & EINTR (got interrupt), read again Else just n < 0 (error occurred), display error message (and terminate child process in err_sys()) Else if n = 0 (receipt of FIN from client, the normal scenario), return 8

TCP Echo Server: str_echo function #include "unp.h" void str_echo(int sockfd) { ssize_t n; char line[MAXLINE]; for ( ; ; ) { if ( (n = Readline(sockfd, line, MAXLINE)) == 0) return; /* connection closed by other end */ Writen(sockfd, line, n); } lib/str_echo.c

TCP Echo Client main() Algorithm Algorithm outline : Check number of commandline arguments It must be 2 (program name and server address) Quit if not 2 (call to sys_quit()) Open socket Fill in internet socket address structure Connect to server Call str_cli() to handle the rest of the client processing Exit when no more user input Note: All errors end up in termination of the client in this function. Real applications may need to recover differently 10

TCP Echo Client: main function #include "unp.h" int main(int argc, char **argv) { int sockfd; struct sockaddr_in servaddr; if (argc != 2) err_quit("usage: tcpcli "); sockfd = Socket(AF_INET, SOCK_STREAM, 0); bzero(&servaddr, sizeof(servaddr)); servaddr.sin_family = AF_INET; servaddr.sin_port = htons(SERV_PORT); Inet_pton(AF_INET, argv[1], &servaddr.sin_addr); Connect(sockfd, (SA *) &servaddr, sizeof(servaddr)); str_cli(stdin, sockfd); /* do it all */ exit(0); } tcpcliserv/tcpcli01.c

TCP Echo Client: str_cli function #include "unp.h" void str_cli(FILE *fp, int sockfd) { char sendline[MAXLINE], recvline[MAXLINE]; while (Fgets(sendline, MAXLINE, fp) != NULL) { Writen(sockfd, sendline, strlen(sendline)); if (Readline(sockfd, recvline, MAXLINE) == 0) err_quit("str_cli: server terminated prematurely"); Fputs(recvline, stdout); } lib/str_cli.c

Normal Startup (1/2) To watch the sequence of client/server To start the server in background : linux% tcpserv01 & [1] To check the status of all sockets on a system (-a) before the client starts : linux% netstat – a Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 *:9877 *:* LISTEN Note : The output above shows only partial results, and the output format may be different from system to system 13

Normal Startup (2/2) To start the client on the same machine (using the loopback address) : linux% tcpcli Then, check the status of all sockets again : linux% netstat – a Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 localhost:9877 localhost:42758 ESTABLISHED tcp 0 0 localhost:42758 localhost:9877 ESTABLISHED tcp 0 0 *:9877 *:* LISTEN Note : The first tcp connection is for the server child, and the second is for the client, and the third is the server parent 14

Normal Termination (1/2) Client 離開 str_cli  執行 exit 離開 main Client 結束時，所有已開啟的 socket 會被 kernel 自動關閉。 Client 的 TCP 會送 FIN 至 server ， server 的 TCP 會回 ACK. Client 到 server child 這一半的 connection 已關閉 Fgets

Normal Termination (2/2) 當 server TCP 收到 FIN 時， server child 正 等在 readline ，故 readline 會傳回 0 ，導致 server child 離開 str_echo 回到 main 。 Server child 執行 exit 會使 server TCP 關閉 另一半 (server child 到 client) 的連結。 Server child 程序結束成為 zombie

Zombie Process ( 另一個系統的輸出 )

Cleaning Up Zombie Processes Child process 變成 zombie 後不會自行消 失 Zombie 保留 child 結束時的狀態以備 parent 取得，因此會佔用 kernel 空間 Parent 應該要主動清除 zombie 必須要處理 kernel 發出的 signal(SIGCHLD)

POSIX Signal Handling Signal (software interrupt): sent by one process to another process (or to itself) or by the kernel to a process SIGCHLD: by the kernel to the parent when a child process is terminated Disposition of a signal: catch the signal by a specified signal handler SIG_IGN: ignore it SIG_DFL: default: terminate or ignore

signal Function That Enables System Call Restart #include "unp.h" Sigfunc *signal(int signo, Sigfunc *func) { struct sigaction act, oact; act.sa_handler = func; sigemptyset(&act.sa_mask); act.sa_flags = 0; if (signo == SIGALRM) { #ifdef SA_INTERRUPT act.sa_flags |= SA_INTERRUPT; /* SunOS 4.x */ #endif } else { #ifdef SA_RESTART act.sa_flags |= SA_RESTART; /* SVR4, 44BSD */ #endif } if (sigaction(signo, &act, &oact) < 0) return(SIG_ERR); return(oact.sa_handler); } 讓被某些 singal 中斷的 system call 能夠 restart Posix 標準 signal disposition 函數 sigaction 使用上較為 複雜，故作者用此 signal 函數來包裝 sigaction lib/signal.c

Signal Function That Enables System Call Restart (cont.) Sigfunc *Signal(int signo, Sigfunc *func) { Sigfunc *sigfunc; if ( (sigfunc = signal(signo, func)) == SIG_ERR) err_sys("signal error"); return(sigfunc); } POSIX signal semantics: 1. Once a signal handler is installed, it remains installed. 2. The signal being delivered is blocked while a signal handler is executing. 3. By default, signals are not queued. Signal 是將 signal 包裝起來的函數

Handling Zombies Whenever we fork children, we must wait for them to prevent them from becoming Zombies To do this, we establish a signal handler to cache SIGCHLD and within the handler we call wait.

Signal Handler for SIGCHLD #include "unp.h" void sig_chld(int signo) { pid_t pid; int stat; pid = wait(&stat); printf("child %d terminated\n", pid); return; } tcpcliserv/sigchldwait.c child 的 process id 呼叫 wait 以免 讓 child 變成 zombie

Interrupted System Calls 當 kernel 發出 SIGCHLD signal 而被 parent 捕捉 時， parent 是 block 在 accept 這個 system call 中。 此 system call 被 interrupt 去執行 handler 當 signal handler return 後，如果此 interrupted system call 沒有被 restart ，此 system call 會傳 回錯誤 (EINTR) 我們定義的 signal 函數有設定要 restart system call EINTR 的錯誤可導致我們設計的 Accept 函數中 止程式執行 (abort)

Handling Interrupted System Calls Definition: Slow system call “ system call that can block forever ” Some kernels automatically restart some interrupted system calls, while some don ’ t We must prepare for slow system calls to return EINTR for ( ; ; ){ clilen = sizeof (cliaddr); if ( (connfd = accept (listenfd, (SA) &cliaddr, &clilen)) < 0) { if (errno == EINTR) continue; /* back to for ( ) */ else err_sys ( “ accept error ” ); } 我們自行 restart accept 這個 system call

Weakness of Wait Unix signals are normally not queued multiple occurrences of the same signal only cause the handler to be called once It ’ s a problem when multiple children terminate at the same time Solution: use waitpid instead of wait in the handler to kill all zombies

Client terminates all five connections catching all SIGCHLD signals in server parent

SIGCHLD Handler UsingWaitpid #include "unp.h" void sig_chld(int signo) { pid_t pid; int stat; while ( (pid = waitpid(-1, &stat, WNOHANG)) > 0) printf("child %d terminated\n", pid); return; } #include pid_t waitpid (pid_t pid, int *statloc, int options); returns: process ID if OK, o, or -1 on error Wait for the first terminated child Not block if there are no terminated children

Final (correct) Version of TCP Echo Server handling SIGCHLD, EINTR from accept, zombies #include "unp.h" int main(int argc, char **argv) { int listenfd, connfd; pid_t childpid; socklen_t clilen; struct sockaddr_in cliaddr, servaddr; void sig_chld(int); listenfd = Socket(AF_INET, SOCK_STREAM, 0); bzero(&servaddr, sizeof(servaddr)); servaddr.sin_family = AF_INET; servaddr.sin_addr.s_addr = htonl(INADDR_ANY); servaddr.sin_port = htons(SERV_PORT); Bind(listenfd, (SA *) &servaddr, sizeof(servaddr)); Listen(listenfd, LISTENQ); tcpcliserv/tcpserv04.c

Final (correct) Version of TCP Echo Server (cont.) Signal(SIGCHLD, sig_chld); /* must call waitpid() */ for ( ; ; ) { clilen = sizeof(cliaddr); if ( (connfd = accept(listenfd, (SA *) &cliaddr, &clilen)) < 0) { if (errno == EINTR) continue; /* back to for() */ else err_sys("accept error"); } if ( (childpid = Fork()) == 0) { /* child process */ Close(listenfd); /* close listening socket */ str_echo(connfd); /* process the request */ exit(0); } Close(connfd); /* parent closes connected socket */ }

Connection Abort Before accept Returns implementation dependent ! In BSD, kernel handles this. accept does not return. In SVR4, accept is returned with EPROTO. In POSIX.1g, accept is returned with ECONNABORTED The server can ignore the error and just call accept again

Crashing of Server Child Process Procedure: 1. Kill the child. Server TCP sends FIN to client TCP, which responds with an ACK. (TCP half-close) (The client process is blocked in fgets when client TCP receives FIN.) 2. SIGCHLD signal is sent to the server parent. 3. User inputs. The client process calls writen to send data to server. 4. The server TCP responds with an RST. 5. The client process returns from readline, 0, when client TCP receives RST. 6. The client process terminates. The client is blocked on the call to fgets when FIN is received. (it should block on input from either source) Solution: Use select or poll to block on either socket or stdio Client 沒辦法同時 等待兩個 input

SIGPIPE Signal when writing to a socket that has received an RST Procedure: 1. The client writes to a crashed server process. An RST is received at the client TCP and readline returns 0 (EOF). 2. If the client ignores the error returned from readline and write more, SIGPIPE is sent to the client process. 3. If SIGPIPE is not caught, the client terminates with no output Problem: Nothing is output even by the shell to indicate what has happened. (Have to use “ echo $? ” to examine the shell ’ s return value of last command.) Solution: 1.Setting the signal disposition to SIG_IGN 2.Catch the SIGPIPE signal for further processing. (handle EPIPE error returned from write).

An Example to Show SIGPIPE To invoke tcpcli11 which has two write operations to show an example of writing to a closed socket The first write to the closed socket is to solicit RST from the server TCP The second write is to generate SIGPIPE from the local process. An sample run : linux% tcpcli Hi there # user input in bold Hi there # echoed back from server # terminate server child process then Bye # then type this line purposely Borken pipe # output by the shell because of SIGPIPE Note: To write to a socket which has received a FIN is OK. However, it is an error to write to a socket hat has received an RST 34

str_cli() – Calling writen() Twice tcpcliserv/str_cli11.c 35

Crash of Server Host Scenario 1. client and server run on different hosts 2. make a connection between client and server 3. client types something to transmit data to the server 4. disconnect the server host from the network (destination unreachable) 5. client types something again. client TCP continuously retx data and timeout around 9 min The client process will then return with the error ETIMEDOUT. If some intermediate router determined that the server host was down and responded with an ICMP “ destination unreachable ” message, the error returned will then be either EHOSTUNREACH or ENETUNREACH To quickly detect: timeout on readline, SO_KEEPALIVE socket option, heartbeat functions

Reboot of Server Host The client does not see the server host shut down Client sends data to server after the server reboots server TCP responds to client data with an RST because it loses all connection information readline returns ECONNRESET

Shutdown of Server Host (by Operator) init process sends SIGTERM to all processes We can catch this signal and close all open descriptors by ourselves init waits 5-20 sec and sends SIGKILL to all processes all open descriptors are closed by kernel

Summary of TCP Example From client ’ s perspective: socket and connect specifies server ’ s port and IP client port and IP chosen by TCP and IP respectively From server ’ s perspective: socket and bind specifies server ’ s local port and IP listen and accept return client ’ s port and IP

TCP Client/Server – Client ’ s Perspective 40

TCP Client/Server – Client ’ s Perspective 41

Data Format: Text Strings server process gets two numbers (in a line of text) from client and returns their sum In str_echo: sscanf converts string to long integer, snprintf converts long back to string

str_echo() – Adding 2 Numbers tcpcliserv/str_echo08.c 43

Data Format: Binary Structure Passing binary structure between client and server does not work when the client and server are run on hosts with different byte orders or sizes of long integer Since different implementations can store the same C datatype differently. Suggestions: pass in string only explicitly define the format of data types (e.g. RPC ’ s XDR -- external data representation)

str_cli() – Sending 2 Binary Int ’ s tcpcliserv/str_cli09.c 45

str_echo() – Adding 2 Binary Int ’ s tcpcliserv/str_echo09.c 46

Beware of Different Byte Orders Due to the big-endian and little-endian implementations, sending binary numbers between different machine architectures may end up with different results An example of two big-endian SPARC machines : solaris% tcpcli # user input in bold 33 # result back from server An example of big-endian SPARC and little-endian Intel machines : linus% tcpcli # user input in bold 3 # It seems to work # oops! It does not work! 47