1. CONCURRENT PROGRAMMING Lecture 5

2. 2 Assignment 1  Having trouble?  Resolve it as early as possible!  Assignment 2:  Handling text-based protocol  HTTP Proxy  Assignment 3: Concurrency  Assignment 4: Distribute file system

3. 3 Last lecture  TCP  Socket()  Question: difference between send() and write()?  MSG_DONTWAIT, MSG_NOSIGNAL

4. 4 Concurrent Programming is Hard!  The human mind tends to be sequential  The notion of time is often misleading  Thinking about all possible sequences of events in a computer system is at least error prone and frequen tly impossible

5. 5 Concurrent Programming is Hard!  Classical problem classes of concurrent programs:  Races: outcome depends on arbitrary scheduling decision s elsewhere in the system Example: who gets the last seat on the airplane?  Deadlock: improper resource allocation prevents forward progress Example: traffic gridlock  Livelock / Starvation / Fairness: external events and/or sy stem scheduling decisions can prevent sub-task progress Example: people always jump in front of you in line

6. 6 Client / Server Session Iterative Echo Server ClientServer socket bind listen rio_readlineb rio_writenrio_readlineb rio_writen Connection request rio_readlineb close EOF Await connection request from next client open_listenfd open_clientfd acceptconnect

7. 7 Iterative Servers  Iterative servers process one request at a time client 1serverclient 2 connect accept connect write read call read close accept write read close Wait for Client 1 call read write ret read write ret read

8. 8 Where Does Second Client Block?  Second client attempts to conne ct to iterative server  Call to connect returns  Even though connection not yet accepted  Server side TCP manager que ues request  Feature known as “TCP listen backlog”  Call to rio_writen returns  Server side TCP manager buff ers input data  Call to rio_readlineb blocks  Server hasn’t written anything for it to read yet. Client socket rio_readlineb rio_writen Connection request open_clientfd connect

9. 9 Fundamental Flaw of Iterative Servers  Solution: use concurrent servers instead  Concurrent servers use multiple concurrent flows to serve multip le clients at the same time User goes out to lunch Client 1 blocks waiting for user to type in data Client 2 blocks waiting to read from server Server blocks waiting for data from Client 1 client 1serverclient 2 connect accept connect write read call read write call read write ret read

10. 10 Creating Concurrent Flows  Allow server to handle multiple clients simultaneously  1. Processes  Kernel automatically interleaves multiple logical flows  Each flow has its own private address space  2. Threads  Kernel automatically interleaves multiple logical flows  Each flow shares the same address space  3. I/O multiplexing with select()  Programmer manually interleaves multiple logical flows  All flows share the same address space  Relies on lower-level system abstractions

11. 11 Concurrent Servers: Multiple Processes  Spawn separate process for each client client 1serverclient 2 call connect call accept call read ret connect ret accept call connect call fgets fork child 1 User goes out to lunch Client 1 blocks waiting for user to type in data call accept ret connect ret accept call fgets writefork call read child 2 write call read end read close...

12. 12 Review: Iterative Echo Server  Accept a connection request  Handle echo requests until client terminates int main(int argc, char **argv) { int listenfd, connfd; int port = atoi(argv[1]); struct sockaddr_in clientaddr; int clientlen = sizeof(clientaddr); listenfd = Open_listenfd(port); while (1) { connfd = Accept(listenfd, (SA *)&clientaddr, &clientlen); echo(connfd); Close(connfd); } exit(0); }

13. 13 int main(int argc, char **argv) { int listenfd, connfd; int port = atoi(argv[1]); struct sockaddr_in clientaddr; int clientlen=sizeof(clientaddr); Signal(SIGCHLD, sigchld_handler); listenfd = Open_listenfd(port); while (1) { connfd = Accept(listenfd, (SA *) &clientaddr, &clientlen); if (Fork() == 0) { Close(listenfd); /* Child closes its listening socket */ echo(connfd); /* Child services client */ Close(connfd); /* Child closes connection with client */ exit(0); /* Child exits */ } Close(connfd); /* Parent closes connected socket (important!) */ } Process-Based Concurrent Server Fork separate process for each client Does not allow any communication between different client handlers

14. 14 Process-Based Concurrent Server (cont)  Reap all zombie children void sigchld_handler(int sig) { while (waitpid(-1, 0, WNOHANG) > 0) ; return; }

15. 15 Process Execution Model  Each client handled by independent process  No shared state between them  Both parent & child have copies of listenfd and connfd Parent must close connfd Child must close listenfd Server Process (Client 1) Server Process (Client 2) Server Process (Listening) Connection Requests Client 1 dataClient 2 data

16. 16 Concurrent Server: accept Illustrated listenfd(3) Client 1. Server blocks in accept, waiting for connection request on listening descriptor listenfd clientfd Server listenfd(3) Client clientfd Server 2. Client makes connection request by calling and blocking in connect Connection request listenfd(3) Client clientfd Server 3. Server returns connfd from accept. Forks child to handle client. Client returns from connect. Connection is now established between clientfd and connfd Server Child connfd(4)

17. 17 Implementation Must-dos With Process-Based Designs  Listening server process must reap zombie children  to avoid fatal memory leak  Listening server process must close its copy of connfd  Kernel keeps reference for each socket/open file  After fork, refcnt(connfd) = 2  Connection will not be closed until refcnt(connfd) == 0

18. 18 View from Server’s TCP Manager Client 1ServerClient 2 cl1>./echoclient greatwhite.ics.cs.cmu.edu 15213 srv>./echoserverp 15213 srv> connected to (128.2.192.34), port 50437 cl2>./echoclient greatwhite.ics.cs.cmu.edu 15213 srv> connected to (128.2.205.225), port 41656 ConnectionHostPortHostPort Listening --- 128.2.220.1015213 cl1128.2.192.3450437128.2.220.1015213 cl2128.2.205.22541656128.2.220.1015213

19. 19 Pros and Cons of Process-Based Designs  + Handle multiple connections concurrently  + Clean sharing model  file tables (yes)  global variables (no)  + Simple and straightforward  – Additional overhead for process control  – Nontrivial to share data between processes  Requires IPC (interprocess communication) mechanisms  FIFO’s (named pipes), System V shared memory and semaphores

20. 20 #2) Event-Based Concurrent Servers Using I/O Multiplexi ng  Server maintains set of active connections  Array of connfd’s  Repeat:  Determine which connections have pending inputs  If listenfd has input, then accept connection Add new connfd to array  Service all connfd’s with pending inputs  Library support  Use library functions to construct scheduler within single process

21. 21 Adding Concurency: Step One  Start with allowing address re-use  Then we set the socket to non-blocking int sock, opts; sock = socket(…); // getting the current options setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &opts, sizeof(opts)); int sock, opts; sock = socket(…); // getting the current options setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &opts, sizeof(opts)); // getting current options if (0 > (opts = fcntl(sock, F_GETFL))) printf(“Error…\n”); // modifying and applying opts = (opts | O_NONBLOCK); if (fcntl(sock, F_SETFL, opts)) printf(“Error…\n”); bind(…); // getting current options if (0 > (opts = fcntl(sock, F_GETFL))) printf(“Error…\n”); // modifying and applying opts = (opts | O_NONBLOCK); if (fcntl(sock, F_SETFL, opts)) printf(“Error…\n”); bind(…);

22. 22 Adding Concurrency: Step Two  Monitor sockets with select()  int select(int maxfd, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, const struct timespec *timeout);  So what’s an fd_set ?  Bit vector with FD_SETSIZE bits  maxfd – Max file descriptor + 1  readfs – Bit vector of read descriptors to monitor  writefds – Bit vector of write descriptors to monitor  exceptfds – Read the manpage, set to NULL  timeout – How long to wait with no activity before retur ning, NULL for eternity

23. 23 How does the code change? if (listen(sockfd, 5) < 0) { // listen for incoming connections printf(“Error listening\n”); init_pool(sockfd, &pool); while (1) { pool.ready_set = &pool.read_set; pool.nready = select(pool.maxfd+1, &pool.ready_set, &pool.write_set, NULL, NULL); if (FD_ISSET(sockfd, &pool.ready_set)) { if (0 > (isock = accept(sockfd,…))) printf(“Error accepting\n”); add_client(isock, &caddr, &pool); } check_clients(&pool); } // close it up down here if (listen(sockfd, 5) < 0) { // listen for incoming connections printf(“Error listening\n”); init_pool(sockfd, &pool); while (1) { pool.ready_set = &pool.read_set; pool.nready = select(pool.maxfd+1, &pool.ready_set, &pool.write_set, NULL, NULL); if (FD_ISSET(sockfd, &pool.ready_set)) { if (0 > (isock = accept(sockfd,…))) printf(“Error accepting\n”); add_client(isock, &caddr, &pool); } check_clients(&pool); } // close it up down here

24. 24 What was pool?  User defined structure used to store information.  A struct something like this: typedef struct s_pool { int maxfd; // largest descriptor in sets fd_set read_set; // all active read descriptors fd_set write_set; // all active write descriptors fd_set ready_set;// descriptors ready for reading int nready;// return of select() int maxi; /* highwater index into client array */ int clientfd[FD_SETSIZE];/* set of active descriptors */ rio_t clientrio[FD_SETSIZE]; /* set of active read buffers */ } pool; typedef struct s_pool { int maxfd; // largest descriptor in sets fd_set read_set; // all active read descriptors fd_set write_set; // all active write descriptors fd_set ready_set;// descriptors ready for reading int nready;// return of select() int maxi; /* highwater index into client array */ int clientfd[FD_SETSIZE];/* set of active descriptors */ rio_t clientrio[FD_SETSIZE]; /* set of active read buffers */ } pool;

25. 25 void init_pool(int sockfd, pool * p); { int i; p->maxfd = -1; for (i=0;i<FD_SETSIZE;i++) p->clientfd[i]=-1; p->maxfd = sockfd; FD_ZERO(&p->read_set); FD_SET(listenfd, &p->read_set); } // close it up down here void init_pool(int sockfd, pool * p); { int i; p->maxfd = -1; for (i=0;i<FD_SETSIZE;i++) p->clientfd[i]=-1; p->maxfd = sockfd; FD_ZERO(&p->read_set); FD_SET(listenfd, &p->read_set); } // close it up down here

26. 26 void add_client(int connfd, pool *p) { int i; p->nready--; for (i = 0; i < FD_SETSIZE; i++) /* Find an available slot */ if (p->clientfd[i] < 0) { /* Add connected descriptor to the pool */ p->clientfd[i] = connfd; Rio_readinitb(&p->clientrio[i], connfd); /* Add the descriptor to descriptor set */ FD_SET(connfd, &p->read_set); /* Update max descriptor and pool highwater mark */ if (connfd > p->maxfd) p->maxfd = connfd; if (i > p->maxi) p->maxi = i; break; } if (i == FD_SETSIZE) /* Couldn't find an empty slot */ app_error("add_client error: Too many clients"); } // close it up down here void add_client(int connfd, pool *p) { int i; p->nready--; for (i = 0; i < FD_SETSIZE; i++) /* Find an available slot */ if (p->clientfd[i] < 0) { /* Add connected descriptor to the pool */ p->clientfd[i] = connfd; Rio_readinitb(&p->clientrio[i], connfd); /* Add the descriptor to descriptor set */ FD_SET(connfd, &p->read_set); /* Update max descriptor and pool highwater mark */ if (connfd > p->maxfd) p->maxfd = connfd; if (i > p->maxi) p->maxi = i; break; } if (i == FD_SETSIZE) /* Couldn't find an empty slot */ app_error("add_client error: Too many clients"); } // close it up down here

27. 27 void check_clients(pool *p) { int i, connfd, n; char buf[MAXLINE]; rio_t rio; for (i = 0; (i maxi) && (p->nready > 0); i++) { connfd = p->clientfd[i]; rio = p->clientrio[i]; /* If the descriptor is ready, echo a text line from it */ if ((connfd > 0) && (FD_ISSET(connfd, &p->ready_set))) { p->nready--; if ((n = Rio_readlineb(&rio, buf, MAXLINE)) != 0) { byte_cnt += n; //line:conc:echoservers:beginecho printf("Server received %d (%d total) bytes on fd %d\n", n, byte_cnt, connfd); Rio_writen(connfd, buf, n); //line:conc:echoservers:endecho } /* EOF detected, remove descriptor from pool */ else { Close(connfd); //line:conc:echoservers:closeconnfd FD_CLR(connfd, &p->read_set); //line:conc:echoservers:beginremove p->clientfd[i] = -1; //line:conc:echoservers:endremove } void check_clients(pool *p) { int i, connfd, n; char buf[MAXLINE]; rio_t rio; for (i = 0; (i maxi) && (p->nready > 0); i++) { connfd = p->clientfd[i]; rio = p->clientrio[i]; /* If the descriptor is ready, echo a text line from it */ if ((connfd > 0) && (FD_ISSET(connfd, &p->ready_set))) { p->nready--; if ((n = Rio_readlineb(&rio, buf, MAXLINE)) != 0) { byte_cnt += n; //line:conc:echoservers:beginecho printf("Server received %d (%d total) bytes on fd %d\n", n, byte_cnt, connfd); Rio_writen(connfd, buf, n); //line:conc:echoservers:endecho } /* EOF detected, remove descriptor from pool */ else { Close(connfd); //line:conc:echoservers:closeconnfd FD_CLR(connfd, &p->read_set); //line:conc:echoservers:beginremove p->clientfd[i] = -1; //line:conc:echoservers:endremove }

28. 28 So what about bit vectors?  void FD_ZERO(fd_set *fdset);  Clears all the bits  void FD_SET(int fd, fd_set *fdset);  Sets the bit for fd  void FD_CLR(int fd, fd_set *fdset);  Clears the bit for fd  int FD_ISSET(int fd, fd_set *fdset);  Checks whether fd’s bit is set

29. 29 What about checking clients?  The code only tests for new incoming connections  But we have many more to test!  Store all your client file descriptors  In pool is a good idea!  Several scenarios  Clients are sending us data  We may have pending data to write in a buffer  Keep the while(1) thin  Delegate specifics to functions that access the appropriate d ata  Keep it orthogonal!

30. 30 Back to that lifecycle… Server Client(s) socket() connect() write( ) read() close() socket() bind() listen() select() write() read() close() read() EOF Connection Request Client / Server Session(s) FD_ISSET(sfd) accept() check_clients() main loop

31. 31 I/O Multiplexed Event Processing 10 clientfd 7 4 12 5 0 1 2 3 4 5 6 7 8 9 Active Inactive Active Never Used listenfd = 3 10 clientfd 7 4 12 5 listenfd = 3 Active Descriptors Pending Inputs Read

32. 32 Time out? int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout);

33. 33 Pros and Cons of I/O Multiplexing  + One logical control flow.  + Can single-step with a debugger.  + No process or thread control overhead.  Design of choice for high-performance Web servers and search engines.  – Significantly more complex to code than process- o r thread-based designs.  – Hard to provide fine-grained concurrency  E.g., our example will hang up with partial lines.  – Cannot take advantage of multi-core  Single thread of control