cs423-cotter1 Concurrency Issues in Client/Server Applications Chapters 15,16, 28

cs423-cotter2 Server Concurrency Control Must choose between Iterative or Concurrent –How many clients may query server? –How often will clients Query? –What mode of query will be used? –How long will service take?

cs423-cotter3 Server Concurrency Control Must choose between Iterative or Concurrent –How many clients may query server? –How often will clients Query? –What mode of query will be used? –How long will service take? If concurrent, must choose the level of concurrency –Apparent or real concurrency –Variable number of client threads –Fixed Maximum number of threads

cs423-cotter4 Server Concurrency Models Thread on demand –One thread per session –One thread per request

cs423-cotter5 Server Concurrency Models Thread on demand –One thread per session –One thread per request Thread Pool –Create thread pool –Allocate threads to queries as needed. –When query has been served, return thread to pool –When all threads busy, queue or discard other queries

cs423-cotter6 Socket Pre-allocation Techniques Associate a thread (or a thread pool) with each different service For TCP, associate each thread with the service socket, such that when a query arrives, one (and only one) thread will have accept unblocked, will create a new socket for this query, handle the query, and then return to accept. For UDP, each request is a single datagram, which gets assigned to a single slave thread.

cs423-cotter7 Concurrent Connection-oriented Master slave Port

cs423-cotter8 Concurrent Connectionless Master slave Port

cs423-cotter9 Delayed Concurrency in Servers Concurrency helpful when the cost of a new process (thread) is less than service time for the request. May vary by request. (Some database queries are short and others are long).

cs423-cotter10 Delayed Concurrency in Servers Concurrency helpful when the cost of a new process (thread) is less than service time for the request. May vary by request. (Some database queries are short and others are long). Start service iteratively and set timer. If timer expires, create new thread to finish service. Start new threads based on service requested (where multiple services are available).

cs423-cotter11 Concurrency in Clients Why? –Easier to program, because concurrency forces modularity –Easier to maintain, also because of modularity –Can interact with several servers at once –Allows the user to interact with server, even when a call is blocked.

cs423-cotter12 Concurrency in Clients Example Database query - “residents on Elm Street” Response may be short (in Lone Jack).... or very long (in KCMO)

cs423-cotter13 Concurrency in Clients Example Database query - “residents on Elm Street” Response may be short (in Lone Jack).... or very long (in KCMO) How does the user know when to abort a “failed” query? Might query server for status... Might terminate current query (if server is too slow) and restart on another server...

cs423-cotter14 Functional Concurrency in Clients Use multiple threads to perform different actions –One thread manages input (keyboard) –One thread manages input to socket –One thread manages output from socket –One thread manages control interaction with server –etc.

cs423-cotter15 Concurrency in Clients Multiple Servers Allows multiple accesses at the same time... so that performance measures (round trip delay, lost packets, etc.) are all taken against the same network environment. Faster than iterative model

cs423-cotter16 Concurrency in Clients Issues Concurrency Model: May use real or apparent concurrency (single threaded or multi-threaded). OS Support for concurrency: Some systems allow sharing memory between threads, while others may not. Functional Decomposition: What operations might be done in parallel, and which need to be done sequentially?

cs423-cotter17 Concurrency in Clients Singly threaded Allows memory sharing between connections May (or may not) overload, depending on rate of data exchange with multiple connections May deadlock!!

cs423-cotter18 Concurrent Client (TCPtecho) #include,,, #defineBUFSIZE4096/* write buffer size*/ #defineCCOUNT64*1024/* default character count*/ #defineWSVERSMAKEWORD(2, 0) #defineMIN(x, y)((x)>(y) ? (y) : (x)) #defineUSAGE"usage: TCPtecho [ -c count ] host1 host2...\n” struct hdat { char*hd_name;/* host name */ SOCKET hd_sock;/* host socket descriptor*/ unsigned hd_rc;/* recv character count */ unsigned hd_wc;/* send character count */ } hdat[FD_SETSIZE];/* fd to host name mapping*/ charbuf[BUFSIZE];/* read/write data buffer*/ voidTCPtecho(fd_set *, int); intreader(struct hdat *, fd_set *); voidwriter(struct hdat *, fd_set *); longmstime(u_long *);

cs423-cotter19 Concurrent Client (TCPtecho) void main(int argc, char *argv[]) { intccount = CCOUNT, i, hcount = 0, fd; unsigned longone = 1; fd_setafds; WSADATAwsdata; if (WSAStartup(WSVERS, &wsdata)) errexit("WSAStartup failed\n"); FD_ZERO(&afds); for (i=1; i<argc; ++i) { if (strcmp(argv[i], "-c") == 0) { if (++i < argc && (ccount = atoi(argv[i]))) continue; errexit(USAGE); }/* else, a host */

cs423-cotter20 Concurrent Client (TCPtecho) fd = connectTCP(argv[i], "echo"); if (ioctlsocket(fd, FIONBIO, &one)) { fprintf(stderr,"can't mark nonblocking (host %s): %d\n", argv[i], GetLastError()); continue; } hdat[hcount].hd_name = argv[i]; hdat[hcount].hd_sock = fd; hdat[hcount].hd_rc = hdat[hcount].hd_wc = ccount; ++hcount; FD_SET(fd, &afds); } TCPtecho(&afds, hcount); WSACleanup(); exit(0);}

cs423-cotter21 Concurrent Client (TCPtecho) void TCPtecho(fd_set *pafds, int hcount) { fd_setrfds, wfds;/* read/write fd sets*/ fd_setrcfds, wcfds;/* read/write fd sets (copy)*/ intfd, hndx, i; for (i=0; i<BUFSIZE; ++i)/* echo data*/ buf[i] = 'D'; memcpy(&rcfds, pafds, sizeof(rcfds)); memcpy(&wcfds, pafds, sizeof(wcfds)); (void) mstime((u_long *)0);/* set the epoch */

cs423-cotter22 Concurrent Client (TCPtecho) while (hcount) { memcpy(&rfds, &rcfds, sizeof(rfds)); memcpy(&wfds, &wcfds, sizeof(wfds)); if (select(FD_SETSIZE, &rfds, &wfds, (fd_set *)0, (struct timeval *)0) == SOCKET_ERROR) errexit("select failed: error %d\n",GetLastError()); for (hndx=0; hndx<hcount; ++hndx) { fd = hdat[hndx].hd_sock; if (FD_ISSET(fd, &rfds)) if (reader(&hdat[hndx], &rcfds) == 0){/* host done*/ for (i=hndx+1; i<hcount; ++i) hdat[i-1]=hdat[i]; hcount--; continue; } if (FD_ISSET(fd, &wfds)) writer(&hdat[hndx], &wcfds); }}}

cs423-cotter23 Concurrent Client (TCPtecho) intreader(struct hdat *phd, fd_set *pfdset) {... } voidwriter(struct hdat *phd, fd_set *pfdset) {... } longmstime(u_long *pms) {... }

cs423-cotter24 Deadlock in Client/Server Systems A situation where computation cannot proceed because a set or 2 or more components in the system is blocked and each component is waiting on another component in the set. Cannot be released through an external input. Often difficult to detect or prove, because it may depend on a particular, unusual sequence of events.

cs423-cotter25 Deadlock Between Client & Server May use request / response model to avoid deadlock. Must ensure that protocol design is “tight”. Remove ambiguities in the design. Assign responsibility for communication synchronization. (One side always starts)

cs423-cotter26 Starvation in Client/Servers Occurs when some clients cannot obtain service because other clients or processes are monopolizing the server resources. Results in unfair allocation of services

cs423-cotter27 Starvation in Client/Servers Occurs when some clients cannot obtain service because other clients or processes are monopolizing the server resources. Results in unfair allocation of services Delays waiting for a client to transmit (blocking on receive) can be managed through timeouts. For busy / malicious clients, servers multithreading or non-blocking can be used.

cs423-cotter28 Livelock in Clients / Servers Like deadlock, livelock results from circular dependency among processes and resources. System is locked into an endless self-generating sequence of commands that ultimately consumes all system resources.

cs423-cotter29 Livelock in Clients / Servers Like deadlock, livelock results from circular dependency among processes and resources. System is locked into an endless self- generating sequence of commands that ultimately consumes all system resources. Example: –File Server with time stamp logging. –Livelock occurs if a remote call to time is used –First file request comes in. Time stamp requires time (which generates an incoming message to file server). Incoming time response looks like a file request, which requires a new time stamp.....

cs423-cotter30 Planning for Concurrency Consider what system resources should be consumed by the server Consider what the demand for the service(s) is Provide limits (threads, processes, memory) that are appropriate. Incorporate logging into the server to track program problems, congestion, etc. Consider “livelock” problems when working with multiple services.