1. ECE 297 Concurrent Servers Process, fork & threads ECE 297

2. Cache How do you handle cache updates? How do you handle cache invalidation? Keep it simple Process-based server

3. ECE 297 file How do you handle concurrent access to files? Careful with writing to the same file in different processes! Process-based server

4. ECE 297 Process versus thread I Process Unit of resource ownership with respect to the execution of a single program Can encompass more than one thread of execution –E.g., Web browser: More than one thread (process) per window/tab, GUI, rendering engine etc. –E.g., Web server: More than one thread for handling requests Thread Unit of execution Belongs to a process Can be traced (i.e., list the sequence of instructions)

5. ECE 297 Process versus thread II A.k.a. lightweight process (LWP), threads, multi- threaded processes

6. ECE 297 Process versus thread III Per process items Address space Global variables Open files Child processes Pending alarms Signal and signal handlers Accounting information Per thread items Program counter Registers Stack

7. ECE 297 Use Processes are largely independent and often compete for resources Use Threads are part of the same “job” and are actively and closely cooperating OS Threads Process 1Process 2 Process 3 Process

8. ECE 297 Threads OS Threads Thread 1’s stack Process

9. Thread-based server Server design alternatives –Thread-per-request –Thread-per-client –Thread-per connection The new thread can access all resources held by the process that created it For example, the cache, open data files, global variables are all available to the threads –Unlike for process-based servers

10. ECE 297 pthreads API overview pthread_create(…): creates a thread pthread_wait(…): waits for a specific thread to exit pthread_exit(…): terminates the calling thread pthread_yield(…): calling thread passes control voluntarily to another thread p is for POSIX

11. Thread priority, initial stack size, …; NULL for defaults Pointer to argument for function ECE 297 pthreads API I #include pthread_create(pthread_t *tid, const pthread_attr_t *attr, void *(*func) (void *), void *arg); Returns 0, if OK, positive Exx on error p is for POSIX Thread ID Function to execute; the actual “thread”

12. ECE 297 pthreads API II pthread_join(pthread_t *tid, void **status) Returns 0, if OK, positive Exx on error p is for POSIX Caller waits for the given thread to terminate If not NULL, a pointer to the return value is stored in this value

13. ECE 297 pthreads API III pthread_exit(void *status) One way for a thread to terminate, others: –The function associated with the thread terminates –Any thread in the process calls exit(…) If a thread is not detached, its exit status and thread ID are retained for a later pthread_join –By default threads are joinable status must not point to an object local to the thread, since this object disappears when the thread terminates p is for POSIX

14. ECE 297 pthreads API IV pthread_self(void) –Returns thread ID to caller pthread_detach(pthread_t thread) –Indicates to system that storage for thread can be reclaimed There are many other pthread API calls, the above should suffice for our purposes p is for POSIX

15. ECE 297 Thread-based server void *thread(void *vargp); int *connfdp; int main(int argc, char **argv) { … pthread_t tid; … listenfd = socket(…); … listen(listenfd, …) // main server loop for( ; ; ) { connfdp = malloc(sizeof(int)); … *connfdp = accept(listenfd, (struct sockaddr *) &clientaddr, &clientlen); pthread_create(&tid, NULL, thread, (void *) connfdp); } // for } // main We create the thread to handle the connected client.

16. ECE 297 The actual thread to handle the client void *thread(void *vargp) { int connfd; // detached to avoid a memory leak pthread_detach(pthread_self()); connfd = *((int *)vargp); free(vargp); // do the work, service the client close(connfd); return NULL; } This is where the client gets serviced

17. ECE 297 listenfd = socket(AF_INET, SOCK_STREAM, 0) … bind(listenfd, …) listen(listenfd, …) for( ; ; ){ … connfd = accept(listenfd, …); … if ( (childPID = fork()) == 0 ){// The Child! close(listenfd); //Close listening socket do the work //Process the request exit(0); } … close(connfd); //Parent closes connfd } Concurrent server template

18. ECE 297 Issues with thread-based servers Must be careful to avoid unintended sharing of variables For example, what happens if we pass the address of connfd to the thread routine? pthread_create(&tid, NULL, thread, (void *)&connfd); Must protect access to intentionally shared data –Here, we got around this by creating a new variable, but in general … Would be a shared variable

19. Complications Imaging a global variable counter in the process –For example the storage server in-memory cache (more complex structure) –Or the connfd variable Let’s dissect the issue in detail !

20. Shared data & synchronization ECE 297 Table What happens if multiple threads concurrently access shared process state (i.e., memory)?

21. Concurrently manipulating shared data Two threads execute concurrently as part of the same process Shared variable (e.g., global variable) –counter = 5 Thread 1 executes –counter++ Thread 2 executes –counter— What are the possible values of counter after Thread 1 and Thread 2 executed? ECE 297 counter

22. ECE 297 Machine-level implementation Implementation of “counter++” register 1 = counter register 1 = register 1 + 1 counter= register 1 Implementation of “counter--” register 2 = counter register 2 = register 2 – 1 counter= register 2

23. ECE 297 Possible execution sequences counter++ counter-- Context Switch counter++ counter-- Context Switch

24. ECE 297 Interleaved execution Assume counter is 5 and interleaved execution of counter++ (P) and counter– (C) T 1 : r 1 =counter(register 1 = 5) T 1 : r 1 = r 1 + 1(register 1 = 6) T 2 : r 2 =counter(register 2 = 5) T 2 : r 2 = r 2 – 1(register 2 = 4) T 1 : counter= r 1 (counter = 6) T 2 : counter= r 2 (counter = 4) The value of counter may be either 4 or 6, where the correct result should be 5. context switch

25. ECE 297 Race condition Race condition: –Several threads manipulate shared data concurrently. The final value of the data depends upon which thread finishes last. In our example (interleaved execution) for c++ last, result would be 6, and for c-- last, result would be 4 (correct result should be 5) To prevent race conditions, concurrent processes must be synchronized.

26. ECE297 The moral of this story The statements counter++; counter--; must each be executed atomically. Atomic operation means an operation that completes in its entirety without interruption. This is achieved through synchronization primitives (semaphores, locks, condition variables, monitors, disabling of IRPs …).

27. Synchronization primitives Semaphore (cf. ECE344) Monitor (cf. ECE344) Condition variable (cf. ECE344) Lock –Prevent data inconsistencies due to race conditions –A.k.a. mutex (mutual exclusion) –Use to protect shared data within a process –Can not be used across processes Need to use semaphore instead ECE 297

28. Mutex: Mutual exclusion pthread_mutex_lock(pthread_mutex_t *mtpr) pthread_mutex_unlock(pthread_mutex_t *mtpr) Returns 0, if OK, positive Exx on error There are other abstractions, but the mutex should suffice for us NB: In ECE344 we learn how to implement locks. ECE 297

29. The pthreads mutex (lock) pthread_mutex_t my_cnt_lock = PTHREAD_MUTEX_INITIALIZER; int counter=0; pthread_mutex_lock( & my_cnt_lock ); counter++; pthread_mutex_unlock( & my_cnt_lock ); … ECE 297

30. Mutex is for mutual exclusion ECE 297 For statically allocated mutexes. pthread_mutex_lock(& my_cnt_lock); counter++; pthread_mutex_unlock(& my_cnt_lock); pthread_mutex_t my_cnt_lock = PTHREAD_MUTEX_INITIALIZER Guaranteed to execute atomically pthread_mutex_lock(& my_cnt_lock); counter--; pthread_mutex_unlock(& my_cnt_lock); Guaranteed to execute atomically

31. ECE 297 Possible execution sequences counter++ counter-- Context Switch lock unlock lock unlock counter++ lock unlock counter-- lock unlock lock

32. Watch out for I For all shared data access you must use a synchronization mechanism For Milestone 4 based on threads, you can get by with the mutexes Other useful mechanisms in pthreads are –pthread_join(…) –pthread_cond_wait(…) & pthread_cond_signal() Bugs due to race conditions are extremely difficult to track down –Non-deterministic behaviour of code ECE 297

33. Watch out for II You can not make any assumption about thread execution order or relative speed Threaded code must use thread-safe functions –Functions that use no static variables, no global variables, don’t return pointers to static variables Otherwise need to protect call to non-thread-safe code with mutexes Non-thread-safe code also called non-reentrant code –Function local data is allocated on the stack Deadlocks –Code halts, as threads may wait indefinitely on locks –Cause is programmer error or poorly written code ECE 297

34. Pros & cons of threads-based servers Probably the simplest option –No zombies, no signal handling, no onerous data structures “Easy” to share data structures between threads –Logging information, data files, cache, … Thread creation is more efficient than process creation Enables concurrent processing of requests from multiple clients

35. ECE 297 Pros & cons cont.’d Unintentional sharing can introduce subtle and hard to reproduce race conditions malloc an argument (struct) for each thread and pass pointer to variable to thread and free after use Keep global variables to a minimum If a thread references a global variable protect it with a mutex or think carefully about whether unprotected variable is safe –e.g., one writer thread vs. multiple readers is OK.