1. CS 3214 Computer Systems Godmar Back Lecture 22

2. Announcements Project 4 due Nov 10 Exercise 9 due Nov 11 CS 3214 Fall 2010

3. MULTI-THREADING CS 3214 Fall 2010

4. Coordinating Multiple Threads Aside from coordinating access to shared items, threads may need to communicate about events –“has event A already happened in another thread?” –aka “precedence constraint”, or “scheduling constraint” Do B after A Must do so –Correctly (never miss that event A has occurred when in fact it has) –Efficiently Don’t waste resources in the process Don’t unnecessarily delay notification of event A CS 3214 Fall 2010

5. Q.: How can thread2 make sure that ‘coin_flip’ has occurred before printing its outcome? CS 3214 Fall 2010 int main() { int i, N = 2; pthread_t t[N]; srand(getpid()); pthread_create(&t[1], NULL, thread2, NULL); pthread_create(&t[0], NULL, thread1, NULL); for (i = 0; i < N; i++) pthread_join(t[i], NULL); return 0; } int main() { int i, N = 2; pthread_t t[N]; srand(getpid()); pthread_create(&t[1], NULL, thread2, NULL); pthread_create(&t[0], NULL, thread1, NULL); for (i = 0; i < N; i++) pthread_join(t[i], NULL); return 0; } int coin_flip; static void * thread1(void *_) { coin_flip = rand() % 2; printf("Thread 1: flipped coin %d\n", coin_flip); return NULL; } static void * thread2(void *_) { printf("Thread 2: flipped coin %d\n", coin_flip); return NULL; } int coin_flip; static void * thread1(void *_) { coin_flip = rand() % 2; printf("Thread 1: flipped coin %d\n", coin_flip); return NULL; } static void * thread2(void *_) { printf("Thread 2: flipped coin %d\n", coin_flip); return NULL; }

6. Thread 2 could “busy- wait” – spin until thread 1 completes the coin flip. Exceptions not withstanding, this is practically never an acceptable solution. CS 3214 Fall 2010 static void * thread2(void *_) { /* Thread 2 spins, "busy-waits" until the coin flip is done. * This is an unacceptable solution. Bad for the planet, too. */ while (!coin_flip_done) continue; printf("Thread 2: flipped coin %d\n", coin_flip); return NULL; } static void * thread2(void *_) { /* Thread 2 spins, "busy-waits" until the coin flip is done. * This is an unacceptable solution. Bad for the planet, too. */ while (!coin_flip_done) continue; printf("Thread 2: flipped coin %d\n", coin_flip); return NULL; } int coin_flip; volatile bool coin_flip_done; static void * thread1(void *_) { coin_flip = rand() % 2; coin_flip_done = true; printf("Thread 1: flipped coin %d\n", coin_flip); return NULL; } int coin_flip; volatile bool coin_flip_done; static void * thread1(void *_) { coin_flip = rand() % 2; coin_flip_done = true; printf("Thread 1: flipped coin %d\n", coin_flip); return NULL; } The somewhat less wasteful variant of busy-waiting: while (!coin_flip_done) sched_yield(); is not acceptable, either. -wastes CPU cycles - is fragile (volatile needed when using –O) - does not document semantics

7. CS 3214 Fall 2010 Semaphores Invented by Edsger Dijkstra in 1960s Counter S, initialized to some value, with two operations: –P(S) or “down” or “wait” – if counter greater than zero, decrement. Else wait until greater than zero, then decrement –V(S) or “up” or “signal” or “post” – increment counter, wake up any threads stuck in P. Semaphores don’t go negative: –#V + InitialValue - #P >= 0 Note: direct access to counter value after initialization is not allowed Counting Semaphores vs Binary Semaphores –Binary: counter can only be 0 or 1 Simple to implement, yet powerful –Can be used for many synchronization problems Source: inter.scoutnet.org

8. CS 3214 Fall 2010 static void * thread2(void *_) { // wait until semaphore is raised, // then decrement, 'down' sem_wait(&coin_flip_done); printf("Thread 2: flipped coin %d\n", coin_flip); } static void * thread2(void *_) { // wait until semaphore is raised, // then decrement, 'down' sem_wait(&coin_flip_done); printf("Thread 2: flipped coin %d\n", coin_flip); } int coin_flip; sem_t coin_flip_done; // semaphore for thread 1 to signal coin flip static void * thread1(void *_) { coin_flip = rand() % 2; sem_post(&coin_flip_done); // raise semaphore, increment, 'up' printf("Thread 1: flipped coin %d\n", coin_flip); } int coin_flip; sem_t coin_flip_done; // semaphore for thread 1 to signal coin flip static void * thread1(void *_) { coin_flip = rand() % 2; sem_post(&coin_flip_done); // raise semaphore, increment, 'up' printf("Thread 1: flipped coin %d\n", coin_flip); } int main() { … sem_init(&coin_flip_done, 0, 0); pthread_create(&t[1], NULL, thread2, NULL); pthread_create(&t[0], NULL, thread1, NULL); … } int main() { … sem_init(&coin_flip_done, 0, 0); pthread_create(&t[1], NULL, thread2, NULL); pthread_create(&t[0], NULL, thread1, NULL); … } Notice the 3 rd argument of sem_init() – it gives the initial value of the semaphore: ‘0’ means the semaphore is used to express scheduling constraint POSIX Semaphores

9. CS 3214 Fall 2010 Implementing Mutual Exclusion with Semaphores Semaphores can be used to build locks Must initialize semaphore with 1 to allow one thread to enter critical section This is not a recommended style, despite of what Bryant & O’Hallaron suggest – you should use a mutex instead [Cantrill & Bonwick 2008]Cantrill & Bonwick 2008 Easily generalized to allow at most N simultaneous threads: multiplex pattern (i.e., a resource can be accessed by at most N threads) sem_t S; sem_init(&S, 0, 1); lock_acquire() { // try to decrement, wait if 0 sem_wait (S); } lock_release() { // increment (wake up waiters if any) sem_post(S); } sem_t S; sem_init(&S, 0, 1); lock_acquire() { // try to decrement, wait if 0 sem_wait (S); } lock_release() { // increment (wake up waiters if any) sem_post(S); }

10. Condition Variables - Intro Besides (and perhaps more so) than semaphores, condition variables are another widely used form to implement ‘signaling’ kinds of coordination/synchronization –In POSIX Threads, Java, C# Based on the concept of a Monitor –ADT that combines protected access to state and signaling Confusing terminology alert: –Word ‘signal’ is overloaded 3 times Semaphore signal (V(), “up”, “post”) Monitor/Condition variable signal (“signal”, “notify”) Unix signals –Word ‘wait’ is overloaded Semaphore wait (P(), “down”) Monitor/Condition variable wait Unix wait() for child process CS 3214 Fall 2010

11. Monitors A monitor combines a set of shared variables & operations to access them –Think of a Java class with no public fields & all public methods carrying the attribute ‘synchronized’ A monitor provides implicit synchronization (only one thread can access private variables simultaneously) –Single lock is used to ensure all code associated with monitor is within critical section A monitor provides a general signaling facility –Wait/Signal pattern (similar to, but different from semaphores) –May declare & maintain multiple signaling queues

12. CS 3214 Fall 2010 Monitors (cont’d) Classic monitors are embedded in programming languages –Invented by Hoare & Brinch-Hansen 1972/73 –First used in Mesa/Cedar System @ Xerox PARC 1978 –Adapted version available in Java/C# (Classic) Monitors are safer than semaphores –can’t forget to lock data – compiler checks this In contemporary C, monitors are a synchronization pattern that is achieved using locks & condition variables –Helps to understand monitor abstraction to use it correctly

13. CS 3214 Fall 2010 Infinite Buffer w/ Monitor monitor buffer { /* implied: struct lock mlock;*/ private: char buffer[]; int head, tail; public: produce(item); item consume(); } monitor buffer { /* implied: struct lock mlock;*/ private: char buffer[]; int head, tail; public: produce(item); item consume(); } buffer::produce(item i) { /* try { lock_acquire(&mlock); */ buffer[head++] = i; /* } finally {lock_release(&mlock);} */ } buffer::consume() { /* try { lock_acquire(&mlock); */ return buffer[tail++]; /* } finally {lock_release(&mlock);} */ } buffer::produce(item i) { /* try { lock_acquire(&mlock); */ buffer[head++] = i; /* } finally {lock_release(&mlock);} */ } buffer::consume() { /* try { lock_acquire(&mlock); */ return buffer[tail++]; /* } finally {lock_release(&mlock);} */ } Monitors provide implicit protection for their internal variables –Still need to add the signaling part

14. Condition Variables Used by a monitor for signaling a condition –a general (programmer-defined) condition, not just integer increment as with semaphores –Somewhat weird: the condition is actually not stored in the variable – it’s typically some boolean predicate of monitor variables, e.g. “buffer.size > 0” –the condition variable itself is better thought of as a signaling queue Monitor can have more than one condition variable Three operations: –Wait(): leave monitor, wait for condition to be signaled, reenter monitor –Signal(): signal one thread waiting on condition –Broadcast(): signal all threads waiting on condition CS 3214 Fall 2010

15. Condition Variables as Queues A condition variable’s state is just a queue of waiters: –Wait(): adds current thread to (end of queue) & block –Signal(): pick one thread from queue & unblock it –Broadcast(): unblock all threads Region of mutual exclusion Enter Exit Wait Signal Wait Signal Note on style: best practice is to leave monitor only once, and near the procedure’s entry.

16. CS 3214 Fall 2010 Bounded Buffer w/ Monitor monitor buffer { condition items_avail; condition slots_avail; private: char buffer[]; int head, tail; public: produce(item); item consume(); } monitor buffer { condition items_avail; condition slots_avail; private: char buffer[]; int head, tail; public: produce(item); item consume(); } buffer::produce(item i) { while ((tail+1–head)%CAPACITY==0) slots_avail.wait(); buffer[head++] = i; items_avail.signal(); } buffer::consume() { while (head == tail) items_avail.wait(); item i = buffer[tail++]; slots_avail.signal(); return i; } buffer::produce(item i) { while ((tail+1–head)%CAPACITY==0) slots_avail.wait(); buffer[head++] = i; items_avail.signal(); } buffer::consume() { while (head == tail) items_avail.wait(); item i = buffer[tail++]; slots_avail.signal(); return i; }

17. CS 3214 Fall 2010 Bounded Buffer w/ Monitor monitor buffer { condition items_avail; condition slots_avail; private: char buffer[]; int head, tail; public: produce(item); item consume(); } monitor buffer { condition items_avail; condition slots_avail; private: char buffer[]; int head, tail; public: produce(item); item consume(); } buffer::produce(item i) { while ((tail+1–head)%CAPACITY==0) slots_avail.wait(); buffer[head++] = i; items_avail.signal(); } buffer::consume() { while (head == tail) items_avail.wait(); item i = buffer[tail++]; slots_avail.signal(); return i; } buffer::produce(item i) { while ((tail+1–head)%CAPACITY==0) slots_avail.wait(); buffer[head++] = i; items_avail.signal(); } buffer::consume() { while (head == tail) items_avail.wait(); item i = buffer[tail++]; slots_avail.signal(); return i; } Q1.: How is lost update problem avoided? Q2.: Why while() and not if()? lock_release(&mlock); block_on(items_avail); lock_acquire(&mlock);

18. cond_signal semantics cond_signal keeps lock, so it leaves signaling thread in monitor waiter is made READY, but can’t enter until signaler gives up lock There is no guarantee whether signaled thread will enter monitor next or some other thread will (who may be already waiting!) –so must always use “while()” when checking condition – cannot assume that condition set by signaling thread will still hold when monitor is reentered by signaled thread This semantics is also referred to as “Mesa-Style” after the system in which it was first used –POSIX Threads, Java, and C# use this semantics CS 3214 Fall 2010

19. Condition Variables vs. Semaphores Condition Variables –Signals are lost if nobody’s on the queue (e.g., nothing happens) –Wait() always blocks Semaphores –Signals (calls to V() or sem_post()) are remembered even if nobody’s current waiting –Wait (e.g., P() or sem_wait()) may or may not block

20. Monitors in C POSIX Threads as well as many custom environments No compiler support, must do it manually –must declare locks & condition vars –must call pthread_mutex_lock/unlock when entering & leaving the monitor –must use pthread_cond_wait/pthread_cond_signal to wait for/signal condition Note: pthread_cond_wait(&c, &m) takes monitor lock as parameter –necessary so monitor can be left & reentered without losing signals pthread_cond_signal() does not –leaving room for programmer error! CS 3214 Fall 2010

21. Monitors in Java synchronized block means –enter monitor –execute block –leave monitor wait()/notify() use condition variable associated with receiver –Every object in Java can function as a condition variable (just like it can function as a lock) –More restrictive than Pthreads/C which allow multiple condition variables (signaling conditions) to be used in connection with a lock protecting state class buffer { private char buffer[]; private int head, tail; public synchronized produce(item i) { while (buffer_full()) this.wait(); buffer[head++] = i; this.notifyAll(); } public synchronized item consume() { while (buffer_empty()) this.wait(); i = buffer[tail++]; this.notifyAll(); return ; } class buffer { private char buffer[]; private int head, tail; public synchronized produce(item i) { while (buffer_full()) this.wait(); buffer[head++] = i; this.notifyAll(); } public synchronized item consume() { while (buffer_empty()) this.wait(); i = buffer[tail++]; this.notifyAll(); return ; }

22. CS 3214 Fall 2010 Monitors in Java, Take 2 Previous slide (bounded buffer) is actually an example of where Java’s built-in monitors suck –Needed “notifyAll()” to make sure one at least one of the right kind of threads was woken up –Unacceptably inefficient Use java.util.concurrent.- locks.Condition instead in cases where multiple condition queues are needed import java.util.concurrent.locks.*; class buffer { private ReentrantLock monitorlock = new ReentrantLock(); private Condition items_available = monitorlock.newCondition(); private Condition slots_available = monitorlock.newCondition(); public /* NO SYNCHRONIZED here */ void produce(item i) { monitorlock.lock(); try { while (buffer_full()) slots_available.await(); buffer[head++] = i; items_available.signal(); } finally { monitorlock.unlock(); } } /* consume analogous */ } import java.util.concurrent.locks.*; class buffer { private ReentrantLock monitorlock = new ReentrantLock(); private Condition items_available = monitorlock.newCondition(); private Condition slots_available = monitorlock.newCondition(); public /* NO SYNCHRONIZED here */ void produce(item i) { monitorlock.lock(); try { while (buffer_full()) slots_available.await(); buffer[head++] = i; items_available.signal(); } finally { monitorlock.unlock(); } } /* consume analogous */ }

23. CS 3214 Fall 2010 A ReadWrite Lock Implementation struct lock mlock; // protects rdrs & wrtrs int readers = 0, writers = 0; struct condvar canread, canwrite; void read_lock_acquire() { lock_acquire(&mlock); while (writers > 0) cond_wait(&canread, &mlock); readers++; lock_release(&mlock); } void read_lock_release() { lock_acquire(&mlock); if (--readers == 0) cond_signal(&canwrite); lock_release(&mlock); } struct lock mlock; // protects rdrs & wrtrs int readers = 0, writers = 0; struct condvar canread, canwrite; void read_lock_acquire() { lock_acquire(&mlock); while (writers > 0) cond_wait(&canread, &mlock); readers++; lock_release(&mlock); } void read_lock_release() { lock_acquire(&mlock); if (--readers == 0) cond_signal(&canwrite); lock_release(&mlock); } void write_lock_acquire() { lock_acquire(&mlock); while (readers > 0 || writers > 0) cond_wait(&canwrite, &mlock); writers++; lock_release(&mlock); } void write_lock_release() { lock_acquire(&mlock); writers--; ASSERT(writers == 0); cond_broadcast(&canread); cond_signal(&canwrite); lock_release(&mlock); } void write_lock_acquire() { lock_acquire(&mlock); while (readers > 0 || writers > 0) cond_wait(&canwrite, &mlock); writers++; lock_release(&mlock); } void write_lock_release() { lock_acquire(&mlock); writers--; ASSERT(writers == 0); cond_broadcast(&canread); cond_signal(&canwrite); lock_release(&mlock); } Note: this is a naïve implementation that may lead to livelock – no guarantees a reader or writer ever enters the locked section even if every threads eventually leaves it

24. CS 3214 Fall 2010 Summary Semaphores & Condition Variables provide signaling facilities –Condition variables are loosely based on “monitor” concept Java/C# provide syntactic sugar Semaphores have “memory” –But require that # of signals matches # of waits –Good for rendezvous, precedence constraints – if problem lends itself to semaphore, use one Always use idiomatic “while (!cond) *_wait()” pattern when using condition variables (in C) or Object.wait() (in Java)