1. Announcement Exercise Set 1 is released.  More problems will be added in next two weeks.  We will post a midterm sample this weekend or so (some problems already appear in the exercise) and a midterm study guide.

2. Lecture 5B: Advanced Synchronization Topics CS170. UCSB. 2015. T. Yang Some of slides are from J. Kubiatowicz (UCB cs162)

3. Topics Use of synchronization primitives  Design advices  Deadlock issue  Application in reader/writer problem Can we use variables to synchronize instead of locks/semaphore?  Too much milk problem Implementation of locks  Interrupts  Hardware primitives

4. Design advices for synchronization Allocate one lock for each shared variable or a group of shared variables Add condition variables (waking up a thread is triggered by some condition) Semaphore is not recommended by AD textbook (Chap. 5.8) and Chap 6.2 has more on design patterns State variables Synchronization variables Public methods Shared Objects

5. Granularity control for synchronization Fine-grain (protection around small data items) vs coarse grain (protection on a big group of items).  Fine-grain: –More performance flexibility and scalability –Possibly more design complexity and synchronization overhead  Coarse-grain –Easy to ensure correctness –Possibly difficult to scale

6. Deadlock and Starvation Deadlock – two or more threads (or processes) are waiting indefinitely for an event that can be only caused by one of these waiting processes Starvation – indefinite blocking.  E.g. A thread is in a waiting queue indefinitely for resources constantly in use by high-priority threads Deadlock  Starvation but not vice versa Res 2 Res 1 Thread B Thread A Wait For Wait For Owned By Owned By

7. Example of Deadlock Let S and Q be two locks: P 0 P 1 Acquire(S); Acquire(Q); Acquire (Q); Acquire (S);. Release (Q); Release(S); Release (S); Release(Q);

8. Deadlock Avoidance Order the locks and always acquire the locks in that order. Eliminate circular waiting P 0 P 1 Acquire(S); Acquire(S); Acquire(Q); Acquire (Q);. Release(Q); Release (Q); Release(S); Release (S);

9. Readers-Writers Problem A data set is shared among a number of concurrent processes.  Readers – only read the data set; never modify  Writers – can both read and write Requirement:  allow multiple readers to read at the same time.  Only one writer can access the shared data at the same time. Reader/writer access permission table: ReaderWriter Reader OKNo WriterNONo R R RW

10. Readers-Writers (First try with 1 lock) writer do { wrt.Acquire(); // wrt is a lock // writing is performed wrt.Release(); } while (TRUE); Reader do { wrt.Acquire(); // Use wrt lock // reading is performed wrt.Release(); } while (TRUE); ReaderWriter Reader?? Writer?? R R RW

11. Readers-Writers (First try with 1 lock) writer do { wrt.Acquire(); // wrt is a lock // writing is performed wrt.Release(); } while (TRUE); Reader do { wrt.Acquire(); // Use wrt lock // reading is performed wrt.Release(); } while (TRUE); ReaderWriter ReaderNO WriterNO R R RW

12. Pthread read-write locks Read/write locks enable multiple readers or one writer to lock a critical section –int pthread_rwlock_init(pthread_rwlock_t *restrict rwlock, const pthread_rwlockattr_t *restrict attr); –int pthread_rwlock_destroy(pthread_rwlock_t **rwlock); pthread_rwlock_rdlock(), pthread_rwlock_wrlock() pthread_rwlock_unlock() R R RW

13. pthread_rwlock_rdlock and pthread_rwlock_wrlock pthread_rwlock_rdlock()  Multiple readers can acquire the lock if no writer holds the lock.  “POSIX states that it is up to the implementation whether to allow a reader to acquire a lock if writers are blocked on the lock.”  pthread_rwlock_tryrdlock() returns if there is a writer. Pthread_rwlock_wrlock  The calling thread acquires the write lock if no other thread (reader or writer) holds the read-write lock rwlock. Otherwise, the thread blocks until it can acquire the lock.  Writers are favored over readers of the same priority to avoid writer starvation.  Pthread_rwlock_trywrlock. The function fails if any thread currently holds rwlock (for reading or writing).

14. RW Lock Implementation from AD Textbook Basic structure of a solution:  Reader() Wait until no writers Access data base Check out – wake up a waiting writer  Writer() Wait until no active readers or writers Access database Check out – wake up waiting readers or writer State variables (Protected by a lock called “lock”): –int AR: Number of active readers; initially = 0 –int WR: Number of waiting readers; initially = 0 –int AW: Number of active writers; initially = 0 –int WW: Number of waiting writers; initially = 0 –Condition okToRead –Conditioin okToWrite R R RW

15. Code for a Reader Reader() { // First check self into system startRead() lock.Acquire(); while ((AW + WW) > 0) {// Is it safe to read? WR++;// No. Writers exist okToRead.wait(&lock);// Sleep on cond var WR--;// No longer waiting } AR++;// Now we are active! lock.release(); // Perform actual read-only access AccessDatabase(ReadOnly); // Now, check out of system doneRead() lock.Acquire(); AR--;// No longer active if (AR == 0 && WW > 0)// No other active readers okToWrite.signal();// Wake up one writer lock.Release(); } Why Release the Lock here?

16. Writer() { // First check self into system. startWrite() lock.Acquire(); while ((AW + AR) > 0) {// Is it safe to write? WW++;// No. Active users exist okToWrite.wait(&lock);// Sleep on cond var WW--;// No longer waiting } AW++;// Now we are active! lock.release(); // Perform actual read/write access AccessDatabase(ReadWrite); // Now, check out of system. doneWrite() lock.Acquire(); AW--;// No longer active if (WW > 0){// Give priority to writers okToWrite.signal();// Wake up one writer } else if (WR > 0) {// Otherwise, wake reader okToRead.broadcast();// Wake all readers } lock.Release(); } Why Give priority to writers? Code for a Writer Why broadcast() here instead of signal()?

17. Time Simulation of Readers/Writers solution Read R1 Read R2 Write W1 Read R3

18. Time Simulation of Readers/Writers solution Read R1 Read R2 Write W1 Read R3 Read R1 Read R2 Write W1 Read R3

19. Simulation of Readers/Writers solution Consider the following sequence of operators:  R1, R2, W1, R3 On entry, each reader checks the following: while ((AW + WW) > 0) {// Is it safe to read? WR++;// No. Writers exist okToRead.wait(&lock);// Sleep on cond var WR--;// No longer waiting } AR++;// Now we are active! First, R1 comes along: AR = 1, WR = 0, AW = 0, WW = 0 Next, R2 comes along: AR = 2, WR = 0, AW = 0, WW = 0 Now, readers make take a while to access database  Situation: Locks released  Only AR is non-zero

20. Simulation(2) Next, W1 comes along: while ((AW + AR) > 0) {// Is it safe to write? WW++;// No. Active users exist okToWrite.wait(&lock);// Sleep on cond var WW--;// No longer waiting } AW++; Can’t start because of readers, so go to sleep: AR = 2, WR = 0, AW = 0, WW = 1 Finally, R3 comes along: AR = 2, WR = 1, AW = 0, WW = 1 Now, say that R2 finishes before R1: AR = 1, WR = 1, AW = 0, WW = 1 Finally, last of first two readers (R1) finishes and wakes up writer: if (AR == 0 && WW > 0)// No other active readers okToWrite.signal();// Wake up one writer

21. Simulation(3) When writer wakes up, get: AR = 0, WR = 1, AW = 1, WW = 0 Then, when writer finishes: if (WW > 0){ // Give priority to writers okToWrite.signal();// Wake up one writer } else if (WR > 0) {// Otherwise, wake reader okToRead.broadcast();// Wake all readers }  Writer wakes up reader, so get: AR = 1, WR = 0, AW = 0, WW = 0 When reader completes, we are finished

22. Questions Can readers starve? Consider Reader() entry code: while ((AW + WW) > 0) {// Is it safe to read? WR++;// No. Writers exist okToRead.wait(&lock);// Sleep on cond var WR--;// No longer waiting } AR++;// Now we are active! What if we erase the condition check in Reader exit? AR--;// No longer active if (AR == 0 && WW > 0)// No other active readers okToWrite.signal(); // Wake up one writer Further, what if we turn the signal() into broadcast() AR--;// No longer active okToWrite.broadcast(); // Wake up one writer Finally, what if we use only one condition variable (call it “ okToContinue ”) instead of two separate ones?  Both readers and writers sleep on this variable  Must use broadcast() instead of signal()

23. Impact of OS thread scheduling on condition wake up Example dequeue code: while (queue.isEmpty()) { dataready.wait(&lock); // If nothing, sleep } item = queue.dequeue();// Get next item  Why didn’t we do this? if (queue.isEmpty()) { dataready.wait(&lock); // If nothing, sleep } item = queue.dequeue();// Get next item Answer: depends on the type of scheduling  Hoare-style scheduling (most textbooks): –Signaler gives lock, CPU to waiter; waiter runs immediately –Waiter gives up lock, processor back to signaler when it exits critical section or if it waits again  Mesa-style scheduling (most real operating systems): –Signaler keeps lock and continues to process –Waiter placed on ready queue with no special priority –Practically, need to check condition again after wait

24. “Too much milk” Great thing about OS’s – analogy between problems in OS and problems in real life Example: People need to coordinate: Arrive home, put milk away3:30 Buy milk3:25 Arrive at storeArrive home, put milk away3:20 Leave for storeBuy milk3:15 Leave for store3:05 Look in Fridge. Out of milk3:00 Look in Fridge. Out of milkArrive at store3:10 Person BPerson ATime Problem from the AD textbook. Slides from J. Kubiatowicz (UCB cs162)

25. Do we have to use lock/condition variable/semaphore? For example: fix the milk problem by putting a lock on the refrigerator  Lock it and take key if you are going to go buy milk  Fixes too much: roommate angry if only wants OJ #$@%@ #$@

26. What do we need to consider? Need to be careful about correctness of concurrent programs, since non-deterministic What are the correctness properties for the “Too much milk” problem??? 1. Safety: Never more than one person buys 2. Liveness: Someone eventually buys when needed Restrict ourselves to use only atomic load and store operations as building blocks

27. Consider property of critical-section solutions during the design 1.Mutual Exclusion – Only one can enter the critical section. Safety property in AD book Safety: Never more than one person buys 2. Liveness: Someone eventually buys when needed -- Progress - If someone wishes to enter their critical section and nobody is in the critical section, then one of them will enter in a limited time --Bounded Waiting - If one starts to wait for entering an critical section, there is a limit on the number of times others entering the section before this thread enters. Unbounded/indefinite blocking  starvation

28. Use a note to avoid buying too much milk:  Leave a note before buying (kind of “lock”)  Remove note after buying (kind of “unlock”) Suppose a computer tries this (remember, only memory read/write are atomic): if (no Milk) { if (no Note) { leave Note; buy milk; remove note; } } Result?  Still too much milk but only occasionally!  Thread can get context switched after checking milk and note but before buying milk! Solution makes problem worse since fails intermittently  Makes it really hard to debug…  Must work despite what the dispatcher does! Too Much Milk: Solution #1

29. if (no Milk) { if (no Note) { leave Note; buy milk; remove note; } } Too Much Milk: Solution #1 if (no Milk) { if (no Note) { leave Note; buy milk; remove note; } } Time

30. if (no Milk) { if (no Note) { leave Note; buy milk; remove note; } } Too Much Milk: Solution #1 if (no Milk) { if (no Note) { leave Note; buy milk; remove note; } } Time

31. Too Much Milk Solution #3 Thread AThread B leave note A;leave note B; while (note B) { //X if (no Note A) { //Y do nothing; if (no Milk) { } buy milk; if (no Milk) { } buy milk;} }remove note B; remove note A; Does this work? Yes. Both can guarantee that:  It is safe to buy, or  Other will buy, ok to quit At point X:  if no note B, safe for A to buy,  otherwise wait to find out what will happen At point Y:  if no note A, safe for B to buy  Otherwise, A is either buying or waiting for B to quit

32. Solution #3 discussion Our solution protects a single “Critical-Section” piece of code for each thread: if (no Milk) { buy milk; } Solution #2 works, but it’s really unsatisfactory  Really complex – even for this simple an example –Hard to convince yourself that this really works  A’s code is different from B’s – what if lots of threads? –Code would have to be slightly different for each thread  While A is waiting, while-loop is consuming CPU time.  This is called “busy-waiting” or “spinning”

33. Where are we going with synchronization? We are going to implement various higher-level synchronization primitives using atomic operations  Everything is pretty painful if only atomic primitives are load and store  Need to provide primitives useful at user-level Load/Store Disable Ints Test&Set Comp&Swap Locks Semaphores Conditions Send/Receive Shared Programs Hardware Higher- level API Programs

34. How to implement Locks? Lock: prevents someone from doing something  Lock before entering critical section and before accessing shared data  Unlock when leaving, after accessing shared data  Wait if locked –Important idea: all synchronization involves waiting –Should sleep if waiting for a long time Atomic Load/Store: get solution like Milk #2  Pretty complex and error prone Alternatively  Interrupt disabling/enabling  Hardware primitives

35. Can this thread hold the CPU so others do not cause a trouble? No context switching. How? Recall: dispatcher gets control in two ways.  Internal: Thread does something to relinquish the CPU  External: Interrupts cause dispatcher to take CPU away Solution: avoid context-switching by:  Preventing external events by disabling interrupts  Avoiding internal events (although virtual memory tricky) How to make sure another thread does nothing affecting this thread?

36. Consequently, naïve Implementation of locks: LockAcquire { disable Ints; } LockRelease { enable Ints; } Problems with this approach:  Can’t let untrusted user do this! Consider LockAcquire(); While(TRUE) {;}  Real-Time system—no guarantees on timing! –Critical Sections might be arbitrarily long  Unresponsiveness in handling I/O or other important events? Naïve use of Interrupt Enable/Disable

37. Better Implementation of Locks by Disabling Interrupts Key idea: maintain a lock variable and impose mutual exclusion only during operations on that variable int value = FREE; Acquire() { disable interrupts; if (value == BUSY) { put thread on wait queue; Go to sleep(); // Enable interrupts? } else { value = BUSY; } enable interrupts; } Release() { disable interrupts; if (anyone on wait queue) { take thread off wait queue Place on ready queue; } else { value = FREE; } enable interrupts; } What is the advantage compared to the previous solution?

38. New Lock Implementation: Discussion Why do we need to disable interrupts at all?  Avoid interruption between checking and setting lock value  Otherwise two threads could think that they both have lock Busy-waiting is minimized: unlike previous solution, the critical section (inside Acquire() ) is very short Who turns interrupts on after sleep()? Acquire() { disable interrupts; if (value == BUSY) { put thread on wait queue; Go to sleep(); // Enable interrupts? } else { value = BUSY; } enable interrupts; } Critical Section

39. How to Re-enable After Sleep()? In scheduler, since interrupts are disabled when you call sleep:  Responsibility of the next thread to re-enable interrupts  When the sleeping thread wakes up, returns to acquire and re-enables interrupts Thread AThread B.. disable ints sleep sleep return enable ints...... disable int sleep sleep return enable ints.. context switch

40. Alternative Solutions with Atomic Hardware instructions Problems with previous solution:  Doesn’t work/scale well on multiprocessor –Disabling interrupts on all processors requires messages and would be very time consuming Alternative: atomic instruction sequences  These instructions read a value from memory and write a new value atomically  Hardware is responsible for implementing this correctly –on both uniprocessors (not too hard) –and multiprocessors (requires help from cache coherence protocol)  Unlike disabling interrupts, can be used on both uniprocessors and multiprocessors

41. Examples of Atomic Instructions with Read- Modify-Write testAndSet (&address) { /* most architectures */ result = M[address]; M[address] = 1; return result; } swap (&address, register) { /* x86 */ temp = M[address]; M[address] = register; register = temp; } compare&swap (&address, reg1, reg2) { /* 68000 */ if (reg1 == M[address]) { M[address] = reg2; return success; } else { return failure; } } load-linked&store conditional(&address) { /* R4000, alpha */ loop: ll r1, M[address]; movi r2, 1; /* Can do arbitrary comp */ sc r2, M[address]; beqz r2, loop; }

42. Atomic TestAndSet Instruction Definition: Fetch the value of a shared variable and then set it to 1 (or TRUE) atomically. boolean TestAndSet (*target) { rv = *target; *target = TRUE; return rv: } Target: False True Target: True True Case 1: Case 2:

43. Spin Lock Solution using TestAndSet Shared boolean variable: lock.  “True” means somebody has acquired the lock.  initialized to FALSE. Solution: while ( TestAndSet (&lock )) ; // Acquire Critical section; lock = FALSE; //Release lock Mutual exclusion? Progress? Bounded waiting? Lock: True True Lock: False True Case 1: lock is not used Case 2: lock is used

44. Thread P0: while(TestAndSet (&lock)); critical section lock = FALSE; } Spin Lock using TestAndSet Thread P1: while(TestAndSet (&lock)); critical section lock = FALSE; Property: Initially Lock=F Lock=T  somebody is in the critical section Lock=F  nobody is in the critical section. Thread P2: while(TestAndSet (&lock)); critical section lock = FALSE;

45. Thread P0: while(TestAndSet (&lock)); critical section lock = FALSE; } Spin Lock Thread P1: while(TestAndSet (&lock)); critical section lock = FALSE; Property: Initially Lock=F Lock=T  somebody is in the critical section Lock=F  nobody is in the critical section. Thread P2: while(TestAndSet (&lock)); critical section lock = FALSE; T F

46. Thread P0: while(TestAndSet (&lock)); critical section lock = FALSE; } Spin Lock Thread P1: while(TestAndSet (&lock)); critical section lock = FALSE; Property: Initially Lock=F Lock=T  somebody is in the critical section Lock=F  nobody is in the critical section. Thread P2: while(TestAndSet (&lock)); critical section lock = FALSE; T TT

47. Thread P0: while(TestAndSet (&lock)); critical section lock = FALSE; } Spin Lock Thread P1: while(TestAndSet (&lock)); critical section lock = FALSE; Property: Initially Lock=F Lock=T  somebody is in the critical section Lock=F  nobody is in the critical section. Thread P2: while(TestAndSet (&lock)); critical section lock = FALSE; T T T

48. Thread P0: while(TestAndSet (&lock)); critical section lock = FALSE; } Mutual exclusion? Assume P0 enters first, and then P1 is also in. Thread P1: while(TestAndSet (&lock)); critical section lock = FALSE; Property: Lock=T  somebody is in the critical section Lock=F  nobody is in the critical section. Conflict, both cannot be true C1: lock was F in last TestAndSet(). TestAndSet() returns F Now lock is T. C2: lock was F in last TestAndSet(). TestAndSet() returns F

49. Thread P0: TestAndSet (&lock)); //get in … Lock=FALSE; //get out TestAndSet(&lock);// get in … Lock=FALSE; //get out TestAndSet(&lock);// get in … Lock=FALSE; //get out TestAndSet(&lock);// get in Bounded Waiting? Thread P1: TestAndSet(&lock);// wait Time

50. Atomic Swap Instruction Definition: exchange values of two variables atomically. void Swap (boolean *a, boolean *b) { boolean temp = *a; *a = *b; *b = temp: }

51. Spin Lock Solution using Swap Shared Boolean variable lock initialized to FALSE; Each process has a local Boolean variable key Solution: key = TRUE; //Acquire while ( key == TRUE) //Acquire Swap (&lock, &key ); Critical section lock = FALSE; //Release Mutual exclusion, progress. bounded waiting? Lock: False Key: True

52. Summary: Busy-waiting Spin lock Positives for this solution  Machine can receive interrupts  Works on a multiprocessor Negatives  Inefficient spin lock –Busy-waiting thread will consume cycles waiting –Waiting thread may take cycles away from thread holding lock (no one wins!)  No guarantee on bounded waiting –Priority Inversion: If busy-waiting thread has higher priority than thread holding lock  no progress!

53. Better Locks with test&set and sleep Can we build test&set locks without busy-waiting?  Can’t entirely, but can minimize!  Idea: only busy-wait to atomically check lock value Note: sleep has to be sure to reset the guard variable  Why can’t we do it just before or just after the sleep? Release() { // Short busy-wait time while (test&set(guard)); if anyone on wait queue { take thread off wait queue Place on ready queue; } else { value = FREE; } guard = false; int guard = false; int value = FREE; Acquire() { // Short busy-wait time while (test&set(guard)); if (value == BUSY) { put thread on wait queue; sleep()& guard=false; } else { value = BUSY; guard = false; } }

54. Summary Concurrent threads are a very useful abstraction  Allow transparent overlapping of computation and I/O  Allow use of parallel processing when available  Scheduling to determine which threads to run next Synchronization of shared variable access with lock, condition variables, & semaphores  Semaphores: –P(): Wait if zero; decrement when becomes non-zero –V(): Increment and wake a sleeping task (if exists)  Use separate semaphore for each constraint Monitors: A lock plus one or more condition variables  Always acquire lock before accessing shared data  Use condition variables to wait inside critical section –Three Operations: Wait(), Signal(), and Broadcast() Important concept: Atomic Operations  An operation that runs to completion or not at all  Used for constructing various synchronization primitives