1. SPL/2010 Guarded Methods and Waiting 1

2. SPL/2010 Reminder! ● Concurrency problem: asynchronous modifications to object states lead to failure of thread computation (pre-condition is not met) 2

3. SPL/2010 Overview ● Techniques to deal with potential failures ● check all invariants and preconditions before performing action ● refuse to perform actions unless they can ensure that these actions will succeed ● exceptions, exception handling code 3

4. SPL/2010 4

5. Policies for failed preconditions/invariants ● Balking. throw exception if precondition fails. try, and if you cannot succeed, fail fast ● Guarded suspension. suspend method invocation (and thread) until precondition becomes true. ● Time-outs. between balking and suspension. Bounded wait for precondition to become true. 5

6. SPL/2010 Today ● guarded suspension ● general suspension mechanisms in Java RTE 6

7. SPL/2010 Guarded suspension and time-outs in concurrent software ● abstract queue supports add() get() size() 7

8. SPL/2010 Guards = special conditionals ● Conditionals in sequential programs: ● if statement check condition on method entry ● if false - no wait - can never become true 8

9. SPL/2010 Queue impl. Balking 9

10. SPL/2010 Guarded suspension ● asynchronous state changes can happen ● precondition may hold in the future ● wait until precondition holds ● guard asserts that another thread will make required state changes… ● Time-outs - soft guards - balking policy if wait for too long 10

11. SPL/2010 11

12. SPL/2010 Java Mechanics for Guarded Suspension ● Wait/Notify constructs - simple, mechanism for suspension in multi-threaded context 1. mechanism semantics 2. higher level constructs to simplify usability 12

13. SPL/2010 wait() ● threads may wait() until some other thread wakes them up ● each object has a wait set - similar to locks ● entities possessing locks and wait sets are called monitors ● maintained internally by the JVM ● set holds threads blocked by wait on object ● until notifications are invoked or waits released ● threads enter the queue of an object, o, by invoking the wait() method of o. A thread wishing to wakeup threads from the queue of o may call o.notify() (or o.notifyAll()) to wakeup just one thread (resp. all of the threads) waiting in the wait set of o. 13

14. SPL/2010 notify() ● threads enter wait queue of object o by invoking the wait() of o. ● thread wakeups threads from queue of o by: o.notify()/o.notifyAll() 14

15. SPL/2010 wait/notify cycle ● o.wait() - wait invocation ● current thread calling wait() is blocked. ● JVM places thread in wait set of o. ● o.notify() - notify invocation ● arbitrarily thread T is removed from o’s wait set ● T is resumed from the point of wait(). ● o.notifyAll() ● like notify except for all threads in o’s wait set 15

16. SPL/2010 Interrupting Threads ● Special JVM exception to threads: ● thread may need to stop waiting for a notify() (example when program needs to exit) ● InterruptedException – wait() method exits immediately 16

17. SPL/2010 Handle InterruptedException 17 ?

18. SPL/2010 Back to queue ● Producer-Consumer design pattern ● Producer produces objects /Consumer needs process ● decouple processes that produce and consume data at different rates ● queue data in producer loop / process in consumer loop at own pace 18

19. SPL/2010 19

20. SPL/2010 ● video player example: ● Producer read the file from disk and give chunks of data to Consumer – add() ● Consumer decodes and play – take() ● Guard add() / take() methods, such that all preconditions are respected – wait() until the preconditions hold. 20

21. SPL/2010 Producer-Consumer Queue 21

22. SPL/2010 22

23. SPL/2010 ● thread trying to get() when queue is empty will wait in wait set of queue ● thread which successfully add()s an object to queue will wake him up (and vice versa) ● What is wrong? ● No synchronization 23

24. SPL/2010 What is wrong? Guard atomicity ● two Consumer threads and one producer The ● queue is empty at first ● consumers execute take() - block (size=0) ● producer calls add() once, notifyAll() ● both consumers wake up. both execute vec_.remove(0) ● precondition does not hold for one of them 24

25. SPL/2010 What is wrong? Wait atomicity ● one producer and one consumer ● first runs the consumer, calls take() ● consumer checks condition, and is stopped right after (scheduler) ● producer calls add() successfully – calls notifyAll() - no thread in wait set at that time ● consumer is resumed and calls this.wait() ● queue not empty-consumer miss notification. 25

26. SPL/2010 Guard Atomicity - while loop 26

27. SPL/2010 Solution? ● while loop ensures after waking up, checks precondition one more time ● no synchronization ● use notify() instead of notifyAll() ? dangerous... 27

28. SPL/2010 Wait Atomicity ● ensure no one calls notify() between precondition check and wait. ● make both check and wait atomic with relation to notify mechanism. ● use single lock to protect them (?) 28

29. SPL/2010 Wait Atomicity ● thread get hold of a lock ● check ● wait() - enter a wait set ● no other thread may get lock and wake us up! 29

30. SPL/2010 Wait Atomicity ● Java solution: ● call to wait() or notifyAll() - calling thread must hold object's monitor (lock). ● enter wait() - thread (involuntarily) releases lock ● before exit wait() - thread must relock object (why?) 30

31. SPL/2010 31

32. SPL/2010 Wait/Notify Mechanism ● thread T calls o.wait() ● JVM checks that T holds o's lock (if not, exception IllegalMonitorState). ● T is blocked, and lock of o is released. ● JVM places T in the wait set associated with o. 32

33. SPL/2010 Wait/Notify Mechanism ● thread T calls o.notify() ● JVM checks that T holds o's lock. if not exception is thrown. ● arbitrarily thread T' is removed from wait set ● T' tries to re-acquire the lock of o (T' blocks until it does). ● T' is resumed from the point of its wait(). 33

34. SPL/2010 Wait/Notify Mechanism ● o.notifyAll() - steps occur simultaneously for all threads in wait set of o. ● Thundering Herd phenomena – all threads try to acquire the lock competing with each other. ● holding the monitor's lock check is done at runtime and not at compile time. 34

35. SPL/2010 35

36. SPL/2010 36

37. SPL/2010 Rules of Thumb for Wait and Notify ● Guarded wait - for each condition that needs to be waited on, write a guarded wait loop that causes the current thread to block if the guard condition is false. 37

38. SPL/2010 Rules of Thumb for Wait and Notify ● Guard atomicity - Condition checks must always be placed in while loops. ● When action is resumed, waiting thread doesn't know if condition is true ● to maintain safety, it must check again. ● thread can wake up without notify, interrupt, or timing out - spurious wakeup. 38

39. SPL/2010 Rules of Thumb for Wait and Notify ● Multiple guard atomicity If there are several conditions which need to be waited for, place them all in the same loop. 39

40. SPL/2010 Rules of Thumb for Wait and Notify ● Don't forget to wake up - ensure liveness, classes must wake up waiting threads. ● when value of field in guard changes - waiting threads must be awakened to recheck conditions 40

41. SPL/2010 Rules of Thumb for Wait and Notify ● notifyAll() - multiple threads wait on same wait set: ● at least two threads waiting for different conditions, ● ensure that no thread misses an event. 41

42. SPL/2010 notify() Vs. notifyAll(): ● queue which has add() / addTwo() methods ● Thread t1 waits for queue in add() checking room for one object, ● thread t2 waits in addTwo() checking room for two objects. ● get() method calls notify() and thread t2 wakes up ● room for one object in queue, t2 goes back to sleep. ● t1 will not be woken up at all, even though there is space in the queue. 42

43. SPL/2010 Sophisticated Primitives ● Wait and Notify: ● basis for higher level constructs. ● general constructs ● Specific mechanisms make code easier to understand and demand less details when using them: ● Semaphores ● Readers-Writers locks. 43

44. SPL/2010 Semaphores ● Object controls bounded number of permits (permit = train ticket): ● Threads ask semaphore for permit (a ticket). ● If semaphore has permits available, one permit is assigned to requesting thread. ● If no permit available, requesting thread is blocked until permit is available – when a thread returns permit received earlier 44

45. SPL/2010 implementation 45

46. SPL/2010 Java Semaphore class properties ● not re-entrant – thread calls acquire() twice, must release() twice! ● semaphore can be released by a thread other than owner (unlike lock) – no ownership. ● constructor accepts fairness parameter – if t1 requested before t2 it will receive first ● services as tryAcquire and managing permits. 46

47. SPL/2010 SimpleQueue vs. Semaphore ● SQ: add(),remove() call notifyAll() call. ● Sem: release() calls notifyAll() method acquire() does not. ● SQ:add(), remove() call wait() ● Sem: method acquire() calls wait() method release() does not. 47

48. SPL/2010 SimpleQueue vs. Semaphore ● SQ: add(),remove() - non-empty preconditions correspond to 2 distinct states of the Queue: empty or full. ● Sem: acquire() has pre-condition and release() has no pre-condition. ● release() does not need to call wait(). ● acquire() call wait() - symmetric call in release() for notifyAll() (for each wait() - corresponding notifyAll() that unblocks). 48

49. SPL/2010 Readers/Writers Example ● allow readers and writers to access a shared resource under different policies ● Policy: ● several readers can access resource together ● only one writer can access resource at given time ● Example: – Readers wait if writer is pending – One writer at a time can access the shared resource, if no readers are reading from it 49

50. SPL/2010 50

51. SPL/2010 51