1. Concurrency and Threads
Acknowledgement: Some slides adapted from a slide deck by Robbie Hott

2. Spot the buzzword!
“A simple, object-oriented, distributed, interpreted, robust, secure, architecture neutral, portable, high-performance, multithreaded, and dynamic language.” [Sun95]

3. Concurrent Programming
On a computer that can only do one instruction at a time, why would we want to program pretending it can do many things at once?
Concurrency: having several computations interleaved or executing simultaneously, potentially interacting with each other

4. Threading Concept Multiple Threads of execution at once
One set of shared data!
Web
Server
Process
Listen();
Thread();
Web
Server
Process
Listen();
Respond(); // …
Request
Request
Respond(); //
Respond(); //

5. Concurrent Programming
Why?
Some problems are clearer to program concurrently
Modularity: Don’t have to explicitly interleave code for different abstractions (especially: user interfaces)
Modeling: Closer map to real world problems: things in the real world aren’t sequential

6. The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software,
Herb Sutter

7. Where is most of the processing power on your PC?
nVIDIA GeForce GTX 470
448 Cores
nVIDIA QuadroPlex 7000 DHIC
1792 cores

8. Simple Example: Counter (in Java)
One Counter object with two instance methods, increment and decrement.
Two Threads, one calls increment 1000 times, the other calls decrement 1000 times.
After each call, they sleep for a random time.
What do you think will be the value of the counter when they both finish?

9. Challenge of Concurrency
Thread 1
DataStore
Thread 2
Thread 3
Coordinated access to
shared data!
Instruction Streams

10. Example: Scheduling Meetings
Alice wants to schedule a meeting with Bob and Colleen
Bob
“When can you meet Friday?”
Alice
“When can you meet Friday?”
Colleen
“11am or 3pm”
“9am or 11am”
Picks meeting time
“Let’s meet at 11am”
Reserves 11am
for meeting

11. Partial Ordering of Events
Sequential programs give use a total ordering of events: everything happens in a determined order
Concurrency gives us a partial ordering of events: we know some things happen before other things, but not total order
Alice asks to schedule meeting before Bob replies
Alice asks to schedule meeting before Colleen replies
Bob and Colleen both reply before Alice picks meeting time
Alice picks meeting time before Bob reserves time on calendar

12. Race Condition Bob Alice Colleen Doug “When can you meet Friday?”
“9, 11am or 3pm”
“9am or 11am”
“9, 11am or 3pm”
Picks meeting time
Reserves 11am
for Doug
“Let’s meet at 11am”
“Let’s meet at 11am”
“Let’s meet at 11am”
“I’m busy then…”

13. Preventing Race Conditions
Use locks to impose ordering constraints
After responding to Alice, Bob reserves all the times in his response until he hears back (and then frees the other times)

14. Locking Bob Alice Colleen Doug “When can you meet Friday?”
“9, 11am or 3pm”
“9am or 11am”
Picks meeting time
Locks calendar
“Let’s meet at 11am”
“3pm”
“Let’s meet at 11am”
“Let’s meet at 3”

15. Deadlocks Doug Bob Alice Colleen Locks calendar for Doug, can’t
“When can you meet Friday?”
Doug
Bob
Alice
Colleen
Locks calendar
for Doug, can’t
respond to Alice
“When can you meet Friday?”
“When can you meet Friday?”
“When can you meet Friday?”
“9, 11am or 3pm”
Can’t schedule
meeting, no
response from
Bob
Locks calendar
for Alice, can’t
respond to Doug
Can’t schedule
meeting, no
response from
Colleen

16. Deadlocks
Deadlock: when computation has stalled because execution units are blocked and waiting on a circular dependency chain.
For example, when two threads wait for the other’s response to finish.
Therefore, neither does.
Other examples?
“When two trains approach each other at a crossing, both shall come to a full stop and neither shall start up again until the other has gone.” —statute passed by the Kansas Legislature (wikipedia)

17. Why are threads hard? Too few ordering constraints: race conditions
Too many ordering constraints: deadlocks,
poor performance, livelocks, starvation
Hard/impossible to reason about modularly
If an object is accessible to multiple threads, need to think about what any of those threads could do at any time!
Testing is even more impossible than it is for sequential code
Even if you test all the inputs, don’t know it will work if threads run in different order 17

18. Challenge of Concurrency
Server
Listener
Request 1
DataStore
Request 2
Request 3
Coordinated access to
shared data!
Instruction Streams

19. Thread safety Concept Why is this important?
The representation of an ADT should be valid (when observed by other entities) even when used concurrently by multiple threads
Why is this important?
ADTs designed so each method preserves representation invariants.
Remember ADT implementation correctness proof
Invariants hold on method entry => they hold on exit
Correctness proof for each methods assumes invariant holds on entry; (and implicitly assumes exclusive access to representation)
Problems will occur: IF invariants fail in the middle of execution of a method, AND concurrent execution of another method call can observe an inconsistent state

20. Potential Solutions Immutable objects No shared store
Functional programming
Scheme without set!, set-car!, set-cdr!
Immutable objects
Use locking discipline
Require thread interleavings to happen in a predictable way

21. Q: Where do I have concurrency problems
public class RGB {
//Values must be between 0 and 255.
private int red; private int green; private int blue;
private String name;
public RGB(int red, int green, int blue, String name) {
check(red, green, blue);
this.red = red; this.green = green; this.blue = blue; this.name = name; }
public void set(int red, int green, int blue, String name) {
this.red = red; this.green = green; this.blue = blue; this.name = name;
public int getRGB() {
return ((red << 16) | (green << 8) | blue);
public void invert() {
red = red; green = green; blue = blue;
name = "Inverse of " + name;
Q: Where do I have concurrency problems

22. There are more problems …
SynchronizedRGB color = new SynchronizedRGB(0, 0, 0, "Black");
...
int myColorInt = color.getRGB(); //Statement 1
String myColorName = color.getName(); //Statement 2

23. Some issues with RGB class
Write/write conflicts
If two threads try to write different colors, result may be a “mix” of R,G,B from two different colors
Read/write conflicts
If one thread reads while another writes, the color that is read may not match the color before or after

24. An immutable version! public class ImmutableRGB {
//Values must be between 0 and 255.
private int red; private int green; private int blue;
private String name;
public RGB(int red, int green, int blue, String name) {
check(red, green, blue);
this.red = red; this.green = green; this.blue = blue; this.name = name; }
public void set(int red, int green, int blue, String name) {
this.red = red; this.green = green; this.blue = blue; this.name = name;
public int getRGB() {
return ((red << 16) | (green << 8) | blue);
public ImmutableRGB invert() {
return new ImmutableRGB(255 - red; green; blue; "Inverse of“
+ name)
An immutable version!

25. Thread Coordination Tools Provided by Java
Synchronized methods
Synchronized statements (code blocks)
Guarded blocks

26. [Slight detour: creating threads in Java]
public class Thread implements Runnable {
// OVERVIEW: A thread is a thread of execution in a program.
// The Java Virtual Machine allows an application to have
// multiple threads of execution running concurrently.
public Thread (Runnable target)
// Creates a new Thread object that will run the target.
public void start ()
// Starts a new thread of execution. Calls the target’s run().
… many other methods } 26

27. [Slight detour: creating threads in Java]
public interface Runnable {
public void run()
When an object implementing interface Runnable is
used to create a thread, starting the thread causes the
object's run method to be called in that separately
executing thread. The general contract of the method run is that it may take any action
whatsoever. } 27

28. Thread Coordination: Tools Provided by Java
Synchronized methods
Synchronized statements (code blocks)
Guarded blocks

29. Thread Coordination: Tools Provided by Java
Synchronized methods
Object monitor
Synchronized statements (code blocks)
Guarded blocks

30. Simple Example: Counter (in Java)
One Counter object with two instance methods, increment and decrement.
Two Threads, one calls increment 1000 times, the other calls decrement 1000 times.
After each call, they sleep for a random time.
What do you think will be the value of the counter when they both finish? 30

31. public class Counter {
private int c = 0;
public void increment() {
c++; }
public void decrement() {
c--;
public int value() {
return c;

32. public class Counter {
private int c = 0;
public synchronized void increment() {
c++; }
public synchronized void decrement() {
c--;
public synchronized int value() {
return c;
Underlying implementation: object monitor/lock acquired when entering synchronized methid

33. Performance issues Why not just synchronize everything?
Performance costs
Possible risks of deadlock from too much locking
Performance in current Sun JVM
Synchronized method are 4 to 6 times slower than non-synchronized
Performance in general
Unnecessary blocking and unblocking of threads can reduce concurrency
Immutable objects can be short-lived, increase garbage collector

34. Acquires monitor for obj
Tools provided by Java
Synchronized methods
Synchronized statements (code blocks)
synchronized(obj) {
... code block }
Guarded blocks
Acquires monitor for obj

35. Is this correct? public class BankAccount { private int balance;
public void transfer(BankAccount b, int amount)
throws InsufficientFundsException {
// MODIFIES: this, b
// EFFECTS: If this account has more than amount value,
// transfers amount from this account to b.
// Otherwise, throws InsufficientFundsException.
if (this.getBalance() > amount) {
b.addFunds(amount);
this.balance = this.balance – amount;
} else { throw new InsufficientFundsException(); } }
Is this correct?
synchronized(this) { }

36. Thread Coordination Tools Provided by Java
Synchronized methods
Synchronized statements (code blocks)
Guarded blocks
obj.wait() / obj.notify(), obj.notifyAll()
Example use: want a thread to start making progress only when a condition is fulfilled
(ugly) alternative: busy waiting
public static void yield()

37. obj.wait()
Gives up lock on obj; puts current thread in waiting set for obj
obj.notify(), obj.notifyAll()
Don’t give up lock; selects one (notify) or all (notifyAll) threads in waiting set for obj and wakes them up (to be scheduled)
Key points:
Thread has to hold the monitor on the object when calling these methods
The awakened thread will not be able to proceed until the current thread relinquishes the monitor on the object

38. wait and notify Thread A Thread B synchronized (o) { o.wait ()
waiting
o.notify ()
awake, but not running
} // end synchronized
can reclaim o lock

39. wait and notify waiting waiting still waiting
Thread C
Thread A
synchronized (o){
Thread B
If multiple threads are waiting on the same object, any one of them can be awakened
o.wait ()
synchronized (o){
waiting
o.wait ()
synchronized (o) {
waiting
o.notify ()
awake,
not running
still waiting
} // end synchronized

40. Back to reasoning about correctness!
wait within synchronized code causes the thread to give up its lock and sleep until notified
Allows another thread to obtain lock and continue processing
State should be valid!
Termination?

41. Now that I have created [lots of] threads how do I stop them?
Wait for termination?
public final void join()
throws InterruptedException

42. Stopping Threads public class java.lang.Thread {
public final void stop()
Deprecated. This method is inherently unsafe.
Forces the thread to stop executing. …The thread represented by this thread is forced to stop whatever it is doing abnormally and to throw a newly created ThreadDeath object as an exception. …
Recommended reading: Why are Thread.stop, Thread.suspend and Thread.resume Deprecated?

43. Why deprecate stop?
What should happen to all the locks a thread owns when it is stopped?
What if an invariant is temporarily broken in a method?

44. Suspending Threads public final void suspend()
Suspends this thread. If the thread is alive, it is suspended and makes no further progress unless and until it is resumed.
Deprecated. This method has been deprecated, as it is inherently deadlock-prone. If the target thread holds a lock on the monitor protecting a critical system resource when it is suspended, no thread can access this resource until the target thread is resumed. If the thread that would resume the target thread attempts to lock this monitor prior to calling resume, deadlock results. Such deadlocks typically manifest themselves as "frozen" processes.

45. Can’t stop, can’t suspend, what can you do?
public void interrupt()
Interrupts this thread.
If this thread is blocked in an invocation of the wait(), wait(long), or wait(long, int) methods of the Object class, or of the join(), join(long), join(long, int), sleep(long), or sleep(long, int), methods of this class, then its interrupt status will be cleared and it will receive an InterruptedException. …
If none of the previous conditions hold then this thread's interrupt status will be set.

46. Being Interrupted public boolean isInterrupted() MODIFIES: nothing
Interrupts are just “polite” requests!
The thread can ignore it and keep going…
Being Interrupted
public boolean isInterrupted()
MODIFIES: nothing
EFFECTS: Returns true iff this thread
has been interrupted.
Good read:

47. you now have new guns to shoot yourself in the foot …
Let’s discuss a few ways:
Nested monitors and deadlock
Priority inversion

48. Nested monitor lockout problem
Problem: Calling a blocking method within a synchronized method can lead to deadlock

49. Nested Monitor Lockout Example
class Stack {
LinkedList list = new LinkedList();
public synchronized void push(Object x) { synchronized(list) {
list.addLast( x ); notify();
} }
public synchronized Object pop() { synchronized(list) {
if( list.size() <= 0 ) wait();
return list.removeLast(); }
Releases lock on Stack object but not lock on list;
a push from another thread will deadlock

50. Preventing nested monitor deadlock
Two programming suggestions
No blocking calls in synchronized methods, or
Provide some non-synchronized method of the blocking object
No simple solution that works for all programming situations

51. you now have new guns to shoot yourself in the foot …
Let’s discuss a few examples:
Nested monitors and deadlock
Priority inversion

52. class IncThread extends Thread {
private Counter c;
public IncThread (Counter p_c) { c = p_c; }
public void run () {
while (true) {
synchronized (c) {
c.increment ();
System.err.println ("Running inc thread: " + currentThread () + …);
c.notify ();
} } } }
class DecThread extends Thread { …
while (c.getValue () <= 0) {
try { c.wait (); } catch (InterruptedException e) { ; } }
c.decrement ();
System.err.println ("Running dec thread: " + …);

53. Counter c = new Counter (); IncThread incThread = new IncThread (c);
DecThread decThread = new DecThread (c);
incThread.setPriority (Thread.NORM_PRIORITY);
incThread.start ();
decThread.setPriority (Thread.MAX_PRIORITY);
decThread.start ();
Running inc thread: Thread[Thread-0,5,main] / Value: 1
Running dec thread: Thread[Thread-1,10,main] / Value: 0

54. Priorities
In general, threads with higher priorities will be scheduled preferentially.
There are no guarantees!
up to the Java scheduler
Thread class:
void setPriority (int newPriority)
// MODIFIES: this
// EFFECTS: Changes the priority of this
// thread to newPriority.

55. Priorities, Priorities
incThread.setPriority (Thread.NORM_PRIORITY);
incThread.start ();
decThread.setPriority (Thread.MIN_PRIORITY);
decThread.start ();
The incThread should run more than the decThread, but there is no guarantee.
Thread.MIN_PRIORITY
Thread.NORM_PRIORITY
Thread.MAX_PRIORITY

56. Mars Pathfinder
Landed on Mars
July 4, 1997
Sojourner Rover

57. New York Times, 15 July 1997
Mary Beth Murrill, a spokeswoman for NASA's Jet Propulsion Laboratory, said transmission of the panoramic shot took “a lot of processing power.” She likened the data overload to what happens with a personal computer “when we ask it to do too many things at once.”
The project manager, Brian Muirhead, said that to prevent a recurrence, controllers would schedule activities one after another, instead of at the same time. It was the second time the Pathfinder's computer had reset itself while trying to carry out several activities at once.
In response, controllers reprogrammed the computer over the weekend to slow down the rate of activities and avoid another reset. But today, about an hour into a two-hour transmission session, it happened again.

58. Priority-Based Scheduling
Scheduler ensures that the highest priority task that can run is always running
Lower priority tasks run only when no higher priority task can run
Standard JavaVM scheduler does not do this, but many operating Systems for embedded systems do including the vxWorks used on the PathFinder.
What could go wrong with priority-based scheduling?

59. Priority Inversion Low Medium High synchronized(r) synchronized(r)
Pre-empts low-priority thread
Waiting on lock r (held by low-priority task)

60. Priority Inversion on Mars
Meterological data task
(low priority)
Bus Management Task
(high priority)
Data collection task
(medium priority)
For details, see Glenn Reeves account:

61. Solutions? Priority Inheritance Priority Ceilings
If a low priority task holds a resource needed by a high priority task, the low priority task temporarily inherits the high task’s priority
Priority Ceilings
Associate minimum priorities with resources: only a high priority task can acquire the lock on an important resource

62. Summary DataStore The challenge: Coordinated access to shared data!
Server
Listener
Request 1
Request 2
Request 3
DataStore
Instruction Streams

63. Thread safety Concept Why is this important?
The representation of an ADT should be valid (when observed by other entities) even when used concurrently by multiple threads
Why is this important?
ADTs designed so each method preserves representation invariants.
Remember ADT implementation correctness proof
Invariants hold on method entry => they hold on exit
Correctness proof for each methods assumes invariant holds on entry; (and implicitly assumes exclusive access to representation)
Problems will occur: IF invariants fail in the middle of execution of a method, AND concurrent execution of another method call can observe an inconsistent state

64. Potential Solutions Immutable objects No shared store
Use locking discipline
Require thread interleavings to happen in a predictable way
Synchronized methods
Object monitor
Synchronized statements (code blocks)
Guarded blocks

65. Done

66. The Dining Philosopher’s Problem

67. The Dining Philosopher’s Problem
What are the issues to avoid?
Deadlock
Starvation

68. The Dining Philosopher’s Problem
How does it look in Java?