Designing a thread-safe class  Store all states in public static fields  Verifying thread safety is hard  Modifications to the program hard  Design for thread safety  Identify variables that form object’s state  Identify invariants on these states  Establish policy for managing concurrent access  Objects and variables have a state space  Smaller, the better

State-dependent operations  Current state dependent on previous state  cannot remove item from empty queue  Single threaded  Precondition does not hold  Operation fails  Concurrent  Precondition may become true later  Wait for action by another thread  Precondition may be invalidated as well

Instance confinement  Encapsulation of data  Object’s methods will access the data  Ensure data is accessed with appropriate lock held  Not escape intended scope  Monitor pattern  Encapsulate all mutable state  Use object’s own intrinsic lock to guard the state Public class PrivateLock { private final Object myLock = new Object(); Widget widget; void someMethod() { synchronized(myLock) { …. } }

Summary  Mutable state  Coordinating access to mutable state  Less mutable, easier to ensure thread safety  Make fields final  If they need not be mutable  Immutable objects are thread-safe  Encapsulation helps thread-safety  Mutable state  Guard with a lock (and the same lock everywhere)  Compound actions are guarded together

Summary (contd)  Mutable variable from multiple threads with no synchronization  Broken program  Include thread-safety in design process  Explicit documentation  Document synchronization policy

Structuring concurrent applications  Tasks  Abstract, discrete unit of works  Benefits  Simplifies program organization  Facilitates error recovery  Promotes concurrency  Natural structure for parallelizing work  Independent activities  State, result or side-effects of other tasks  Each task represents fraction of application processing capacity

Executing tasks sequentially Class WebServer { public static void main(…) { ServerSocket socket = new ServerSocket(80); while(true) { Socket connection = socket.accept(); handleRequest(connection); }  Simple and correct  Perform poorly in production  Might work if handleRequest returned immediately

Explicitly creating threads for tasks Class WebServer { …. Serversocket socket = new ServerSocket(80); while(true) { final Socket connection = socket.accept(); Runnable task = new Runnable() { public void run() { handleRequest(connection); } }; new Thread(task).start(); }  For each connection, new thread is created

Consequences  Task processing is offloaded from main thread  Wait for new connections  Tasks can be processed in parallel  Multiple requests serviced simultaneously  Task-handling code must be thread-safe

Disadvantages of unbounded thread creation  Thread lifecycle overhead  Thread creation  Thread teardown  Resource consumption  Threads consume resources  Some threads may sit idle consuming resources  Stability  Limit on how many threads can be created  OutOfMemoryError is possible

Decoupling  Tasks are logical units of work  Threads to run tasks asynchronously  Decouple task submission and task execution  Task submission – producer  Task execution – consumer  Pre-determined number of threads created  Testing possible for stability  Bounded queue of tasks

Execution policies  In what thread will tasks be executed  Order of execution of tasks  Concurrent tasks  Queuing length of the tasks  Task that should be evicted  How application should be notified  Action before/after executing a task

Cancellation and shutdown  Stop tasks/threads earlier  How?  Immediately?  Why?  User requested cancellation  Time-limited activities  Best solution within a time  Application events  Decomposition of problem space  Errors  e.g., disk full  Shutdown

Task cancellation  public void foo() {  while(!cancelled) {  doSomething(); } public void bar() { cancelled = true; }  Eventually foo() exits  What is the type of cancelled?

Interruption  Each thread has a boolean interrupted status  Some blocking methods detect interrupts and return early with exception  Well behaved methods  Return the interrupt status to the calling method to handle  Poorly behaved methods  Swallow the interrupts  Interruption policy  How thread should interpret interrupt?  What needs to be done?  What units of work are considered atomic?  Time of reaction to interrupts

Thread Pools  Dependent tasks  Constraints on execution policy  Avoid liveness problems  Tasks that exploit thread confinement  Single threaded executor  Thread safety is not required  Response-time sensitive tasks  Long running tasks to thread pool with few threads  Tasks that use ThreadLocal  Not to be used to communicate between tasks

Thread Pools (contd.)  Thread starvation deadlock  Tasks depend on other tasks in thread pool  Second task sits on work queue  First task waits for result of second task  Long-running tasks  Fewer threads in thread pool  Responsiveness suffers

Sizing thread pools  Ideal size  Type of submitted tasks  Characteristics of the deployment system  Not hard-coded  Computed dynamically  Configuration mechanism  Too big  Threads compete for resources  Too small  Throughput suffers

Sizing thread pools (contd.)  Number of processors  Amount of memory  Type of task  Computation  I/O  Some combination  Requires scarce resource  Compute-intensive tasks  N+1 threads on N processor system  I/O based tasks  More threads in the pool

Optimal pool size  N_cpu = number of CPUs  U_cpu = target CPU utilization, 0 <= U_cpu <= 1  W/C = ratio of wait time to compute time N_threads = N_cpu * U_cpu * (1 + W/C)  Other parameters  Memory, file handles, etc