1 Thread Pools Representation and Management of Data on the Internet

2 Why Thread Pools? Thread pools help achieve optimum resource utilization Common in server applications where –The processing of each individual task is short- lived and the number of requests is large –The overhead of creating a new thread for each request is significant –Servers should not spend more time and consume more system resources creating and destroying threads than processing actual user requests –Creating too many threads in one JVM can cause the system to run out of memory

3 Reusing Threads By reusing threads for multiple tasks: –The thread-creation overhead is spread over many tasks –The delay introduced by thread creation is eliminated

4 A Wrong Implementation Consider the case where –There is a pool of threads –Each task asks for a thread when starting and returns the thread to the pool after finishing –When there are no available threads in the pool the thread that initiates the task waits till the pool is not empty –What is the problem here?

5 Work Queue Worker Threads wait() Q is Empty All the worker threads wait for tasks A Possible Solution

6 Work Queue Worker Threads Task The number of worker threads is fixed. When a task is inserted to the queue, notify is called.

7 Work Queue Worker Threads notify() The number of worker threads is fixed. When a task is inserted to the queue, notify is called.

8 Work Queue Worker Threads The task is executed by the tread

9 Work Queue Worker Threads The task is executed by the tread Task notify()

10 Work Queue Worker Threads The remaining tasks are executed by the tread

11 Work Queue Worker Threads When a task ends the tread is released While the Q is not empty, take the task from the Q and run it (if the Q was empty, wait() would have been called)

12 Work Queue Worker Threads A new task is executed by the released tread

13 public class WorkQueue { private final int nThreads; private final PoolWorker[] threads; private final LinkedList queue; public WorkQueue(int nThreads) { this.nThreads = nThreads; queue = new LinkedList(); threads = new PoolWorker[nThreads]; for (int i=0; i<nThreads; i++) { threads[i] = new PoolWorker(); threads[i].start(); }

14 public void execute(Runnable r) { synchronized(queue) { queue.addLast(r); queue.notify(); } Why is synchronized needed? What does the notify wake?

15 private class PoolWorker extends Thread { public void run() { Runnable r; while (true) { synchronized(queue) { while (queue.isEmpty()) { try { queue.wait(); } catch (InterruptedException ignored) { } } r = (Runnable)queue.removeFirst(); } Why is synchronized needed?

16 // If we don't catch RuntimeException, // the pool could leak threads try { r.run(); } catch (RuntimeException e) { // You might want to log something here } Why is run called and not start? What happens if we don’t catch the Runtime Exception?

17 Risks in Using Thread Pools A deadlock that is caused when –there are no free threads, –the tasks that are running wait for some task T, and –T waits for a free thread (since the number of running threads is limited) Threads consume memory and if they are active the context switch consumes time (can significantly slowdown the system performance) May cause starvation of threads for other tasks

18 Risks in Using Thread Pools Incorrect implementation may cause –Threads that wait and never being notified –Thread leakage – threads that are taken from the pool and never being returned –Tasks that are stalled for a very long time (e.g., waiting for a user response where the user already gone home …) –Too many requests that arrive at the same time and consume too much resources

19 Some Solutions Do not queue tasks that wait synchronously for results from other tasks If the program must wait for a resource, such as an I/O completion, specify a maximum wait time, and then fail or re-queue the task for execution at a later time Tune the thread pool size according to the number and characterizations of expected tasks