1. Lecture 21 Concurrency Introduction
(D&D 23)

2. Goals By the end of this lesson, you should:
Understand a simplified view of the CPU core/thread interactions
Be able to describe why one might want to use threads
Be able to describe the difference between a Java thread and OS-level threads
Understand the life cycle of Java threads

3. Central Processing Unit (CPU)
Processor Architecture
Threads
OS Threads
Thread States
Summary
Core 1
Core 2
Core 3
Core 4
Cache
Cache
Cache
Cache
Main Memory

4. Component Breakdown
Processor Architecture
Threads
OS Threads
Thread States
Summary
Central Processing Unit (CPU): The CPU consists of multiple components to perform operations, we will focus on the cores and shared cache.
Core: A core acts as a computation unit and provides a context for a Thread to operate. Each core in a CPU may run one thread at a time. A core will usually have its own cache as well.
Process: This is what starts when you run your Java program. When a process is started, resources are allocated by the OS for it to use. This starts with one executing thread.
Thread: One or more threads exist within a process and can be executed to complete a task.
Task: A set of program instructions that are defined by the written code.

5. Why Threads?
Processor Architecture
Threads
OS Threads
Thread States
Summary
Efficiency: Having just a single (OS level) thread means sticking to a single CPU/core. Other CPUs/cores remain idle.
Threads can use the other CPUs/cores: more efficient use of computing resource.
Responsiveness: A single-threaded GUI application cannot respond to events while it is carrying out computing-intensive tasks in the background.
Multiple threads can dedicated one thread to the GUI and one or more to the backend (e.g., in the Model-View-Controller structure)
See also our multithreaded server
Task separation: A single thread can only tend to a single task at a time.
Multiple threads can interleave tasks

6. Where do people use threads?
Processor Architecture
Threads
OS Threads
Thread States
Summary
Server applications having to support multiple clients / sessions
Client applications operating multiple sessions or connections (e.g., web browsers)
Graphical user interfaces that are the frontend for computing- and/or I/O-intensive tasks
Image/video processing (operations on many pixels/blocks can be carried out in parallel on multiple CPUs)
Graphics / games: same reasons as 2 and 3 above
AI / machine learning
Search
…basically everywhere!

7. Thread vs. processes Processor Architecture Threads OS Threads
Thread States
Summary
Each program we run consists of at least one process on the operating system.
Each process on an operating system gets its own:
method call stack. A process’ methods generally only invoke methods in this process.
program counter (a variable that keeps track of which statement in the program the process is currently executing)
Memory. A process generally only reads from and writes to its own memory heap.
Processes can spawn other processes, and can communicate / exchange data via shared memory.
Threads also get their own call stack and program counter, but generally share the memory heap with all other threads of the same process.

8. Multi-Threaded Programs
Processor Architecture
Threads
OS Threads
Thread States
Summary
How do threads physically run on the cores we explored earlier?
Computers with access to multiple cores, or multiple CPUs, may have each thread running on its own Core/CPU
Core 1
Thread 1
Core 2
Thread 2
If a computer only has access to a single core, multiple threads may be run on one core. However, only one thread can run at a time. This is known as interleaving where each thread has control for a succession of time slices
Thr 1
Thr 2
Thr 2
Thr 1
Thr 2

9. Multi-Threaded Programs
Processor Architecture
Threads
OS Threads
Thread States
Summary
Threads can be forced to terminate, or can temporarily cease their execution. This may occur in two ways.
Voluntarily: The thread may issues a sleep or yield command to perform cooperative scheduling. The benefits of threads acting this way is to prevent “Selfish” threads handling all of the resources. Consider cars merging on a highway as an analogy.
Enforced: The OS scheduler may interrupt a thread through the use of preemptive time slicing. There are multiple algorithms for efficiently scheduling the runtime of threads, based on their estimated runtime and resources used. Consider traffic lights directing the flow of traffic as an analogy.

10. OS-supported threads vs. Java threads
Processor Architecture
Threads
OS Threads
Thread States
Summary
OS-supported threads run each thread on the same CPU / core. Threads from the same process may run on different CPUs.
Java threads run as OS-supported threads, but the JVM typically switches them between different OS-supported threads over time.
Ralf Haeusler used to explain it like this:
A Java thread object is like a boat captain,
an OS-supported thread object is like a boat,
in an OS-defined universe where
Captains are repeatedly shifted between boats,
Captains command at most one boat at any given time, and
Boats persist much longer than Captains

11. Multi-Threaded Programs
Processor Architecture
Threads
OS Threads
Thread States
Summary
OS level threads: Each thread runs on one CPU only
ProgD
Thread 1
ProgA
Thread 2
ProgA
Thread 2
ProgA
Thread 2
ProgD
Thread 1
ProgC
Thread 2
CORE 2
ProgA
Thread 1
ProgB
Thread 1
ProgA
Thread 1
ProgC
Thread 1
ProgA
Thread 1
ProgB
Thread 1
ProgA
Thread 1
CORE 1
Time

12. Multi-Threaded Programs
Processor Architecture
Threads
OS Threads
Thread States
Summary
Java threads: Each thread runs on one CPU at a time but can continue on another
ProgD
Thread 1
ProgA
Thread 1
ProgA
Thread 2
ProgA
Thread 1
ProgD
Thread 1
ProgC
Thread 1
CORE 2
ProgA
Thread 1
ProgB
Thread 1
ProgA
Thread 2
ProgC
Thread 1
ProgA
Thread 2
ProgB
Thread 1
ProgA
Thread 1
CORE 1
Time

13. Java threads: life stages
Processor Architecture
Threads
OS Threads
Thread States
Summary
new (created, never started)
waiting
(waiting for notification from another thread)
wait()
start()
notify(), notifyAll()
timed waiting
(as waiting, but with timeout)
wait(timeout), sleep()
runnable
ready (started but not running)
notify(), notifyAll() (or timeout expires)
blocked
(waiting: to acquire lock, for an interrupt, or completion of an I/O operation)
other thread enters synchronized method first, or I/O request
transition managed by OS
other thread(s) exit synchronized method,
I/O request completes, interrupt()
running
terminated
(thread has completed its task)
run() returns

14. ready (started but not running)
Thread States
Processor Architecture
Threads
OS Threads
Thread States
Summary
At the OS level, the Runnable state actually encompasses two states, ready and running. These are hidden from the JVM which only views the Runnable state.
When a thread enters the Runnable state, at the OS level it starts in the ready state. It will move to the running state once it has been assigned a timeslice on a processor by the OS. This is known as dispatching the thread.
The process of determining how the thread will be dispatched is known as thread scheduling.
runnable
ready (started but not running)
transition managed by OS
running

15. OS View of Runnable State
Processor Architecture
Threads
OS Threads
Thread States
Summary
New: All threads being in the New state. It remains this way until the program starts the thread, placing it in the Runnable state.
Waiting: A thread may place itself in a Waiting state when waiting for another thread to perform a task. It will remain this way until it is notified by another thread.
Timed Waiting: Specify a timeout interval where the thread will stop waiting and switch back to the Runnable state. Consider a program backing up a harddrive for waiting uses.
Blocked: A thread in the Runnable state will switch to Blocked when it attempts to perform a task that it can’t perform immediately. It must wait until the other task blocking it completes before it returns to Runnable
Terminated: A thread moves to the Terminated state when it successfully completes its task or it otherwise terminates, say due to an error.

16. Next Lecture
Using Threads In Java (D&D 23)