Lecture 21 Concurrency Introduction (D&D 23)

Goals By the end of this lesson, you should: Understand a simplified view of the CPU core/thread interactions Be able to describe why one might want to use threads Be able to describe the difference between a Java thread and OS-level threads Understand the life cycle of Java threads

Central Processing Unit (CPU) Processor Architecture Threads OS Threads Thread States Summary Core 1 Core 2 Core 3 Core 4 Cache Cache Cache Cache Main Memory https://images.anandtech.com/doci/11729/intel_core_i7-678_678x452.png

Component Breakdown Processor Architecture Threads OS Threads Thread States Summary Central Processing Unit (CPU): The CPU consists of multiple components to perform operations, we will focus on the cores and shared cache. Core: A core acts as a computation unit and provides a context for a Thread to operate. Each core in a CPU may run one thread at a time. A core will usually have its own cache as well. Process: This is what starts when you run your Java program. When a process is started, resources are allocated by the OS for it to use. This starts with one executing thread. Thread: One or more threads exist within a process and can be executed to complete a task. Task: A set of program instructions that are defined by the written code.

Why Threads? Processor Architecture Threads OS Threads Thread States Summary Efficiency: Having just a single (OS level) thread means sticking to a single CPU/core. Other CPUs/cores remain idle. Threads can use the other CPUs/cores: more efficient use of computing resource. Responsiveness: A single-threaded GUI application cannot respond to events while it is carrying out computing-intensive tasks in the background. Multiple threads can dedicated one thread to the GUI and one or more to the backend (e.g., in the Model-View-Controller structure) See also our multithreaded server Task separation: A single thread can only tend to a single task at a time. Multiple threads can interleave tasks

Where do people use threads? Processor Architecture Threads OS Threads Thread States Summary Server applications having to support multiple clients / sessions Client applications operating multiple sessions or connections (e.g., web browsers) Graphical user interfaces that are the frontend for computing- and/or I/O-intensive tasks Image/video processing (operations on many pixels/blocks can be carried out in parallel on multiple CPUs) Graphics / games: same reasons as 2 and 3 above AI / machine learning Search …basically everywhere!

Thread vs. processes Processor Architecture Threads OS Threads Thread States Summary Each program we run consists of at least one process on the operating system. Each process on an operating system gets its own: method call stack. A process’ methods generally only invoke methods in this process. program counter (a variable that keeps track of which statement in the program the process is currently executing) Memory. A process generally only reads from and writes to its own memory heap. Processes can spawn other processes, and can communicate / exchange data via shared memory. Threads also get their own call stack and program counter, but generally share the memory heap with all other threads of the same process.

Multi-Threaded Programs Processor Architecture Threads OS Threads Thread States Summary How do threads physically run on the cores we explored earlier? Computers with access to multiple cores, or multiple CPUs, may have each thread running on its own Core/CPU Core 1 Thread 1 Core 2 Thread 2 If a computer only has access to a single core, multiple threads may be run on one core. However, only one thread can run at a time. This is known as interleaving where each thread has control for a succession of time slices Thr 1 Thr 2 Thr 2 Thr 1 Thr 2

Multi-Threaded Programs Processor Architecture Threads OS Threads Thread States Summary Threads can be forced to terminate, or can temporarily cease their execution. This may occur in two ways. Voluntarily: The thread may issues a sleep or yield command to perform cooperative scheduling. The benefits of threads acting this way is to prevent “Selfish” threads handling all of the resources. Consider cars merging on a highway as an analogy. Enforced: The OS scheduler may interrupt a thread through the use of preemptive time slicing. There are multiple algorithms for efficiently scheduling the runtime of threads, based on their estimated runtime and resources used. Consider traffic lights directing the flow of traffic as an analogy.

OS-supported threads vs. Java threads Processor Architecture Threads OS Threads Thread States Summary OS-supported threads run each thread on the same CPU / core. Threads from the same process may run on different CPUs. Java threads run as OS-supported threads, but the JVM typically switches them between different OS-supported threads over time. Ralf Haeusler used to explain it like this: A Java thread object is like a boat captain, an OS-supported thread object is like a boat, in an OS-defined universe where Captains are repeatedly shifted between boats, Captains command at most one boat at any given time, and Boats persist much longer than Captains

Multi-Threaded Programs Processor Architecture Threads OS Threads Thread States Summary OS level threads: Each thread runs on one CPU only ProgD Thread 1 ProgA Thread 2 ProgA Thread 2 ProgA Thread 2 ProgD Thread 1 ProgC Thread 2 CORE 2 ProgA Thread 1 ProgB Thread 1 ProgA Thread 1 ProgC Thread 1 ProgA Thread 1 ProgB Thread 1 ProgA Thread 1 CORE 1 Time

Multi-Threaded Programs Processor Architecture Threads OS Threads Thread States Summary Java threads: Each thread runs on one CPU at a time but can continue on another ProgD Thread 1 ProgA Thread 1 ProgA Thread 2 ProgA Thread 1 ProgD Thread 1 ProgC Thread 1 CORE 2 ProgA Thread 1 ProgB Thread 1 ProgA Thread 2 ProgC Thread 1 ProgA Thread 2 ProgB Thread 1 ProgA Thread 1 CORE 1 Time

Java threads: life stages Processor Architecture Threads OS Threads Thread States Summary new (created, never started) waiting (waiting for notification from another thread) wait() start() notify(), notifyAll() timed waiting (as waiting, but with timeout) wait(timeout), sleep() runnable ready (started but not running) notify(), notifyAll() (or timeout expires) blocked (waiting: to acquire lock, for an interrupt, or completion of an I/O operation) other thread enters synchronized method first, or I/O request transition managed by OS other thread(s) exit synchronized method, I/O request completes, interrupt() running terminated (thread has completed its task) run() returns

ready (started but not running) Thread States Processor Architecture Threads OS Threads Thread States Summary At the OS level, the Runnable state actually encompasses two states, ready and running. These are hidden from the JVM which only views the Runnable state. When a thread enters the Runnable state, at the OS level it starts in the ready state. It will move to the running state once it has been assigned a timeslice on a processor by the OS. This is known as dispatching the thread. The process of determining how the thread will be dispatched is known as thread scheduling. runnable ready (started but not running) transition managed by OS running

OS View of Runnable State Processor Architecture Threads OS Threads Thread States Summary New: All threads being in the New state. It remains this way until the program starts the thread, placing it in the Runnable state. Waiting: A thread may place itself in a Waiting state when waiting for another thread to perform a task. It will remain this way until it is notified by another thread. Timed Waiting: Specify a timeout interval where the thread will stop waiting and switch back to the Runnable state. Consider a program backing up a harddrive for waiting uses. Blocked: A thread in the Runnable state will switch to Blocked when it attempts to perform a task that it can’t perform immediately. It must wait until the other task blocking it completes before it returns to Runnable Terminated: A thread moves to the Terminated state when it successfully completes its task or it otherwise terminates, say due to an error.

Next Lecture Using Threads In Java (D&D 23)