1. Auburn University http://www.eng.auburn.edu/~xqin
COMP7330/7336 Advanced Parallel and Distributed Computing Thread Basics
Dr. Xiao Qin
Auburn University
Feedback on homework 3. TBC=8
Slides are adopted from Drs. Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar

2. What is a thread?
A thread is a lightweight process

3. What is a process?
Unit of resource ownership (A process is allocated?)
a virtual address space to hold the process image
control of some resources (files, I/O devices...)
Unit of dispatching (A process is an execution path through one or more programs:)
execution may be interleaved with other processes
the process has an execution state and a dispatching priority
Reference:
Unit of resource ownership-- A process is allocated:a virtual address space to hold the process image
control of some resources (files, I/O devices...)
Unit of dispatching- A process is an execution path through one or more programs:execution may be interleaved with other processes
the process has an execution state and a dispatching priority
If we treat these two characteristics as being independent (as does modern OS theory):
The unit of resource ownership is usually referred to as a process or task. This Processes have:
a virtual address space which holds the process image.
protected access to processors, other processes, files, and I/O resources.
The unit of dispatching is usually referred to a thread or a lightweight process. Thus a thread:
Has an execution state (running, ready, etc.)
Saves thread context when not running
Has an execution stack and some per-thread static storage for local variables
Has access to the memory address space and resources of its process
all threads of a process share this when one thread alters a (non-private) memory item, all other threads (of the process) sees that a file open with one thread, is available to others
Reference:

4. Threads have some advantages of (multi) processes Creation/Termination
Less time to create a new thread than a process, because the newly created thread uses the current process address space.
Less time to terminate a thread than a process. (Why?)
Reference:
Unit of resource ownership-- A process is allocated:a virtual address space to hold the process image
control of some resources (files, I/O devices...)
Unit of dispatching- A process is an execution path through one or more programs:execution may be interleaved with other processes
the process has an execution state and a dispatching priority
If we treat these two characteristics as being independent (as does modern OS theory):
The unit of resource ownership is usually referred to as a process or task. This Processes have:
a virtual address space which holds the process image.
protected access to processors, other processes, files, and I/O resources.
The unit of dispatching is usually referred to a thread or a lightweight process. Thus a thread:
Has an execution state (running, ready, etc.)
Saves thread context when not running
Has an execution stack and some per-thread static storage for local variables
Has access to the memory address space and resources of its process
all threads of a process share this when one thread alters a (non-private) memory item, all other threads (of the process) sees that a file open with one thread, is available to others

5. Threads have some advantages of (multi) processes switch and communication
Q1 Why it takes less time to switch between two threads within the same process?
because the newly created thread uses the current process address space.
Why the communication overheads among multiple threads is low?
The threads share everything: address space. Data produced by one thread is immediately available to all the other threads.
Reference:
Unit of resource ownership-- A process is allocated:a virtual address space to hold the process image
control of some resources (files, I/O devices...)
Unit of dispatching- A process is an execution path through one or more programs:execution may be interleaved with other processes
the process has an execution state and a dispatching priority
If we treat these two characteristics as being independent (as does modern OS theory):
The unit of resource ownership is usually referred to as a process or task. This Processes have:
a virtual address space which holds the process image.
protected access to processors, other processes, files, and I/O resources.
The unit of dispatching is usually referred to a thread or a lightweight process. Thus a thread:
Has an execution state (running, ready, etc.)
Saves thread context when not running
Has an execution stack and some per-thread static storage for local variables
Has access to the memory address space and resources of its process
all threads of a process share this when one thread alters a (non-private) memory item, all other threads (of the process) sees that a file open with one thread, is available to others

6. Multithreading Reference: http://www.cs.cf.ac.uk/Dave/C/node29.html
Unit of resource ownership-- A process is allocated:a virtual address space to hold the process image
control of some resources (files, I/O devices...)
Unit of dispatching- A process is an execution path through one or more programs:execution may be interleaved with other processes
the process has an execution state and a dispatching priority
If we treat these two characteristics as being independent (as does modern OS theory):
The unit of resource ownership is usually referred to as a process or task. This Processes have:
a virtual address space which holds the process image.
protected access to processors, other processes, files, and I/O resources.
The unit of dispatching is usually referred to a thread or a lightweight process. Thus a thread:
Has an execution state (running, ready, etc.)
Saves thread context when not running
Has an execution stack and some per-thread static storage for local variables
Has access to the memory address space and resources of its process
all threads of a process share this when one thread alters a (non-private) memory item, all other threads (of the process) sees that a file open with one thread, is available to others
Reference:

7. Single-threaded vs. Multithreaded Models Q2: What are their differences?
Reference:
Unit of resource ownership-- A process is allocated:a virtual address space to hold the process image
control of some resources (files, I/O devices...)
Unit of dispatching- A process is an execution path through one or more programs:execution may be interleaved with other processes
the process has an execution state and a dispatching priority
If we treat these two characteristics as being independent (as does modern OS theory):
The unit of resource ownership is usually referred to as a process or task. This Processes have:
a virtual address space which holds the process image.
protected access to processors, other processes, files, and I/O resources.
The unit of dispatching is usually referred to a thread or a lightweight process. Thus a thread:
Has an execution state (running, ready, etc.)
Saves thread context when not running
Has an execution stack and some per-thread static storage for local variables
Has access to the memory address space and resources of its process
all threads of a process share this when one thread alters a (non-private) memory item, all other threads (of the process) sees that a file open with one thread, is available to others
Reference:

8. Thread Basics
All memory in the logical machine model of a thread is globally accessible to every thread.
The stack corresponding to the function call is generally treated as being local to the thread for liveness reasons.
A logical machine model with both global memory (default) and local memory (stacks).
Such a flat model may result in very poor performance since memory is physically distributed in typical machines.

9. Overview of Programming Models
Process based models assume that all data associated with a process is private, by default, unless otherwise specified.
Lightweight processes and threads assume that all memory is global.
Directive based programming models extend the threaded model by facilitating creation and synchronization of threads.

10. Example
A thread is a single stream of control in the flow of a program. A program like:
for (row = 0; row < n; row++)
for (column = 0; column < n; column++)
c[row][column] =
dot_product( get_row(a, row),
get_col(b, col));
can be transformed to:
create_thread( dot_product(get_row(a, row), get_col(b, col)));
Q3: What is the difference between the above two code segments?
The for loop has n^2 iterations, each of which can be computed independently.
The thread is an instance of a function that returns before the function has finished executing.

11. A logical machine model
The logical machine model of a thread-based programming paradigm.

12. Thread Basics Threads provide software portability.
Inherent support for latency hiding. (Why?)
Scheduling and load balancing.
Ease of programming and widespread use.

13. The POSIX Thread API
Commonly referred to as Pthreads, POSIX has emerged as the standard threads API, supported by most vendors.
The concepts discussed here are largely independent of the API and can be used for programming with other thread APIs (NT threads, Solaris threads, Java threads, etc.) as well.

14. Thread Basics: Creation
Pthreads provides two basic functions for specifying concurrency in a program:
#include <pthread.h>
int pthread_create (
pthread_t *thread_handle,
const pthread_attr_t *attribute,
void * (*thread_function)(void *),
void *arg);
int pthread_join (
pthread_t thread,
void **ptr);
The function pthread_create invokes function thread_function as a thread
When an attribute object is not specified, it is NULL, and the default thread is created with the following attributes:
It is unbounded
It is nondetached
It has a default stack and stack size
It inhetits the parent's priority
pthread_join - wait for thread termination. The value passed to pthread_exit is returned in the located pointed by ptr.
The function above makes sure that its parent thread does not terminate until it is done. This function is called from within the parent thread and the first argument is the thread ID of the thread to wait on and the second argument is the return value of the thread on which we want to the parent thread to wait. If we are not interested in the return value then we can set this pointer to be NULL.
Reference:

15. Thread Basics: Termination
A thread can terminate in three ways :
If the thread returns from its start routine. pthread_join()
If it is canceled by some other thread. The function used here is pthread_cancel().
If its calls pthread_exit() function from within itself.
pthread_join - wait for thread termination
The function above makes sure that its parent thread does not terminate until it is done. This function is called from within the parent thread and the first argument is the thread ID of the thread to wait on and the second argument is the return value of the thread on which we want to the parent thread to wait. If we are not interested in the return value then we can set this pointer to be NULL.
Reference:

16. An Example: Hello World
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <pthread.h>
void *thread_function(void *arg);
char message[] = "Hello World";
int main() {
int res;
pthread_t a_thread;
void *thread_result;
......
Show the fork-and-join model on the whiteboard
See programming/thread/thread1.c

17. An Example: Hello World (cont.)
......
res = pthread_create(&a_thread, NULL,
thread_function, (void *)message);
if (res != 0) {
perror("Thread creation failed");
exit(EXIT_FAILURE); }
printf("Waiting for thread to finish...\n");
res = pthread_join(a_thread, &thread_result);
perror("Thread join failed");
printf("Thread joined, it returned %s\n",
(char *)thread_result);
printf("Message is now %s\n", message);
exit(EXIT_SUCCESS);
Show the fork-and-join model on the whiteboard

18. Thread Basics: Creation and Termination (Example)
Q4: How do threads communicate communicate with the parent process?
#include <pthread.h>
#include <stdlib.h>
#define MAX_THREADS 512
void *compute_pi (void *);
int hits[MAX_THREADS], total_hits, total_misses, ...
main() {
...
pthread_t p_threads[MAX_THREADS];
pthread_attr_t attr;
pthread_attr_init (&attr);
for (i=0; i< num_threads; i++) {
hits[i] = i;
pthread_create(&p_threads[i], &attr, compute_pi,
(void *) &hits[i]); }
pthread_join(p_threads[i], NULL);
total_hits += hits[i];
Show the fork-and-join model on the whiteboard

19. Thread Basics: Creation and Termination (Example)
void *compute_pi (void *s) {
int seed, i, *hit_pointer;
double rand_no_x, rand_no_y;
int local_hits;
hit_pointer = (int *) s;
seed = *hit_pointer;
local_hits = 0;
for (i = 0; i < sample_points_per_thread; i++) {
rand_no_x =(double)(rand_r(&seed))/(double)((2<<14)-1);
rand_no_y =(double)(rand_r(&seed))/(double)((2<<14)-1);
if (((rand_no_x - 0.5) * (rand_no_x - 0.5) +
(rand_no_y - 0.5) * (rand_no_y - 0.5)) < 0.25)
local_hits ++;
seed *= i; }
*hit_pointer = local_hits;
pthread_exit(0);