1. Loop Parallelism and OpenMP CS433 Spring 2001
Laxmikant Kale

2. Loop parallelism Observation:
Most programs that need to be parallelized derive their complexity (in terms of the number of operations executed) from iterative behavior, expressed in loops
If we can execute multiple iterations of a loop in parallel…
We can often, indeed, do that
What are the conditions?
Sometimes, the code needs a bit of restructuring
Several research efforts
Also, automating the parallelization of loops via compilers
But that is not our focus in this lecture

3. OpenMP evolution Standardization effort: Involves many vendors:
Included shared memory programming as well
I.e. subsumes Pthreads
Involves many vendors:
Intel, SGI, HP, DEC/Compaq, ..
Language bindings:
Fortran, C, C++

4. Overview Basic view: Parallel threads Worksharing: Shared variables:
The program is single threaded, but it spawns parallel subtasks (threads) once in a while
Note the distinction with Pthreads model
There, each thread could be executing completely different code
But typically, it doesn’t
Parallel threads
Worksharing:
parallel loops
sections
Shared variables:
shared, private, firstprivate, lastprivate, ..

5. Spawning threads
Executes a section of code, by spawning multiple threads to execute it.
Each thread gets a copy of all private variables
Separate from the “main program”’s copy
(so, there are k+1 copies if there are k threads fired)
Threads may uses locks and barriers
calls are different than PThreads
double A[1000];
omp_set_num_threads(4);
#pragma omp parallel
{ int ID = omp_thread_num();
pooh(ID,A); }
Some Example/s from sc99 Tutorial at OpenMP web site.

6. Work-sharing: for construct
Do all the iterations of a loop in parallel
We will use Prof. Padua’s slides for this
Not just the syntax and semantics of the parallel loop construct
But how to write code, by transforming sequential loops, so they are parallel
Slides (Section 4.1 to 4.4), and (4.7)
Also read the OpenMP C manual
#pragma omp parallel
#pragma omp for schedule(static)
for(i=0;I<N;i++)
{ a[i] = a[i] + b[i];}

7. Work-sharing: Sections
Parallel Loops allow identical sections of code (with different loop indices) to execute in parallel
Sections allow different sections of code to be run by parallel threads
#pragma omp parallel
#pragma omp sections {
X_calculation();
#pragma omp section
y_calculation();
z_calculation(); }