Loop Parallelism and OpenMP CS433 Spring 2001 Laxmikant Kale

Loop parallelism Observation: Most programs that need to be parallelized derive their complexity (in terms of the number of operations executed) from iterative behavior, expressed in loops If we can execute multiple iterations of a loop in parallel… We can often, indeed, do that What are the conditions? Sometimes, the code needs a bit of restructuring Several research efforts Also, automating the parallelization of loops via compilers But that is not our focus in this lecture

OpenMP evolution Standardization effort: Involves many vendors: Included shared memory programming as well I.e. subsumes Pthreads Involves many vendors: Intel, SGI, HP, DEC/Compaq, .. Language bindings: Fortran, C, C++

Overview Basic view: Parallel threads Worksharing: Shared variables: The program is single threaded, but it spawns parallel subtasks (threads) once in a while Note the distinction with Pthreads model There, each thread could be executing completely different code But typically, it doesn’t Parallel threads Worksharing: parallel loops sections Shared variables: shared, private, firstprivate, lastprivate, ..

Spawning threads Executes a section of code, by spawning multiple threads to execute it. Each thread gets a copy of all private variables Separate from the “main program”’s copy (so, there are k+1 copies if there are k threads fired) Threads may uses locks and barriers calls are different than PThreads double A[1000]; omp_set_num_threads(4); #pragma omp parallel { int ID = omp_thread_num(); pooh(ID,A); } Some Example/s from sc99 Tutorial at OpenMP web site.

Work-sharing: for construct Do all the iterations of a loop in parallel We will use Prof. Padua’s slides for this Not just the syntax and semantics of the parallel loop construct But how to write code, by transforming sequential loops, so they are parallel Slides 20-45 (Section 4.1 to 4.4), and 55-59 (4.7) Also read the OpenMP C manual http://polaris.cs.uiuc.edu/~padua/cs320 http://polaris.cs.uiuc.edu/~padua/cs320/guidec.pdf #pragma omp parallel #pragma omp for schedule(static) for(i=0;I<N;i++) { a[i] = a[i] + b[i];}

Work-sharing: Sections Parallel Loops allow identical sections of code (with different loop indices) to execute in parallel Sections allow different sections of code to be run by parallel threads #pragma omp parallel #pragma omp sections { X_calculation(); #pragma omp section y_calculation(); z_calculation(); }