csinparallel.org Patterns and Exemplars: Compelling Strategies for Teaching Parallel and Distributed Computing to CS Undergraduates Libby Shoop Joel Adams Dick Brown

csinparallel.org Today’s messages Parallel Design Patterns provide an established, practical set of principles for teaching PDC “Exemplar” example applications with multiple implemented solutions provide motivation for students and teaching materials for instructors Patterns and Exemplars fit together naturally and are ready for deployment

csinparallel.org Parallel Design Patterns Following on the original Gang of Four design patterns work Active work on parallel design patterns and parallel pattern languages: Catalog parallel patterns used in solutions and describe a methodology for using the pattern

csinparallel.org Past Work Lea : – Java Concurrency Patterns book Mattson, Saunders, and Massingil : – PPLP book Ralph Johnson et al. : – Parallel Programming Patterns online; books of Visual C++,.NET examples Oretega-Arjona book McCool, Reinders, and Robison book Kreutzer, Mattson, et al. : – Our Pattern Language (OPL) online ParaPLoP Workshop on Parallel Programming Patterns ParaPLoP ‘

csinparallel.org Pattern Approach Using existing design knowledge when designing new parallel programs Leads to parallel software systems that are: – modular, adaptable, understandable and evolve easily Also provides an effective problem-solving framework and a guide for teaching about good parallel solutions

csinparallel.org PATTERNLETS

csinparallel.org Patternlets… … are minimalist, scalable, executable programs, each illustrating a particular pattern’s behavior: – Minimalist so that students can grasp the concept without non-essential details getting in the way – Scalable so that students see different behaviors as the number of threads changes – Executable so that Instructors can use it in a live-coding demo Students can use it in a hands-on exercise Patternlets let students see the pattern in action

csinparallel.org Existing Patternlets (so far) OpenMP – Fork-Join – SPMD – Master-Worker – Parallel For Loop (blocks) – Parallel For Loop (stripes) – Reduction – Private – Atomic – Critical – Critical2 – Sections – Barrier MPI – SPMD – Master-Worker – Message Passing – Parallel For Loop (stripes) – Parallel For Loop (blocks) – Broadcast – Reduction – Scatter – Gather – Barrier

MPI Patternlets OpenMP Patternlets

csinparallel.org /* masterWorker.c (MPI) … */ #include int main(int argc, char** argv) { int id = -1, numProcs= -1, length = -1; char hostName[MPI_MAX_PROCESSOR_NAME]; MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD, &id); MPI_Comm_size(MPI_COMM_WORLD, &numProcs); MPI_Get_processor_name (hostName, &length); if ( id == 0 ) { // process with ID == 0 is the master printf("Greetings from the master, #%d (%s) of %d processes\n”, id, hostName, numProcs); } else { // processes with IDs > 0 are workers printf("Greetings from a worker, #%d (%s) of %d processes\n”, id, hostName, numProcs); } MPI_Finalize(); return 0; }

csinparallel.org Sample Executions $ mpirun -np 1./masterWorker Greetings from the master, #0 (node-01) of 1 processes $ mpirun –np 8./masterWorker Greetings from the master, #0 (node-01) of 8 processes Greetings from a worker, #1 (node-02) of 8 processes Greetings from a worker, #5 (node-06) of 8 processes Greetings from a worker, #3 (node-04) of 8 processes Greetings from a worker, #4 (node-05) of 8 processes Greetings from a worker, #7 (node-08) of 8 processes Greetings from a worker, #2 (node-03) of 8 processes Greetings from a worker, #6 (node-07) of 8 processes

csinparallel.org /* masterWorker.c (OpenMP) … */ #include int main(int argc, char** argv) { int id = -1, numThreads = -1; // #pragma omp parallel { id = omp_get_thread_num(); numThreads = omp_get_num_threads(); if ( id == 0 ) { // thread with ID 0 is master printf(”Greetings from the master, #%d of %d threads\n\n”, id, numThreads); } else { // threads with IDs > 0 are workers printf(”Greetings from a worker, #%d of %d threads\n\n”, id, numThreads); } return 0; }

csinparallel.org Sample Executions $./masterWorker// pragma omp parallel disabled Greetings from the master, #0 of 1 threads $./masterWorker// pragma omp parallel enabled Greetings from a worker, #1 of 8 threads Greetings from a worker, #2 of 8 threads Greetings from a worker, #5 of 8 threads Greetings from a worker, #3 of 8 threads Greetings from a worker, #6 of 8 threads Greetings from the master, #0 of 8 threads Greetings from a worker, #4 of 8 threads Greetings from a worker, #7 of 8 threads

csinparallel.org EXEMPLARS

csinparallel.org Motivation Everyone in CS needs PDC Not everyone is naturally drawn to PDC topics How shall we motivate every CS undergraduate to learn the PDC they will need for their careers?

csinparallel.org Motivation Everyone in CS needs PDC Not everyone is naturally drawn to PDC topics Proposal: Teach PDC concepts with compelling applications. Some CS students draw by concepts and tech Other CS students drawn by the applications How shall we motivate every CS undergraduate to learn the PDC they will need for their careers?

csinparallel.org Exemplars An exemplar is: A representative applied problem plus multiple code solutions implemented in various PDC technologies, with commentary

csinparallel.org Exemplar A (from EAPF Practicum) Compute π via numerical integration Implemented solutions – Serial – Shared memory (OpenMP, TBB, pthreads, Windows Threads, go language) – Distributed computing (MPI) – Accelerators (CUDA, Array Building Blocks) Comments: – Flexible uses: demo, concepts, tech, compare – But not a compelling application

csinparallel.org Exemplar B (from EAPF Practicum) Drug design Implemented solutions – Serial – Shared memory (OpenMP, boost threads, go lang) – Map-reduce framework (Hadoop)

csinparallel.org Exemplar B (from EAPF Practicum) Comments – Compelling application – Molecular dynamics, docking algorithm – Substitute for docking algorithm to score ligands: (score is maximal match count) Relates to genetic alignment algorithm Multiple ways to scale: # ligands, ligand length, # cores Random strings with random lengths for variable computational load per ligand

csinparallel.org Exemplars + Patterns Exemplar implementations offer a rich opportunity for learning patterns Examples – π as area (among 8 PDC implementations): Data Decomposition, Geometric Decomposition; Parallel For Loop, Master-Worker, Strict Data Parallel, Distributed Array; SIMD, Thread Pool, Message Passing, Collective Communication, Mutual Exclusion – Drug design (among 4 PDC implementations): Map-Reduce; Data Decomposition; Parallel For Loop, Fork- Join, BSP, Master-Worker, Task Queue, Shared Array, Shared Queue; Thread Pool, Message Passing, Mutual Exclusion

Drug design π as area

csinparallel.org Conclusion Patterns – a meaning for “parallel thinking,” best practice from industry Patternlets – minimalist, scalable, executable programs, each illustrating a particular pattern’s behavior Exemplars – motivation, hands-on/demo, teaching resource, opportunities for PDC These are naturally combined and ready for deployment