Parallel and Distributed Algorithms Spring 2005 Johnnie W. Baker

Presentations Professor Johnnie W. Baker Instructor Will give most presentations Guest Lecturers from Parallel Processing Group Occasional lecture in areas of expertise Occasionally cover classes when I am away Hopefully, we will have a grader from this group.

Two Primary Textbooks Parallel Programming in C with MPI and OpenMP Michael Quinn, author Published by McGraw Hill in 2004 Used in both PDC and PDA Parallel Computation: Models and Methods Selim Akl, author Prentice Hall, 1997 Access to an online copy will be provided. There will also be some supplementary handouts.

Additional References There will also be some supplementary handouts provided, as needed. Another excellent reference (not required) for parts of the PDA course is Introduction to Parallel Computing (Second Edition) Authors are Grama, Gupta, Karypis, Kumar Addison Wesley, 2003. A more advanced reference for some topics in Quinn A useful online textbook (more applicable to PDC) is Designing and Building Parallel Programs Ian Foster, Addison Wesley, 1995 The website for this book is http://www-unix.mcs.anl.gov/dbpp/

Two Complementary Courses Parallel & Distributed Computing (Fall) Parallel Architectures Parallel Languages Parallel Programming Algorithm Examples for some architectures Parallel & Distributed Algorithms (Spring) Important Models of Computation Designing Efficient Algorithms for Various Models PDC and PDA can be taken in either order More natural for PDC to be taken first However, students often take PDA first

Limited Overlap in PDC & PDA Allows PDC and PDA to be taken in either order. Performance Evaluation and Limits for Parallel Computation Some general topics required for both courses. More practical coverage needed for programming in PDC More theoretical considerations in PDA MPI Language Covered as a programming language in PDC Only a subset used in algorithms covered in PDA No programming assignments in PDA

Prerequisites The prerequisite for this course is A course in the design and analysis of algorithms such as CS 4/56101. Or Permission Alternately, students who have the following course should also have an adequate background for this course. CS 6/76105 Parallel and Distributed Computing

Assignments and Grading Homework assignments Problems assigned for most chapters Probably 5-7 different assignments No programming assignments Course Grade Based on homework, midterm, and final Approximate weights (assuming grader) Homework 40% Midterm Exam 30% Final Exam 30%

Major Topics Covered in PDA (Not necessarily in order covered) General topics Analysis of parallel computation Limits for parallel computation PRAM model and algorithms Algorithms for some important interconnection networks e.g. linear arrays, 2D mesh, hypercube Bus-Based models and typical algorithms Task/Channel Model algorithms (using MPI) BSP (Bulk Synchronous Model) and algorithms KSU’s associative model and algorithms

Major Topics in Companion Course (PDC) Fundamental concepts in parallel computation. Synchronous Computation SIMD, Vector, Pipeline Computing Associative and Multi-Associative Computing ASC Language and Programming MultiC Language and Programming Fortran 90 and HPF Languages Asynchronous (MIMD) Shared Memory Computation OpenMP language Symmetric Multiprocessors or SMPs Asynchronous (MIMD) Distributed Memory Computation Communications MPI Language and Programming Architectures Interconnection Networks (synchronous and asynchronous) Specific Computer Examples for above computation paradigms MIMD-SIMD Comparisons in Real-Time Applications