1. Parallel and Distributed Algorithms (CS 6/76501) Spring 2007
Johnnie W. Baker

2. Presentations Professor Johnnie W. Baker
Instructor
Will give most presentations
Guest Lecturers from Parallel Processing Group
Occasional lecture in areas of expertise
Occasionally cover classes when I am away
Perhaps a lecture by a visiting professor

3. Two Textbooks Primary Textbook (online copy provided)
Parallel Computation: Models and Methods
Selim Akl, author
Prentice Hall, 1997
A textbook focusing on multiprocessor models and algorithms
Parallel Programming in C with MPI and OpenMP, Michael Quinn, McGraw Hill, 2004
Introduction to Parallel Computing, Grama, Gupta, Karypis, Kumar, 2nd Edition, Addison Wesley, 2003
There will also be some supplementary handouts.

4. Two Complementary Courses
Parallel & Distributed Computing (Fall)
Parallel Architectures
Parallel Languages
Parallel Programming
Algorithm Examples for some architectures
Parallel & Distributed Algorithms (Spring)
Important Models of Computation
Designing Efficient Algorithms for Various Models
PDC and PDA can be taken in either order
More natural for PDC to be taken first
However, students often take PDA first

5. Limited Overlap in PDC & PDA
Allows PDC and PDA to be taken in either order.
Performance Evaluation and Limits for Parallel Computation
Some general topics required for both courses.
More practical coverage needed for programming in PDC
More theoretical considerations in PDA
MPI Language
Used as a programming language in PDC
Only a subset used in algorithm descriptions in PDA
No programming in PDA

6. Prerequisites
The prerequisite for this course is one of the following:
An undergraduate or graduate course in the design and analysis of algorithms such as CS 4/56101.
Or CS 6/76105 Parallel and Distributed Computing
Or permission

7. Assignments and Grading
Homework assignments
Problems assigned for most chapters
Probably 5-7 different assignments
No programming assignments
Course Grade based on
Homework (class presentations)
Midterm
final
Approximate weights
Homework %
Midterm Exam 30%
Final Exam %

8. Overview of Topics
Parallel Algorithms are key ingredient to solving a wide range of problems on various parallel systems
Important parallel models capture the essence of existing or project future parallel systems.
Will include both synchronous & asychronous models.
Asychronous models & algorithms are closely related to distributed computing
Typical algorithms studied are for basic areas
searching, sorting, graphs, matrices, computational geometry
A key course for those planning on working in parallel & distributed computing area.

9. Major Topics Covered in PDA (Not necessarily in order covered)
General topics
Analysis of parallel computation
Limits for parallel computation
PRAM model and algorithms
Algorithms for some important interconnection networks
e.g. linear arrays, 2D mesh, hypercube
Bus-Based & optical models & typical algorithms
Task/Channel Model algorithms (using MPI)
BSP (Bulk Synchronous Model) and algorithms
KSU’s associative model and algorithms

10. Any Questions?

11. Major Topics in Companion Course (PDC)
Fundamental concepts in parallel computation.
Synchronous Computation
SIMD, Vector, Pipeline Computing
Associative and Multi-Associative Computing
ASC Language and Programming
MultiC Language and Programming
Fortran 90 and HPF Languages
Asynchronous (MIMD) Shared Memory Computation
OpenMP language
Symmetric Multiprocessors or SMPs
Asynchronous (MIMD) Distributed Memory Computation
Communications
MPI Language and Programming
Architectures
Interconnection Networks (synchronous and asynchronous)
Specific Computer Examples for above computation paradigms
MIMD-SIMD Comparisons in Real-Time Applications