1. Parallel and Distributed Algorithms Spring 2005
Johnnie W. Baker

2. Presentations Professor Johnnie W. Baker
Instructor
Will give most presentations
Guest Lecturers from Parallel Processing Group
Occasional lecture in areas of expertise
Occasionally cover classes when I am away
Hopefully, we will have a grader from this group.

3. Two Primary Textbooks Parallel Programming in C with MPI and OpenMP
Michael Quinn, author
Published by McGraw Hill in 2004
Used in both PDC and PDA
Parallel Computation: Models and Methods
Selim Akl, author
Prentice Hall, 1997
Access to an online copy will be provided.
There will also be some supplementary handouts.

4. Additional References
There will also be some supplementary handouts provided, as needed.
Another excellent reference (not required) for parts of the PDA course is
Introduction to Parallel Computing (Second Edition)
Authors are Grama, Gupta, Karypis, Kumar
Addison Wesley, 2003.
A more advanced reference for some topics in Quinn
A useful online textbook (more applicable to PDC) is
Designing and Building Parallel Programs
Ian Foster, Addison Wesley, 1995
The website for this book is

5. Two Complementary Courses
Parallel & Distributed Computing (Fall)
Parallel Architectures
Parallel Languages
Parallel Programming
Algorithm Examples for some architectures
Parallel & Distributed Algorithms (Spring)
Important Models of Computation
Designing Efficient Algorithms for Various Models
PDC and PDA can be taken in either order
More natural for PDC to be taken first
However, students often take PDA first

6. Limited Overlap in PDC & PDA
Allows PDC and PDA to be taken in either order.
Performance Evaluation and Limits for Parallel Computation
Some general topics required for both courses.
More practical coverage needed for programming in PDC
More theoretical considerations in PDA
MPI Language
Covered as a programming language in PDC
Only a subset used in algorithms covered in PDA
No programming assignments in PDA

7. Prerequisites The prerequisite for this course is
A course in the design and analysis of algorithms such as CS 4/56101.
Or Permission
Alternately, students who have the following course should also have an adequate background for this course.
CS 6/76105 Parallel and Distributed Computing

8. Assignments and Grading
Homework assignments
Problems assigned for most chapters
Probably 5-7 different assignments
No programming assignments
Course Grade
Based on homework, midterm, and final
Approximate weights (assuming grader)
Homework %
Midterm Exam 30%
Final Exam %

9. Major Topics Covered in PDA (Not necessarily in order covered)
General topics
Analysis of parallel computation
Limits for parallel computation
PRAM model and algorithms
Algorithms for some important interconnection networks
e.g. linear arrays, 2D mesh, hypercube
Bus-Based models and typical algorithms
Task/Channel Model algorithms (using MPI)
BSP (Bulk Synchronous Model) and algorithms
KSU’s associative model and algorithms

10. Major Topics in Companion Course (PDC)
Fundamental concepts in parallel computation.
Synchronous Computation
SIMD, Vector, Pipeline Computing
Associative and Multi-Associative Computing
ASC Language and Programming
MultiC Language and Programming
Fortran 90 and HPF Languages
Asynchronous (MIMD) Shared Memory Computation
OpenMP language
Symmetric Multiprocessors or SMPs
Asynchronous (MIMD) Distributed Memory Computation
Communications
MPI Language and Programming
Architectures
Interconnection Networks (synchronous and asynchronous)
Specific Computer Examples for above computation paradigms
MIMD-SIMD Comparisons in Real-Time Applications