Chapter 1 Introduction and General Concepts

References Selim Akl, Parallel Computation: Models and Methods, Prentice Hall, 1997, Updated online version available through website. Selim Akl, The Design of Efficient Parallel Algorithms, Chapter 2 in “Handbook on Parallel and Distributed Processing” edited by J. Blazewicz, K. Ecker, B. Plateau, and D. Trystram, Springer Verlag, Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar, Introduction to Parallel Computing, 2 nd Edition, Addison Wesley, Harry Jordan and Gita Alaghband, Fundamentals of Parallel Processing: Algorithms Architectures, Languages, Prentice Hall, Michael Quinn, Parallel Programming in C with MPI and OpenMP, McGraw Hill, Michael Quinn, Parallel Computing: Theory and Practice, McGraw Hill, 1994 Barry Wilkenson and Michael Allen, Parallel Programming, 2 nd Ed.,Prentice Hall, 2005.

Outline Need for Parallel & Distributed Computing Flynn’s Taxonomy of Parallel Computers –Two Main Types of MIMD Computers Examples of Computational Models Data Parallel & Functional/Control/Job Parallel –Granularity Analysis of Parallel Algorithms –Elementary Steps: computational and routing steps –Running Time & Time Optimal –Parallel Speedup –Speedup –Cost and Work –Efficiency Linear and Superlinear Speedup Speedup and Slowdown Folklore Theorems Amdahl’s and Gustafon’s Law

Reasons to Study Parallel & Distributed Computing Sequential computers have severe limits to memory size –Significant slowdowns occur when accessing data that is stored in external devices. Sequential computational times for most large problems are unacceptable. Sequential computers can not meet the deadlines for many real-time problems. Many problems are distributed in nature and natural for distributed computation