Parallel Algorithm Design & Analysis Course Dr. Stephen V. Providence Motivation, Overview, Expectations, What’s next.

Dr. Stephen V. Providence Parallel Algorithms 2 Motivation The most important element of parallel computation and the primary focus of this course is the parallel algorithm, without which no problem could be solved in parallel.

Dr. Stephen V. Providence Parallel Algorithms 3 Motivation The manner in which algorithms occupy a central place in computer science, parallel algorithms occupy the center in parallel computation. Given a problem to be solved in parallel, a parallel algorithm defines how the problem can be solved on the given parallel computer. (i.e. Grid Computing System)

Dr. Stephen V. Providence Parallel Algorithms 4 Motivation Specifically, parallel computation must specify how the problem is divided into sub-problems, how the processors communicate, and how the partial solutions are combined to produce the final answer.

Dr. Stephen V. Providence Parallel Algorithms 5 Motivation An understanding of the role of parallel algorithms and a good knowledge of the techniques of designing and analyzing them are essential to anyone entering the field of HPC or Computational Science.

Dr. Stephen V. Providence Parallel Algorithms 6 Overview To be taught in a manner analogous to a course on sequential algorithms, –this course will apply a similar course schema to parallel algorithm design and analysis. –there are topics unique to parallel algorithms.

Dr. Stephen V. Providence Parallel Algorithms 7 Overview Start with simple searching and sorting algorithms. Then design them for parallel execution over given parallel topology (hypercube, mesh, grid), –details in the syllabus. Progress made to more advanced algorithms, –such as: sorting networks, polynomial products, DFT and FFT. –details in the syllabus.

Dr. Stephen V. Providence Parallel Algorithms 8 Overview The textbook will outline the principles and give students and in-hand guide. Handouts may be ed to students along with technical data from journal articles. The following texts are under consideration: –S.G. Akl, Parallel Computation: Models and Methods (reference). –S.G. Alk, The Design and Analysis of Parallel Algorithms (source) –M.J. Quinn, Parallel Programming in C with MPI and OpenMP (practical).

Dr. Stephen V. Providence Parallel Algorithms 9 Overview Programming in Parallel is an essential component of the course. All algorithms must be coded and run on an available parallel computing system. –The binding language of choice is MPI. –We may use C or HPF (ForTran), probably C. –HPC++ and Java are possibilities (not Perl).

Dr. Stephen V. Providence Parallel Algorithms 10 Overview This course lends itself well to NCREN; –much of the work can be done online, parallel codes easily demonstrated over NCREN, lectures explaining algorithm functionality can be given over NCREN, presentations of complex parallel computing concepts can be given visually (animations), –exams and the like may be done off-line.

Dr. Stephen V. Providence Parallel Algorithms 11 Overview Culmination of course: –work on real research project, –execute algorithm design cycle, 1.preliminary algorithms designed and analyzed, 2.prototype codes created and tested, 3.codes verified and run, –extend and refine, –repeat design cycle.

Dr. Stephen V. Providence Parallel Algorithms 12 Overview Real world problem: –research in sparse matrix by vector products, –current research relies on matrix transformations, –use of structured matrices (Vandermonde, Cauchy, Toeplitz, Hankel, and matrices of related structure), –create diagonally dominant block matrices, –utilize super-fast linear system solvers, –all derived algorithms are highly parallelizable (sic), –applications to material science, CFD, and pure mathematics, details in the syllabus.

Dr. Stephen V. Providence Parallel Algorithms 13 Expectations All students will appreciate the relevance and utility of parallel algorithms in HPC; –their realization of parallel algorithm, importance will force them to seek the relevant algorithm first, then code the efficient solution (same as sequential).

Dr. Stephen V. Providence Parallel Algorithms 14 Expectations They will be able to solve problems more efficiently and know the limits of their solution (time and memory constraints); –NP-hard and NP-complete problems, –principle of parallel synergy (not limited by Amdahl’s speedup rule), –real time parallel computing

Dr. Stephen V. Providence Parallel Algorithms 15 Expectations They will have the facility to extend well- known algorithms to solve specific problems; –many interesting problems require modification of parallel algorithm to exploit hidden parallelism, –often these modifications are minor.

Dr. Stephen V. Providence Parallel Algorithms 16 Expectations Focus is on algorithms: –not parallel architecture, –not parallel programming. Although this course uses elements of both, it will give the student a foundation for work in HPC.

Dr. Stephen V. Providence Parallel Algorithms 17 What’s next Tasks: –set timetable –generate syllabus, lecture schedule, assignment, quiz and exam schedule, grading rules, –choose textbook (listed above), –select computing platform (IBM p-series 690), –programming language (C or ForTran 90/95 w/ MPI, HPF, HPC++…).

Dr. Stephen V. Providence Parallel Algorithms 18 What’s next Contact: Dr. Stephen V. Providence North Carolina A&T State University Department of Computer Science McNair Hall E. Market St. Greensboro, NC 27411