Parallel and Distributed Algorithms (CS 6/76501) Spring 2010 Johnnie W. Baker

Overview and Syllabus

Presentations Professor Johnnie W. Baker –Instructor –Will give most presentations Guest Lecturers from Parallel Processing Group –Occasional lecture in areas of expertise –Occasionally cover classes when I am away

Two Primary Textbooks Parallel Computation: Models and Methods –Selim Akl, author –Prentice Hall, 1997 –Access to an online copy will be provided. Parallel Algorithms –Henri Casanova, Arnaud Legrand, & Yves Roberts –CRC Press (A Chatman & Hall Book) There will also be some supplementary handouts.

Prerequisites The prerequisite for this course is one of the following: An undergraduate course in algorithms such as CS 4/ CS 6/76105 Parallel and Distributed Computing Permission

Overview of Topics Parallel Algorithms are key ingredient to solving a wide range of problems on various parallel systems Sequential algorithms are designed for one standard model called RAM –Random Access Machine Due to the wide variety of parallel & distributed systems, multiple computational models are needed to described the different types of systems Important parallel models capture the essence of existing and projected future parallel systems.

Overview (cont.) Will include both synchronous & asychronous models. –Asychronous models & algorithms are closely related to distributed computing Typical algorithms studied are for basic topics –searching, sorting, graphs, matrices, computational geometry While focus will be on parallel algorithms, many of the models, algorithms, and principles are applicable to distributed systems as well. Key course for those planning on working in parallel & distributed computing

Major Topics Covered in PDA (Not necessarily listed in order covered) General topics –Analysis of parallel computation –Limits for parallel computation PRAM model and algorithms –Parallel Random Access Machine (or Parallel “RAM”) –More published algorithms than other models –Formerly the “standard parallel model” Algorithms for some important interconnection networks –linear arrays and rings –2D mesh –hypercube

Major Topics in PDA (cont) Bus-Based & optical models & typical algorithms –Mesh with multiple broadcast –Reconfigurable mesh –Linear array with optical buses BSP (Bulk Synchronous Model) and algorithms –Essentially a combined computational & programming model. –Has an extensive software library that can be used in programs KSU’s associative model and algorithms –Air Traffic Control algorithms included

Benefits Most large software projects must be implemented on a parallel or distributed system –Needed for memory space –Needed to obtain reasonable speed Parallel systems are much more efficient for computational intensive applications –Distributed systems are much slower due to greater communication bottlenecks due to distributed database and greater synchronization problems Efficient algorithms and software is key to effective use of parallel & distributed systems.

Benefits (cont) There is a wide choice of thesis and dissertation topics in parallel & distributed computing area –Professors sponsoring parallel research are Baker, Farrell, and Walker –Farrell and Ruttan use parallel computation heavily as a tool in applications (bioinformatics, scientific, etc.) –Several professors in distributed areas as well. Students who are working on thesis or dissertation in another area can also benefit from this course. –Parallel systems often needed to handle computational intensive systems (e.g., bioinformatics)

Two Complementary Courses Parallel & Distributed Computing (usually in Fall) –Parallel Architectures –Parallel Languages –Parallel Programming –Algorithm Examples for some architectures Parallel & Distributed Algorithms (Alternate Springs) –Important Models of Computation –Designing Efficient Algorithms for Various Models PDC and PDA can be taken in either order –More natural for PDC to be taken first –However, students often take PDA first

Limited Overlap in PDC & PDA Allows PDC and PDA to be taken in either order. Performance Evaluation and Limits for Parallel Computation –Some general topics required for both courses. –More practical coverage needed for programming in PDC –More theoretical considerations in PDA Associative Computing Model –Programming focus in PDC –Primary focus in PDA is on associative algorithms

Major Topics in Companion Course (PDC) Fundamental concepts in parallel computation. Synchronous Computation –SIMD, Vector, Pipeline Computing –Associative and Multi-Associative Computing –ASC Language and Programming –MultiC Language and Programming –Fortran 90 and HPF Languages Asynchronous (MIMD) Shared Memory Computation –OpenMP language –Symmetric Multiprocessors or SMPs Asynchronous (MIMD) Distributed Memory Computation –Communications –MPI Language and Programming Architectures –Interconnection Networks (synchronous and asynchronous) –Specific Computer Examples for above computation paradigms MIMD-SIMD Comparisons in Real-Time Applications

Assignments and Grading Homework assignments –Problems assigned for most chapters –Probably 5-7 different assignments –No programming assignments Course Grade based on Homework (& class presentations) Midterm final –Approximate weights Homework 30% Midterm Exam 35% Final Exam 35%

Disabilities Information In accordance with university policy, if you have a documented disability and require accomodations to obtain equal access to this course, please contact the instructor at the beginning of the semester or when given an assignment for which an accomodation is required. Students with disabilities must verify their elgibility through the Office of Student Disability Services (SDS) in the Michael Schwartz Student Services Center ( ).

Plagiarism Information Plagiarism of any type will not be tolerated and will be dealt with in accordance to the University's Administrative policy and procedures regarding student cheating and plagiarism. See the University Statement on Academic Dishonesty for more details. Additionally, unattributed copying from another webpage is also considered plagiarism. Also, see the Computer Science Department Academic Policy involving Programming. Statement on Academic DishonestyAcademic Policy involving Programming

Attendance Information It is important that students attend class regularly. Material for the slides for this course is often obtained from multiple sources. While my slides will contain a lot of information, I can not include all of the information that will be covered in class. Those who can not attend regularly should discuss this situation with me in advance.