CIS4930/CDA5125 Parallel and Distributed Systems Florida State University CIS4930/CDA5125: Parallel and Distributed Systems Instructor: Xin Yuan, 168 Love, Office hour: 10:30am-11:20am M, W, F, or by appointment Course website:

Course objectives Systematically introduce parallel and distributed systems (PDS) and programming Expose contemporary PDS technologies  Processors, networking, system software, and programming paradigms Study the trends of technology advances in PDS. At the end of the course, the students are expected to  Explain general concepts and important issues in parallel architectures, programming, and algorithms.  Understand the challenges in parallel and distributed computing  Program with different programming paradigms for different architectures. CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

Topics covered Architecture –Architecture classification –SIMD, MIMD, shared memory (SMP, multicore, SMT, heterogeneous multi-core, many-core), distributed memory (cluster, massively parallel, grid and cloud). Programming –Vector programming (SSE), Thread, OpenMP, MPI, CUDA, and OpenCL Algorithm –PRAM, BSP Advanced –Power and others CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

Textbok Required: none Reference books: –D. E. Culler and J. P. Singh, "Parallel Computer Architecture, A Hardware/Software Approach," Morgan Kaufmann, –Maurice Herlihy and Nir Shavit, "The Art of Multiprocessor Programming," Morgan Kaufmann, A significant portion of the course materials come from online resources. –MPI: –OpenMP: –CUDA: NVIDIA CUDA Programming GuideNVIDIA CUDA Programming Guide –OpenCL: OpenCL Programming Guide for the CUDA ArchitectureOpenCL Programming Guide for the CUDA Architecture CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

Course schedule (tentative) Week 1: Introduction and architectures classification Week 2: Performance Week 3: SIMD/Vector architectures, SSE and SIMD intrinsics Week 4: Shared memory architectures Week 5: Shared memory programming, thread Week 6: Shared memory programming, OpenMP Week 7: MIMD, distributed memory architectures, MPI Week 8: communication hardware and Interconnection networks Week 9: Stream architectures, CUDA/OpenCL Week 10: Spring break Week 11: CUDA/OpenCL Week 12: Performance monitoring and Optimization Week 13: PRAM and BSP Week 14: Advanced topics Week 15: Project presentations Week 16: Final exam CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

Prerequisite COP 4610: Operating systems –No requirement for background knowledge of parallel and distributed computing –The class is designed to minimized the dependence on COP4610. CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

Grading 25 % - homework, programming assignment, class participation. –Homework includes paper critics –Programming assignments can be done by groups of two. 25% - course project –Research or development –Groups of two –End product: report, software, and presentation. 20% - Midterm 30% - Final Final target letter grade: half A’s and half B’s. CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

Course project Development projects –Parallelizing a fairly significant application (preferably some well known efficient sequential code). Evaluation projects –E.g. Evaluating SSE performance on different processors, evaluate different synchronization algorithms on one or more processors Research projects –Survey an emerging area (e.g. techniques to improve power efficiency in multi-core machines, emerging programming models for multi-core systems). – Develop new techniques in any area of PDS (e.g. new communication algorithms, new synchronization algorithms, new scheduling schemes, etc) CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

Course policies Attendance: required. Late assignments: not accepted without a valid excuse. Missed exam: following the university rules. –Let me know when you need to miss an exam ASAP. Incomplete grade: –Miss the final with an accepted excuse –Due to extraordinary circumstances with appropriate documentation. CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

Academic Integrity –No copying from anywhere –Don’t ask others for solutions and don’t give solutions to others. Violation –The university requires all violations to be reported. –First violation with level 1 agreement: 0 for the particular assignment/exam and the lowering of one letter (A->B) for course final grade. –Second violation: resolved through the office of the Dean and the Faculties CIS4930/CDA5125 Parallel and Distributed Systems Florida State University

Accommodation for Disabilities 1.Register with and provide documentation to the Student Disability Resource Center (SDRC). 2.Bring a letter to me from the SDRC `indicating you need academic accomodations. This should be done within the first week of class CIS4930/CDA5125 Parallel and Distributed Systems Florida State University