Early Adopter Introduction to Parallel Computing: Research Intensive University: 4 th Year Elective Bo Hong Electrical and Computer Engineering Georgia Institute of Technology

2 Syllabus – Overview (1) Format –A 4 th year elective course on parallel computing. Target Audience –students who have background in computer organization and programming, and would like to learn parallel computing. NSF/IEEE-TCPP PDC Curriculum

3 Syllabus – Overview (2) Topical Coverage: –Parallel computing systems –Parallel programming –Parallel algorithms Expected Outcomes: –Upon completion of the course, the students are expected to (1) understand parallel computing systems, and (2) be able to design algorithms and implement programs for such systems. NSF/IEEE-TCPP PDC Curriculum

4 Syllabus – Topical Coverage (1) TopicsBLOOM#HoursSource Parallel Architectures Stacks and thread executionK1Addition to the proposed curriculum Multi-threading on shared memory computers C1Addition to the proposed curriculum Memory Hierarchy and cache coherency K1NSF-TCPP Curriculum Architecture (C) Snoopy based cache coherency C2NSF-TCPP Curriculum Architecture (N) Directory based cache coherency A2NSF-TCPP Curriculum Architecture (N) Interconnect, topologies and routing A1NSF-TCPP Curriculum Architecture (C/K) Massive multi-threading and GPU K2Addition to the proposed curriculum NSF/IEEE-TCPP PDC Curriculum

5 Syllabus – Topical Coverage (2) TopicsBLOOM#HoursSource Parallel Programming SIMDK1NSF-TCPP Curriculum Programming (K) Shared MemoryK2NSF-TCPP Curriculum Programming (A) Message PassingK2NSF-TCPP Curriculum Programming (C) SynchronizationA1NSF-TCPP Curriculum Programming (A) Pthread LibraryK3NSF-TCPP Curriculum Programming (C) MPIK3Briefly mentioned in the proposed curriculum CUDAK3Briefly mentioned in the proposed curriculum NSF/IEEE-TCPP PDC Curriculum

6 Syllabus – Topical Coverage (3) TopicsBLOOM#HoursSource Parallel Algorithms AsymptoticsK1NSF-TCPP Curriculum Algorithm (C) Speedup, efficiency, cost, and scalability K2NSF-TCPP Curriculum Algorithm (C/K/N) Broadcast, reduction, scatter and gather C3NSF-TCPP Curriculum Algorithm (C/A) Matrix multiplicationK3NSF-TCPP Curriculum Algorithm (optional) Solving linear systemsK2NSF-TCPP Curriculum Algorithm (optional) Sorting (merge, bitonic, quick)K/C3NSF-TCPP Curriculum Algorithm (C) Graphs (minimum spanning tree) C2Addition to the proposed curriculum FFTK/C3Addition to the proposed curriculum NSF/IEEE-TCPP PDC Curriculum

7 Teaching Experience: Melding with Old Topics Students knew uni-thread execution mechanisms before coming to the course. The lectures analyzed the role of stacks and address spaces for uni-threads, and then extended the analysis to multiple threads. Students found it easy to understand the execution of multi-threaded programs, which further extends to other parallel computing models. NSF/IEEE-TCPP PDC Curriculum

8 Evaluation (1) Pedagogy 1.Gradual transition from sequential experience to parallel thinking 2.Explanation of abstract concepts via concrete examples (e.g. cache coherency) Student feedbacks –Item 1 is important for accepting parallel thinking –Item 2 is helpful for understanding of the materials NSF/IEEE-TCPP PDC Curriculum

9 Evaluation (2) Learning outcome –Evaluated through programming projects (pthread/MPI/CUDA intro, finding prime numbers, solving linear systems, sorting) and a final survey on current topics With the course, the students –can approach a problem with parallel thinking –understand the mechanisms of different parallel architectures –can design and analysis simple parallel algorithms –can implement and evaluate parallel algorithms using pthread, MPI, and CUDA NSF/IEEE-TCPP PDC Curriculum

10 Evaluation (3) Appropriateness of topical overages –Students find that The topics are comprehensive, and it is appreciated Some topics are complicated (cache coherency, CUDA programming, synchronization, etc) –Students suggest that Add some brief discussions on the current state-of-the- art of parallel computing Include more hands-on programming projects to help understand the concepts NSF/IEEE-TCPP PDC Curriculum

11 Deviation from the Proposed Curriculum and the Impact Additions to the curriculum –Thread execution mechanisms facilitates the sequential-to-parallel transition –Massive multi-threading and GPU helps cover the emerging field of GPU computing –Minimum spanning tree and FFT discusses two useful applications of parallel computing Expected level of understanding of the topics –Slight adjusted to fit the comprehensive introductory course NSF/IEEE-TCPP PDC Curriculum

12 Summary With the introductory course, the students achieved the expected learning outcome. The comprehensiveness and level of difficulty of the course are appreciated by the students. NSF/IEEE-TCPP PDC Curriculum