Parallel and Distributed Computing Early ( and Often) Steven Bogaerts (speaker), Kyle Burke, Brian Shelburne Department of Mathematics & Computer Science Wittenberg University Springfield, OH Melissa Smith Department of Electrical & Computer Engineering Clemson University Clemson, SC Eric Stahlberg CCR Bioinformatics Core National Cancer Institute Bethesda, MD First NSF/TCPP Workshop on Parallel and Distributed Computing Education (EduPar-11) May 16 th, 2011 NSF grant CCF , SHF:Small:RUI:Collaborative Research: Accelerators to Applications – Supercharging the Undergraduate Computer Science Curriculum PIs: Eric Stahlberg, Melissa Smith, Steven Bogaerts

PDC Concepts Throughout the CS Curriculum Steven Bogaerts Algorithms Programming Languages Computational Modeling CS2 CS1 CS0 or Computer Literacy

Computer Literacy Steven Bogaerts Scratch

Computer Literacy Steven Bogaerts

Computer Literacy – PDC Overview Steven Bogaerts Concepts The world is “obviously” parallel. Big-picture descriptions of some applications. Terminology, history Physical activities Low-level: binary adder Higher-level: card sorting Communication Shared memory vs. message passing

Computer Literacy – Curriculum Initiative Connections Steven Bogaerts Cross-Cutting Why and what is PDC? Concurrency topics : Non-determinism Power Current/Hot/Advanced Topics Cluster Cloud/Grid Algorithms : Algorithmic Problems : Broadcast Asynchrony Programming : Semantics and Correctness Issues : Tasks and Threads Architecture : Classes : Multicore

CS1 Steven Bogaerts Parallelism/Concurrency can be used as a medium for teaching core computer science topics.

Analogy: Object-Oriented Programming Steven Bogaerts Years ago:  OOP was a new paradigm  Should OOP be taught in introductory computer science courses?  Should OOP topics be “sprinkled” into existing courses?  If so, which topics should be taken out to make room? Now:  Not just an advanced OOP course  CS1  Data structures  Graphics  … Topics taught through the medium of OO

Goals (CS1 and Beyond) Steven Bogaerts To prepare students for the “impending ordinariness” (Doug Lea) of parallelism and concurrency. Minimize the amount of material that must be removed to “make room.” Prevent the necessity of adding and staffing another course.

CS1 Steven Bogaerts Introduction similar to Literacy course Terminology, high-level concepts, physical activities Basic use of Python multiprocessing module Flexible application to many kinds of problems Minimal additional syntax

Python Multiprocessing Module Steven Bogaerts

Python Multiprocessing Module Steven Bogaerts lockVar = Lock() lockVar.acquire() … lockVar.release() procVar.join()

Parallelism/Concurrency as a Medium Steven Bogaerts Classes and objects Parameter passing Modularity and abstraction “Sandbox” for exploration of other concepts: Computing terms of the quadratic formula Searching or sorting a list Pattern matching Clustering Simulations of parallel processes in nature … Key idea: “Do what you usually do, just do (some of) it in parallel.”

Computational Modeling – Motivations Computational models are increasing in complexity Data available for constructing computational models is rapidly growing Parallel computing platforms are readily available Extreme Scale computing (exascale) platforms require new algorithms built from the ground up using parallel computing concepts Industry needs better performing models and parallel and accelerated computing are the only real options

Computational Modeling – Enhancement Objectives Primary Provide examples of easy parallel computing for models (embarrassingly parallel) Provide introduction to use of 2D arrays in parallel computing environments Keep it straightforward and accessible Secondary Provide introduction to contention and overhead

Computational Modeling – Approach Use Computational Models and Methods course 200 level (sophomore level) Pre-requisite: Calculus, CS1 Implementation using Mathematica Parallel Computing Toolkit Mathematica is proven effective for computational models and methods Functional programming Ready with built-in parallel programming primitives starting with version 7 Machine-independent Select examples and provide a lab experience using primitives

Computational Modeling – Trial Lessons Learned Students can master easy primitives Students can even become comfortable with parallelizing matrix access by index Mathematica more difficult for high-level parallelism “Empty Kernel” starting point requires significant distribution of data and functions to start parallelism

Computational Modeling – Simple Parallel Examples Data reduction (simple) Sum[a,b,c,…] ParallelSum[a,b,c,…] Product[a,b,c,…] ParallelProduct[a,b,c,…] Matrices (data sharing) Shared[{a,b,c}] Other new fun options to try ParallelPlot3D[] ParallelDensityPlot[] ParallelContourPlot[]

Programming Language Courses Exploration of programming paradigms Include HPC languages! Language: Supports multi-threading Alleviates common concurrency issues Simplifies vital parallel patterns Steven Bogaerts

Computer Organization Courses Already implicitly include parallelism! Combinatorial circuits Pipelines, super-scalar architectures I/O with interrupts and DMA Make the inherent parallelism more explicit Consider VHDL Example: Carry look-ahead adder (parallel propagation of carrys) vs. ripple carry adder Steven Bogaerts

Conclusion Early work in applying parallelism/concurrency as a medium for learning computer science. Developments in: CS1 Programming languages Computer organization Steven Bogaerts