1. Parallelization Technology v 0.2 Parallel-Developers Discussion 6/29/11

2. Origins Indebted to HPCWire article by Michael Wolfe http://www.hpcwire.com/hpcwire/2011- 03- 08/compilers_and_more_programming_at _exascale.html http://www.hpcwire.com/hpcwire/2011- 03- 08/compilers_and_more_programming_at _exascale.html

3. Parallelization Tech Instruction-level Thread level Process level Language level Design Patterns Numerical Libraries Virtualization

4. Instruction-level Vectorization: 3DNow!, SSE, AVX Instruction (VLIW / FMA) Pipelining Requires tuned libraries, assembly language, or tunable code Potential portability constraints Array Building Blocks

5. Libraries POSIX Threads (pthreads) OpenMP TBB MPI Hadoop

6. Language UPC / CoArray Fortran X10 / Chapel Cilk OpenCL / CUDA Haskell / F# Matlab / Mathematica Java / Go / Erlang / Scala

7. Functional-ish Languages Thread – functions parallelize well due to language assumptions as well as compiler Powerful type system and dimensional analysis (free at runtime due to compile- time checking) OCaml/ Functory Erlang Haskell Scala F#

8. Matlab / Mathematica Low barrier to entry Potentially significant overhead for environment Matlab: – Distributed Computing Toolkit on the Client, Distributed Computing Engine on the Worker – Can run independent workers or shared-workload – Matlab Compiler for stupid parallelization Mathematica – Native support, GridMathematica option – ParallelDo