1. 1 © 2012 The MathWorks, Inc. Parallel computing with MATLAB

2. 2 Going Beyond Serial MATLAB Applications MATLAB Desktop (Client) Worker

3. 3 Programming Parallel Applications (CPU)  Built-in support with toolboxes Ease of Use Greater Control

4. 4 Example: Optimizing Cell Tower Position Built-in parallel support  With Parallel Computing Toolbox use built-in parallel algorithms in Optimization Toolbox  Run optimization in parallel  Use pool of MATLAB workers

5. 5 Tools Providing Parallel Computing Support  Optimization Toolbox  Global Optimization Toolbox  Statistics Toolbox  Signal Processing Toolbox  Neural Network Toolbox  Image Processing Toolbox  … B LOCKSETS Directly leverage functions in Parallel Computing Toolbox www.mathworks.com/builtin-parallel-support

6. 6 Agenda  Task parallel applications  GPU acceleration  Data parallel applications  Using clusters and grids

7. 7  Ideal problem for parallel computing  No dependencies or communications between tasks  Examples: parameter sweeps, Monte Carlo simulations Independent Tasks or Iterations Time blogs.mathworks.com/loren/2009/10/02/using-parfor-loops-getting-up-and-running/

8. 8 Example: Parameter Sweep of ODEs Parallel for-loops  Parameter sweep of ODE system –Damped spring oscillator –Sweep through different values of damping and stiffness –Record peak value for each simulation  Convert for to parfor  Use pool of MATLAB workers

9. 9 The Mechanics of parfor Loops Pool of MATLAB Workers a = zeros(10, 1) parfor i = 1:10 a(i) = i; end a a(i) = i; Worker 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9

10. Agenda  Task parallel applications  GPU acceleration  Data parallel applications  Using clusters and grids

11. 11 What is a Graphics Processing Unit (GPU)  Originally for graphics acceleration, now also used for scientific calculations  Massively parallel array of integer and floating point processors –Typically hundreds of processors per card –GPU cores complement CPU cores  Dedicated high-speed memory * Parallel Computing Toolbox requires NVIDIA GPUs with Compute Capability 1.3 or higher, including NVIDIA Tesla 20-series products. See a complete listing at www.nvidia.com/object/cuda_gpus.html www.nvidia.com/object/cuda_gpus.html

12. 12 Core 1 Core 3Core 4 Core 2 Cache Performance Gain with More Hardware Using More Cores (CPUs) Using GPUs Device Memory GPU cores Device Memory

13. 13 Example: Mandelbrot set  The color of each pixel is the result of hundreds or thousands or iterations  Each pixel is independent of the other pixels  Hundres of thousands of pixels

14. 14 Real-world performance increase Solving a wave equation Intel Xeon Processor X5650, NVIDIA Tesla C2050 GPU Grid Size CPU (s) GPU (s) Speedup 64 x 640.10040.35530.28 128 x 1280.19310.33680.57 256 x 2560.58880.42171.4 512 x 5122.81630.82433.4 1024 x 102413.47972.49795.4 2048 x 204874.99049.95677.5

15. 15 Programming Parallel Applications (GPU)  Built-in support with toolboxes  Simple programming constructs: gpuArray, gather  Advanced programming constructs: arrayfun, spmd  Interface for experts: CUDAKernel, MEX support Ease of Use Greater Control www.mathworks.com/help/distcomp/run-cuda-or-ptx-code-on-gpu www.mathworks.com/help/distcomp/run-mex-functions-containing-cuda-code

16. 16 Agenda  Task parallel applications  GPU acceleration  Data parallel applications  Using clusters and grids

17. 17 Big data: Distributed Arrays T OOLBOXES B LOCKSETS Distributed Array Lives on the Cluster Remotely Manipulate Array from Desktop 11 26 41 12 27 42 13 28 43 14 29 44 15 30 45 16 31 46 17 32 47 17 33 48 19 34 49 20 35 50 21 36 51 22 37 52

18. 18 Big Data: Distributed Arrays Pool of MATLAB Workers y = distributed(rand(10)); Column 1:3 of y Worker Column 7:8 of y Column 9:10 of y Column 4:6 of y

19. 19 Demo: Approximation of π

20. 20 Programming Parallel Applications (CPU)  Built-in support with toolboxes  Simple programming constructs: parfor, batch, distributed  Advanced programming constructs: createJob, labSend, spmd Ease of Use Greater Control

21. 21 Agenda  Task parallel applications  GPU acceleration  Data parallel applications  Using clusters and grids

22. 22 Working on C3SE

23. 23 Apply for a project with SNIC

24. 24 © 2012 The MathWorks, Inc.