1. MATLAB Parallel Computing Toolbox A.Hosseini Email: s.a.hosseini86@gmail.com Course : Professional Architecture

2. Of 372 Product Overview solve computationally and data intensive problems using multicore processors, GPUs, and computer clusters. The toolbox provides twelve workers to execute applications locally without changing the code MDCS software allows you to run as many MATLAB workers on a remote cluster

3. Of 373 Key Problems Addressed by Parallel Computing Running Parallel for-Loops (parfor) Executing Batch Jobs in Parallel Partitioning Large Data Sets

4. Of 374 Running Parallel for-Loops (parfor) Many applications involve multiple repetitive segments of code. Often you can use for-loops to solve these cases. The ability to execute code in parallel, on one computer or on a cluster of computers, can significantly improve performance in many cases Many iterations Long iterations

5. Of 375 Executing Batch Jobs in Parallel When working interactively in a MATLAB session, you can offload work to a MATLAB worker session to run as a batch job. MATLAB session is not blocked. The MATLAB worker can run either on the same machine as the client, or if using MATLAB Distributed Computing Server, on a remote cluster machine.

6. Of 376 Partitioning Large Data Sets If you have an array that is too large for your computer's memory, it cannot be easily handled in a single MATLAB session. Parallel Computing Toolbox software allows you to distribute that array among multiple MATLAB workers, so that each worker contains only a part of the array.

7. Of 377 Introduction to Parallel Solutions Interactively Run a Loop in Parallel Run a Batch Job Run a Batch Parallel Loop Distributing Arrays and Running SPMD

8. Of 378 Interactively Run a Loop in Parallel This section shows how to modify a simple for-loop so that it runs in parallel. for i=1:1024 A(i) = sin(i*2*pi/1024); End plot(A) for i=1:1024 A(i) = sin(i*2*pi/1024); End plot(A) matlabpool open local 3 parfor i=1:1024 A(i) = sin(i*2*pi/1024); End plot(A) matlabpool close matlabpool open local 3 parfor i=1:1024 A(i) = sin(i*2*pi/1024); End plot(A) matlabpool close

9. Of 379 Run a Batch Job To offload work from your MATLAB session to another session, you can use the batch command. This example uses the for-loop from the last section inside a script. edit mywave for i=1:1024 A(i) = sin(i*2*pi/1024); End job = batch('mywave') wait(job) load(job, 'A') plot(A) destroy(job) edit mywave for i=1:1024 A(i) = sin(i*2*pi/1024); End job = batch('mywave') wait(job) load(job, 'A') plot(A) destroy(job)

10. Of 3710 Run a Batch Parallel Loop combine the abilities to offload a job and run a parallel loop

11. Of 3711 Distributing Arrays and Running SPMD Distributed Arrays (see page 9 on document) Single Program Multiple Data(SPMD) (see page 34 on document)

12. Of 3712 Interactive Parallel Computation with pmode(P45) pmode lets you work interactively with a parallel job running simultaneously on several labs. provides a desktop with a display for each lab running the job. When you exit your pmode session, its job is effectively destroyed, and all information and data on the labs is lost. Starting another pmode session always begins from a clean state.

13. Of 3713 Run Parallel Jobs Interactively Using pmode

14. Of 3714 Profiling Parallel Code The parallel profiler provides an extension of the profile command and the profile viewer specifically for parallel jobs, to enable you to see how much time each lab spends evaluating each function and how much time communicating or waiting for communications with the other labs.profile mpiprofile on Execution time of each function on each lab Execution time of each line of code in each function Amount of data transferred between each lab Amount of time each lab spends waiting for communications

15. Of 3715 Viewing Parallel Profile Data P>> R1 = rand(16, codistributor()) P>> R2 = rand(16, codistributor()) P>> mpiprofile on P>> P = R1*R2 P>> mpiprofile off P>> mpiprofile viewer

16. Of 3716 The function summary report Column HeaderDescription CallsHow many times the function was called on this lab Total TimeThe total amount of time this lab spent executing this function Self TimeThe time this lab spent inside this function, not within children or subfunctions Total Comm TimeThe total time this lab spent transferring data with other labs, including waiting time to receive data Self Comm Waiting TimeThe time this lab spent during this function waiting to receive data from other labs Total Interlab DataThe amount of data transferred to and from this lab for this function Computation Time Ratio The ratio of time spent in computation for this function vs. total time (which includes communication time) for this function Total Time PlotBar graph showing relative size of Self Time, Self Comm Waiting Time, and Total Time for this function on this lab

17. Of 3717 Parallel computing with MATLAB Parallel Computing Toolbox (formerly Distributed Computing Toolbox) should be installed on the computer where you write your applications. MATLAB Distributed Computing Server (formerly MATLAB Distributed Computing Engine) should be installed on each computer of the cluster that performs the computation.

18. Of 3718 job A job is some large operation that you need to perform in your MATLAB session. A job is broken down into segments called tasks. You decide how best to divide your job into tasks. You could divide your job into identical tasks, but tasks do not have to be identical.

19. Of 3719 client this is on the machine where you program MATLAB. The client uses Parallel Computing Toolbox software to perform the definition of jobs and tasks. MATLAB Distributed Computing Server software is the product that performs the execution of your job by evaluating each of its tasks and returning the result to your client session.

20. Of 3720 job manager the part of the engine that coordinates the execution of jobs and the evaluation of their tasks. The job manager distributes the tasks for evaluation to the server's individual MATLAB sessions called workers.

21. Of 3721 Basic Parallel Computing Configuration

22. Of 3722 Life Cycle of a Job

23. Of 3723 Job StageDescription Pending You a job on the scheduler with the createJob function in your client session of Parallel Computing Toolbox software. The job's first state is pending. This is when you define the job by adding tasks to it. createcreateJob Queued When you execute the submit function on a job, the scheduler places the job in the queue, and the job's state is queued. The scheduler executes jobs in the queue in the sequence in which they are submitted, all jobs moving up the queue as the jobs before them are finished. You can change the order of the jobs in the queue with the promote and demote functions.submitpromote demote Running When a job reaches the top of the queue, the scheduler distributes the job's tasks to worker sessions for evaluation. The job's state is running. If more workers are available than necessary for a job's tasks, the scheduler begins executing the next job. In this way, there can be more than one job running at a time. Finished When all of a job's tasks have been evaluated, a job is moved to the finished state. At this time, you can retrieve the results from all the tasks in the job with the function getAllOutputArguments.getAllOutputArguments Failed When using a third-party scheduler, a job might fail if the scheduler encounters an error when attempting to execute its commands or access necessary files. DestroyedWhen a job's data has been removed from its data location or from the job manager, the state of the job in the client is destroyed. This state is available only as long as the job object remains in the client.

24. Of 3724 Parallel Configurations for Cluster Access You create and modify configurations through the Configurations Manager. You access the Configurations Manager using the Parallel pull-down menu on the MATLAB desktop. Select Parallel > Manage Configurations to open the Configurations Manger.

25. Of 3725 The first time you open the Configurations Manager, it lists only one configuration called local, which at first is the default configuration and has only default settings.

26. Of 3726 Creating and Modifying User Configurations In the Configurations Manager, select New > jobmanager. This specifies that you want a new configuration whose type of scheduler is a job manager

27. Of 3727 Creating and Modifying User Configurations Enter a configuration name MyJMconfig1 and a description as shown in the following figure. In the Scheduler tab, enter the host name for the machine on which the job manager is running and the name of the job manager.

28. Of 3728 Creating and Modifying User Configurations In the Jobs tab, enter the maximum and minimum number of workers. This specifies that for jobs using this configuration, they require at least four workers and use no more than four workers. Therefore, the job runs on exactly four workers, even if it has to wait until four workers are available before starting.

29. Of 3729 Creating and Modifying User Configurations To create a similar configuration with just a few differences, you can duplicate an existing configuration and modify only the parts you need to change: In the Configurations Manager, right-click the configuration MyJMconfig1 in the list and select Duplicate.

30. Of 3730 Exporting and Importing Configurations To export a parallel configuration: In the Configurations Manager, select (highlight) the configuration you want to export. Click File > Export. Configurations saved in this way can then be imported by other MATLAB software users: In the Configuration Manager, click File > Import.

31. Of 3731 Validate Configurations Open the Configurations Manager by selecting on the desktop Parallel > Manage Configurations. In the Configurations Manager, click the name of the configuration you want to test in the list of those available. Click Start Validation.

32. Of 3732 The configuration listing displays the overall validation result for each configuration. The following figure shows overall validation results for one configuration that passed and one that failed. The selected configuration is the one that failed.

33. Of 3733 How to submit program for run? Create a Scheduler Object Create a Job Create Tasks Submit a Job to the Scheduler Retrieve the Job's Results

34. Of 3734 Creating Jobs sched = findResource('scheduler','type','local'); (create an object in your local MATLAB session representing the local scheduler.) job1 = createJob(sched) (This statement creates a job in the scheduler's data location) get(job1) ( see all the properties of this job object) Sched (The scheduler's display now indicates the existence of your job) createTask(job1, @rand, 1, {{3,3} {3,3} {3,3} {3,3} {3,3}}); (Tasks define the functions to be evaluated by the workers during the running of the job) get(job1,'Tasks') submit(job1) (To run your job and have its tasks evaluated) waitForState(job1) results = getAllOutputArguments(job1); (to retrieve the results from all the tasks in the job) sched job1 get(job1,'Tasks')

35. Of 3735 Books

36. Of 3736 Books(Cont.)

37. Of 3737 Thanks for your attention Any Question?