1. Matlab for Engineers Compiling & Deploying Code Gari Clifford © 2010 - 2014 Centre for Doctoral Training in Healthcare Innovation Institute of Biomedical Engineering Department of Engineering Science University of Oxford Supported by the RCUK Digital Economy Programme grant number EP/G036861/1

2. Cluster Computing - Introduction The University (OSC) cluster info can be found here: http://www.osc.ox.ac.uk/content/services http://www.osc.ox.ac.uk/content/services Prof. Ventikos’ group runs the IBME cluster to which you can have access (contact him or Nick Hawker) It is a Linux-based machine cluster… Windows code (dll’s, exe’s.Net programs) will not work on this … so first try to compile your code under the virtual Linux OS on your computer. (Start VMWare) If you can’t get it to compile, then you will have no chance of running it on the cluster If you can compile it, then you have some hope …

3. The Linux environment The first step is that you need to be able to launch your simulations from the command line in Linux. I.e. you need to be at the shell command line in Linux, where you will type one command (Matlab or whatever it would be) and then it is running. So first - log into the cluster: ssh -X yourname@fbg-cluster.eng.ox.ac.uk yourname is a username specific to the cluster – see Nick Hawker Under windows, try using putty to connect securely using ssh -X parameter passes your x-window screen environment... Again, when in windows, you will need to run an x-window emulator if you need a GUI But you really don’t need to. E.g. In Matlab, you can run: matlab –nodisplay –nojvm This will give you a terminal only interaction and no java virtual machine Temporary network disconnections or local system reboots / crashes will not affect the remote job! You can still ‘print’ figures in a batch job!

4. Good (nice) behaviour Using nice If you preface any command with nice -N where N is an integer, then you deprecate the priority of the task so that other critical tasks can take priority. On the cluster it will manage this for you, but on your own machine you will need to manage this yourself – don’t load up your machine so that you can’t even log in yourself! E.g. nice -9 matlab –nodisplay will deprecate the priority that this particular Matlab job has over other processes on the computer.

5. Good (clever) behaviour Using ps, grep & kill Don’t leave jobs running that are no longer needed, (such as a script with the incorrect parameters, or one that is in an infinite loop). kill will end a task. ps will list all the tasks grep witll filter the list for keywords E.g.: ps –ef | grep matlab This lists all the tasks with their PIDs. Locate the PID then kill the task using kill -9 123456 where 123456 is the PID top is another useful command and lists the jobs which are using up the most resources

6. Good (clever) behaviour Using screen Screen creates a virtual ‘shell’... From which you can detach, and leave something still running – try it now under Linux: Type screen at a command prompt Start matlab: nice -9 matlab –nodisplay –nojvm Start off a long loop: for(i=1:10^5);fprintf(‘%i ‘,i);pause(1);end Type :CTL-A + CTL+D.... this detacheds you from the virtual shell You can log out and back in again if you like Type screen –r at a command prompt on the machine on which you ran the above Matlab loop. This reconnects you to the terminal (Don’t forget to kill this job by using ‘kill’ or reconnecting to the virtual shell and typing CTL+C You can start multiple screens – when you type screen –r it will give you a list of all screens – just append the number of the relevant screen to your command to get the right virtual shell. E.g.: screen –r 192342

7. Matlab on the cluster Use a secure shell to log on to the cluster.. E.g. ssh -X gari@fbg-cluster.eng.ox.ac.uk If you want to use Matlab and if you haven't done already, create your compilation environment using mbuild -setup Now try a test - create your function, e.g. Test2.m : function y=test2(x) if isdeployed() x=str2num(x); end y=x.^2+3; disp(y); end Then use mcc at the Matlab command line to compile your code... e.g. type: mcc -m test2.m This creates an executable (test in Linux and test.exe in Windows) Now create a shell script to run this executable, say called run_test.sh

8. mcc restrictions There are some limitations and restrictions using the MATLAB Compiler. Therefore mcc (Version 3.0) cannot compile: M-files containing scripts. M-files that use objects. Calls to the MATLAB Java interface. M-files that use the MATLAB commands input or eval to manipulate workspace variables. Input and eval calls that do not use workspace variables will compile and execute properly. M-files that use the MATLAB command exist with 2(!) input arguments. M-files that load text files. Data exchange via files is best done using binary / MAT- files. This ensures for example that files created under Linux would be readable under Windows (look out of lsb/msb byte swapping issues). As a consequence of some bugs and known unresolved problems there exist some workarounds to cope with this problems:

9. mcc restrictions - workarounds As a consequence of some bugs and known unresolved problems there exist some workarounds to cope with this problems: Sometimes the mcc doesn’t find, for some reasons, M-files of external toolboxes, even though the correct path was set within a -I statement. Therefore these M-files must be copied to the same directory as the main M-files that need this helper functions. Many functions of the Image Processing Toolbox, i.e. those functions for morphological operations, are not found by the compiler. To enable compiling this functions, these M- files must be copied into the working directory. Functions that use the MATLAB command imlincomb (linear combination of images), i.e. many functions of the Image Processing Toolbox, would return a compiler error when generating C++ stand-alone applications. Therefore functions that use functionality of the Image Processing Toolbox can not be translated to C++ code in general. As a workaround, generate C stand-alone applications instead of C++ stand- alone applications.

10. Matlab on the cluster The run the code at the fbg-cluster command line: Guest00@fbg-cluster:~> sh run_test.sh /opt/matlab 2,5,6,7,8,9,9999 ------------------------------------------ Setting up environment variables --- LD_LIBRARY_PATH is.:/opt/matlab/runtime/glnxa64:/opt/matlab/bin/glnxa64:/opt/matlab/sys/os/glnxa64:/ opt/matlab/sys/java/jre/glnxa64/jre/lib/amd64/native_threads:/opt/matlab/sys/ja va/jre/glnxa64/jre/lib/amd64/server:/opt/matlab/sys/java/jre/glnxa64/jre/lib/amd6 4/client:/opt/matlab/sys/java/jre/glnxa64/jre/lib/amd64 Columns 1 through 6 7 28 39 52 67 84 Column 7 99980004 (This is the output you would expect.)

11. Matlab on the cluster Note also run_test.sh is the file you produce from compiling under Matlab, which requires two options: - the path to Matlab: /opt/matlab - and the arguments of the Matlab function: ( a vector == 2,5,6,7,8,9,9999 in this case). You can then run the code on the cluster using: qsub testMat.sh Where testMat.sh is a script I made up that looked like this: #!/bin/bash #$ -N -test-job-by-smoot #$ -cwd -V./run_test.sh /opt/matlab 2,5,6,7,8,9,9 > tmp.dat Note test-job-by-smoot is the name of the test that hit the queue (type qstat at the command line to find out if your job is running). Change this to something else that identifies your script.

12. Matlab on the cluster more testMat.sh #!/bin/bash #$ -N -test-job-by-smoot #$ -cwd -V./run_test.sh /opt/matlab 2,5,6,7,8,9,9 > tmp.dat The > tmp.dat command is a Linux shell redirect (it dumps the terminal output of that script to a file, which in this case is called tmp.dat)

13. More help …. A good guide / reference on Linux is the ‘Linux in a Nutshell’ book. Do read this. You can also Google any command and find a good description of it. Also – use the man pages... Almost every command has a manual page which describes the function and syntax of the command (similar to Matlab’s ‘help’ command). E.g.: > man grep Other useful functions: emacs - a powerful editor (can work within a shell without X-windows) more – allows you to view the contents of a file without invoking an editor locate – helps you find a file on the system ls – lists files in a director, or recursively pwd - ‘print working director’ (where am I?)

14. Cluster Computing – further info More info on how to submit jobs to the cluster and how to manage your code in a responsible manner can be found here: http://code.google.com/p/fbg-cluster/wiki/ManualIntro Note the links to Linux command line tools. Don't be scared - it's easy :) Remember – you need a separate user account for this machine - please request your own individual accounts from Nick Hawker or Prof Ventikos. More info on the cluster can be found here: http://code.google.com/p/fbg-cluster/ The section you most likely need is the one on submitting serial jobs.

15. Appendix: README.TXT under Windows MATLAB Compiler 1. Prerequisites for Deployment Verify the MATLAB Compiler Runtime (MCR) is installed and ensure you have installed a compatible version (e.g. 7.14). If the MCR is not installed, run MCRInstaller, located in: *\toolbox\compiler\deploy\win32\MCRInstaller.exe For more information on the MCR Installer, see the MATLAB Compiler documentation. NOTE: Prior to running your deployed application, verify the user locale and system locales have the same value on the Microsoft Windows platform. If these values are not the same, you might see an I18n warning message and garbled text.

16. README.TXT 2. Files to Deploy and Package Files to package for Standalone ================================ -test.exe -MCRInstaller.exe -include when building component by selecting "include MCR" option in deploytool -This readme file 3. Definitions MCR - MATLAB Compiler uses the MATLAB Compiler Runtime (MCR), which is a standalone set of shared libraries that enable the execution of M-files. The MCR provides complete support for all features of MATLAB without the MATLAB GUI. When you package and distribute an application to users, you include supporting files generated by the builder as well as the MATLAB Compiler Runtime (MCR). If necessary, run MCRInstaller to install version 7.14 of MCR. For more information about the MCR, see the MATLAB Compiler documentation.

17. README.TXT NOTE: is the directory where MCR is installed on the target machine. 4. Appendix A. On the target machine, add the MCR directory to the system path specified by the target system's environment variable. i. Locate the name of the environment variable to set, using the table below: Operating System Environment Variable ================ ==================== Windows PATH ii. Set the path by doing one of the following: NOTE: is the directory where MCR is installed on the target machine. On Windows systems:. Add the MCR directory to the environment variable by opening a command prompt and issuing the DOS command: set PATH= \v714\runtime\win32;%PATH%

18. README.TXT Alternately, for Windows, add the following pathname: \v714\runtime\win32 to the PATH environment variable, by doing the following: 1. Select the My Computer icon on your desktop. 2. Right-click the icon and select Properties from the menu. 3. Select the Advanced tab. 4. Click Environment Variables. NOTE: On Windows, the environment variable syntax utilizes backslashes (\), delimited by semi-colons (;).