Intel Xeon Phi Training - Introduction Rob Allan Technology Support Manager The Hartree Centre

The Hartree Centre, based at STFC's Daresbury Laboratory, was founded in 2012 principally to stimulate the uptake of HPC and to support UK industry. We are working with a number of high profile partners, including IBM, Intel, Unilever, Rolls-Royce, etc. We have a large variety of state of the art computation, data analysis and “cognitive” systems. All access is “project based”, you need to have a project set up first. This can be in a number of ways: Pay for access Be part of a commercial project Be part of an academic “call for proposals” Be part of a development project Enquiries: or

The Iden Cluster All our computer systems are named after obsolete British car manufacturers. Iden: 84 nodes of Intel 24 core IvyBridge 64 GB memory per node (host processor) 42 nodes with attached Xeon Phi 5110P (60 cores) 8 GB memory per Xeon Phi Mellanox FRD InfiniBand between hosts Iden and Napier are attached to the Calcott login and management cluster. They share common login nodes and LSF batch queue system. For information, see:

1) your course id is of the form xxxyy-dxp75 2) you need to look up your password in SAFE 3) log in using the course id and password – easy! – Tip: if you are asked to change your password, just put it in backwards 4) open a terminal (right click in background and select "Open in Terminal". Now initialise the environment for Intel software source ~/../shared/intel/parallel_studio_xe_ /psxevars.sh intel64 Logging onto the Workstations

Logging onto Iden 1) From the workstations open a terminal (right click in background and select "Open in Terminal". 2) ssh into one of the Phase-2 login nodes (generic phase2.wonder.hartree.stfc.ac.uk). IP Address DNS Name phase2-login1.wonder.hartree.stfc.ac.uk phase2-login2.wonder.hartree.stfc.ac.uk phase2-login3.wonder.hartree.stfc.ac.uk phase2-login4.wonder.hartree.stfc.ac.uk Now initialise the environment for Intel software for the Xeon Phi cp ~/../shared/bashconfig ~/. source ~/bashconfig module load intel/ _mic

Accessing a Xeon Phi There are 4 steps to physically access a Xeon Phi on the system: 1) SSH onto a login node (as above) 2) get interactive access to a Phi host node (see next) 3) set up a key pair (see next) 4) SSH to the MIC co-processor The Xeon Phis are attached to 42 of the nodes of Iden in chassis "1A" or "1C". Host nodes have alternate numbers idb1a01 to idb1a41 or idb1c02 to idb1c42.

Interactive Shell on a Host Node An interactive shell on an available host node can be started as follows: bsub -q phiq -U -Is /bin/bash source /etc/profile.d/modules.sh After a few seconds this should return a command prompt. If this does not happen it may be that the node is already in use (access is exclusive). To check, use a command like this bjobs - uall | grep idb1 which will identify any users of chassis 1a or 1c using the phiq. Note that the second line is sourcing the module environment which will be needed later.

To SSH into a MIC co-processor you will need a key pair if you do not already have one. # generate key pair and add it to authorized_keys cd ~/.ssh ssh-keygen -f mic.key -t rsa -b 1024 ^RTN cat mic.key.pub >> authorized_keys # add the IdentityFile line to ~/.ssh/config echo "IdentityFile ~/.ssh/mic.key" | cat >> config # ensure its only you that can read it chmod 600 config # now try logging onto the MIC ssh `hostname`-mic0 SSH onto the Phi

Copying Files The workstations are connected to a file server and have a shared file system on /gpfs/home/training. You can log onto any workstation and your home directory will be /gpfs/home/HCT00032/dxp75/xxxyy-dxp75. On the iDataPlex cluster your home file system is also /gpfs/home/HCT00032/dxp75/xxxyy-dxp75 although this is not the same GPFS system, and files will have to be copied between the two. To copy files from the workstation to Iden, first open a local terminal on the workstation by using a right mouse click in the background and selecting “open in terminal”, then do the following. cd ~ scp -r * phase2.hartree.stfc.ac.uk:.