1. SMiLE Lehrstuhl für Betriebssysteme SCI Low-level programming Lab-Session Joachim Worringen Lehrstuhl für Betriebssysteme RWTH Aachen Martin Schulz Lehrstuhl für Rechnertechnik und Rechnerorganisation Technische Universität München

2. Joachim Worringen & Martin Schulz, SCI Summer School, Oct 1-3, Dublin 2 Outline  Lab Setup  Experiments with the SISCI API  Compile and run SISCI segment mapping code  Small modifications  Experiments with the SMI API  Compile and run “helloworld”  Run and modify “test_regions” & “pass_pointer”  Experiments with CML+  Compile and run ringtest code

3. SMiLE Lehrstuhl für Betriebssysteme SCI Low-level programming Lab-Session  SCI Summer School Lab Setup Experiments with the SISCI API Experiments with the SMI API Experiments with CML+

4. Joachim Worringen & Martin Schulz, SCI Summer School, Oct 1-3, Dublin 4 Setup for Low-Level Lab  Ringlets with 2 machines each  Node IDs: 4 & 8 or 12 &16  Make sure you know the correct Ids  Use the Dolphin tools @ /opt/dolphin/sbin/scitools  Experiments on 2 node clusters  Log in both of them and start shells  Don‘t run more than one application at a time  Conflicts may occur  Create subdirectories for application generations

5. SMiLE Lehrstuhl für Betriebssysteme SCI Low-level programming Lab-Session SCI Summer School Lab Setup  Experiments with the SISCI API Experiments with the SMI API Experiments with CML+

6. Joachim Worringen & Martin Schulz, SCI Summer School, Oct 1-3, Dublin 6 SISCI Setup  SISCI distribution  /opt/dolphin  You might want to install environment variables  $(SCI_ROOT)= /opt/dolphin/  $(SCI_LIB)= $(SCI_ROOT)/lib  $(SCI_INCLUDE)= $(SCI_ROOT)/include  Get sisci.c from ~schulz2k/sisci/sisci.c  We will go through it together  Compile code (use appropriate flags)  Experiment with it on two nodes

7. Joachim Worringen & Martin Schulz, SCI Summer School, Oct 1-3, Dublin 7 SISCI sample code  Basic SCI segment operation  Master: Create a local segment  Slave: Map segment from master  Step 1: Initialize and Open  Read local SCI ID  Step 2a, Master: Create Segment  Use SCI ID as identifier  Step 2b, Slave: Connect to Segment  Wait until connection is done

8. Joachim Worringen & Martin Schulz, SCI Summer School, Oct 1-3, Dublin 8 SISCI sample code (cont.)  Access to global array  Master writes entries 1-15  Slave writes entries 16-31  “Synchronization”  Master waits (within this time, slave must be done)  Afterwards, print the whole array  Clean-Up  Remove all mappings / segments before exiting  Similar to shmem.c from SISCI distribution

9. Joachim Worringen & Martin Schulz, SCI Summer School, Oct 1-3, Dublin 9 Task during the lab  Play with this simple mapping code  Compile with appropriate flags  -I /opt/dolphin/include for SISCI API  demolib.a and libsisci.a (order!)  Start code on Slave:./sisci  Start code on Master:./sisci  Modifications  Play with actual data transfer  Add real synchronization  E.g. flag synchronization

10. Joachim Worringen & Martin Schulz, SCI Summer School, Oct 1-3, Dublin 10 Useful information  SISCI header file with full prototypes  $(SCI_INCLUDE)/sisci_api.h  SISCI files required for compilation  Libraries: $(SCI_LIB)  libsisci.a  demolib.a  Include files: $(SCI_INCLUDE)  Application startup  Node A: sisci  Node B: sisci

11. SMiLE Lehrstuhl für Betriebssysteme SCI Low-level programming Lab-Session SCI Summer School Lab Setup Experiments with the SISCI API  Experiments with the SMI API Experiments with CML+

12. Joachim Worringen & Martin Schulz, SCI Summer School, Oct 1-3, Dublin 12 SMI Lab Session  The slides for the SMI lab session are available on the tutorial homepage as a separate file.

13. SMiLE Lehrstuhl für Betriebssysteme SCI Low-level programming Lab-Session SCI Summer School Lab Setup Experiments with the SISCI API Experiments with the SMI API  Experiments with CML+ Lehrstuhl für Betriebssysteme

14. Joachim Worringen & Martin Schulz, SCI Summer School, Oct 1-3, Dublin 14 CML+ Setup  CML+ distribution at  ~schulz2k/packages/cmlplus.tgz  Same as available from SMiLE server  Unpack the Tar-File  Creates cmplus subdirectory tar –xzf cmplus.tgz  Binary distribution of CML+ and Headerfile  Test-Code  Makefile

15. Joachim Worringen & Martin Schulz, SCI Summer School, Oct 1-3, Dublin 15 To get started...  Take a look at the header file  Functions and Constants exported  Very lean API  Messaging Functionality  cml_nbsend()  cml_blreceiver()  cml_nbmatch()  Interrupts  cml_trigger()

16. Joachim Worringen & Martin Schulz, SCI Summer School, Oct 1-3, Dublin 16 The sample code  Ring test  Establish a logical ring between all nodes  Send one message through ring  Functionality  Master: Create message, send it, and wait  Slave: Simply forward a message  Parameters  cmltest

17. Joachim Worringen & Martin Schulz, SCI Summer School, Oct 1-3, Dublin 17 Compiling and Running  Change in the cmlplus directory  Modify makefile  Variable SISCI_DIR – /opt/dolphin/  Compile by typing make  Start the code on all nodes  Local identifiers can be picked freely  The receiver-ids have to form a logical ring  Example: cmltest s 1 2 cmltest m 2 1

18. Joachim Worringen & Martin Schulz, SCI Summer School, Oct 1-3, Dublin 18 Further Experiments  Take a look at all parameters  Identify the parameters of send and receive  Modify some of them  Overlap communication  Don‘t use blocking receive  Check for arrived messages before receiving  While (cml_nbmatch()<=1) do_useful_work()  Use strides and different block sizes

19. SMiLE Lehrstuhl für Betriebssysteme SCI Low-level programming Closing remarks Joachim Worringen Lehrstuhl für Betriebssysteme RWTH Aachen Martin Schulz Lehrstuhl für Rechnertechnik und Rechnerorganisation Technische Universität München Lehrstuhl für Betriebssysteme

20. Joachim Worringen & Martin Schulz, SCI Summer School, Oct 1-3, Dublin 20 Closing remarks... Thank you for your attention !  Further Information  http://smile.in.tum.de/ http://smile.in.tum.de/  http://www.lfbs.rwth-aachen/joachim/users/ http://www.lfbs.rwth-aachen/joachim/users  Contact us via email  Joachim Worringen: joachim@lfbs.rwth-aachen.de@lfbs.rwth-aachen.de  Martin Schulz: schulzm@in.tum.deschulzm@in.tum.de Any further questions ?