1. Introduction to Charm++ Machine Layer Gengbin Zheng Parallel Programming Lab 4/3/2002

2. Machine Layer Overview Startup and initialization; Low level communication CmiFreeSendFn, etc CmiSyncBroadcastAllFn(), etc SMP support Multiple worker threads and one comm thread; Thread creation and state initialization;

3. Supported Platforms Charm version OS Communication Compiler ------------------- --------- ------------------- -------------------- net-linux PC Linux UDP/Myrinet GNU compiler net-linux-ia64 IA64 Linux UDP/Myrinet GNU compiler net-axp Alpha UDP GNU compiler net-cygwin Win32/cygwin UDP GNU compiler net-win32 Win32 UDP MS Visual C++ net-sol Solaris UDP GNU compiler net-irix IRIX UDP GNU compiler net-hp HP-UX UDP GNU compiler net-ppc-darwin MacOS X UDP GNU C++ compiler mpi-linux PC Linux MPI GNU compiler mpi-linux-ia64 IA64 Linux MPI GNU compiler mpi-axp Alpha MPI GNU compiler mpi-linux-axp Alpha Linux MPI GNU compiler mpi-origin Origin2000 MPI C++ compiler mpi-sp AIX MPI AIX xlC Compiler origin2000 Origin2000 shared-mem SGI C++ compiler t3e Cray T3E shared-mem Cray C++ compiler

4. Communication Architecture Communication API (Send/Recv) Net MPI Shmem UDP (machine-eth.c) TCP (machine-tcp.c) Myrinet (machine-gm.c)

5. Machine Layer Files Layout Configuration files for each main platform: conv-mach.h conv-common.h conv-mach-opt.h from: smp, scyld, bluegene, gm, vmi, tcp … cc, cxx, kcc, cc64, acc, ecc … Implementations: common.net, common.mpi, common.shmem machine.c

6. How ‘build’ works build charm++ net-linux gm smp kcc bluegene build charm++ net-linux gm smp kcc bluegene Sort gm, smp and bluegene Mkdir net-linux-bluegene-gm-smp-kcc Cat conv-mach-[kcc|bluegene|gm|smp].h to conv-mach-opt.h Cat conv-mach-[kcc|bluegene|gm|smp].sh to conv-mach-opt.sh Gather files from common.net, etc (Makefile) Make charm++ under net-linux-bluegene-gm-smp-kcc/tmp

7. Parallel Program Startup Net version - nodelist Charmrun node compute node Rsh/ssh (IP, port) my node (IP, port) Broadcast all nodes (IP, port) compute node Rsh/ssh (IP, port)

8. Converse life ConverseInit Global variables initialization; Start worker threads; ConverseRunPE for each Charm PE Per thread initialization; Loop into scheduler: CsdScheduler()

9. Message formats Net version #define CMK_MSG_HEADER_BASIC { CmiUInt2 d0,d1,d2,d3,d4,d5,hdl,d7; } Dgram Headerlength handlerxhandler MPI version #define CMK_MSG_HEADER_BASIC { CmiUInt2 rank, root, hdl,xhdl,info,d3; } infohandlerxhandlerrankrootd3

10. SMP support MPI-smp as an example Create threads: CmiStartThreads Worker threads work cycle See code in mahcine-smp.c Communication thread work cycle See code in machine-smp.c