1. HackLatt 20081 MILC Code Basics Carleton DeTar First presented at Edinburgh EPCC HackLatt 2008 Updated 2013

2. HackLatt 20082 MILC Code Capabilities Molecular dynamics evolution –Staggered fermion actions (Asqtad, Fat7, HISQ, etc) –Clover fermion action –Pure gauge –Schroedinger functional Hadron spectroscopy –Staggered mesons and baryons –Clover mesons and baryons –Mixed staggered/clover mesons –Static/light spectroscopy –Quarkonium spectroscopy (S and P-wave) Current matrix elements –Leptonic decay (fpi, fB, fD) –Semileptonic decay (heavy-light) Miscellaneous –Topological charge –Dirac matrix eigenvectors and eigenvalues –Nonperturbative renormalization of currents

3. HackLatt 20083 Two views of the code. Powerful and capable.

4. HackLatt 20084 Mathias Gruenewald: Temptation of St Anthony (1515) Or a path to confusion and misery.

5. HackLatt 20085 Supported File Formats Gauge configuration file formats –MILC, SciDAC (ILDG), NERSC, Fermilab Dirac propagator file formats –USQCD, Fermilab Staggered propagator file formats –USQCD, MILC, Fermilab

6. HackLatt 20086 Supported SciDAC C-Coded Packages QIO (I/O) QMP (Message passing) QLA (linear algebra – single processor) QDP/C (linear algebra – data parallel) QOPQDP (“Level 3” optimized) (More in the next session) http://usqcd.jlab.org/usqcd-software/

7. HackLatt 20087 Precision Global single or double precision Mixed precision in some applications Portability Any scalar machine Any MPP machine with MPI GPU, Intel Xeon Phi (somewhat)

8. HackLatt 20088 MILC Code Organization Application directories –With compilation targets Library directory –Linear algebra routines Shared procedures (“generic”) directories –Shared across applications

9. HackLatt 20089 MILC Code Organization Application directories: examples –cd ks_imp_dyn (application) make su3_rmd (target) (Asqtad R algorithm) make su3_spectrum (another target) (staggered spectroscopy) –cd ks_imp_rhmc (application) make su3_rhmc (Asqtad RHMC algorithm) make su3_rhmc_hisq (target) (HISQ algorithm) –cd clover_invert2 (application) make su3_clov (clover spectroscopy, etc.)

10. HackLatt 200810 MILC Code Organization Shared procedures directories: examples –generic (common to all applications) –generic_ks (common to staggered fermion applications) –generic_wilson (common to clover and Wilson fermion apps)

11. HackLatt 200811 Building the MILC Code Download source http://www.physics.utah.edu/~detar/milc_qcd.html http://www.physics.utah.edu/~detar/milc_qcd.html Unpack Configure Build Check

12. HackLatt 200812 Building the MILC Code Unpack –tar –xvzf milc_qcd-7.7.10-a7.tar.gz

13. HackLatt 200813 Building the MILC Code Configure (crude old fashioned!) –Copy default Makefile to application directory cd ks_imp_rhmc cp../Makefile. –Edit (example in next slide) Makefile../libraries/Make_vanilla../include/config.h

14. HackLatt 200814 #---------------------------------------------------------------------- # 2. Architecture # Compiling for a parallel machine? Uncomment this line. #MPP = true #---------------------------------------------------------------------- # 3. Precision # 1 = single precision; 2 = double PRECISION = 1 #---------------------------------------------------------------------- # 4. Compiler # Choices include mpicc cc gcc pgcc g++ ifeq ($(strip ${MPP}),true) CC = /usr/local/mvapich/bin/mpicc else CC = gcc endif Editing the Makefile -- examples

15. HackLatt 200815 Optimization Possibilities Build plain MILC version Build with SciDAC optimized QOP support –Requires installing the SciDAC packages. (See second tutorial and code home page). Build with GPU QUDA support –Requires installing QUDA and QUDA-MILC packages. (See the code home page) Build with OpenMP directives on loops –Very rudimentary and spotty. May need to edit some code by hand to insert OMP macros.

16. HackLatt 200816 #---------------------------------------------------------------------- # 5. Compiler optimization level OPT = -O3 #---------------------------------------------------------------------- # 6. Other compiler optimizations (depending on compiler) OCFLAGS = # Compiling with OpenMP? OMP = true #---------------------------------------------------------------------- # 10. SciDAC package options WANTQOP = true # turns on all optimized QOPQDP modules #---------------------------------------------------------------------- # 14. GPU/QUDA Options WANTQUDA = true # turns on possible QUDA selections WANT_FN_CG_GPU = true # turns QUDA CG WANT_FL_GPU = true # QUDA link fattening WANT_FF_GPU = true # QUDA fermion force term WANT_GF_GPU = true # QUDA gauge force ifeq ($(strip ${MPP}),true) CC = /usr/local/mvapich/bin/mpicc else CC = gcc endif Otimization choices controlled in the Makefile -- examples

17. HackLatt 200817 Building and Checking the Code Build (for example) –make su3_rmd Check single precision su3_rmd –make check “PROJS=su3_rmd” “PRECLIST=1” Check all targets in this directory –make check

18. HackLatt 200818 Running the code su3_rhmc outputfile su3_rhmc inputfile > outputfile su3_rhmc inputfile outputfile

19. HackLatt 200819 Sample parameter input (su3_rmd) prompt 0 nflavors1 2 nflavors2 1 nx 16 ny 16 nz 16 nt 64 iseed 5682304 warms 0 trajecs 2 traj_between_meas 1 beta 6.76 mass1 0.05 mass2 0.5 u0 0.8441 microcanonical_time_step 0.02 steps_per_trajectory 4 max_cg_iterations 300 max_cg_restarts 5 error_per_site.000005 error_for_propagator.000002 npbp_reps 1 prec_pbp 1 reload_serial../../binary_samples/lat.sample.l4444 save_serial_scidac lat.test.scidac molecular dynamics measurements lattice dimensions

20. HackLatt 200820 Summary The MILC code is versatile and portable I have given a brief overview of the code structure I have touched on the procedures for building and running the code

21. HackLatt 200821 Tutorial 1 Goals Run precompiled code Modify the input parameters Build a different target Modify the Makefile and build Download and unpack the code (if time)