Computational chemistry packages (efficient usage issues?) Jemmy Hu SHARCNET HPC Consultant Summer School June 3, 2016 /work/jemmyhu/ss2016/chemistry/

Chemistry Software managed by SHARCNET Electronic Structure TheoryMolecular Dynamics Gaussian (09, 03) -commercialAMBER -commercial ADF/Band -commercialGromacs NWChemCPMD Quantum ExpressoNAMD ORCA Gamess/Firefly Abinit MolproVASP TurbomoleCHARMM User/group owned software (commercial)

Nature of the software SoftwareParallelPurpose GaussianThreaded (OpenMP) ADFMPI NWChemMPI sw.org/index.php/Main_Page Quantum ExpressoMPI VASPMPI vasp/59-about-vasp ORCAMPI

Open Software used by 2 or more groups – add to SHARCNET software stack with software usage instruction/example webpage online Software used by one group – user installs in his/her own space with help from SHARCNET if necessary SHARCNET owned commercial software – free Self owned software – user’s response for license issues Software are installed as module package, setup in your bash configuration file, i.e.,.bash_profile file under /home/userid/ module load gaussian/g09_D.01 General policy for software

module load packages Login to a cluster, e.g., ssh Open the shell configuration file, e.g., vi.bash_profile Add the module load lines at the end of the file module load gaussian/g09_D.01 module load adf/ module load nwchem/6.5 module load espresso/5.1.1 module load orca/3.0 Save the file Exit Re-login Goto your /work directory: cd /work/userID Copy the files cp –rf /work/jemmyhu/ss2016/chemistry.

System Info Parallel Computer Memory Architectures Hybrid Distributed-Shared Memory (Cluster solution)

Size of Threaded Jobs on specific system SystemNodesCPU/Node (threads) Memory GB (Max for full cpus usage) orcaOpteron16 or 2430 sawXeon815 brownXeon816 Job uses less than the max threads and memory on a node will be preferable

Gaussian 09 issues Latest version: g09-D.01 Input sample File location Memory Disk Restart Utilities

#p b3lyp/ g(2df,p) opt freq > 0 1 c o 1 co co= (leave an blank line at the end) G09 input file: simplest case File name: CO-b3lyp-1.com

Real jobs should be submitted to the threaded queue using sqsub command First check the files under your /work and /scratch directory open two windows, one for /work, one to view files in /scratch For testing purpose today, we run jobs from command line on the dev. node on saw (saw-dev1, …, saw-dev6) ssh saw-dev4 run g09 CO-b3lyp-1.com >& CO-b3lyp-1.log & Output file CO-b3lyp-1.log Runtime files (5 of them,.rwf is big) /scratch/userID Job uses defaults [1 cpu, minimum memory (256MB?)] Only.log output file will be generated in the run directory Runtime files will be removed automatically once finished successfully, otherwise they will stay. Run job, file structure

Specify checkpoint file: name.chk %chk=CO-b3lyp-1chk #p b3lyp/ g(2df,p) opt freq > 0 1 c o 1 co co= Input file: CO-b3lyp-1chk.com Run on dev: g09 CO-b3lyp-1chk.com >& CO-b3lyp-1chk.log & Files: CO-b3lyp-1chk.log, CO-b3lyp-1chk.chk

G09 input file: CO-b3lyp-4.com, use 4 cpus %nosave %mem=500MB %chk=CO-b3lyp-4 %nproc=4 #p b3lyp/ g(2df,p) opt freq > 0 1 c o 1 co co=1.1281

Run Job in the queue %mem=500MB (--mpp=2x%mem, memory per job, for all cpus) %nproc=4 (-n 4) Submit command is sqsub -q threaded -n 4 -r 1.0d –mpp=2g -o CO out g09 CO com What’s happening afterwards? -sqjobs to see job status, jobID -If runs, where are the files?.log and.chk files are generated in the run directory runtime files,.rwf, etc will be in /scratch/yourID/jobID/ -Failed? remove the runtime files, rm –rf /scratch/yourID/jobID. correct input file, re-run

%rwf=CO-b3lyp-4rwf %nosave %mem=500MB %chk=CO-b3lyp-4rwf %nproc=4 #p b3lyp/ g(2df,p) opt freq > 0 1 c o 1 co co= The named.rwf file will be generated in the current directory Input file with named.rwf: CO-b3lyp-4rwf.com

Checkpointing 1..chk can be used to restart an Optimization job, or used for property (freq, etc) calculation afterwards. 2. In G09,.rwf can be used to restart a frequency calculation.

Restart (see examples) Restart opt job from.chk Restart freq from.rwf %rwf=/scratch/jemmyhu/Cu_test.rwf %NoSave %chk=Cu_test.chk %mem=500mb %nproc=4 #p restart (one blank line) %NoSave %chk=Cu_test.chk %mem=500mb %nproc=4 #B3LYP/DGDZVP Opt Geom=AllCheck Guess=Read (one blank line)

Gaussian Utilities Utilities formchk cubegen freqchk …… use the same version direct run from command line NBO

Review Input file, memory (%mem) /cpu (%nproc) sqsub (-n, --mpp), sqjobs, sqkill Use job info from scheduler File location Failed jobs File cleanup

ADF ADF is frequently used for studying transition metal complexes and molecules with heavy atoms. ADF offers unique capabilities to predict molecular properties of organic electronics materials.transition metal complexesheavy atomsorganic electronics ADF, BAND, DFTB

Use ADF on SHARCNET Available on major clusters, viz-stations Versions: , Setup, add module load line to.bash_profile file, i.e., module load adf/ Examples /opt/sharcnet/adf/ /examples

module show adf/ /opt/sharcnet/modules/adf/ : module-whatis Provides ADF binary for x86_64 Centos 6 systems conflict adf setenv ADFROOT /opt/sharcnet/adf setenv ADFHOME /opt/sharcnet/adf/ setenv ADFBIN /opt/sharcnet/adf/ /bin setenv ADFRESOURCES /opt/sharcnet/adf/ /atomicdata setenv SCMLICENSE /opt/sharcnet/adf/ /license setenv SCM_TMPDIR /scratch/jemmyhu setenv SCM_USETMPDIR yes prepend-path PATH /opt/sharcnet/adf/ /bin Default settings

Two input options name.inp (one step a time) use a runscript ‘ADF’ or ‘BAND’, to bypass the runtime files to /scratch/username/jobid/ sqsub -q mpi --nompirun -n 8 --mpp=2g -r 1.0d -o GO_H2O-adf.log ADF GO_H2O-adf.inp sqsub -q mpi --nompirun -n 8 --ppn=4 --mpp=2g -r 1.0d -o GO_H2O-adf.log ADF GO_H2O-adf.inp sqsub -q mpi --nompirun -n 8 --mpp=2g -r 1.0d -o name_band.log BAND name_band.inp

/home/jemmyhu/work/ss2016/chemistry/examples/adf GO_H2O-adf.inp

sqsub -q mpi --nompirun -n 8 --ppn=4 --mpp=2g -r 1.0d -o NMR_B3LYP.log./NMR_B3LYP.run Input for multi-steps: name.run runtime files are under /scratch/username/

Restart TAPE13 It serves as a checkpoint file. TAPE13 can be used to restart the calculation at a point not too far before the fatal condition occurred TAPE13 is upgraded during the calculation but discarded upon normal termination. To keep TAPE13, -use the SAVE key TAPE21 Upon normal (i.e. program-controlled) termination of a calculation, the TAPE21 result file can be used for restart purposes When a crash occurs, however, chances are that TAPE21 has not correctly been closed and that its data structure is inconsistent

ADF GUI GUI software to run on viz-workstations vdi-centos6.user.sharcnet.ca viz-uwo10.sharcnet.ca viz-uwo11.sharcnet.ca Turn on X11 locally, then login to he workstation adfview

pwd /home/jemmyhu/work/ss2016/chemistry/examples/nwchem ls h2o_hu.nw sqsub -q mpi -n 4 -r 3.0d --mpp=4g -o h2o_hu.log nwchem h2o_hu.nw RUN nwchem on SHARCNET

Research group owns the license Installation guideline

What else? Question? Discussion?