1. Black
All good PowerPoint starts with black.

2. Compiling VTK-m/Ascent on Stampede2 with Spack
Kenneth Moreland, Sandia National Laboratories

3. Issue 1: Spack sets up compilers wrong
Spack finds gcc and Intel in your path
But, both rely on their respective module to be loaded
So, you download Spack, ask it to install a simple package (like libelf), and it fails with an error like
configure: error: installation or configuration problem: C compiler cannot create executables.

4. Issue 1 Solution: Add module to compiler configuration
Apparently Spack gets it on some systems (like Cray); not Stampede
This should be pushed back to the Spack team
From compilers.yaml:
- compiler:
environment: {}
extra_rpaths: []
flags: {}
modules: [intel/18.0.2]
operating_system: centos7
paths:
cc: /opt/intel/compilers_and_libraries_ /linux/bin/intel64/icc
cxx: /opt/intel/compilers_and_libraries_ /linux/bin/intel64/icpc
f77: /opt/intel/compilers_and_libraries_ /linux/bin/intel64/ifort
fc: /opt/intel/compilers_and_libraries_ /linux/bin/intel64/ifort
spec:
target: x86_64

5. Issue 2: Spack wants to compile CMake
The point of CMake is that you have one thing that can manage configuration
Spack wants to compile its own version, which requires the installation of about a dozen dependent packages (including autotools), which defeats the purpose
Also failed to build because gnutils couldn’t seem to get guile correctly
Many HPC systems (including Stampede) have CMake installed, and it is trivial to download and use your own binary (for x86 architecture)

6. Issue 2 Solution: Configure external packages for CMake
You can point to either an external module or executable for CMake
Configuration is in packages.yaml
It would save a lot of people a lot of time if Spack did this automatically.
CMake is ubiquitous enough to have a special CMake set up like compilers
packages: cmake:
modules: cmake/3.10.2
paths: ~/local/stow/cmake Linux-x86_64
buildable: False

7. Issue 3: Spack Intel-mpi configuration broken for Stampede
Spack makes some assumptions about how Intel-mpi is installed that is violated with Stampede
Cyrus can provide more details

8. Issue 3 Solution: Workaround by setting up mvapich2 to use Intel-mpi
Thanks to Cyrus for the solution!
Spack has an open issue to address this
In packages.yaml:
packages:
all:
providers:
mpi:
- mvapich2
mvapich2:
buildable: False
modules:
mvapich2: impi

9. Issue 4: Executables compiled with Intel compiler module need module loaded to run
Ascent needs the python module and several other related modules
If python is compiled with the intel compiler on Stampede2 (with, for example, the intel/ module), you need that compiler module loaded to run
Otherwise you get: error while loading shared libraries: libimf.so: cannot open shared object file: No such file or directory
This happens in modules like py-setuptools, which run python scripts but do not compile anything
Nothing compiled, no compiler module loaded

10. Issue 4 Solution: Set compiler configuration to add rpath to compiler-provided libraries
Using --dirty flag to spack might also work
Open Spack issue:
- compiler:
environment: {}
extra_rpaths:
- /opt/intel/compilers_and_libraries_ /linux/compiler/lib/intel64_lin
flags: {}
modules: [intel/18.0.2]
operating_system: centos7
paths:
cc: /opt/intel/compilers_and_libraries_ /linux/bin/intel64/icc
cxx: /opt/intel/compilers_and_libraries_ /linux/bin/intel64/icpc
f77: /opt/intel/compilers_and_libraries_ /linux/bin/intel64/ifort
fc: /opt/intel/compilers_and_libraries_ /linux/bin/intel64/ifort
spec:
target: x86_64

11. Issue 5: Ascent package in Spack does not work
Ironic since Ascent source build scripts use Spack for compiling
But the Ascent build wants to pull its own version of Spack and attempt to add patches to get it to work in known places
I don’t want to do that because
I want my Ascent build to work with other things I build with Spack
It is easier to fix Spack issues in my own environment rather than Ascent’s
I would rather submit fixes to Spack and have it fixed for everyone than submit to Ascent and have it work only for Ascent builds
Main issue is that Ascent requires specific commits for VTK-m, which are not captured in the Spack repository

12. Issue 5 Solution: WIP
I was working on attempting to set up a version package of Ascent
Requires setting up dependencies to correct versions of VTK-m and VTK-h
These might be “random” git commits.
This should be easier to manage once VTK-m and VTK-h merge.