Installing your Spack environment¶
As in the previous example, we need to spec and then install our
environment.
Spack find¶
Let's look at the output of spack find in the activated environment:
spack find output
[spack-python] $ spack find -x -p
==> In environment /gpfs/scratch/abc123/spack-environments/spack-python
==> 3 root specs
-- no arch / no compilers ---------------------------------------
- libsndfile - py-pip - python@3.12
==> 0 installed packages
==> 0 concretized packages to be installed (show with `spack find -c`)
So, in the context of the environment, the "root specs" are listed but not yet
installed ("==> 0 installed packages").
Speccing the environment¶
Always run a spec command before install!
You should always check what is about to be installed using the spec
command as detailed below and then move onto the install command.
Let's run a spec command in the activated environment:
spack spec output (click to expand)
[spack-python] $ spack spec
- libsndfile@1.2.2~alsa~external-libs~sqlite build_system=autotools platform=linux os=rocky9 target=x86_64_v4 %c,cxx=gcc@12.2.0
[^] ^compiler-wrapper@1.0 build_system=generic platform=linux os=rocky9 target=x86_64_v4
[e] ^gcc@12.2.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=rocky9 target=x86_64
- ^gcc-runtime@12.2.0 build_system=generic platform=linux os=rocky9 target=x86_64_v4
[e] ^glibc@2.34 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] ^gmake@4.4.1~guile build_system=generic platform=linux os=rocky9 target=x86_64_v4
[^] ^gcc-runtime@11.4.1 build_system=generic platform=linux os=rocky9 target=x86_64_v4
[^] ^pkgconf@2.2.0 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] py-pip@23.1.2 build_system=generic platform=linux os=rocky9 target=x86_64_v4
[^] ^gcc-runtime@12.2.0 build_system=generic platform=linux os=rocky9 target=x86_64_v4
[e] ^glibc@2.34 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] ^python-venv@1.0 build_system=generic platform=linux os=rocky9 target=x86_64_v4
[^] python@3.12.1+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic platform=linux os=rocky9 target=x86_64_v4
[^] ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=rocky9 target=x86_64_v4
[^] ^diffutils@3.10 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] ^expat@2.6.2+libbsd build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] ^libbsd@0.12.1 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] ^libmd@1.0.4 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] ^gdbm@1.23 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] ^gettext@0.22.5+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] ^libxml2@2.10.3+pic~python+shared build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] ^tar@1.34 build_system=autotools zip=pigz platform=linux os=rocky9 target=x86_64_v4
[^] ^pigz@2.8 build_system=makefile platform=linux os=rocky9 target=x86_64_v4
[^] ^zstd@1.5.6+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=rocky9 target=x86_64_v4
[^] ^libffi@3.4.6 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] ^libxcrypt@4.4.35~obsolete_api build_system=autotools patches:=4885da3 platform=linux os=rocky9 target=x86_64_v4
[^] ^perl@5.38.0+cpanm+opcode+open+shared+threads build_system=generic patches:=714e4d1 platform=linux os=rocky9 target=x86_64_v4
[^] ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=rocky9 target=x86_64_v4
[^] ^ncurses@6.5~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=rocky9 target=x86_64_v4
[^] ^openssl@3.3.0~docs+shared build_system=generic certs=mozilla platform=linux os=rocky9 target=x86_64_v4
[^] ^ca-certificates-mozilla@2023-05-30 build_system=generic platform=linux os=rocky9 target=x86_64_v4
[^] ^readline@8.2 build_system=autotools patches:=bbf97f1 platform=linux os=rocky9 target=x86_64_v4
[^] ^sqlite@3.43.2+column_metadata+dynamic_extensions+fts~functions+rtree build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] ^util-linux-uuid@2.38.1 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] ^xz@5.4.6~pic build_system=autotools libs:=shared,static platform=linux os=rocky9 target=x86_64_v4
[^] ^zlib-ng@2.1.6+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=rocky9 target=x86_64_v4
This time everything is already in upstream ([^]) apart from libsndfile and
a small gcc-runtime package, which in the output above are marked with -
meaning that the package is missing and needs to be installed personally.
Installing the environment¶
Once you are happy with the output of the spack spec command above, you can
move onto a spack install command. Remember, you should add
-j ${SLURM_NTASKS} to your command to ensure that you match the installation
process threading to your salloc core request:
spack install output (click to expand)
[spack-python] $ spack install -j ${SLURM_NTASKS}
==> Concretized 3 specs
- si47cwg libsndfile@1.2.2~alsa~external-libs~sqlite build_system=autotools platform=linux os=rocky9 target=x86_64_v4 %c,cxx=gcc@12.2.0
[^] oqjjbmb ^compiler-wrapper@1.0 build_system=generic platform=linux os=rocky9 target=x86_64_v4
[e] c6glrz7 ^gcc@12.2.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=rocky9 target=x86_64
- 5or73vm ^gcc-runtime@12.2.0 build_system=generic platform=linux os=rocky9 target=x86_64_v4
[e] vjgsh5e ^glibc@2.34 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] xchit5a ^gmake@4.4.1~guile build_system=generic platform=linux os=rocky9 target=x86_64_v4
[^] llid4hw ^gcc-runtime@11.4.1 build_system=generic platform=linux os=rocky9 target=x86_64_v4
[^] 7wg26bz ^pkgconf@2.2.0 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] inullmb py-pip@23.1.2 build_system=generic platform=linux os=rocky9 target=x86_64_v4
[^] w77gg5r ^gcc-runtime@12.2.0 build_system=generic platform=linux os=rocky9 target=x86_64_v4
[e] xri56vc ^glibc@2.34 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] fcsyl32 ^python-venv@1.0 build_system=generic platform=linux os=rocky9 target=x86_64_v4
[^] sd2mpei python@3.12.1+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic platform=linux os=rocky9 target=x86_64_v4
[^] uj4wyhx ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=rocky9 target=x86_64_v4
[^] 7zermel ^diffutils@3.10 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] je4ggnn ^expat@2.6.2+libbsd build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] zb23l3j ^libbsd@0.12.1 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] y6ppafl ^libmd@1.0.4 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] bx77xc6 ^gdbm@1.23 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] udcuonu ^gettext@0.22.5+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] q6zmsq6 ^libxml2@2.10.3+pic~python+shared build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] ivzmnos ^tar@1.34 build_system=autotools zip=pigz platform=linux os=rocky9 target=x86_64_v4
[^] somkvv4 ^pigz@2.8 build_system=makefile platform=linux os=rocky9 target=x86_64_v4
[^] my7tyw6 ^zstd@1.5.6+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=rocky9 target=x86_64_v4
[^] yc4oqfz ^libffi@3.4.6 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] 4mszlpj ^libxcrypt@4.4.35~obsolete_api build_system=autotools patches:=4885da3 platform=linux os=rocky9 target=x86_64_v4
[^] eyk53wh ^perl@5.38.0+cpanm+opcode+open+shared+threads build_system=generic patches:=714e4d1 platform=linux os=rocky9 target=x86_64_v4
[^] cogxxtx ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=rocky9 target=x86_64_v4
[^] 4n2uzha ^ncurses@6.5~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=rocky9 target=x86_64_v4
[^] filwsx6 ^openssl@3.3.0~docs+shared build_system=generic certs=mozilla platform=linux os=rocky9 target=x86_64_v4
[^] muc5zmk ^ca-certificates-mozilla@2023-05-30 build_system=generic platform=linux os=rocky9 target=x86_64_v4
[^] ea7drzj ^readline@8.2 build_system=autotools patches:=bbf97f1 platform=linux os=rocky9 target=x86_64_v4
[^] t5xgpdh ^sqlite@3.43.2+column_metadata+dynamic_extensions+fts~functions+rtree build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] nkfrifq ^util-linux-uuid@2.38.1 build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[^] rwn7pno ^xz@5.4.6~pic build_system=autotools libs:=shared,static platform=linux os=rocky9 target=x86_64_v4
[^] g2yruc3 ^zlib-ng@2.1.6+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=rocky9 target=x86_64_v4
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/none-none/compiler-wrapper/1.0-oqjjbmb
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-11.4.1/gcc/12.2.0-6frskzg (external gcc-12.2.0-c6glrz75exefrjwq5gurxmxiqkc34ubt)
[+] /usr (external glibc-2.34-vjgsh5eoloariofarhttteo73mj5rgql)
[+] /usr (external glibc-2.34-xri56vcyzs7kkvakhoku3fefc46nw25y)
==> No binary for gcc-runtime-12.2.0-5or73vm4wrifsxjiarbogboeue7zdq25 found: installing from source
==> Installing gcc-runtime-12.2.0-5or73vm4wrifsxjiarbogboeue7zdq25 [5/32]
==> No patches needed for gcc-runtime
==> gcc-runtime: Executing phase: 'install'
==> gcc-runtime: Successfully installed gcc-runtime-12.2.0-5or73vm4wrifsxjiarbogboeue7zdq25
Stage: 0.00s. Install: 0.72s. Post-install: 0.11s. Total: 0.91s
[+] /gpfs/scratch/abc123/spack/apps/linux-rocky9-x86_64_v4/none-none/gcc-runtime/12.2.0-5or73vm
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/gcc-runtime/12.2.0-w77gg5r
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-11.4.1/gcc-runtime/11.4.1-llid4hw
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/libxcrypt/4.4.35-4mszlpj
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/libmd/1.0.4-y6ppafl
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/libffi/3.4.6-yc4oqfz
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/util-linux-uuid/2.38.1-nkfrifq
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-11.4.1/bzip2/1.0.8-uj4wyhx
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-11.4.1/zlib-ng/2.1.6-g2yruc3
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-11.4.1/zstd/1.5.6-my7tyw6
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-11.4.1/ncurses/6.5-4n2uzha
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-11.4.1/xz/5.4.6-rwn7pno
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-11.4.1/gmake/4.4.1-xchit5a
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/libbsd/0.12.1-zb23l3j
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/openssl/3.3.0-filwsx6
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-11.4.1/pigz/2.8-somkvv4
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-11.4.1/readline/8.2-ea7drzj
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-11.4.1/libxml2/2.10.3-q6zmsq6
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-11.4.1/pkgconf/2.2.0-7wg26bz
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/expat/2.6.2-je4ggnn
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-11.4.1/tar/1.34-ivzmnos
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/sqlite/3.43.2-t5xgpdh
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-11.4.1/gdbm/1.23-bx77xc6
==> No binary for libsndfile-1.2.2-si47cwglqaeguj6qku263iupuxnub7fu found: installing from source
==> Installing libsndfile-1.2.2-si47cwglqaeguj6qku263iupuxnub7fu [28/32]
==> Fetching https://mirror.spack.io/_source-cache/archive/37/3799ca9924d3125038880367bf1468e53a1b7e3686a934f098b7e1d286cdb80e.tar.xz
[100%] 730.76 KB @ 52.7 MB/s
==> No patches needed for libsndfile
==> libsndfile: Executing phase: 'autoreconf'
==> libsndfile: Executing phase: 'configure'
==> libsndfile: Executing phase: 'build'
==> libsndfile: Executing phase: 'install'
==> libsndfile: Successfully installed libsndfile-1.2.2-si47cwglqaeguj6qku263iupuxnub7fu
Stage: 0.15s. Autoreconf: 0.00s. Configure: 12.61s. Build: 3.05s. Install: 0.42s. Post-install: 0.06s. Total: 16.56s
[+] /gpfs/scratch/abc123/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/libsndfile/1.2.2-si47cwg
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-11.4.1/gettext/0.22.5-udcuonu
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/python/3.12.1-sd2mpei
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/python-venv/1.0-fcsyl32
[+] /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/py-pip/23.1.2-inullmb
==> Updating view at /gpfs/scratch/abc123/spack-environments/spack-python/.spack-env/view
Spack has
concretized
the environment specs and then used upstream or existing personal installs for
everything except libsndfile, which it has installed into the
install_tree: root: we defined in the config: section of our
spack.yaml file.
If we now run our find command again, we see the "root specs" are now
installed. Also listed as "installed" are any required dependencies that have
been taken from our upstream:
spack find output
[spack-python] $ spack find -x -p
==> In environment /gpfs/scratch/abc123/spack-environments/spack-python
==> 3 root specs
-- no arch / no compilers ---------------------------------------
[+] libsndfile [^] py-pip [^] python@3.12
-- linux-rocky9-x86_64_v4 / %c,cxx=gcc@12.2.0 -------------------
libsndfile@1.2.2 /gpfs/scratch/abc123/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/libsndfile/1.2.2-si47cwg
-- linux-rocky9-x86_64_v4 / no compilers ------------------------
openssl@3.3.0 /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/openssl/3.3.0-filwsx6
python@3.12.1 /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/python/3.12.1-sd2mpei
sqlite@3.43.2 /share/apps/rocky9/spack/apps/linux-rocky9-x86_64_v4/gcc-12.2.0/sqlite/3.43.2-t5xgpdh
==> 4 installed packages
==> 0 concretized packages to be installed (show with `spack find -c`)