Page History

...

A possible solution is to create the conda environment in a path that exists on a shared storage that can be accessed by the worker nodes.
The shell script below can be used to setup a conda virtual environment in which the mini-conda installation is already on /opt/exp_software/<experiment>.

$ cat$ cat /storageopt/gpfsexp_datasoftware/juno/test_conda/conda.sh
export CONDA_EXE='/cvmfs/juno.ihep.ac.cn/sw/anaconda/Anaconda3-2020.11-Linux-x86_64/bin/conda'
export _CE_M=''
export _CE_CONDA=''
export CONDA_PYTHON_EXE='/cvmfs/juno.ihep.ac.cn/sw/anaconda/Anaconda3-2020.11-Linux-x86_64/bin/python'
[...]

...

$ conda create -p /storageopt/gpfsexp_datasoftware/juno/test_conda/ python=3.8.0 anaconda

the conda virtual environment has been created in /storageopt/gpfsexp_datasoftware/juno/test_conda/ for the 3.8 version of Python.
After this, the screen will show

"Continue creating environment (y / [n])?"

so that a confirm is requested.
In this case, the submit description file and the executable file have to be in the folder /storageopt/gpfsexp_datasoftware/juno/test_conda/.
The content of the submit description file is:

...

$ cat test.sh
#!/bin/bash
/storageopt/gpfsexp_datasoftware/juno/test_conda/bin/python --version
/storageopt/gpfsexp_datasoftware/juno/test_conda/bin/python -c 'import h5py'
/storageopt/gpfsexp_datasoftware/juno/test_conda/bin/python -c 'import matplotlib'

...