...
In the following, few utilization examples are given in order to practice with these concepts.
- Examples
In the following, some examples of submit files to help the user get comfortable with Slurm. See them as a controlled playground to test some of the features of Slurm
Simple batch submit
#!/bin/bash
#
#SBATCH --job-name=tasks1
#SBATCH --output=tasks1.txt
#SBATCH --nodelist=hpc-200-06-[17-18]
#SBATCH --ntasks-per-node=8
#SBATCH --time=5:00
#SBATCH --mem-per-cpu=100
srun hostname -s
the output should be something like (hostnames may change):
hpc-200-06-17
hpc-200-06-17
hpc-200-06-17
hpc-200-06-17
hpc-200-06-17
hpc-200-06-17
hpc-200-06-17
hpc-200-06-17
hpc-200-06-18
hpc-200-06-18
hpc-200-06-18
hpc-200-06-18
hpc-200-06-18
hpc-200-06-18
hpc-200-06-18
hpc-200-06-18
As we can see, the --ntasks-per-node=8 was interpreted by slurm as “reserve 8 cpus on node 17 and 8 on node 18 and execute the job on those cpus”:
it is quite useful to see how the output would look in case of --ntasks=8 was used instead of --ntasks-per-cpu. In that case, the output should be something like:
hpc-200-06-17
hpc-200-06-18
hpc-200-06-17
hpc-200-06-18
hpc-200-06-17
hpc-200-06-18
hpc-200-06-17
hpc-200-06-18
As we can see the execution involved 8 CPUs only and the payload was organized to minimize the burden over the nodes.
Simple MPI submit
#!/bin/bash
#
#SBATCH --job-name=test_mpi_picalc
#SBATCH --output=res_picalc.txt
#SBATCH --nodelist=... #or use --nodes=...
#SBATCH --ntasks=8
#SBATCH --time=5:00
#SBATCH --mem-per-cpu=1000
srun picalc.mpi
If the .mpi file is not available on compute nodes, which will be the most frequent scenario, computation is going to fail. That happens because Slurm does not take autonomously the responsibility of transfering files over compute nodes.
On way of acting might be to secure copy the executable over the desired nodelist, which can be feasible only if 1-2 nodes are involved. Otherwise, Slurm offers a srun option which may help the user.
build the srun command as follows:
srun --bcast=~/picalc.mpi picalc.mpi
Where the --bcast option copies the executable to every node by specifying the destination path. The final part of srun (the real instruction) should be then coherent with path you specified with bcast.
- Additional Informations
- complete overview of hardware specs per node: http://wiki.infn.it/strutture/cnaf/clusterhpc/home
- ask for support via E-mail: hpc-support@lists.cnaf.infn.it