Job submission

While locuaz may be ran on a PC, it was developed to run on UNIX-based clusters where multiple GPUs are available, and since these usually also include a workload manager, here are 2 sample submission scripts for the 2 most popular workload managers.

Running from SLURM

Here’s an example script with SLURM:

#!/bin/bash
#SBATCH -N1
#SBATCH -n4
#SBATCH --cpus-per-task=32
#SBATCH --gres=gpu:4
#SBATCH --time=24:00:00
#SBATCH --job-name locuaz
#SBATCH -o salida_locuaz
#SBATCH -e error_locuaz
#SBATCH --exclusive

cd $SLURM_SUBMIT_DIR
module load profile/lifesc
module load autoload gromacs/2021.4
source /m100/home/userexternal/pbarlett/.bashrc
conda activate locuaz

locuaz config.yaml

Running from PBS

And another one with PBS:

#!/bin/bash
#PBS -N locuaz
#PBS -l walltime=00:15:00
#PBS -l select=1:ncpus=20:ngpus=2:mpiprocs=20
#PBS -q debug

cd $PBS_O_WORKDIR
export OMP_NUM_THREADS=4
module load gromacs/2021.4
module load mpi
source /home/pbarletta/.bashrc
conda activate locuaz

locuaz.py config.yaml

Running from within an apptainer container

locuaz is also offered as an apptainer (formerly known as singularity) container. To download a specific version do:

apptainer pull oras://ghcr.io/pgbarletta/locuaz.sif:0.6.1

Currently, Apptainer doesn’t offer any progress bar, so just wait a while. The container weighs around 3Gb.

Binding GROMACS paths to the container

The locuaz container doesn’t come with its own version of GROMACS, since it assumes it’ll be run in an HPC cluster with an optimized version of it. For this reason, locuaz needs to be able to find the installed GROMACS.

From within the container, locuaz will find the gmx binary, but it won’t find the GROMACS libraries, since apptainer only exposes (binds) a subset of the host directories to the container.

By setting the variable APPTAINER_BIND or SINGULARITY_BIND to a comma separated list of the necessary paths, locuaz will be able to use GROMACS.

Take as an example:

export APPTAINER_BIND="/usr/local/gromacs,/lib/x86_64-linux-gnu,/usr/local/cuda-12.2/lib64,/etc/alternatives/"
apptainer exec --nv locuaz.sif locuaz config.yaml

To know which paths you need to bind in your specific platform, run the command:

ldd `which gmx`

which will tell you the location of the libraries gmx calls. For more info, check this blog post.