Run HADDOCK2.4 using slurm

1

Hi,
I am using HADDOCK local version 2.4 successfully and now I want to start running HADDCOK using slurm. I read the tutorial online but did not understand it. Is this what I should do?
This it what I have now:
in run.cns
{===>} queue_1=“/bin/tcsh”;
{===>} cns_exe_1=“/home/qnt/drorimi2/software/cns_solve_1.3/intel-x86_64bit-linux/source/cns_solve-2305211622.exe”;
{===>} cpunumber_1=18;

and in config.local
set CNSTMP=/home/qnt/drorimi2/software/cns_solve_1.3/intel-x86_64bit-linux/source/cns_solve-2305211622.exe
set QUEUETMP=/bin/tcsh
set NUMJOB=4
set QUEUESUB=QueueSubmit_concat.py

Should I change in run.cns to {===>} queue_1=“sbatch”;
My script for slurm is:
#!/bin/bash

#SBATCH -p prod # Submit to ‘stdmemq’ Partitiion or queue
#SBATCH -J HADDOCK_4_4_5_f_6_1 # Name the job as ‘MPItest’
#SBATCH -o MCSA1test-%j.out # Write the standard output to file named ‘jMPItest-<job_number>.out’
#SBATCH -e MCSA1test-%j.err # Write the standard error to file named ‘jMPItest-<job_number>.err’
#SBATCH -t 0-10:00:00 # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
#SBATCH --nodes=1 # Request N nodes
#SBATCH --ntasks-per-node=18 # Request n cores or task per node
#SBATCH --mem-per-cpu=1GB # Request 4GB RAM per core

pwd # prints current working directory
date # prints the date and time

/usr/lib64/mpich/bin/mpirun -np --ntasks-per-node ??? <run_HADDOCK_slurm.sh> run_HADDOCK_slurm.out #run the MPI job

What should I have the executable file be? (instead of ???) and did I change run.cns correctly? is there any other file I should change? Where should I put the code provided in HADDOCK2.4 manual - Frequently Asked Questions – Bonvin Lab?

Thank you

2

There are two ways you could run it:

1) You start haddock2.4 on the master node and let HADDOCK submit individual jobs to the batch system.

For this you would define in run.cns:

{===>} queue_1=“sbatch <any options you might want to add>”;
{===>} cns_exe_1=“/home/qnt/drorimi2/software/cns_solve_1.3/intel-x86_64bit-linux/source/cns_solve-2305211622.exe”;
{===>} cpunumber_1=18;

Possibly you will need the full path to sbatch. And you could define as many cpus as you are allowed to use.
The corresponding config file would look like:

set CNSTMP=/trinity/home/enmr/software/cns/cns_solve_1.31-UU/intel-x86_64bit-linux/bin/cns
set QUEUETMP=“sbatch <any options you might want to add>"
set NUMJOB=50
set QUEUESUB=QueueSubmit_concat.py

2) You submit the full HADDOCK process to a node and use X cores on that node.
In that scenario you would write a sbatch script that requests one node and X cores, and starts haddock. No MPI! You will first need to call haddock2.4 once to create the rundir and edit the run.cns file. Then submit to the batch system


#!/bin/bash
#SBATCH -p prod # Submit to ‘stdmemq’ Partitiion or queue
#SBATCH -J HADDOCK_4_4_5_f_6_1 # Name the job
#SBATCH -o MCSA1test-%j.out # Write the standard output to file named ‘jMPItest-<job_number>.out’
#SBATCH -e MCSA1test-%j.err # Write the standard error to file named ‘jMPItest-<job_number>.err’
#SBATCH -t 0-10:00:00 # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
#SBATCH --nodes=1 # Request N nodes
#SBATCH --ntasks-per-node=18 # Request n cores or task per node
#SBATCH --mem-per-cpu=1GB # Request 4GB RAM per core

date # prints the date and time

source <path-to-haddock-installation/haddock_configure.sh

cd <haddock-rundir>

haddock2.4 >haddock.out

And in run.cns you would have:


{===>} queue_1=“/bin/tcsh”;
{===>} cns_exe_1=“/home/qnt/drorimi2/software/cns_solve_1.3/intel-x86_64bit-linux/source/cns_solve-2305211622.exe”;
{===>} cpunumber_1=18;

Making sure of course that the path to the executable and the haddock installation are accessible from a node.

Cheers

Alexandre

3

Thank you!
I am trying the second way
my batch code looks like this:
"
#!/bin/bash

#SBATCH -p prod # Submit to ‘stdmemq’ Partitiion or queue
#SBATCH -J HADDOCK_4_4_5_f_6_1 # Name the job
#SBATCH -o MCSA1test-%j.out # Write the standard output to file named ‘jMPItest-<job_number>.out’
#SBATCH -e MCSA1test-%j.err # Write the standard error to file named ‘jMPItest-<job_number>.err’
#SBATCH -t 0-10:00:00 # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
#SBATCH --nodes=1 # Request N nodes
#SBATCH --ntasks-per-node=18 # Request n cores or task per node
#SBATCH --mem-per-cpu=1GB # Request 4GB RAM per core

date # prints the date and time
source /home/qnt/drorimi2/software/haddock2.4-2023-01/haddock_configure.sh
cd /home/qnt/drorimi2/full_analize_40PDB/7pll/HADDOCK/run_7pll_4_4_5_flex_6_1
haddock2.4 >&haddock.out
"

And the run.param is just as you wrote (without changing)
However, this is the error I get:
"
which: no naccess in (/home/qnt/drorimi2/anaconda3/envs/haddock2.4/bin:/usr/condabin:/private/chem/qchem/5.4.2r/exe:/private/chem/qchem/5.4.2r/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/dell/srvadmin/bin:/opt/intel/mkl/10.2.4.032/lib/em64t:/opt/intel/impi/4.1.0/bin64:/opt/hpmpi/bin:/home/qnt/majort/bin/babel/openbabel-2.2.3/bin:/private/chem/bin:/home/qnt/drorimi2/bin:/home/qnt/drorimi2/bin/wham:/home/qnt/drorimi2/bin/perl:.:/home/qnt/drorimi2/anaconda3/bin)
which: no pales in (/home/qnt/drorimi2/anaconda3/envs/haddock2.4/bin:/usr/condabin:/private/chem/qchem/5.4.2r/exe:/private/chem/qchem/5.4.2r/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/dell/srvadmin/bin:/opt/intel/mkl/10.2.4.032/lib/em64t:/opt/intel/impi/4.1.0/bin64:/opt/hpmpi/bin:/home/qnt/majort/bin/babel/openbabel-2.2.3/bin:/private/chem/bin:/home/qnt/drorimi2/bin:/home/qnt/drorimi2/bin/wham:/home/qnt/drorimi2/bin/perl:.:/home/qnt/drorimi2/anaconda3/bin)
which: no tensor2 in (/home/qnt/drorimi2/anaconda3/envs/haddock2.4/bin:/usr/condabin:/private/chem/qchem/5.4.2r/exe:/private/chem/qchem/5.4.2r/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/dell/srvadmin/bin:/opt/intel/mkl/10.2.4.032/lib/em64t:/opt/intel/impi/4.1.0/bin64:/opt/hpmpi/bin:/home/qnt/majort/bin/babel/openbabel-2.2.3/bin:/private/chem/bin:/home/qnt/drorimi2/bin:/home/qnt/drorimi2/bin/wham:/home/qnt/drorimi2/bin/perl:.:/home/qnt/drorimi2/anaconda3/bin)
"

how can this be fixed?
In general, before I run anything in HADDOCK I do:

"
cd /home/qnt/drorimi2/software/haddock2.4-2023-01/
source /home/qnt/drorimi2/anaconda3/etc/profile.d/conda.csh
conda activate /home/qnt/drorimi2/anaconda3/envs/haddock2.4
./install.csh config.local
source ./haddock_configure.csh
"

but when I added it to the code it did not work

"
/home/qnt/drorimi2/anaconda3/etc/profile.d/conda.csh: line 1: setenv: command not found
/home/qnt/drorimi2/anaconda3/etc/profile.d/conda.csh: line 2: setenv: command not found
/home/qnt/drorimi2/anaconda3/etc/profile.d/conda.csh: line 3: setenv: command not found
/home/qnt/drorimi2/anaconda3/etc/profile.d/conda.csh: line 4: setenv: command not found
/home/qnt/drorimi2/anaconda3/etc/profile.d/conda.csh: line 33: syntax error near unexpected token "${1}"' /home/qnt/drorimi2/anaconda3/etc/profile.d/conda.csh: line 33: switch ( “${1}” )’

CommandNotFoundError: Your shell has not been properly configured to use ‘conda activate’.
"

4

maybe the issue is that I am not running the sh configuration file?
when I do that it also doesn’t work. the file looks like
#!/bin/bash

HADDOCK configuration file

HADDOCK=“/home/qnt/drorimi2/software/haddock2.4-2023-01”
HADDOCKTOOLS=“$HADDOCK/tools”
PYTHONPATH=“${PYTHONPATH}:$HADDOCK”
alias haddock2.4=“$(which python2.7) $HADDOCK/Haddock/RunHaddock.py”

Define location of third party software

NACCESS=which naccess
PALES=which pales
PROFIT=which profit
TENSOR2=which tensor2

export HADDOCK HADDOCKTOOLS PYTHONPATH NACCESS PALES PROFIT TENSOR2

is this correct?

5

Define doesn’t work…

6

When I run

conda activate haddock2.4
./install.csh config.local
source ./haddock_configure.sh

the error is
HADDOCK=/home/qnt/drorimi2/software/haddock2.4-2023-01: Command not found.
HADDOCKTOOLS=/home/qnt/drorimi2/software/haddock2.4-2023-01/tools: Command not found.
PYTHONPATH=/home/qnt/drorimi2/software/haddock2.4-2023-01:/home/qnt/drorimi2/software/haddock2.4-2023-01:/home/qnt/drorimi2/software/haddock2.4-2023-01: Command not found.
Illegal variable name.

the haddock_configure.sh file looks like:
#!/bin/bash

HADDOCK configuration file
HADDOCK=“/root/software/haddock2.4-2021-05”
HADDOCKTOOLS=“$HADDOCK/tools”
PYTHONPATH=“${PYTHONPATH}:$HADDOCK”
alias haddock2.4=“$(which python) $HADDOCK/Haddock/RunHaddock.py”
#alias haddock2.4=“/home/enmr/software/python/bin/python $HADDOCK/Haddock/RunHaddock.py”

Define location of third party software
NACCESS=which naccess
PALES=which pales
PROFIT=which profit
TENSOR2=which tensor2

export HADDOCK HADDOCKTOOLS PYTHONPATH NACCESS PALES PROFIT TENSOR2

and config.local
#!/bin/bash

HADDOCK configuration file
HADDOCK=“/root/software/haddock2.4-2021-05”
HADDOCKTOOLS=“$HADDOCK/tools”
PYTHONPATH=“${PYTHONPATH}:$HADDOCK”
alias haddock2.4=“$(which python) $HADDOCK/Haddock/RunHaddock.py”
#alias haddock2.4=“/home/enmr/software/python/bin/python $HADDOCK/Haddock/RunHaddock.py”

Define location of third party software
NACCESS=which naccess
PALES=which pales
PROFIT=which profit
TENSOR2=which tensor2

export HADDOCK HADDOCKTOOLS PYTHONPATH NACCESS PALES PROFIT TENSOR2

(priviesly I used ./haddock_configure.csh and it works, but does not work under the slurm code that looks like this:
#!/bin/bash

#SBATCH -p prod # Submit to ‘stdmemq’ Partitiion or queue
#SBATCH -J HADDOCK_4_4_5_f_6_1 # Name the job
#SBATCH -o MCSA1test-%j.out # Write the standard output to file named ‘jMPItest-<job_number>.out’
#SBATCH -e MCSA1test-%j.err # Write the standard error to file named ‘jMPItest-<job_number>.err’
#SBATCH -t 0-10:00:00 # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
#SBATCH --nodes=1 # Request N nodes
#SBATCH --ntasks-per-node=18 # Request n cores or task per node
#SBATCH --mem-per-cpu=1GB # Request 4GB RAM per core

date # prints the date and time

cd /home/qnt/drorimi2/software/haddock2.4-2023-01/
source /home/qnt/drorimi2/anaconda3/etc/profile.d/conda.csh
conda activate /home/qnt/drorimi2/anaconda3/envs/haddock2.4
./install.csh config.local
source ./haddock_configure.csh

cd /home/qnt/drorimi2/full_analize_40PDB/7pll/HADDOCK/run_7pll_4_4_5_flex_6_1

#Run full analysis
haddock2.4 >&haddock.out

Is this correct? why is the code not working?

7

Are you running under bash or tcsh?
If the tcsh version work use that one, but then you slurm script must also use tcsh for things to work

I don’t get those errors when trying under bash

You only need to run once the install.csh script.

Try editing your haddock_configure.sh script and add export commands, e.g.

export HADDOCK=“/root/software/haddock2.4-2021-05”
export HADDOCKTOOLS=“$HADDOCK/tools”
export PYTHONPATH=“${PYTHONPATH}:$HADDOCK”
alias haddock2.4=“$(which python) $HADDOCK/Haddock/RunHaddock.py”Define location of third party software

export PALES=which pales
export PROFIT=which profit
export TENSOR2=which tensor2

8

ysing bash
I changed it to
#!/bin/bash

HADDOCK configuration file

export HADDOCK=“/home/qnt/drorimi2/software/haddock2.4-2023-01”
export HADDOCKTOOLS=“$HADDOCK/tools”
export PYTHONPATH=“${PYTHONPATH}:$HADDOCK”
alias haddock2.4=“$(which python2.7) $HADDOCK/Haddock/RunHaddock.py”

Define location of third party software

#NACCESS=which naccess
#PALES=which pales
#PROFIT=which profit
#TENSOR2=which tensor2

export NACCESS=which naccess
export PALES=which pales
export PROFIT=which profit
export TENSOR2=which tensor2

export HADDOCK HADDOCKTOOLS PYTHONPATH NACCESS PALES PROFIT TENSOR2

and when I run just the file theerror is
export: Command not found.
export: Command not found.
export: Command not found.
Illegal variable name.