RAxML multi-threading option in HPC cluster

Question

[MAPK]

I am trying to run raxml (https://sco.h-its.org/exelixis/web/software/raxml/hands_on.html) on HPC cluster using multiple cpus using a pbs script below, but for some reason it is only using one cpu and is very slow. I have selected 60 cpus with -T 60, but the process only seems to use a single cpu. Is there anything I am missing here? Thanks for your help in advance.

#!/bin/bash

#PBS -N RaxML
#PBS -l nodes=5:ppn=12
#PBS -l walltime=20:00:00
#PBS -j oe

cd $PBS_O_WORKDIR

raxmlHPC-PTHREADS-SSE3 -T 60 -m PROTGAMMAJTT -p 12345 -# 20 -s aligned_fasta_seq.fas -n T12

Answer 1

[MAPK]

So I figured out the answer to this problem. The multithreading in cluster works only when we use raxmlHPC-MPI (MPI version). Here is the command I used (note mpiexec used to execute in multiple cores): mpiexec raxmlHPC-MPI -N $PBS_NP -m PROTGAMMAJTT -p 12345 -s test_mpkk.fas -n T13

Also, -T option is not used in MPI version which should be replace with -N option.

Answer 2

[Michael]

-# 20 -s aligned_fasta_seq.fas -n T12

I know the -# is an option to raxml to set the number of bootstrap iterations.

Try using -N 20 or -N 100 instead, or quote the # in the command to avoid that this is interpreted as a comment, this doesn't seem to be the problem here but anyway.

Whether parallelized programs will use multiple threads depends on their implementation and the environment.

You are using raxmlHPC- with PTHREADS where most likely you need to use a version built with MPI (raxmlHPC-MPI...)

You requested 5 nodes with 12 cpus each, but that is not the same as having 60 cpus in total on a single computer. raxml-PTHREADS can only use multiple threads on a single computer/node (having 60 CPUs for example) because pthreads cannot communicate across nodes (afaik). In addition, tree searches are run in parallel in the process, so you need to run at least 60 bootstrap iterations to fully profit from 60 cpus, right?

I don't have a PBS cluster to test this at hand, so these are just suggestions from my side.

[MAPK]

Thanks Michael for clarifying.