STAR aligner error

Question

[M.]

Hi all,

I hope you are well.

I have a slurm job for a STAR alignment job, as follows:

#!/bin/bash
#SBATCH --job-name=STARalignment
#SBATCH --account=investor
#SBATCH --qos=short_investor
#SBATCH --partition=short_investor
#SBATCH --cpus-per-task=12
#SBATCH --mem=60G
#SBATCH --array=0-39
#SBATCH -o /alignment/SlurmOutputs/%x-%j-slurm.out
#SBATCH -e /alignment/SlurmOutputs/%x-%j-slurm.err

module load star-2.7.0e-gcc-9.2.0-vynasg3

cd /alignment

# Get all the _trimmed.fq.gz files in the single-ended directory
input_files=($(ls /trimming/single_ended/concatenated_technical_replicates/*_trimmed.fq.gz))

# Get the file for this array task
FILE=${input_files[$SLURM_ARRAY_TASK_ID]}

# Extract the base name of the file (without the directory and extension)
BASENAME=$(basename "$FILE" _trimmed.fq.gz)

# Print the basename
echo "Processing file: $BASENAME"

# alignment
STAR --runThreadN 12 \
--readFilesCommand zcat \
--readFilesIn $FILE \
--genomeDir ./genomeIndex \
--outSAMtype BAM SortedByCoordinate \
--outFileNamePrefix single_ended_mapped/$BASENAME

In this job, I am getting the following two errors:

Even though the errors look like insufficient resource errors, it doesn't work to increase the resource amounts. And, when I use these exact resource amounts for the files raising errors, the process gets done without any disruption. I see these errors only when I sumbit the job with slurm arrays for all of the files at the same time. And the most strange thing is that I run this script multiple times, and the files that are causing errors change with every run.

Do you have any idea why this could be happening?

Thanks in advance.

[Ram]

Talk to your HPC sysadmin - array jobs might be getting different kinds of nodes compared to standalone jobs.

[M.]

I already told them but they couldn't come up with a useful solution :/

[Ram]

If they can't help you, we probably can't do much either. But we can optimize your STAR command. BAM SortedByCoordinate is the worst setting you can have for BAM output, switch to BAM Unsorted. You can always sort later using samtools. Also, set RAM consumption parameters explicitly.