STAR index preparation fails

Question

[prabin.dm]

I am struggling to prepare genome index in STAR. Any help will be appreciated.

This is the job report

Jul 08 11:32:04 ..... started STAR run
Jul 08 11:32:04 ... starting to generate Genome files
Jul 08 11:33:55 ... starting to sort Suffix Array. This may take a long time...
Jul 08 11:34:38 ... sorting Suffix Array chunks and saving them to disk...

------------------------------------------------------------
Sender: LSF System <lsfadmin@c04b08>
Subject: Job 5059239: <STAR_index_v5> in cluster <..............> Exited

Job <STAR_index_v5> was submitted from host <c33b06> by user <......> in cluster <.......> at Sun Jul  8 11:32:02 2018.
Job was executed on host(s) <3*c04b08>, in queue <long>, as user <.......> in cluster <.......> at Sun Jul  8 11:32:03 2018.
                            <3*c01b04>
</home/xxxx> was used as the home directory.
</project/> was used as the working directory.
Started at Sun Jul  8 11:32:03 2018.
Terminated at Sun Jul  8 11:35:03 2018.
Results reported at Sun Jul  8 11:35:03 2018.

Your job looked like:

# LSBATCH: User input
#!/bin/bash

#BSUB -q long   # partition name
#BSUB -W 4:00 # hours:minutes runlimit after which job will be killed
#BSUB -n 6      # number of cores requested -- this needs to be greater than or equal to the number of cores you plan to use to run your job
#BSUB -R "rusage[mem=3072] span[ptile=3]"
#BSUB -J "STAR_index_v5"        # Job name
#BSUB -o logs/%J%.out   # File to which standard out will be written
#BSUB -e logs/ %j.err   # File to which standard err will be written

module load star/2.5.3a


STAR --runThreadN 6 \
--runMode genomeGenerate \
--genomeDir star_genome_mm10 \
--genomeFastaFiles Mus_musculus/UCSC/mm10/Sequence/WholeGenomeFasta/genome.fa \
--sjdbGTFfile Mus_musculus/UCSC/mm10/Annotation/Genes/genes.gtf \
--sjdbOverhang 99

TERM_MEMLIMIT: job killed after reaching LSF memory usage limit.
Exited with exit code 137.

Resource usage summary:

    CPU time :                                   223.64 sec.
    Max Memory :                                 9216 MB
    Average Memory :                             4252.58 MB
    Total Requested Memory :                     18432.00 MB
    Delta Memory :                               9216.00 MB
    Max Swap :                                   -
    Max Processes :                              4
    Max Threads :                                10
    Run time :                                   180 sec.
    Turnaround time :                            181 sec.

The output (if any) is above this job summary.

Answer 1

[h.mon]

TERM_MEMLIMIT: job killed after reaching LSF memory usage limit. Exited with exit code 137.

Your job is being killed by LSF resource manager, because it is exceeding the memory limit you asked - about 9Gb. STAR is quite memory hungry, if this is a large genome (such as the human genome), you need about 32-34Gb to build the index. You have to ask for more memory and / or use a different queue.

[prabin.dm]

Thanks. I just submitted the same job asking more memory. Hopefully it works.

[prabin.dm]

Thanks again. That did work, although, the job report shows 60GB memory use. Thank you!

[h.mon]

If an answer was helpful you should upvote it, if the answer resolved your question you should mark it as accepted.