Bowtie2 run out of memory

Question

[anon]

Hi All,

I tried to run bowtie2 with some simulated paired-end sequences, and I got a strange error. I'm using a virtual machine, I got for this VM environment 8 Gb RAM, so I assumed that this could be enough even in case of human data... Besides, before this run I used bowtie2 successfully. Anyone experienced the same before?

My bowtie2 command:

/usr/local/bin/bowtie2 \
  -t \
  -p 4 \
  -D 20 \
  -R 3 \
  -N 0 \
  -L 20 \
  -i S,1,0.50 \
  --rg-id "control_0_D4" \
  --rg "LB:Library_control_0_D4" \
  --rg "PL:Illumina" \
  --rg "PU:PE-1234567890" \
  --rg "SM:control_0_D4" \
  -x /usr/local/share/bcbio/genomes/Hsapiens/GRCh37/bowtie2/human_g1k_v37 \
  -1 ./fastq/control_0_D4_1.fq \
  -2 ./fastq/control_0_D4_2.fq 2 >> ./bowtie2/bowtie2.log |\
    samtools view -bhS - | \
    samtools sort - ./bowtie2/control_0_D4;
samtools index ./bowtie2/control_0_D4.bam

and the result of this:

Out of memory allocating plen[] in Ebwt::read() at bt2_io.cpp:272
Overall time: 00:00:00
Error: Encountered exception: 'std::bad_alloc'
Command: /usr/local/bin/../Cellar/

bowtie2/2.2.5/bin/bowtie2-align-s --wrapper basic-0 -t -p 4 -D 20 -R 3 -N 0 -L 20 -i S,1,0.50 --rg-id control_0_D4 --rg LB:Library_control_0_D4 --rg PL:Illumina --rg PU:PE-1234567890 --rg SM:control_0_D4 -x /usr/local/share/bcbio/genomes/Hsapiens/GRCh37/bowtie2/human_g1k_v37 -1 ./fastq/control_0_D4_1.fq -2 ./fastq/control_0_D4_2.fq
(ERR): bowtie2-align exited with value 1

[Sean Davis]

I'd grab a bigger VM. 8GB is tiny for genomics work. You can always watch "top" output during execution on a bigger machine and then scale to what you see there.

[anon]

but before this run, I used this configuration with succes, even when Bowtie2 run for human sequence alignment

[Devon Ryan]

Which version of bowtie2 are you using? There were a couple versions that would rapidly consume RAM sometimes, but not always. I think both Felix Krueger and I ran into problems like that with bisulfite datasets, but it's possible that a similar weird behavior can be invoked with regular data.

[anon]

I'm using the 2.2.5 version

[Devon Ryan]

Have you tracked memory usage as it processes one of the problematic samples? With the cases of this that I'd seen before, memory usage was stable for most of the run and then suddenly spiked exponentially at some point (I don't recall the exact cause, something to do with Ns). It'd be good to know if you see something like that or if you just have a gradual increase in memory consumption.

[anon]

The 'funny' thing is that bowtie2 executed almost like a sec. I think that perhaps my fastq data isn't correct. Thanks for your consideration!

[Devon Ryan]

That would seem likely then.