Maximum size limit for indexing genome with bowtie2 with large index option

Question

[Sitara]

Hello all

I have a fasta file with 100 GB of data and I am trying to create a bowtie2 index. It is showing the following error:

Ran out of memory; automatically trying more memory-economical
parameters. Using parameters --bmax 54 --dcv 4096

Any help would be appreciated. Thanks in advance

[Rob]

How much memory does the machine have? 100GB is quite large, so you'll need a reasonable amount of memory to even create the index.

[Sitara]

I am running on a 256 gb RAM machine with 35 cores

[Rob]

So it's quite possible that there is not enough free memory to build the index with Bowtie2's default settings. You have only ~2.5x as much memory as the size of the input file, and this is likely a shared machine where other processes may be using memory. You can attempt to reduce Bowtie2's indexing size using parameters like --packed and --offrate 7 or --offrate 8 (these latter ones will make mapping slower). Also, it's worth noting that it looks like the message you posted isn't an error. It's a message that lets you know the default parameters are inadequate, and the wrapper is trying to change the parameters to make the index building succeed on your machine. --bmax and --dcv are other parameters that affect the memory required for indexing, and so it's setting them to more conservative values.

[Sitara]

Thank you. Will look into it.

[h.mon]

Is this 100Gb a single genome? What are you trying to do?

[Sitara]

It is a multiple genome fasta file

[ATpoint]

...and why would you align to such a collection?