memory consumption of bwa-mem

Question

[Lars]

Hi all, I am working with Human amplicon data and have been using bwa mem (0.7.15) up to now. Unfortunately, on some datasets the memory consumption exceeds the 8GB RAM I have at spare.

Can someone tell me why this is the case?

I have already tried to modify the parameter (-c or -t) but none of them made it able to run it. Could it be caused by parts of the data aligning to repeat regions/low complexity regions?

If I cannot find a solution to it, I will switch to an alternative aligner that does not need that really high amount of RAM.

[5heikki]

8 GB is not really high amount of RAM. Don't most bottom of the line laptops ship with 8 GB these days? Hell, my phone has 6 GB RAM..

[Lars]

Thank you for this information about your phone with 6GB RAM. I cannot see any use in this comment.

I have a HPC with 2 TB RAM and 32 cores. But this amplicon project has to run on a laptop with exact 8GB RAM. It looks like bwa-mem is the wrong tool then. Thank you anyway.

[5heikki]

Phone having 6 GB RAM goes to show how exhausting 8 GB of RAM is not "incredible high memory consumption" as in the original title of your post..

[Lars]

I already changed it. Wanted to attract people. ;)

[GenoMax]

You should specify what you are aligning against. If it is not human genome then you could take a look at BBMap.

[Lars]

I align against the human genome.

[GenoMax]

But this amplicon project has to run on a laptop with exact 8GB RAM.

Then you have to consider aligning against a subset of the human genome. Perhaps just the regions you are amplifying.

[Lars]

That is not an option. It has to be the complete human genome and on exactly this laptop. My question: Is it possible with bwa-mem, by using some specific parameters?

[GenoMax]

Did you see Heng Li's (author of bwa) answer that I had linked below?

[Lars]

Yes, I know this comment of Heng Li and I tried it, but it didn't work.

[GenoMax]

If you are using human genome to align against then ~6 G is minimally required by bwa for one thread. Other aligners actually will need more than 8 G.

[David Langenberger]

Invest in your future and get a better machine.

Answer 1

[d-cameron]

Close to 8GB is a fairly typical index size for a human reference gnome [1]. If poor performance is acceptable, or the memory usage is only just over 8GB, you may wish to considering increasing and/or adding additional swap space to allow bwa to run.

Be aware that amplicon sequencing has artefacts not present in WGS sequencing that your pipeline may need to address [2,3].

[1] https://doi.org/10.1093/bioinformatics/bts605

[2] http://bmcgenomics.biomedcentral.com/articles/10.1186/1471-2164-15-1073

[3] https://academic.oup.com/nar/article/43/6/e37/2453415/Insight-into-biases-and-sequencing-errors-for