memory consumption of bwa-mem
1
2
Entering edit mode
7.7 years ago
Lars ★ 1.1k

Hi all, I am working with Human amplicon data and have been using bwa mem (0.7.15) up to now. Unfortunately, on some datasets the memory consumption exceeds the 8GB RAM I have at spare.

Can someone tell me why this is the case?

I have already tried to modify the parameter (-c or -t) but none of them made it able to run it. Could it be caused by parts of the data aligning to repeat regions/low complexity regions?

If I cannot find a solution to it, I will switch to an alternative aligner that does not need that really high amount of RAM.

bwa-mem mapping memory RAM • 12k views
ADD COMMENT
2
Entering edit mode

8 GB is not really high amount of RAM. Don't most bottom of the line laptops ship with 8 GB these days? Hell, my phone has 6 GB RAM..

ADD REPLY
2
Entering edit mode

Thank you for this information about your phone with 6GB RAM. I cannot see any use in this comment.

I have a HPC with 2 TB RAM and 32 cores. But this amplicon project has to run on a laptop with exact 8GB RAM. It looks like bwa-mem is the wrong tool then. Thank you anyway.

ADD REPLY
1
Entering edit mode

Phone having 6 GB RAM goes to show how exhausting 8 GB of RAM is not "incredible high memory consumption" as in the original title of your post..

ADD REPLY
0
Entering edit mode

I already changed it. Wanted to attract people. ;)

ADD REPLY
0
Entering edit mode

You should specify what you are aligning against. If it is not human genome then you could take a look at BBMap.

ADD REPLY
0
Entering edit mode

I align against the human genome.

ADD REPLY
0
Entering edit mode

But this amplicon project has to run on a laptop with exact 8GB RAM.

Then you have to consider aligning against a subset of the human genome. Perhaps just the regions you are amplifying.

ADD REPLY
0
Entering edit mode

That is not an option. It has to be the complete human genome and on exactly this laptop. My question: Is it possible with bwa-mem, by using some specific parameters?

ADD REPLY
0
Entering edit mode

Did you see Heng Li's (author of bwa) answer that I had linked below?

ADD REPLY
0
Entering edit mode

Yes, I know this comment of Heng Li and I tried it, but it didn't work.

ADD REPLY
2
Entering edit mode

If you are using human genome to align against then ~6 G is minimally required by bwa for one thread. Other aligners actually will need more than 8 G.

ADD REPLY
0
Entering edit mode

Invest in your future and get a better machine.

ADD REPLY
0
Entering edit mode
7.7 years ago
d-cameron ★ 2.9k

Close to 8GB is a fairly typical index size for a human reference gnome [1]. If poor performance is acceptable, or the memory usage is only just over 8GB, you may wish to considering increasing and/or adding additional swap space to allow bwa to run.

Be aware that amplicon sequencing has artefacts not present in WGS sequencing that your pipeline may need to address [2,3].

[1] https://doi.org/10.1093/bioinformatics/bts605

[2] http://bmcgenomics.biomedcentral.com/articles/10.1186/1471-2164-15-1073

[3] https://academic.oup.com/nar/article/43/6/e37/2453415/Insight-into-biases-and-sequencing-errors-for

ADD COMMENT

Login before adding your answer.

Traffic: 2063 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6