Bwa Bwasw Memory Footprint With Increasing Z Value?

Question

[2184687-1231-83-]

What is the supposed behavior of bwa bwasw with increasing -Z values in terms of memory requirements?

For example, I've been doing some tests trying to align 1000bp sequences to the rather small yeast genome (I tried both indexed with default method and specifying -a bwtsw) and it goes up very quickly to multi-GB RAM usage:

~/src/github/bwa/bwa bwasw -z 100000 saccharomycescerevisiae.bwtsw.fa.gz homosapiens.tss.1000.fasta > /tmp/100000.sam

~/src/github/bwa/bwa bwasw -z 10000 saccharomycescerevisiae.bwtsw.fa.gz homosapiens.tss.1000.fasta > /tmp/10000.sam

~/src/github/bwa/bwa bwasw -z 1000 saccharomycescerevisiae.bwtsw.fa.gz homosapiens.tss.1000.fasta > /tmp/1000.sam

~/src/github/bwa/bwa bwasw -z 100 saccharomycescerevisiae.bwtsw.fa.gz homosapiens.tss.1000.fasta > /tmp/100.sam

Any ideas what I may be doing wrong?

[lh3]

Don't use -Z over 100; otherwise most memory are unused.

Answer 1

[Alex Paciorkowski]

I thought this was the expected behavior for bwasw with increasing values of -z. This decreases the error rate for alignment, but brings a higher cost of CPU usage.

I looked at Table 1 in Li & Durbin's 2010 paper. For sequences of 1000bp going from 10% to 2% error rate increases the CPU sec on their 2.5 GHz CPU. Even though there is not always a relationship between CPU time and RAM, in this case there is since with a higher -z more sequence is held in RAM at any given time.

Is your RAM usage increasing far out of proportion to this?

[Arelicorlaior]

What is the basis for RAM usage? CPU usage is fine, it's RAM usage I don't understand.

[Alex Paciorkowski]

In this case I suppose higher -z value gets more sequence held in RAM