"Optimal" setting of CPU and max_memory in Trinity

Question

[al-ash]

I'm wondering if there is any guideline (or and advice from experienced user) on the --CPU and --max_memory parametres in Trinity (v2.3). Will suboptimal setting of these parametred dramatically influence the computational time?

E.g. I'm running assembly of 10M illumina paired reads (from insect) on a machine with 20 CPUs and 100GB RAM - what might be reasonable settings in such a case?

[GenoMax]

Have you seen this?

[al-ash]

Yes, but I'm not really sure whether it means, that the optimal performance will be reached when I set the max_memory as close to the expected peak RAM requirement as possible...I'm still wondering what effect will have if I set, for example, the max_memory much higher than required according to the "1M~1GB RAM" rule of thumb (I can imagine that Trinity could crash if I set the max_memory way too low).

[GenoMax]

Generally the issue is that people try to run this analysis on a server that does not meet the minimum specs laid out in that document.

Since you have 10M reads and 100GB of RAM that definitely satisfies the minimum requirement. Having/using more memory for the job is beneficial since it would lead to less use of swap space on disk, which can slow things down. Assigning more memory will not be harmful. If you are the only user on this machine you could use all of the resources. If you share the machine with others then you would want to be considerate and use as much resources (with a minimum of 15-20G RAM) as others can spare.

[al-ash]

Ok, I did not realize that limiting the RAM might be because of not completely draining the machine - this makes sense. Thanks for clarification.

Since we touched this topic, I'm actually wondering - what if I set CPUs exceding the actual number of available CPUs? Would you expect some drop in trinity performance?

[GenoMax]

what if I set CPUs exceding the actual number of available CPUs? Would you expect some drop in trinity performance?

Likely. You will have threads that would be waiting for a core to become available before they can do any work.

[pld]

It is going to take a while, give it everything you've got. I wouldn't overthink this, you have a single small machine. I'd be more concerned about not having enough ram.

Answer 1

[euduca]

See this:

http://trinityrnaseq.github.io/performance/mem.html

See the options "--bflyCalculateCPU", "--bflyCPU", "--bflyHeapSpaceMax", "--normalize_reads" (quality trimming and the in silico normalization are currently not compatible), and "--max_memory.