ABYSS can not handel my project on 32G RAM

Question

[kk.mahsa]

i have 40x paired end (2*100) reads (genome size is approximately 2G) and I wanna de novo assembled them using ABYSS. i ran ABYSS on 32G RAM computer and it worked for 5 days without any results and i killed running due to some problems. i used ABYSS v 2.0.0. in abyss manual page mentioned that abyss v2.0.0 could handle human genome (76x) on 34G RAM while it can not done my project. why?

ABYSS, could give me good scaffold using paired end reads or it need mate end reads to do that?

thanks

mahasha

[Medhat]

can you support any log data about at which phase was the software running? and yes mate pairs or other long reads like SMRT PacBio or nanopore will give you better results

[kk.mahsa]

used command was:

abyss-pe k=64 n=5 s=200 name=myscaffolds lib='pe1 pe2' pe1='read1_1.fastq read1_2.fastq' pe2='read2_1.fastq read2_2.fastq'

abyss reaction is:

Reading pe1='read1_1.fastq read1_2.fastq' ...

5 days and only Reading pe1='read1_1.fastq read1_2.fastq' ... !!!

is my command wrong?

[Medhat]

add this as comment not an answer;
as you are using

ABYSS-P: parallel (MPI) de Bruijn graph assembler

The np option of abyss-pe specifies the number of processes to use for the parallel MPI job. Without any MPI configuration, this will allow you to use multiple cores on a single machine.

did you tried to use np=10 (or any number) where this controls number of cores used by the software, also try to use v=-vv to know exactly what is happening

[kk.mahsa]

thanks Medhat, i will use np option and tell you about its result.

[GenoMax]

kk.mahsa : What hardware config do you have available on this machine?

[Medhat]

n: minimum number of pairs required for building contigs

not related to core usage

[kk.mahsa]

Processor: intel Core i7-4770k CPU@3.50GHz×8

CPU(s): 8

Core(s) per socket: 4

socket(s): 1

CPU family: 6

Thread per core: 2

[Medhat]

I think you can run 10 normally

[GenoMax]

That CPU has 4 cores and 8 threads. So using 10 may not be optimal.

[GenoMax]

What is the size of your data files?

Please use ADD COMMENT/ADD REPLY when responding to existing posts to keep threads logically organized.

[kk.mahsa]

sum of data (reads) is approximately 200G

[GenoMax]

In absence of any specific error messages only possibility is that the RAM you have is not enough for this amount of data. Do you also have at least 2x swap space allocated? While you did quote from Abyss site that quote must have some finer points that may not have been explicitly mentioned.

Can you take half of the data and see if the program makes progress? Keep an eye on top to see what happens to the memory. I am assuming you are the only user on this machine and nothing else is running (that can use memory)?

[kk.mahsa]

is it possible that i assembled each lane separately and then merge them into one final file? there are any programs to do that?