Metavelvet fails at Scaffolding stage

Question

[jamesT]

Hello,

I'm trying to assemble partial genomes from a metagenome with approximately 15 million paired end reads, which look like:

>SN7001163:100:D237CACXX:2:1108:5829:37887/1 N:0:AGGCAGA
CCCTCATCCGGCACGCCATCGAGAAGGGGGCGCACTGGATGACGCTCGAGGCGCGCGTCTCCAACACCGTGGCGCACGCGCTGTACCGCAAGTACACCTTC
>SN7001163:100:D237CACXX:2:1108:13068:98533/2 N:0:AGGCAGA
GTCCCGCCCAGCCGGGCCGCCGAGCGCGATCCCCTGGACGTCGTCGCGACGATGAGCGTCTTCGCGGAGTTCGCCAGGGCGGTGGGCGGCGACCTCGTCTC

I was following the tutorial on the Meta-velvet website, and compiled Velvet and Metavelvet using:

make 'MAXKMERLENGTH=101' 'OPENMP=1' 'LONGSEQUENCES=1'

I then ran Velveth:

velveth velvet/ 61 -fasta -shortPaired myreads.fasta

With a k-mer length of 61.

This seemed to run correctly, and created the necessary files. I then ran Velvetg:

velvetg velvet/ -exp_cov auto -ins_length 260

(I'm not sure exactly about my insert length - this was an Illumnia HiSeq run?), which also seemed to work, and produced contigs.fa and Graph2. The final output of this run was:

[188.416726] Estimated Coverage = 3.862745
[188.416758] Estimated Coverage cutoff = 1.931373
Final graph has 78220 nodes and n50 of 360, max 32930, total 22293747, using 3764027/13750351 reads

However, I then ran meta-velvetg ./velvet/ | tee logfile, and the results were:

[meta-velvetg] Check command line options...
    OK. Your command line options seem to be good.

[meta-velvetg] Load meta-graph ...
[0.000001] Reading read set file ./velvet//Sequences;
[2.006028] 13750351 sequences found
[14.165548] Done
[29.950580] Reading graph file ./velvet//Graph2
[29.950634] Graph has 102502 nodes and 13750351 sequences
[meta-velvetg] Category = 'short1'    Ave. = -1, SD = -1
[meta-velvetg] Category = 'short2'    Ave. = -1, SD = -1
[meta-velvetg] Category = 'long'    Ave. = -1, SD = -1
[meta-velveth] ...done (load meta-graph).

[meta-velvetg] Estimate coverage parameters...
[mate-velvetg] Estimate expected coverage ... yes.    Expected coverage = 3.86275
[mate-velvetg] Estimate expected coverages ... yes.
[32.550552] Writing into stats file ./velvet//meta-velvetg.Graph2-stats.txt...
[MetaHisto] First valley = 3
[MetaHisto] Largest peak coverage = 3 (frequency count = 3.2506e+06)
[MetaHisto] Noise cutoff coverage = 325060
[MetaHisto] Find 1-th coverage peak: 10 (frequency count = 2.2049e+06)

[meta-velvetg] Warning: Can't find multiple coverage peaks.
[meta-velvetg] Trun on single coverage peak mode. 

[MetaGraph] 1-th coverage peak = 3.86275
[meta-velvetg] Estimate coverage cutoff ... yes.    Coverage cutoff = 1.93137
[meta-velvetg] ...done (estimate coverage parameters).

[meta-velvetg] Remove low & high coverage nodes ...
[meta-velvetg] Min. coverage cutoff for short reads        = 1.93137
[meta-velvetg] Min. coverage cutoff for long reads         = -1
[meta-velvetg] Max. coverage cutoff for short & long reads = -1
[meta-velvetg] Min. contig length                          = -1
[VelvetGraph]  === Remove low coverage nodes ===
[32.788521] Removing contigs with coverage < 1.931373...
[32.804994] Concatenation...
[32.837935] Renumbering nodes
[32.837940] Initial node count 102502
[32.840015] Removed 23989 null nodes
[32.840018] Concatenation over!
[32.841552] Concatenation...
[32.843933] Renumbering nodes
[32.843936] Initial node count 78513
[32.844192] Removed 0 null nodes
[32.844194] Concatenation over!
[VelvetGraph]  === Remove high coverage nodes ===
[VelvetGraph]  === Clip tips hardly ===
[32.844225] Clipping short tips off graph, drastic
[32.857123] Concatenation...
[32.859803] Renumbering nodes
[32.859805] Initial node count 78513
[32.861166] Removed 126 null nodes
[32.861169] Concatenation over!
[32.861170] 78387 nodes left
[meta-velvetg] ...done (remove low & high coverage nodes).

[meta-velvetg] Scaffolding based on paired-end information ...
[MetaGraph]  === Scaffolding with single peak mode ===
[VelvetGraph]  === Rock Bank  ===
[32.861196] Read coherency...
[32.862602] Identifying unique nodes
[32.864256] Done, 61600 unique nodes counted
[32.864258] Trimming read tips
[32.866858] Renumbering nodes
[32.866860] Initial node count 78387
[32.867108] Removed 0 null nodes
[32.867110] Confronted to 0 multiple hits and 0 null over 0
[32.867111] Read coherency over!
[VelvetGraph]  === Create read paring array ===
[VelvetGraph]  === Detach dubious reads ===
[VelvetGraph]  === Activate gap markers ===
[VelvetGraph]  === Scaffolding ===

Meta-velvet created the two stats files, but not meta-velvetg.contigs.fa! I think it may have crashed at the Scaffolding stage. I'm running this analysis on an ubuntu machine with 60 GB of RAM. Can anyone provide any help as to what may be going wrong?

Thanks!

Answer 1

[afia]

Hi James,

I am Afia, a developer of MetaVelvet-SL. Thanks for your interest using MetaVelvet.

For your case, please also define the insert length when running MetaVelvet, for example:

velveth velvet 61 -fasta -shortPaired myreads.fasta
velvetg velvet -exp_cov auto -ins_length 260
​meta-velvetg velvet  -ins_length 260 | tee logfile

Your read size is 100 bp, isn't it? The k-mer size should be less than 100bp. Then, you only use short reads without the longreads. In order to use the memory efficiently, please re-install MetaVelvet, for example using k-mer size =71. Anyway, it's not mandatory.

make clean
make 'MAXKMERLENGTH=101'

Let me know if you have any other question or comments. You can join the MetaVelvet Google groups metavelvet[at]googlegroups.com or directly send message to us meta[at]dna.bio.keio.ac.jp

Thanks!

[jamesT]

Hi afia, thanks for your reply. I re-compiled velvet and meta-velvet with a max k-mer size of 61 and reran both as you did above. This time Metavelvet quit at a different line:

[meta-velvetg] Load meta-graph ...
[0.000001] Reading read set file ./velvet//Sequences;
[2.101447] 13750351 sequences found
[14.592552] Done
[30.005095] Reading graph file ./velvet//Graph2
[30.005149] Graph has 282865 nodes and 13750351 sequences
[meta-velvetg] Category = 'short1'    Ave. = 260, SD = 26
[meta-velvetg] Category = 'short2'    Ave. = -1, SD = -1
[meta-velvetg] Category = 'long'    Ave. = -1, SD = -1
[meta-velveth] ...done (load meta-graph).

[meta-velvetg] Estimate coverage parameters...
[mate-velvetg] Estimate expected coverage ... yes.    Expected coverage = 2.73826
[mate-velvetg] Estimate expected coverages ... yes.
[34.655362] Writing into stats file ./velvet//meta-velvetg.Graph2-stats.txt...
[MetaHisto] First valley = 3
[MetaHisto] Largest peak coverage = 3 (frequency count = 7.46521e+06)
[MetaHisto] Noise cutoff coverage = 746521
[MetaHisto] Find 1-th coverage peak: 9 (frequency count = 801957)
[MetaHisto] Find 2-th coverage peak: 13 (frequency count = 1.81906e+06)
[MetaGraph] 1-th coverage peak = 13
[MetaGraph] 2-th coverage peak = 9
[meta-velvetg] Estimate coverage cutoff ... yes.    Coverage cutoff = 4.5
[meta-velvetg] ...done (estimate coverage parameters).

[meta-velvetg] Remove low & high coverage nodes ...
[meta-velvetg] Min. coverage cutoff for short reads        = 4.5
[meta-velvetg] Min. coverage cutoff for long reads         = -1
[meta-velvetg] Max. coverage cutoff for short & long reads = -1
[meta-velvetg] Min. contig length                          = -1
[VelvetGraph]  === Remove low coverage nodes ===
[35.305708] Removing contigs with coverage < 4.500000...
[35.466570] Concatenation...
[35.780907] Renumbering nodes
[35.780915] Initial node count 282865
[35.783945] Removed 264984 null nodes
[35.783948] Concatenation over!
[35.784480] Concatenation...
[35.785712] Renumbering nodes
[35.785714] Initial node count 17881
[35.785784] Removed 0 null nodes
[35.785785] Concatenation over!
[VelvetGraph]  === Remove high coverage nodes ===
[VelvetGraph]  === Clip tips hardly ===
[35.785822] Clipping short tips off graph, drastic
[35.791554] Concatenation...
[35.809028] Renumbering nodes
[35.809031] Initial node count 17881
[35.809543] Removed 2890 null nodes
[35.809546] Concatenation over!
[35.809547] 14991 nodes left
[meta-velvetg] ...done (remove low & high coverage nodes).

[meta-velvetg] Scaffolding based on paired-end information ...
[MetaGraph]  === Scaffolding with multi-peak mode ===
[MetaGraph]  === Create read paring array ===
[MetaGraph]  === Detach dubious reads ===
[MetaGraph]  === Activate gap markers ===
[MetaGraph]  === Split into subgraphs ===
[MetaGraph] 1-th round
[MetaGraph] Primary coverage   = 13
[MetaGraph] Secondary coverage = 9

Any idea what I might be doing wrong? It did create meta-velvetg.Graph2-stats.txt and meta-velvetg.split-stats.txt. Here's what my k-mer peaks look like, perhaps the assembly just isn't good enough to scaffold?

[benkhedhermaryem044]

I'm trying to assemble partial genomes from a metagenome ; I was following the tutorial on the Meta-velvet website. I've compiled both velvet 1.2.03 and metavelvet-1.2.02:

make 'MAXKMERLENGTH=101' 'OPENMP=1' 'LONGSEQUENCES=1'

I then ran Velveth:

velveth velvet/ 61 -shortPaired myreads.fasta

With a k-mer length of 61 but I get this message:

Velvet can't handle k-mers as long as 61! We'll stick to 31 if you don't mind.

Answer 2

[lukas]

I have encountered very similar problem with meta-velvetg segfaulting at scaffolding=no (meta-velvetg 1.2.0.9).

Due to some technical issues i'm doing incremental assemby. I assemble 1 library at a time. Afterwards I use its output metavelvetg-contigs.fa and V_unused.fa to reassemble with next library. To make it work I need LONGSEQUENCES. In order to force LONGSEQUENCES I had to make brutal change in the code (global.h), because the makefile still does not parse, and run.c is not even compiled:

make 'LONGSEQUENCES=1' 'OPENMP=1'

The meta-velvetg seems to crash at scaffolding. It doesn't matter if I put scaffolding=no, or even delete -ins_length parameter. It crashes all the same.

Now I'm testing 1.2.0.10. Again no longseq, or openmp in mekafile parser. For openmp I can rationalize the lack, but for longsequences... no idea why it's missing in makefile, while rest of the code is there.

Answer 3

[alfbenpa]

Hi guys,

I solved the "segmentation fault" error by compiling metavelvet in a old 12.04 Ubuntu with g++ 4.6.2. I've reviewed several blogs and problems it seems to be linked with compiler versions from 4.8 -->.

After compiling metavelvet in 4.6.2 (have to be done in 12.04 because downgrade g++ in 14.04 or 16.04 produce errors) I copied the whole folder and paste into my 16.04 environment and now it produces the "meta-velvetg.contigs.fa" file with no errors during scaffolding.

I hope this works for all of you!

Cheers

Alfonso

[Vimac]

Thank you Alfonso! Now let's see where I can find and old Ubuntu machine :)

[Vimac]

Hi Alfonso, I tried to search for a ubuntu 12 computer here at my department but they are all 14 or above :( So I was wondering if you could send me your compiled meta-velvet via mail (kaapi(a)hotmail.com) Thanks!

Answer 4

[afia]

Hi James,

Is your read size uniform? 100 bp for all? The k-mer peaks looks good. I think it's not problem and the scaffolding should be run well.

Anyway, please try to change the kmer size, for example = 57, and see what happen?

velveth velvet 57 -fasta -shortPaired myreads.fasta
velvetg velvet -exp_cov auto -ins_length 260
​meta-velvetg velvet  -ins_length 260 | tee logfile

[jamesT]

Hi afia, thanks for your help. Yes, the read size is uniform. Unfortunately, the same thing happens with different kmer sizes - metavelvet quits at the ==scaffolding== line and the file isn't created. When I run metavelvet with -scaffolding = no, it prints out:

[33.745285] Starting pebble resolution...
Segmentation fault (core dumped)

I've been starting from scratch and trying it over, but I keep getting the same thing. It's a clean install of ubuntu 14.04. To install prereqs, I've been running:

sudo apt-get install build-essential g++
sudo apt-get install zlib*

[afia]

Hi James,

I am sorry for my late respond. I would like to make sure, you use the last version of MetaVelvet (MetaVelvet 1.2.02) and the last version of Velvet (Velvet 1.2.10)?

Answer 5

[yankang1992]

I finally solved this issues. Meta-Velvet should compile with gcc4.6 or lower.

[sidrairshad29]

Hey i have generated the contigs using velvet. Could you please tell me the next steps to whole genome assembly? What is the step next to scaffolds? Could please suggest me any tool compatible with windows taking contigs as input and giving scaffold OR fnall assembly

[Vimac]

Hi Yanjang1992, I have a similar problem with Metavelvet and I was wondering what you did to run it with gcc4.6. I currently have gcc4.9 and for what i have seen it is not that easy to downgrade to gcc4.6. Could you give a bit more info on how you did it. Thanks