Local Blast Error

Question

[bry.th]

I'm trying to use blast agains a de novo trinity assembled transcriptome.

blastn -query /trimmed_concat_trinity_assembly.fa \
-db nt \
-outfmt 11 \
-max_target_seqs 5 \
-num_threads 24 \
-out /trimmed_concat_trinity_assembled_blastn.nt_20210203.asn

It runs for quite a while (>60 hours) before I get the following error message:

Error: (802.4) [blastn] [Cannot close serializing output stream] Exception: basic_ios::clear: iostream error
BLAST failed to write output

However, the output file is present (and absolutely enormous! More than 4 x the assembled transcriptome, btu I haven't run out of space, so that doesn't seem to be it).

Any suggestions?

[5heikki]

Why are you trying to write to ROOT/trimmed_concat_trinity_assembled_blastn.nt_20210203.asn? Is your query file really in ROOT/trimmed_concat_trinity_assembly.fa?

[bry.th]

No, I trimmed the filepaths to make it more readable

[Mensur Dlakic]

My guess is that there is some kind of file size limit (4Gb? 64Gb?) that is either BLAST-dependent or OS-dependent. The actual size of your existing file may tell you what that limit is, assuming there is one.

I suggest you divide your input file into chunks and search them separately, or specify an -evalue that will limit the output size.

[lieven.sterck]

Also double check if you really need the asn output format (it's not the most space efficient one).

And a word of caution: be careful when using the -max_target_seqs 5 parameter, it might not give you the result you expect.

[bry.th]

I'm following a methodology from a paper (https://academic.oup.com/icb/article/56/6/1103/2647085 doi 10.1093/icb/icw107). They output in asn and then convert to fmt. Is that unwise? What would you recommend -max_target_seqs to be (and why? I'm very new to this)?

Thanks for the feedback

[lieven.sterck]

that's OK than, just double checking.

for the max_target_seq parameter, you should google around for it, but in a nutshell: you can not expect that it will return the best x matches when you set that parameter.

here is a good start to read up on it: https://blastedbio.blogspot.com/2015/12/blast-max-target-sequences-bug.html

[bry.th]

This link is a great place to start, thanks so much! I had no idea it was so potentially problematic