blast running for several splitted query

Question

[seta]

Hi all,

I'm getting confused with a so basic issue. I have a large query (about 50 MB) that should be exposed to blastx. For simplifying, I split it into several files (say named x00, x01, x02, etc), now I'm not sure about the right command to run the blast job for these queries. Thanks for sharing your commands

Answer 1

[Ram]

If you have access to an HPC, run an array job - that's the best way to get this done fastest.

Serial processing is to just use a loop:

for num in $(seq 1 10)
do
    blastx input_${num} database.blastdb >output.out
done

You can use GNU parallel or different script files to deal with each BLASTX run.

If you have access to HPC, look up job arrays - these supply an ARRAY_ID or some such iterator variable value to each job in the array, and you can then use this array id to control which input file is used by that job. The command in the HPC script would look like:

blastx input_${ARRAY_ID} database.blastdb >output.out

and the command to submit the script would include the range of the ARRAY_ID variable like so:

qsub -t 1-10 job.pbs #assuming your HPC used PBS

HTH

[GouthamAtla]

-num_threads would not work?

[karl.stamm]

Threading the blast operation doesn't do much because most of the work is still run serially. It only parallelizes some of the overhead, on the assumption each thread will be disk IO bound. To force it and let the OS worry about disk IO, we manually run several instances. You can test it and see you get speedup until 2-4 processes are running, then they slow down regardless of CPU count.

[seta]

Thanks so much for your prompt reply Ram. I'll try it. I heard from you that the speed of blastall for doing blastx for small query is much better than ncbi-blast+ in your experience. Could you please let us if you have even compered the results of two program for the same query file, they were identical or not?

[Ram]

Yes, I did experience that blastall for smaller query sequences was faster than blast+, but that was in 2013, it might not be a valid observation today - blast+ might have been optimized.

I am sorry - I did not compare the results. This was early in my HPC experience, so I was making a ton of mistakes and blast+ was taking too long per learning cycle. Also, I just wanted to get it done and wasn't looking to learn such nuanced matters, sorry.

[seta]

Thanks Ram.

Answer 2

[Antonio R. Franco]

Try using Blast2Go as well. You can run different Blasts and get even more information, such as mapping, domain, etc