Problem Running Blast Jobs.

Question

[Anjan]

I started to run blast, locally on my machine, on 4 files with 1323, 210, 501, 166 fasta sequences each.

For all jobs except the first one blast returned an error:

blastall(7004) malloc: *** mmap(size=2097152) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug

The first job is currently still running as I'm writing this.

The exact blastall command used is as follows:

blastall -p blastn -d /Users/anjan/blastdb/nt -i <input.filename.fasta>  -o RPMBBP10.fasta.blast.out -m 8 -b 10000.

The version of Blast is 2.2.17. I'm running this on a Mac server with 2 Quad-core Intel processors with 16GB RAM. Anyone has any ideas on what the source of the problem is?

Answer 1

[Drio]

Is it possible you compiled or downloaded a 32bit version of the blast binary? If compiling from source, try to add "-arch x86_64" in your compiler flags. You should be able to add that in your configure file or directly in your Makefile if there is no configure.

[Neilfws]

This is almost certainly the explanation. The nt database is large (several GB) so BLAST will fail on 32-bit systems or when using 32-bit BLAST by mistake. See these troubleshooting notes - http://defindit.com/readme_files/blast_failed_to_allocate.html - and mail list thread with a similar problem - http://lists.apple.com/archives/fortran-dev/2008/Jun/msg00002.html.

Answer 2

[Michael]

You are running a pretty outdated version of the Blast binaries! Try the latest version: Blast+ v. 2.2.25 (yours is 2.2.17) from ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/ or the latest legacy blast (only if you must, I don't see a reason to use this): ftp://ftp.ncbi.nlm.nih.gov/blast/executables/release/LATEST/

Answer 3

[Chris Evelo]

You seem to run out of memory (I guess Pierre realized that and added the "memory" tag for that reason). You could try to ask for less alignments. -b 250 is the default if I remember correctly. Why would you want 10.000 alignments anyway?

[Anjan]

Chris, the sequences I am blasting are used for diagnostics. Typically there are hundreds of these deposited by clinical labs around the world. Hence the (somewhat arbitrary) 10000 limit on the number of hits to return.

[Chris Evelo]

OK makes sense. But you could still try a run without the -b parameter to see whether that causes the problem (along with a number of other things that were suggested)

[Anjan]

yes good point. Thanks.

Answer 4

[Joachim]

The error code tells you that the requested memory was not available (http://lists.apple.com/archives/Darwin-dev/2008/May/msg00028.html).

Try running the jobs separately and use 'top' to monitor the memory footprint of each blast job. I am not sure which permissions you have on the server and whether you got access to other installed memory monitoring tools, so you could use modified versions of Code 2 -- 4 as given in http://joachimbaran.wordpress.com/2011/04/03/flystockdb-preprocessing-improvements/ as a poor man's memory-footprinting solution.

Once you have an idea on your memory needs, you can start dividing your current blast-jobs into more but smaller jobs.

Answer 5

[Pierre Lindenbaum]

Yes, as Chris said it could be a memory problem. But as the origin of the problem was a call to mmap and the error code=12 is a CORE_FILE_OPEN .

#define CORE_FILE_OPEN    12  /* Error opening file */

it could also be an I/O problem.

[Anjan]

problem does not occur when command is run with a file with 7 sequences. that suggests that it may not be an IO issue....

[Joachim]

I think that is not the error code you are looking for.. handwave The errno 12 above is related to mmap and not the one in ncbierr.h.

[Pierre Lindenbaum]

ok Joachim. I was not sure about it. But anyway, an error for mmap could be related to I/O.