Blast 2.2.24+ Database Error

Question

[Niek De Klein]

When running local BLAST on a big database (all human proteins) I get the following error:

proteinsApplicationError: Command 'blast-2.2.24+/bin/blastp.exe -query "query.fasta" -
db "human-proteins.fasta" -out blastOutput.xml -evalue 0.01 -outfmt 5'       
returned     non-zero exit status 2, 'BLAST Database error: CSeqDBAtlas::MapMmap: 
While mapping file [human-proteins.fasta.psq] with 5550749 bytes allocated, caught 
exception:'

When googling I only got to the source code of seqdbatlas: http://www.ncbi.nlm.nih.gov/IEB/ToolBox/CPP_DOC/lxr/source/src/objtools/blast/seqdb_reader/seqdbatlas.cpp in which I found:

1214         if (expt.length()) {
1215             // For now, if I can't memory map the file, I'll revert to
1216             // the old way: malloc a chunk of core and copy the data
1217             // into it.
1218             
1219             if (expt.find(": Cannot allocate memory") == expt.npos) {
1220                 expt = string("CSeqDBAtlas::MapMmap: While mapping file [") + 
                     (*m_Fname) + "] with " +
1221                     NStr::UInt8ToString(atlas->GetCurrentAllocationTotal()) +
1222                     " bytes allocated, caught exception:" + expt;
1223                 
1224                 SeqDB_ThrowException(CSeqDBException::eFileErr, expt);
1225             }
1226         }

My knowledge of C++ is limited, but what I get from this is that there isn't enough memory on the PC to run BLAST over that size of a database. Is this correct and if so, is there a way to run this BLAST without getting a computer with a bigger memory? If it's not correct, what is the error that I'm getting?

Thanks in advance, Niek

Answer 1

[Noam N. Kremen]

For large blast databases you will most likely need a 64bit operating system with blast compiled accordingly (default for blast is 32bit, If I'm not mistaken). Note that even then, a part of your blast db will be cached to disk, resulting in very slow performance.

[Noam N. Kremen]

Yes, seems like a memory problem. If your current operating system is 32bit and/or your blast version is 32bit and/or the DB you are searching is over 4GB, it is very likely to be a memory problem.

[Niek De Klein]

Thanks for the suggestion, however I'm not sure if I got the error correctly. Do you agree it is a memory problem?

[Niek De Klein]

Thanks a lot! Is there any way to test before hand if using a 64 bit is going to be enough, before aquiring a 64 bit machine?

[Michael Schubert]

Memory is addressed bytewise, which means for 32 bit you can have 232 byte-files, for 64 bit 264. That is 4 GB for 32 bit and 17179869184 GB for 64 bit. Note that your filesystem has a limit as well, that is likely to be lower (2 TB for ext3).

[Niek De Klein]

@Noam N. Kremen I'm still not sure if it is a memory problem, since there was no error running it with Maize or Soybean proteome. Here are the sizes of the fasta files used to make the database: human -22125 kb maize -30429 kb soybean -25262 kb

Answer 2

[Niek De Klein]

Fixed it by breaking the query file up in two and using blast 2.2.25 instead of 2.2.24