error using update_blastdb.pl downloading the compressed nr BLAST database from NCBI

Question

[liyong]

Hello all,

I am trying to use update_blastdb.pl (ncbi-blast-2.13.0+) to download the compressed nr BLAST database, the command is as following update_blastdb.pl --decompress nr. It shows the following error: "Downloading nr.00.tar.gz...corrupt download, trying again."

In the manual, the author mentioned that "if you run into any problems with this invocation, please try the –passive option, which is enabled by default in BLAST+ 2.8.1 and following", I am using blast-2.13.0+, so the --passive option should be default. Any suggestions how I could make this work?

Thanks a lot!

[iraun]

Few months ago I run into the same issue and I ended up downloading the database "manually" using:

wget -c https://ftp.ncbi.nlm.nih.gov/blast/db/FASTA/nr.gz

But maybe someone can provide the actual solution.

[liyong]

Thanks for the comments! This might be a good option. I will try to troubleshoot it first, if can't work it out, will try the "manually" downloading later. Thanks.

[GenoMax]

If you are behind a firewall it is possible that an intrusion prevention device may be causing the corruption. If that is the case you may need to work with your local network administrators to find a solution.

[liyong]

That's very good to know. I am using ssh connect to a remote Linux machine, not sure whether I am behind a firewall or not. I will contact our admin to double check. Thanks for the tips.

Answer 1

[liyong]

Also, I emailed ncbi's support team, to help people with the same questions, I post their response here:

"It can't hurt to try adding the '--passive' option. These failures are usually due to some internet disruption 
or timeout somewhere.

 The first volume, nr.00.tar.gz, is much bigger than the others, about 23GB. You might see if you can 
 download that file directly from the ftp site, https://ftp.ncbi.nlm.nih.gov/blast/db/

 I would use an ftp client, or wget from a command line.

 If that succeeds, you can try update_blastdb.pl in the same directory. The script should recognize the 
 presence of nr.00.tar.gz and skip it."

Regards

[rgray]

Hi, Did you achieve a solution for this problem? I am having a similar issue with the nt database. Thanks!

Answer 2

[Istvan Albert]

you can also switch sources, try a different source

SYNOPSIS
    update_blastdb.pl [options] blastdb ...

OPTIONS
    --source
      Location to download BLAST databases from (default: auto-detect
      closest location). Supported values: ncbi, aws, or gcp.

but do try with --passive on as well.

[liyong]

Thank you, Istvan. I will give it a try.