Retrieve genome in fasta format from ncbi

Question

[tim.ivanov.92]

What is the way to retrieve genomes from ncbi via biopython? I am able to get a record for my genome of interest, and also i am able to download it manually from search

But how to download it inside a script?

from Bio import Entrez
Entrez.email = "[e-mail]"
handle = Entrez.esearch(db="genome", term="Drosophila eugracilis[Orgn]", idtype="acc")
record = Entrez.read(handle)
for i in record.keys():
    print i,record[i]

[Joe]

You need efetch, see for example:

https://stackoverflow.com/a/26347810/3691040

[tim.ivanov.92]

the problem with using nucleotide db is empy resulting list of id's:

search_term = 'Drosophila eugracilis[orgn] AND complete genome[title]'
handle = Entrez.esearch(db='nucleotide', term=search_term)
genome_ids = Entrez.read(handle)['IdList']
print genome_ids
>>> []

and when using efetch with this genome id - it finds something else:

records=Entrez.efetch(db="nucleotide", id=6863, rettype="gb", retmode="text")
print(records.read())

LOCUS       X51384                   564 bp    DNA     linear   INV 15-NOV-2007
DEFINITION  Caenorhabditis elegans DNA encoding U4-3 snRNA.

which is not a genome

[Joe]

Yeah that's weird, I see the same thing.

Have you tried your query directly on NCBI's website first to see what you get?

[Joe]

It seems you need to use db="genome" not nucleotide in your efetch.

ID 6863 in Nucleotide points to that sRNA, but that same ID number in Genome does correctly point to that Drosophila species (or find out what ID the drosophila genome is using inside nucleotide)

[tim.ivanov.92]

this is true - i used id from db=genome - i first found in in ncbi web server but when i change db to genomes in last request - it says

HTTPErrorTraceback (most recent call last)
<ipython-input-255-7978eb8b163d> in <module>()
----> 1 records=Entrez.efetch(db="genome", id=6863, rettype="gb", retmode="text")
      2 print(records.read())

/uge_mnt/home/tim_ivanov/pythonlibs/biopython-1.71/Bio/Entrez/__init__.pyc in efetch(db, **keywords)
    178             # more than about 200 IDs
    179             post = True
--> 180     return _open(cgi, variables, post=post)
    181 
    182 

/uge_mnt/home/tim_ivanov/pythonlibs/biopython-1.71/Bio/Entrez/__init__.pyc in _open(cgi, params, post, ecitmatch)
    528             handle = _urlopen(cgi)
    529     except _HTTPError as exception:
--> 530         raise exception
    531 
    532     return _binary_to_string_handle(handle)

HTTPError: HTTP Error 400: Bad Request

[Sej Modha]

Hi there,

It is not possible to download the sequences directly from genome database, you will need to link to the actual sequence holding record using elink.

[tim.ivanov.92]

Can you please give an example of using it in pipe with efetch to download genome? or point me on a tutorial page with it

Answer 1

[Sej Modha]

esearch/elink commands would be different depending on the genome of interest.

For example, following command can be used to download assembly in the fasta format using NCBI Unix utils:

esearch -db genome -query "6863"|elink -target assembly|esummary|xtract -pattern FtpPath_RefSeq -element FtpPath_RefSeq

This should give you the URL where assembly files are saved and relevant files can be downloaded from that URL.

[tim.ivanov.92]

Thank you, that does it!

Answer 2

[atiestorage]

Other aproximation in shell:

for org in \<br>
 "Agrobacterium tumefaciens" \<br>
 "Bacillus anthracis" \<br>
 "Escherichia coli" \ <br>
"Neisseria gonorrhoeae" \<br>
 "Pseudomonas aeruginosa" \<br>
 "Shigella flexneri" \ <br>
"Streptococcus pneumoniae"<br>
 do<br>
echo "Download URL for: $org"<br>
data=$(esearch -db genome -query "$org [ORGN]" | efetch -format docsum | xtract -pattern DocumentSummary -element Id)<br>
esearch -db genome -query "$data"|elink -target assembly|esummary|xtract -pattern DocumentSummary -element FtpPath_GenBank<br>
sleep 1<br>
done<br>

You can also play with the query and search by taxid and many other options
All is here:
https://www.ncbi.nlm.nih.gov/books/NBK25501/