NCBI Eutils error forwarding request

Question

[sklimek23]

I am having an issue connecting to the NCBI entrez eutils service. All my requests give the error message:

"error":"error forwarding request","api-key":"xxxxxxxxxx","type":"api-key", "status":"ok"

I am using the GET method to get the content of the url request. The thing is that my url requests where working fine the day before and I did not change anything in my code.

Would this be an issue with the NCBI website or something on my end? I do not know how or where to check the status of their serivces.

Edit by GenoMax: Redacted NCBI api-key posted in the question.

[Bioinformatician_in_trouble]

I guess it is the issue with the website as I am getting the same message when I tried 5 mins ago.

[GenoMax]

This is definitely a problem on NCBI's end. While it is likely to get corrected please put a ticket in using the support center.

[zhuzhuzhu]

Hi, do you know how to solve this problem? My website doesn't work now.

[GenoMax]

NCBI needs to fix whatever the problem is on their end. So please put a support ticket in at NCBI Help desk at link I posted above.

[kraman2]

Not working again: https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=nucleotide&id=KJ567057&rettype=gb&retmode=text

[migueldealmeida]

FYI, I am using Pubmed API for years now and today I got the same error. Found this post by googling the error, so maybe it's a problem on Pubmed's end?

[jimhavrilla]

ditto, had this issue with NIH Federal Reporter before, had to write several emails and it took a week to fix.

[kathka]

It might be the entire eutils serveice which are down, I'm developping some entrez-direct code into a pipeline, so first I suspected me being blocked for making a mistake (an esearch run with ~10 queries).

In terminal i receive a long 500 Internal Server Error, while in browser i receive a

{"error":"error forwarding request","api-key":"[MYIP]","type":"ip",
"status":"ok"}

[danjhayman]

I get exactly the same error just from using a single query, so I think you may be right about eutils

[danjhayman]

I'm having this problem too. Definitely wasn't happening yesterday evening. I'm guessing the problem is with Pubmed.

[kathka]

In addition, I checked their ftp, there doesn't seem to be any new releases this month, so wouldn't expect it to be version-related.

[zhuzhuzhu]

the same situation happened to my website！！！！！！！！！！！！！！！！！！！

[martin.damsbo]

Seven weeks later, and this problem still lingers?! I get the same erroneous response on roughly 20% of my 'Identical Proteins' Efetch queries - the same ones every time. Did anybody find a workaround, or are we still waiting for NCBI to fix it? Is there an open ticket at NCBI, or any other estimate on when this will be resolved?

[GenoMax]

This may be a very specific issue in your case. If you are doing a large number of queries build in some wait using sleep. I tried a small query and there was no problem getting a response.

[martin.damsbo]

It's a problem with specific proteins, regardless of the order or timing of requests. And it's on the "sequence level" (I'm looking for identical proteins), as many alternate accessions for the same protein sequence will yield the same failed result.

Example protein identified by these accessions: PAY79320.1, AAL22298.1, ESB76257.1, RIH13185.1, KXA03582.1 Enter any of these in the following Efetch query to see it fail: https://www.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=protein&id=PAY79320.1&rettype=ipg&retmode=text&api_key=<insert-api-key-here>

[GenoMax]

Since you are querying ipg (Identical Protein Group) database you are getting a representative protein sequence as denoted by WP accession. It represents multiple species.

$ esearch -db ipg -query "PAY79320.1" | efetch -format fasta
>WP_000447529.1 50S ribosomal protein L22 [Proteobacteria]
METIAKHRHARSSAQKVRLVADLIRGKKVSQALDILTYTNKKAAVLVKKVLESAIANAEHNDGADIDDLKVTKIFVDEGP
SMKRIMPRAKGRADRILKRTSHITVVVSDR

$ esearch -db ipg -query "AAL22298.1" | efetch -format fasta
>WP_000447529.1 50S ribosomal protein L22 [Proteobacteria]
METIAKHRHARSSAQKVRLVADLIRGKKVSQALDILTYTNKKAAVLVKKVLESAIANAEHNDGADIDDLKVTKIFVDEGP
SMKRIMPRAKGRADRILKRTSHITVVVSDR

If you were to query protein database with same accession numbers you will get different sequences (edit: sequence is not different but the headers are so these are from two different organisms) for each accession

$ esearch -db protein -query "AAL22298.1" | efetch -format fasta
>AAL22298.1 50S ribosomal subunit protein L22 [Salmonella enterica subsp. enterica serovar Typhimurium str. LT2]
METIAKHRHARSSAQKVRLVADLIRGKKVSQALDILTYTNKKAAVLVKKVLESAIANAEHNDGADIDDLK
VTKIFVDEGPSMKRIMPRAKGRADRILKRTSHITVVVSDR

$ esearch -db protein -query "PAY79320.1" | efetch -format fasta
>PAY79320.1 50S ribosomal protein L22 [Shigella flexneri]
METIAKHRHARSSAQKVRLVADLIRGKKVSQALDILTYTNKKAAVLVKKVLESAIANAEHNDGADIDDLK
VTKIFVDEGPSMKRIMPRAKGRADRILKRTSHITVVVSDR

[martin.damsbo]

You should get identical sequences regardless of which of the accessions used. This is also what you are getting from your example, despite claiming to show the opposite?

The real problem, however, is that the Efetch queries fail on certain protein sequences - and on all accessions denoting that sequence.

[GenoMax]

If you noticed the fasta headers in case of protein database searches they are different as compared to IPG searches. I have clarified that in my comment above.

Searches using Entrezdirect did not fail for me as shown by examples above.

[martin.damsbo]

What I'm interested in retrieving are the IPG reports on identical proteins, hence my "&rettype=ipg" in the original failing URL. Using your suggested Eutils command line tools (under CygWin on Windows), this would then be: esearch -db protein -query "PAY79320.1" | efetch -format ipg , which gives me the similar kind of error:

curl: (22) The requested URL returned error: 500 Internal Server Error ERROR: curl command failed ( Wed, Jan 27, 2021 11:03:06 ) with: 22 https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi -d query_key=1&WebEnv= MCID_601139ed155d522b5b101eaf&retstart=0&retmax=1&db=protein&rettype=ipg&retmode =text&tool=edirect&edirect=14.5&edirect_os=CYGWIN_NT&email=<redacted> ERROR: FAILURE ( Wed, Jan 27, 2021 11:02:55 ) nquire -url https://eutils.ncbi.nlm.nih.gov/entrez/eutils/ efetch.fcgi -query_key 1 -WebEnv MCID_601139ed155d522b5b101eaf -retstart 0 -retmax 1 -db protein -rettype ipg -retmode text -tool edirect -edirect 14.5 -edirect_os CYGWIN_NT -email <redacted> EMPTY RESULT

If I try your suggestion to retrieve the FASTA sequence (which was not my original intent, but just to compare), I get similar kind of errors in 1-4 retries (automatic retries conducted by Curl), and a result is returned in roughly half of the cases?

It seems to be tied to Efetch only, as the first Esearch command gives the correct result every time: esearch -db protein -query "PAY79320.1"

Answer 1

[rajvaghela23]

the error is gone and now it is working as expected.

[Bioinformatician_in_trouble]

I still get the error, anyone on the same boat?

[yz0017]

Update: It's on and off. Seems working again.

I've been having this problem for 4 hours. global query and esearch work fine. However, efetch still has issues.

[GenoMax]

Folks please send in problem tickets NCBI Help desk. Until NCBI hears from multiple people they may not escalate the issue.