getBM error while using bioMart

Question

[MatStat]

Hi all,

This script used to work for me and now it doesn't anymore:

ensembl = useEnsembl(biomart = "ensembl", dataset = "hsapiens_gene_ensembl", mirror = "useast")


getBM( attributes = c("ensembl_gene_id", "entrezgene_id", "hgnc_symbol"),
                  filters = "ensembl_gene_id",
                   values = TPM_data$ensembl,
                   mart = ensembl
                  )

This is the error I am getting:

Error: failed to load external entity "http://www.ensembl.org/info/website/archives/index.html?redirect=no"

I tried to solve this by replacing mirrors but it didn't help.

Can anyone please help me to solve this?

Error: failed to load external entity "http://www.ensembl.org/info/website/archives/index.html?redirect=no"

[koen.vdberge]

I'm having the exact same issue. Could the servers be down?

[MatStat]

I was also wondering if they are down since I'm using it quite often and had no problems with this script in the past.

[dango147]

Same here. Same error, same script I always use. Changing mirrors doesnt help

[MatStat]

I've contacted Ensembl helpdesk now and described the problem. I will update if they'll answer and if the problem is actually central. Unless of course it will be solved here by someone.

[MatStat]

Update: I have received the following answer from Ensembl helpdesk:

Hi,

Thanks for reporting the error with the Ensembl website. Apologies for the problems. The issue with Biomart is now fixed.

Best

[xiaoyonf]

I have encountered the same error in R version 4.0.2 with biomaRt 2.44.1. Please help!

[Kevin Blighe]

Hi, can you please try just:

ensembl = useEnsembl(biomart = "ensembl", dataset = "hsapiens_gene_ensembl")

Also, which version of R and biomaRt are you using?

[MatStat]

Hi, Thank you for the prompt reply.

useEnsembl worked for me already. But I did try the line you wrote anyway without the mirror. No error in that line but when I try getBM it doesn't work and gives the same error.

  ensembl = useEnsembl(biomart = "ensembl", dataset = "hsapiens_gene_ensembl")



TPM_countdata$ensembl <- sapply( strsplit( rownames(TPM_countdata), split="nn+" ), "[", 1 )

#apply gene names on a result table
genemap <- getBM( attributes = c("ensembl_gene_id", "entrezgene_id", "hgnc_symbol"),
                  filters = "ensembl_gene_id",
                  values = TPM_countdata$ensembl,
                  mart = ensembl
                  )

Using: R version 4.0.2 (2020-06-22) bioMart 2.44.1

[Kevin Blighe]

I see. What is the output of:

str(TPM_countdata$ensembl)

?

Also, from where are you running this command (on a cluster)?

[MatStat]

Hi Kevin, Well for the first question, it is just a vector containing strings with the names of the genes as "ensembl":

> head(TPM_countdata$ensembl)

[1] "ENSG00000000003" "ENSG00000000419" "ENSG00000000457" "ENSG00000000460" [5] "ENSG00000000938" "ENSG00000000971"

I'm not sure what do you mean in the second question (sorry).

Best

[xiaoyonf]

It now worked for me. But I have to unload ‘GenomicFeatures’ package, which seems autoload biomaRt that cause version conflict. Thanks!

[user17394]

We experienced the same http connectivity errors with three Ensembl mirrors yesterday (20200809; useast, uswest and asia). It appears that Ensembl has corrected the error on their end, as bioMart versions <= 2.45.5 still work as expected (e.g. biomaRt_2.42.0 on CentOS 7).

Answer 1

[Mike Smith]

My initial assement is that something has changed on the Ensembl website, which now doesn't allow access via http. If you go to http://www.ensembl.org in your browser it will automatically redirect you to use https; the code in biomaRt is failing rather than following the re-direct.

Can you try using the latest development version from GitHub, and let me know if it improves the situation. You can install this via:

BiocManager::install('grimbough/biomaRt')

[Emily]

This is correct. There are https changes and we're attempting to get this fixed today.

[kwhiggins27]

I'm still having trouble using BiomaRt version 2.45.6 and R version 4.0.3. Is anyone else still running into this issue? Any ideas about workarounds?

[Sandip Darji]

Same issue with R 4.0.3 and biomaRt 2.45.7. They haven't been able to fix it.

EDIT: Restarted R session and the issue is fixed.

[xiaoyonf]

I have tried, but still not work. Does the library loading use library(biomaRt)?

[Mike Smith]

I would suggest starting a new R session. Then you can run:

## This install the latest version of biomaRt
BiocManager::install('grimbough/biomaRt')
## This loads the newly installed package
library(biomaRt)
##  Check the version of the package - the latest is 2.45.5
packageVersion('biomaRt')

If that all looks good, run your biomaRt query and let me know if you still see the error.

[dango147]

Worked for me! Thanks

[MatStat]

This has worked for me as well. Thank you for the solution.

Hopefully that would work for the rest.

All the best.

[HB]

Hi, I also encountered the same problem. I tried what you suggested and got:

Error: Failed to install 'biomaRt' from GitHub: (converted from warning) installation of package ‘stringi’ had non-zero exit status

Any suggestions?

[MatStat]

Hi,

Try to install the package: 'stringi'. Then, try again and see if it works.

Best

[hopeful-M]

Hi! I tried what you suggest. With the last version of the genome I'd like to investigate it works. But when I try with any less recent release it doesn't work and reports me this error:

Error in bmRequest(request = request, verbose = verbose) : Not Found (HTTP 404).

Can anybody please help me? Thanks!

[Mike Smith]

Can you provide an example of the code that leads to the error?

[hopeful-M]

For example, if I try to use the last version of human genome (v101) it works, but when I try with another one like version 99, it doesn't. Here I paste the code that I use:

library(biomaRt) ensembl = useEnsembl(biomart="ensembl", dataset = "hsapiens_gene_ensembl", version=99)

Thank you so much for your help!

[Mike Smith]

This should now be fixed in biomaRt version 2.45.9.

[wangxujun87]

Hi, I also tiied the lastest version and met the same problems ensembl = useEnsembl(biomart="ensembl", dataset = "hsapiens_gene_ensembl", version=98) Error in bmRequest(request = request, verbose = verbose) : Not Found (HTTP 404). or mart = useMart( host="https://sep2019.archive.ensembl.org", + biomart="ensembl", + dataset="hsapiens_gene_ensembl") Error in bmRequest(request = request, verbose = verbose) : Not Found (HTTP 404). biomart version is 2.45.9 and R version is 4.0.3. Is there any suggestions to solve the problems? thanks so much

[Mike Smith]

Currently, if I go to https://sep2019.archive.ensembl.org/ I get the message:

Server Status The Ensembl web service you requested is temporarily unavailable. We are working to restore the service as soon as possible, and apologise for any inconvenience caused.

This is a larger Ensembl issue, rather than something we can fix in biomaRt.

[davide.pernici]

I still have the same damn error even after this (biomaRt version changed), can anyone help me?

packageVersion('biomaRt')

[1] ‘2.45.8’

getBM(filters="ensembl_gene_id", attributes=c("ensembl_gene_id", "external_gene_name"), values = tmp$genes, mart = ensembl)

Error: failed to load external entity "http://www.ensembl.org/info/website/archives/index.html?redirect=no"

[Mike Smith]

This seems a weird one. If you run the function listEnsemblArchives() do you also get the error? That is the function that tries to read that URL, and in version 2.45.8 it should not be trying to read the http version.

[davide.pernici]

Just woke up today with the same code working now, so... problem solved I guess

[mat.lesche]

Hey,

I installed the package and still have the error

humanmart <- useEnsembl(biomart = "ensembl", dataset = "hsapiens_gene_ensembl", version = "96")
Error in bmRequest(request = request, verbose = verbose) : 
  Not Found (HTTP 404).

humanmart <- useMart(host='apr2019.archive.ensembl.org', dataset="hsapiens_gene_ensembl", biomart='ENSEMBL_MART_ENSEMBL')
Error in bmRequest(request = request, verbose = verbose) : 
  Not Found (HTTP 404).

[Mike Smith]

This code works for me at the moment,but it looks like some of the other archive sites are down (you get a 'down for maintenance' page in a browser) and will produces this same error. Hopefully this is only a temporary thing, but I think it is a server-side Ensembl issue rather than something I can fix in biomaRt or a setting you need.