hg38 goldenPath cpgIsland is down?

Question

[mariatmaria]

Hello all,

I came up with an error when building annotations for the hg38 CpG islands. I have run this chunk plenty of times, with no similar error. The code looks like this:

library(annotatr)
annots=c("hg38_cpg_islands","hg38_cpg_shelves","hg38_cpg_shores","hg38_cpg_inter") 

annotations = build_annotations(genome = "hg38", annotations = annots)
Building CpG islands... 
Error in open.connection(11L, "rb") : HTTP error 403.

Then, I explored the source code of build_annotations() and I found that the goldenPath path called for CpG islands is:

con = "http://hgdownload.cse.ucsc.edu/goldenpath/hg38/database/cpgIslandExt.txt.gz"

By trying to reach this website, I get this error:

While when I find the URL through searching, it is slightly different and I can normally connect and find the cpgislandExt file listed. This URL is:

https://hgdownload.soe.ucsc.edu/goldenpath/hg38/database/

So my questions are:

• Do you think that the permission error is coming from my connection, or is there something wrong with the UCSC server?
• How could I resolve this, so that I can build my annotations successfully?
• How is the difference in the URLs explained? (soe vs cse)

Thanks a lot in advance and let me know if something does not make sense.

[Pierre Lindenbaum]

something wrong with the UCSC server?

this. Same problem here.

How could I resolve this,

wait

[mariatmaria]

waiting still..

[GenoMax]

Have you tried changing the URL in the source code. It sounds like your institution is preventing you from connecting over a plain http link.

soe = School of Engineering at UCSC
cse = Computer science and Engineering at UCSC

Genome browser is hosted by Dr. David Haussler's lab. He is a faculty member in SOE/CSE.

[mariatmaria]

FYI: After waiting for 6 days the problem was resolved, with no change from my side. Thus indicating it comes from the server.

Thanks all for your help :)

Answer 1

[Maximilian Haeussler]

Sorry, due to the bad weather last week, we had a multi-level power failure, which caused trouble on the XFS file system of this server (the exact problem is not diagnosed yet). We put in place a backup server, but a few files were missing there and the full restoration from the backups and DNS changes took a few days.

The problem that you found probably has to do with DNS caching and that you were hitting once the old server and once the new server, because the changes to the DNS names can take 24 hours or more to make it around the world to all DNS server.

We will make this server more reliable in the future, by having them always in sync and making the switchover faster.

Answer 2

[cmdcolin]

I think goldenPath is case sensitive, also use https

this link works https://hgdownload.soe.ucsc.edu/goldenPath/hg38/database/

this also https://hgdownload.soe.ucsc.edu/goldenPath/hg38/database/cpgIslandExt.txt.gz

[mariatmaria]

Hello! I know this link is working, it is also stated on my post. The one not working is this one: http://hgdownload.cse.ucsc.edu/goldenpath/hg38/database/ Still today annotation is not working for me, giving the same error.