Between 10th March 2020 and 16th April 2020 the EMBL-EBI data centre will be migrating to a new physical location resulting in a period of reduction in availability and functionality in a number of Ensembl services.
What will be affected?
- There will be a temporary loss of all tools, including VEP and BLAST, at the beginning and end of the migration period. Jobs will not be saved during the whole migration period.
- Accounts will not be available
- BLAT will not be available
- Vertebrate archives from e88 to e98 inclusive and all non-vertebrate archives will not be available
- Our FTP site (ftp.ensembl.org) will be served from an alternative server at ftp://ftp.ebi.ac.uk/ensemblorg/
Please see below for more details and ways to mitigate this downtime. If your service is not listed above then you should assume it will be available during this period.
VEP and other tools
We have an alternative server set up for VEP and other tools, such as BLAST, Assembly Converter and the LD Calculator. It is not possible for jobs to be copied between these servers. This means that if you have jobs saved to your session or account prior to the start of migration, you will not be able to see them during the migration, but they will come back once normal service resumes. Similarly, any jobs you run during the migration period will disappear when we return to the normal servers.
We recommend downloading the data from any jobs you wish to keep before the migration occurs, and during this period. Since accounts will not be available, you cannot save to your account.
We will advertise the time of the switch over in advance as any jobs that are running when it happens will be lost. We recommend that you resubmit any lost jobs after the switch. We will close the input forms so you cannot start any new jobs in the hour before the switch, but some larger jobs may still be running from before this.
If you are using the offline version of tools, such as the VEP, these will not be affected.
Accounts and configurations
If you have an Ensembl account, this will not be accessible during the migration, as these are not enabled on our back-up servers. You will not be able to access any saved bookmarks, jobs, custom tracks or configurations, or save any bookmarks, jobs, custom tracks or configurations within Ensembl during this time.
Our main servers and back-up servers will not communicate with each other. This means that if you have a session running at the time when we switch between the servers, any configuration you have set up will not be saved and you will need to reconfigure when you refresh or return to the page. This includes any custom data tracks that you are viewing in Ensembl, which will need to be re-attached. We recommend taking a screenshot of your current configuration so that it can be easily replicated.
BLAT
The BLAST-like alignment tool, BLAT, which is the default nucleotide sequence search tool used by BLAST/BLAT in Ensembl, will not be available during the migration. Jobs will default to BLAST instead, which may mean jobs take longer to run. The BLAST/BLAT tool will experience the same server shift as VEP.
Archives
Some of our archive sites will be completely unavailable. You will not be able to access any archive websites for non-vertebrates. For vertebrates, archives e88-e98 inclusive will be inaccessible.
FTP
If you download flat files for your large scale analysis from our FTP site, you will find that the usual site is not available. You will instead be able to access these from ftp://ftp.ebi.ac.uk/ensemblorg/, where you will find the files organised into the same structure.
This means that any direct links to the FTP site, both from our own site and any you have saved, will not work, but you can access the same files by changing “ftp.ensembl.org/pub/” in your URL to “ftp.ebi.ac.uk/ensemblorg/pub/”, for example “ftp.ensembl.org/pub/current_fasta/homo_sapiens/cdna/Homo_sapiens.GRCh38.cdna.all.fa.gz” will not work, but “ftp.ebi.ac.uk/ensemblorg/pub/current_fasta/homo_sapiens/cdna/Homo_sapiens.GRCh38.cdna.all.fa.gz” will.
This mirror of our flat files will remain active after the migration should you wish to use this server to access our data.
Why is this happening?
The EMBL-EBI data centres are moving to new premises. This means that the servers will be turned off, physically moved and reinstalled. While some of our services have alternative servers that can be used in the event of a data centre shutdown, some are only served by the single data centre and we have no way of providing them otherwise.
When exactly will this occur?
Shut-down will begin on the 10th March. Because of the distance to travel and the complex nature of the servers that need to be reconnected in the right way, we anticipate that services will start to come back online from the 16th April. We will communicate when things go down and as they are restored using Biostars, our blog, our dev and announce mailing lists and our Twitter and Facebook accounts.
Dates of data centre migration changed to 10th March-4th April to reflect current estimates of completion of works.
Are people in UK allowed to work on this? Or the physical move is done. Only reconfig remains?
The physical move was completed last week and we are currently in the process of rebooting different systems and slowly bringing the services back in an organised way. We actually hope that many of the services will be available again before the 4th April but it's a work in progress.
Hi Ben,
Has access to by ascp been disabled? I'm trying to run some downloads using the python script provided by Ben Woodcroft here. His script generates the following ascp command:
But currently fails with the following error:
This is the same problem that has been reported elsewhere: https://www.biostars.org/p/427157/
What I'm confused about is the ftp paths pulled down by Ben Woodcroft's script seem to be accurate, and I can download directly from the ftp link with wget etc without modifying as specified above. It's just ascp that seems to fail. Do I need to modify these paths somehow too? Or is this service offline?
Thanks in advance for your help! Jesse
Downloads from ENA (aspera) are not available during the move. You should use SRA for now. It does not support aspera but does make downloads available via AWS.
Hi, thanks for your reply. I'm still a bit confused though - if downloads are not available during the move, then how come I can still access the ftp site and download from there?
Parts of Ensembl are available albeit with reduced functionality.
ascp is now working for me as of Apr 9th!
The dates of data centre migration have been changed to 10th March-9th April to reflect current estimates of completion of works.
Dates of data centre migration changed to 10th March-16th April to reflect current estimates of completion of works.