ENA bulk download all fastq files in study
2
0
Entering edit mode
7.3 years ago

Hi I want to bulk download all fastq files in a study from ENA http://www.ebi.ac.uk/ena/data/view/PRJEB3334

It is possible to click to the experiment and then download them but this would be very slow... any way to bulk download?

ena fastq • 8.8k views
ADD COMMENT
0
Entering edit mode
7.3 years ago
GenoMax 148k

You can click on "Bulk download files" button on the page you linked above. Alternatively, you can click on "TEXT" button and then open the file that gets downloaded. Links for all fastq files from the dataset are in this file.

ADD COMMENT
0
Entering edit mode

thanks - sorry what I actually mean to as was how to get all of those text files for all studies for this link: http://www.ebi.ac.uk/ena/data/view/PRJNA235852

I know how to go from there using wget with the text file.

ADD REPLY
0
Entering edit mode

Unless I am misinterpreting the statement above the second part of my answer has that info. 10th column in the file that gets downloaded using the TEXT link has the ftp links that you can use with wget. You can get them with a simple for bash loop or via multiple jobs, if you are using a cluster.

ADD REPLY
0
Entering edit mode

It seems as though your project is actually a "Parent project" with many components. So, while you can get texts for them individually, but not all at once. I don't have an exact fix but in my experimenting with wget while trying to familiarize myself with FTP I have a makeshift answer that may work. But, honestly, it may be easier to just download each individually, as this will require some parsing on top of the download.

Concisely, I looked at 2 urls from different projects and found what they cease to have in common. Based on that you can get an FTP directory and get creative with wget. Based on that, this may work

wget -r -l2 -A.txt ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR116/

(r is recursive, l3 tells it to go as far as 2 folders deep,and -A tells it what file type to look for) If you ultimately just want all the fastq.gz for this, I think you could also do this:

wget -r -l2 -A .gz ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR116/

I'll repeat that I mostly found this while looking for solutions myself, and there is almost definitely a quicker way, but this seemed to work

ADD REPLY

Login before adding your answer.

Traffic: 2365 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6