For some bioinformatic applications we use and also for molecular dynamics, we need to apply for supercomputing time. I would like to get feedback and opinions from you; which calls do you follow, which supercomputing centers do you apply to, etc. As of now I only know DEISA in Europe.
Why not using a cloud-computing services provider like AWS?
In our company (Era7 Bioinformatics) we use most of their services: EC2, S3, EBS, SQS... and recently CloudFormation) on a daily basis.
In our case it's vital for semiautomatic genome annotation processes and transcriptomics between others.
Cheers,
Pablo
Hi Liam,
First of all I'd like to point out that the site: https://gocdb.africa-grid.org/ does not work (at least right now...)
Once this has been said, I must say that I totally disagree with the statement 'Cloud and Grid are essentially the same thing'. Using Grid seems to me like a cumbersome process, (applying to become a user, using middleware that can vary between different centers...)
On the contrary, with AWS you just pay for what you use, there are no ties, and you can change anything at any moment.
@Pablo - We'll agree to disagree about the definition :-), although I agree that the usability of AWS is perhaps easier. The nice thing about the GRID, especially if you have very large datasets, is that you DO NOT pay for it. Each organisation agrees to contribute what resources it has. There are many physics groups that analyse massive datasets and I think with some scripts the whole process could be automated fairly quickly. I guess it depends on a users requirements.
You could apply to become a user of the GRID. In SA we make use of the resources made available by http://roc.africa-grid.org/. It is the same as cloud computing although people would argue with me about this. Essentially, all the collaborating institutions combine their cluster computing resources with high speed fiber and storage elements. You should be able to find more information on the site, and the corresponding collaborators in the EU, US, ME and SE-Asia. It might not be high performance computing as such, but it is distributed and if your job can be paralleled, broken into smaller chunks or can use multiple nodes at once, this might be something you should look at.
yes, I'd argue about this :)
at least for what you cite: https://gocdb.africa-grid.org/ I don't see how this relates to AWS
@ Flow - you would generally find out using the interface how many core hours you can get, this can be automated in the submission script and you find out within 5 minutes. I only have experience with the SAGrid, and the time between application and award was never more than 5 minutes.
I met the bioinformaticians at the MCA. They use the Dutch Life Science Grid but I don't know if it is distinct from DEISA.
see their paper:
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3018473
Initial steps towards a production "platform for DNA sequence analysis on the grid"
It looks like the Dutch Grid is connected to the other clusters on the grid. All you really need to do is follow the wiki's which most of the sites will have. Initially you would need to become part of a Virtual Organisation (VO), in my case, SAGrid, you would have to check with your local institutions. Depending on the agreements with your VO, you would have certain access rights to certain machine hardware in certain countries at certain institutions, but if that is not enough for your needs, you can always request membership at other VOs which may have hardware more suited to your analysis. Luckily, all the interactions with these machines are done using easy to use middleware of one variety or another.
I have best choice from CSC Finland as they are national and European grid and supercomputing resources.
Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
version 2.3.6
How much CPU time are you looking for?
as much as possible, millions of core hours
thanks for all the answers