Galaxy Cloudman - Best Way To Customize (And Share) Cloudman

Question

[Applet]

We want to use Galaxy CloudMan to set up an SGE (Sun Grid Engine) instance and run our Bioinformatics pipeline using Amazon's EC2 system.

We require a certain version of R, our set of Perl/Python/R scripts, and a few less common Bioinformatics libraries, and Python libraries.

Is there a way to install all of this stuff once on an Amazon Machine image (AMI) and have CloudMan use it? I'm reading the set up instructions here and it's not clear to me.

We also want to share this set up with other researchers (they would use their own Amazon credentials). Would that just involve making the custom AMI public?

[Applet]

I did find this regarding customization, but it's still not clear how to share my customizations.

http://wiki.g2.bx.psu.edu/Admin/Cloud/Customize%20Galaxy%20Cloud

Answer 1

[Enis Afgan]

After you've customized your instance (as described on the wiki page you pointed to in the comment), click the green 'Share-an-instance' icon next to name of the cluster on the main CloudMan console. You will then be presented with a box that describes how and what will happen when you share your instance.

Note that instance customization (and sharing) encapsulate tools and customizations that are contained to the file system listed on the Admin page. If any of the changes require system-level changes (i.e., new AMI), this method will not work.

Hope this helps and ask if you have any more questions.

Enis

[Enis Afgan]

Yes, the sharing functionality lets you define an environment (tools, data) and then share it with others. When you share an instance, a share string is generated (e.g., cm-b53c6f1223f966914df347687f6fc818/shared/2011-11-29--01-44). This string is provided under the Shared Instances list of the sharing box. You share this string with your colleagues. Then, they start a new CloudMan cluster and under the initial cluster configuration box click on 'More startup options' and paste the string into the Share-an-instance box. In a few minutes they will have a replica of your exact cluster.

[Applet]

Thanks. So that would let me share it with other people? How would they access my saved filesystem?

[Applet]

Where is the "initial cluster configuration box" and the "more startup options" button? It's probably somewhere obvious that I missed.

[Applet]

@Enis Afgan, Hi Enis. Where are the stored customizations for my share string stored? Would I be charged by Amazon for the storage, is that expensive?

[Enis Afgan]

The customizations are stored on a new EBS snapshot and in S3 in the cluster's bucket. The S3 part is very small so it would not cost much. The EBS snapshots (currently) cost 10 cents per gigabyte per month so, depending on the size of the initial data volume, it could add up.