Forum:Can Galaxy Be Used Under The Hood? How About The Galaxy Appliance?
4
1
Entering edit mode
11.4 years ago
seidel 11k

I was intrigued by the recent announcement of a Galaxy Appliance (also written about at rnaseq-blog), because it sounds like an easy way to give a good tool for genomic analysis to the average biologist (and because it's a step up in abstraction). With 16 Intel cores, 100 GB SS drive, 384 GB of memory, 16 TB of storage, and priced just under $20k, I could see many departments reaching for it as an "out of the box" solution to a general problem. Whether or not it's a good idea to get such a box, if a box appeared down the hall, I would hate for such a box to sit idle.

I have limited experience with Galaxy, but so far my experience is that it is used solely through a web interface. This, despite the fact that one can create workflows and custom routines. So here's my question: since Galaxy can be used via web-interface to complete a workflow, isn't there any way for more advanced analysts to set up/run the same workflows without using the web front end? Perhaps some kind of Galaxy API? I can imagine a frame work that would allow things to be accomplished under the hood, and then still be accessible via web front end, allowing advanced users (err, people who prefer command lines) to share the same resource with more general users.

If there's no way to do the above, a related question would be, is there any method for using idle resources on a computational appliance like that describe above? I can see departments buying the Galaxy appliance, and then having it sit idle while it gains users (or simply goes unused much of the time). But most of all I'm intrigued by the idea that an "appliance" meant to make things easy (and run pipelines) for general users, should also have a hood latch for those who want to rev the engine by pulling on the accelerator cable with their fingers (i.e. and run the same pipelines) - I just don't know if it's possible.

galaxy • 4.4k views
ADD COMMENT
0
Entering edit mode

I changed the title slightly to be more specific because I don't think it was clear until after a few posts that you were only interested in managing Galaxy on the Galaxy Appliance.

ADD REPLY
0
Entering edit mode

I changed it yet again (I hope you don't mind), as I was specifically trying to address the layers of using Galaxy via API, using Galaxy on an appliance via API, and using an appliance in general via API (perhaps for something not related to Galaxy). I didn't know if the first layer existed, but if it did, it implies the other two should. Anyway, at least now it forms a proper sentence :)

ADD REPLY
0
Entering edit mode

I'm glad you changed it to what you want, I was just trying to help based on some of the comments/answers. Anyway, this is a really interesting topic that I have learned about through this post.

ADD REPLY
0
Entering edit mode

Why is this Forum? I think it is a regular question.

ADD REPLY
0
Entering edit mode

Because it was abstract and not really focused. I'd never heard of an "appliance" for analyzing genomic data, most people wouldn't have experience with such a thing, and I was looking not so much for an answer, but for discussion on the topic of what role an "appliance" might have in a community such as ours. Not to mention, I'm usually at odds with people in determining what constitutes a "regular question" (I tend to have a very generous interpretation). Posting in Forum avoids that (I hope).

ADD REPLY
3
Entering edit mode
11.4 years ago
Peter 6.0k

Yes, you can run Galaxy tools etc via the Galaxy API http://wiki.galaxyproject.org/Learn/API

ADD COMMENT
0
Entering edit mode

It's good to know there's an API! When I mentioned this problem to my more sophisticated Galaxy friends prior to posting, they didn't know about it. I wonder if it will be available on the "appliance". Pity the articles above don't mention it, as it would also be a way to rope in savvy users.

ADD REPLY
1
Entering edit mode

Yes the Galaxy API is enabled on the appliance so you could run workflows from the command-line or a client app. Advanced users can also connect with ssh and submit their own jobs if they wanted to bypass the API.

ADD REPLY
2
Entering edit mode
11.4 years ago

since Galaxy can be used via web-interface to complete a workflow, isn't there any way for more advanced analysts to set up/run the same workflows without using the web front end?

yes, with a venerable software named Make .

see also qmake

ADD COMMENT
0
Entering edit mode

Thanks Pierre. I currently use Make for my pipelines, and it's wonderful in its efficiency, and somewhat esoteric to most of my friends :) However, I'm curious to know if/how I would be able to use Make on a galaxy appliance.

ADD REPLY
0
Entering edit mode

ok sorry, I didn't understand the context of your question :-)

ADD REPLY
1
Entering edit mode
11.4 years ago
Kraut ▴ 230

The Galaxy Appliance is an open platform and allows you to use the hardware resources however you see fit. To that end we install and configure Sun Grid Engine, specifically opengridscheduler, as the resource manager for galaxy. Jobs submitted through the web will be run through gridengine. Power users who don't want to use the web at all can submit jobs directly to gridengine using qsub or qmake as Pierre mentioned. The appliance works as a standalone system for data intensive jobs or can be integrated with existing infrastructure. It's extremely flexible. The linked article is only a press release. More technical details will be coming soon. I'm happy to answer questions here in the meantime.

Disclaimer: I one of the developers behind the Slipstream Appliance.

ADD COMMENT
0
Entering edit mode
11.4 years ago
SES 8.6k

I think Peter's answer is exactly what you were looking for, but I wanted to also recommend Taverna as a good method for generating workflows and running them programmatically (you can do this in any language locally, but there is also a Taverna API). There is also a Taverna-Galaxy project which seems like it aims to provide some interoperabilty between these two workflow systems. This is a really nice idea for sharing workflows generated with different tools, though I haven't personally had the need to explore this feature. Workflows are great for documenting methods, and if you are using a lot of web services in your workflow then they are easy to share (e.g., through myExperiment) and import. Though, these tools are configurable so I'm sure you can easily create a workflow around your own local set up.

ADD COMMENT
0
Entering edit mode

Yes, I was intrigued by Taverna after seeing a poster on it at the ISMB a few years back. Perhaps I should edit the title of my post above - I'm not asking how to automate workflows, I'm wondering specifically about the Galaxy Appliance, because it's the first time I've seen this kind of abstraction approach to a general genomic analysis problem (essentially a device that runs Galaxy, and only Galaxy). Thus if we have a method for running the device, that would be powerful. (maybe Taverna can run the Galaxy device :)

ADD REPLY
0
Entering edit mode

I see, I just focused on your specific questions, i.e., "...isn't there any way for more advanced analysts to set up/run the same workflows without using the web front end?" and thought you were looking for general approaches for managing these tools programmatically. It's not clear to me if this is possible on the "appliance" but maybe someone working on this project will show up and answer those questions.

ADD REPLY

Login before adding your answer.

Traffic: 2015 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6