Pegasus Vs Taverna

Question

[Jeremy Leipzig]

Can someone compare and contrast the following scientific workflow frameworks?

• Taverna http://www.taverna.org.uk/

• Pegasus http://pegasus.isi.edu/

Answer 1

[donal.k.fellows]

What sort of criteria did you have in mind? Yes, they're both (fine) workflow systems but they're rather different from each other.

Pegasus is focused on coordinating programs that transform files into other files, which is definitely a powerful paradigm. In tooling terms, it works through command line tools only: if you're working with complex workflows, you'll _definitely_ want to have an extra tool around it to generate the workflow for you from some higher-level description. The tooling is such that it is relatively easy to use it behind websites, with other programs, etc. Because you typically generate the workflow programatically, you can easily adapt the workflow according to your particular requirements (provided you're a programmer).

Taverna is much more of an integrated system, and it works at a higher level of abstraction — not files, but rather the data that you might find in them, and not programs, but rather the operations you might do with them. It comes as a GUI workbench for creating the workflows, running them and visualizing them, a command line tool for just running them, and a server for running workflows when you need more power (e.g., to support a portal). The workflows are comparatively awkward to edit by mechanisms other than using the workbench.

To look at the other things you mention in your comment:

• workflow and makefile concept features — Neither system is make and neither works like it. Yes, they all support branching, parallel execution and data merging, but some of the other things you mention under this just don't make sense.
• ease of development — It really depends on who you are and and how you like to develop. In my opinion, Pegasus is perhaps better for workflow creators who are principally programmers (because it has a less complex workflow format to generate) and Taverna is perhaps better for workflow creators who are principally scientists (because it comes with a GUI tool for the job).
• support — Both are supported. Both are producing new versions. There's multiple ways to get support in both cases. I'd call this one a score draw…
• cluster support and tracking — I'm not quite sure what you mean by this. The problem is that when you're dealing with workflows that may potentially be accessing many remote services (that might themselves be cloud-hosted), the meaning of clustering gets very… slippery… and I really don't know what you mean by tracking.
• workflow visualization — Taverna does this; has done for many years. There may be a separate tool for this with Pegasus, but I don't know it (and the different pattern of workflow that is often associated with Pegasus is considerably more difficult to visualize for various reasons, notably that Pegasus workflows are frequently using very wide span-outs; this is a problem that Taverna avoids by representing such things as a single processing element that has multiple data items flowing through it).

_Declaration: I'm one of the Taverna developers._

[Jeremy Leipzig]

by tracking I mean when run on a cluster using a job scheduler does the workflow system track what a process is doing on a node and present this to the user?

[steve]

Looks like this is an old question and the answer-er here has not been around for a while; does anyone else know if Taverna is able to use something like an HPC cluster running SGE? It would have to submit jobs e.g. with qsub and then monitor them for completion, etc.

Answer 2

[Alaninmcr]

What criteria do you want them compared with?

[Jeremy Leipzig]

OK well, for a start: - Workflow and makefile concept features: branch, merge, validatation, reentrancy, implicit rules, etc. - Ease of development - Ease of use - Support (forums, bug-tracking, etc...) - Cluster support and tracking - Workflow visualization

[Jeremy Leipzig]

or whatever you want to comment on, Alan. Beggars can't be choosers.