Is there a good way to test CWL tools and workflows?

Question

[Carlos Borroto]

I'm getting started writing CWL tools and workflows. I would like to get a good testing framework setup from the beginning. Is there something that can help testing CWL? How are you testing CWL?

I saw cwltest exists. However, the documentation is very sparse. I haven't figured out what should go inside test-descriptions.yml. It is also not clear to me if this tool is just checking for syntax errors, or will it actually try to run a tool or a complete workflow.

Thanks, Carlos

[Michael R. Crusoe]

cwltest is for the execution of CWL tools and workflows and comparing their outputs with an expected result or an expected failure.

ionox0 comment should be changed to be an answer, as that is basically my response.

[Martín]

In case someone else lands here looking for the elusive cwltest documentation:

• There's a short description in the CWL GitHub repo's CONFORMANCE_TESTS.md
• As recommended in the (still open) request for better docs, probably the best is to browse through CWL's conformance_tests.yaml.

Answer 1

[Michael R. Crusoe]

https://www.commonwl.org/#Editors_and_viewers lists tools to assist with CWL composition, including a graphical user interface and plugins for many popular IDEs (Atom, vim, emacs, Visual Studio Code, IntelliJ, gedit). Some of these plugins will do a basic syntax check for you as well.

Answer 2

[ionox0]

Added as answer per @mr-c comment

Some suggestions would be:

1. Testing with cwltool and the --js-console and --debug options. Also using --tmp-outdir-prefix, -leave-tmpdir, and --leave-outputs to inspect results from intermediate steps

2. Having small sample inputs that can be used to run the workflows quickly when making changes

3. Testing the individual tools before the whole workflows

4. cwltool --validate to only validate the workflow and not run it

5. cwltool --print-dot and cwltool --print-rdf to show graph of the workflow