What's the best way to handle multi-file inputs in a CWL workflow?
0
0
Entering edit mode
7.5 years ago
oneillkza ▴ 110

I have a scenario in which I'm working with hundreds of files at once, wanting to process them through two kinds of steps:

  1. Within-file preprocessing (one file in, one file out).
  2. Across-file analyses (e.g. clustering, ie many files in, one or a few files out).

My question(s) are:

a. Is there a way in CWL to do something analogous to Makefile wildcards? e.g. in a CWL workflow to specify to run a tool once each on all the files in a directory?

b. Is there something similar at the tool level? The solution we came up with for clustering was to pass in a csv file specifying each filename to read in (along with some metadata for the subsequent heatmap). The potential issue with this solution is that the CWL runner isn't aware of all the files the tool is actually working on.

I tried looking through the examples, and couldn't find a good one for either of these cases.

cwl • 2.1k views
ADD COMMENT
0
Entering edit mode

So it looks like what I'm asking for is also asked for in How process inputs based on a filename pattern using CWL

Michael has said that this is under development, but not yet implemented.

ADD REPLY

Login before adding your answer.

Traffic: 2973 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6