scatter over optional argument

Question

[ionox0]

Is it possible to scatter over an optional argument (of type string[]?) when the argument isn’t provided?

The message that I get from cwltool when trying this is:

[workflow standard_bam_generation] starting step module_1
Unhandled exception
Traceback (most recent call last):
  File "/home/johnsoni/virtualenvs/pipeline_test/lib/python2.7/site-packages/cwltool-1.0.20180306140409-py2.7.egg/cwltool/workflow.py", line 401, in try_make_job
    emptyscatter = [shortname(s) for s in scatter if len(inputobj[s]) == 0]

[drkennetz]

Is there a reason you don't provide the argument? It seems like you should include it if you want to use it as an input. The error is returning that you are scattering over nothing.

[ionox0]

I want to be able to provide an array of adapter sequences to scatter over (string[]: [adapter1, adapter2, ...]), but if the adapters are not provided I would like to be able to provide another argument to the tool to specify it to use the default illumina adapter (string: --illumina), in which case I'd like the scatter to be ignored.

[drkennetz]

Is the string you want to scatter over an output from a previous step?

[ionox0]

No it's not an output from a previous step

[stian]

Could you use default: [] to avoid the un-scatterable null (aka None in Python)?

Answer 1

[bogdan.gavrilovic]

Hi,

There's two issues here.

1) Looks like the input for the step (workflow input) was not a list, but null(not defined). The scatter method must work with a list on input. At least an empty list should be provided [ ].
2) If you scatter over an empty list, 0 commands will be executed (since list length is 0) and the output of the step would be an empty list [ ].

To implement the use-case you have, you could do the following:
1) Set the default value for the optional input to --illumina
2) Set scatter as you did already
3) When running a workflow without adapters provided, on workflow input set adapters: [null]

What this would do is:
- Recognize the adapter list has one element
- Create one job (run the command once with that element from the list)
- Since the element is null, the default value would be used

Hope this helps.

[ionox0]

Thank you @bogdan!

Would it also work with multiple elements in the list? As in would the --illumina value get applied to multiple jobs if adapters: [null]? Or will it only process the first pair of fastqs?

[ionox0]

I assume that to apply the default value to all of the inputs we would need to specify this adapters: [null, null, null...]?

[bogdan.gavrilovic]

Yes, that is correct.