[cwl] referencing to list of paths in an array of files or direcories

Question

[grange]

I try to refer to a list of input paths in a CommandLineTool. In essence, my tool looks like this:

class: CommandLineTool
cwlVersion: v1.0

baseCommand:
  - python
inputs:
  - id: inpfiles
    type: 'Directory[]'

outputs: []
label: _do_magic
arguments:
  - position: 0
    prefix: '-c'
    valueFrom: |
      import magic
      magic.main(["$(inputs.inpfiles.path)"])

The main function of the magic script (not my script...) expects a dict of paths as input (I will probably have to play with the separator setting as well for this). The code I have now works for a single Directory object, but doesn't work. It does work when I refer to a single entry by using $inputs.inpfiles[0].path.

So my question is: how should I get a list of paths in an array of Directory objects?

Answer 1

[Michael R. Crusoe]

Hello grange,

Since you're embedding your own code, you can get a JSON dump of the Directory object and walk over the "listing" attribute to extract the paths you need.

Try inpfiles=$(inputs.inpfiles) in the Python block

Answer 2

[grange]

Works like a charm! Thanks!

Answer 3

[grange]

Actually, the issue now is that the variable that is passed contains a lot of data (including the list of files for each directory) which causes python to bail out with an OSError due to the argument list being too long. Any hints on keeping the data that is put on the command line short? I am using cwltool to run it, but I don't think that makes much of a difference.

[grange]

Answering my own question. Curious to see if this is indeed the neatest option:

class: CommandLineTool cwlVersion: v1.0

baseCommand:
  - python
inputs:
  - id: inpfiles
    inputBinding:
        position: 0
    type: 'Directory[]'

outputs: []
label: _do_magic
arguments:
  - position: 0
    prefix: '-c'
    valueFrom: |
      import sys
      import magic
      magic.main(sys.argv[1:])

[Michael R. Crusoe]

You can have it written to a file and then load json in the Python section

requirements:
  InitialWorkDirRequirement:
    listing:
    - entry: '{"_": $(inputs)}'
      entryname: inputs.json