Can I use output eval with CWL to create the list of output files (i.e. replace the glob results)?

Question

[mavrommatis.konstantinos]

Hi,

I have a tool that is creating a number of files and stores their names in an another file summary.txt. The program generates a number of files with various (random) filenames, and I want to select only few based on some filters based on values inside the summary.txt file.

Is there a way to load the contents of this summary.txt file (using loadContents: true etc) and select the list of the files from it and return it as an array of Files? In essence I would like to replace the list that glob * would return with my list of files.

I have tried something like:

class: CommandLineTool

cwlVersion: v1.0

<various lines>

outputs:
  - id: outputFileList
    type: 'File[]'
    outputBinding:
      loadContents: true
      glob: summary.txt
      outputEval: |

        ${
          var str = self[0].contents
          var file=[]
          var lines=str.split(/\n/)
          for( var i = 1 ; i < lines.length; i ++) {
            if(lines[i] === '' ){continue}
              var fields=lines[i].split(",")
              file.push( runtime.outdir + "/" + fields[4] )
          }
          return( file )
        }

Which returns the list of the files, but they are treated as list of Strings, and downstream tools cannot access them.

Thanks in advance for your help

[Tom]

This javascript code only concatenates strings and returns them, not actual files. I am not sure if what you ask can be done in a single CommandLineTool. I suspect you need a workflow to solve this.

*edit: Let us know if you need suggestions on how to build said workflow.

Answer 1

[michael.franklin]

I'm very late to the party, but you need to return an array of file objects with the minimum properties path and basename:

file.push({
  path: runtime.outdir + "/" + fields[4],
  basename: fields[4]
})

Answer 2

[kaushik.ghose]

Hi! I suspect that the best way to do this is to have a small bash script that runs after the main script, reads summary.txt and copies those files to a new directory say outputs and the CWL declares the output to be that directory. The bash script can be either part of the docker image or injected in as a InitialWorkingDirectoryRequirement