CWL: combine two arrays as workflow step input

Question

[well309]

I'd like to know how can I combine two arrays (of string or File) into a single one in Common Workflow Language. For example:

cwlVersion: v1.0
class: Workflow
inputs:
  files_1: File[]
  files_2: File[]
outputs:
  task1_output:
    type: File
    outputSource: task1/outfile
steps:
  task1:
    in:
      files: [files_1, files_2] # it does not work
    out: [outfile]

Input example:

files_1:
  - class: File
  path: fileA_1.fastq
  - class: File
  path: fileB_1.fastq
files_2:
  - class: File
  path: fileA_2.fastq
  - class: File
  path: fileB_2.fastq

Answer 1

[Michael R. Crusoe]

Hello well309,

Consider using the MultipleInputFeatureRequirement

cwlVersion: v1.0
class: Workflow
requirements:
  MultipleInputFeatureRequirement: {}
inputs:
  files_1: File[]
  files_2: File[]
outputs:
  task1_output:
    type: File
    outputSource: task1/outfile
steps:
  task1:
    in:
      files:
        source: [ files_1, files_2 ]
        linkMerge: merge_flattened
    out: [outfile]

[well309]

Thank you Michael. I added a working example followed by the error message.

Tool example (du):

cwlVersion: v1.0
class: CommandLineTool
baseCommand: [du]
inputs:
  files:
    type: File[]
    inputBinding: {}
outputs:
  outfile:
    type: stdout

Workflow with MultipleInputFeatureRequirement:

cwlVersion: v1.0
class: Workflow
requirements:
  MultipleInputFeatureRequirement: {}
inputs:
  files_1: File[]
  files_2: File[]
outputs:
  task1_output:
    type: File
    outputSource: task1/outfile
steps:
  task1:
    run: du.cwl
    in:
      files: [files_1, files_2]
    out: [outfile]

Error message from cwl-runner:

/usr/local/bin/cwl-runner 1.0.20170622090721
Resolved 'multiple_arrays_input.cwl' to 'file:///home/welliton/multiple_arrays_input.cwl'
Tool definition failed validation:
multiple_arrays_input.cwl:6:3: Source 'files_1' of type {"type": "array", "items": "File"} is incompatible
multiple_arrays_input.cwl:16:7:   with sink 'files' of type {"type": "array", "items": "File"}
multiple_arrays_input.cwl:16:7:   sink has linkMerge method merge_nested
multiple_arrays_input.cwl:7:3: Source 'files_2' of type {"type": "array", "items": "File"} is incompatible
multiple_arrays_input.cwl:16:7:   with sink 'files' of type {"type": "array", "items": "File"}
multiple_arrays_input.cwl:16:7:   sink has linkMerge method merge_nested

[Michael R. Crusoe]

try

      files:
        source: files_1, files_2
        linkMerge: merge_flattened

[well309]

Thank you Michael. Here the code that worked.

cwlVersion: v1.0
class: Workflow
requirements:
  MultipleInputFeatureRequirement: {}
inputs:
  files_1: File[]
  files_2: File[]
outputs:
  task1_output:
    type: File
    outputSource: task1/outfile
steps:
  task1:
    run: du.cwl
    in:
      files:
        source: [files_1, files_2]
        linkMerge: merge_flattened
    out: [outfile]

[Michael R. Crusoe]

Yay! I've updated my response to correct for the missing brackets for files.source -- thanks!