Mixing CWL workflow versions (including draft-3 tool in v1.0 workflow)

Question

[Peter vH]

I'm working on a CWL v1.0 workflow, using the GGR-cwl trimmomatic tool (https://github.com/Duke-GCB/GGR-cwl/blob/master/trimmomatic/trimmomatic.cwl) as one of the steps. That tool is a draft-3 specification one, which I understand is still parsed by cwltool.

Here is my current workflow file:

class: Workflow
requirements:
  - class: StepInputExpressionRequirement
inputs:
  reads1: File
  reads2: File
  slidingw:
    type: string
    default: "30"
  minl:
    type: int
    default: 20
outputs:
  trimmed_reads:
    type: File
    outputSource: trim_reads/output_read1_trimmed_file

steps:
  trim_reads:
    run: { $import: "../../tools/trimmomatic/trimmomatic.cwl" }
    in:
      input_read1_fastq_file: reads1
      input_read2_fastq_file: reads2
      slidingwindow: slidingw
      minlen: minl
      end_mode:
        valueFrom: "PE"


    out: [output_read1_trimmed_file]

This, however, fails to validate, I think because the $import results in mixing draft-3 and v1.0 CWL. Is there any way to use draft-3 CWL with v1.0?

More detail:

The errors are extensive but start as follows:

/cip0/software/x86_64/miniconda-3.19.0/bin/cwltool 1.0.20160714182449
Tool definition failed validation:
Validation error in object file:///net/ceph-mon1.sanbi.ac.za/sanbi/scratch/pvh/cwl/workflows/trim/trim.cwl
  Could not validate as `CommandLineTool` because
    could not validate field `outputs` because
      At position 0
        could not validate field `outputSource` because it is not recognized and strict is True, valid fields are: label, secondaryFiles, format, streamable, doc, id, outputBinding, type

    missing required field `baseCommand`
    could not validate field `steps` because it is not recognized and strict is True, valid fields are: id, inputs, outputs, requirements, hints, label, doc, cwlVersion, class, baseCommand, arguments, stdin, stderr, stdout, successCodes, temporaryFailCodes, permanentFailCodes
  Could not validate as `ExpressionTool` because
    could not validate field `outputs` because
      At position 0
        could not validate field `outputSource` because it is not recognized and strict is True, valid fields are: label, secondaryFiles, format, streamable, doc, id, outputBinding, type

    missing required field `expression`
    could not validate field `steps` because it is not recognized and strict is True, valid fields are: id, inputs, outputs, requirements, hints, label, doc, cwlVersion, class, expression
  Could not validate as `Workflow` because
    could not validate field `steps` because
      the value `[{'id': 'file:///net/ceph-mon1.sanbi.ac.za/sanbi/scratch/pvh/cwl/workflows/trim/trim.cwl#trim_reads'
        'in': [{'id': u'file:///net/ceph-mon1.sanbi.ac.za/sanbi/[...]`
       is not a valid type in the union, expected one of:
      - array of <WorkflowStep>, but
         At position 0
          could not validate field `run` because
            the value `{'arguments': [{'position': 1,
                            'shellQuote': False,
                            'valueFrom': '$("-Djava.io.tmpdir="+runtime.tmpdir)'},
                           {'posi[...]`
             is not a valid type in the union, expected one of:
            - string, but
               the value `{'arguments': [{'position': 1,
                              'shellQuote': False,
                              'valueFrom': '$("-Djava.io.tmpdir="+runtime.tmpdir)'},
                             {'posi[...]` is not string
            - CommandLineTool, but
               could not validate field `inputs` because
                At position 0
                  could not validate field `description` because it is not recognized and strict is True, valid fields are: label, secondaryFiles, format, streamable, doc, id, inputBinding, default, type

              could not validate field `outputs` because
                At position 4
                  could not validate field `description` because it is not recognized and strict is True, valid fields are: label, secondaryFiles, format, streamable, doc, id, outputBinding, type

              could not validate field `description` because it is not recognized and strict is True, valid fields are: id, inputs, outputs, requirements, hints, label, doc, cwlVersion, class, baseCommand, arguments, stdin, stderr, stdout, successCodes, temporaryFailCodes, permanentFailCodes

Note the reference to field description that is valid in draft-3 but not v1.0. The equivalent draft-3 workflow seems to be accepted as valid:

cwlVersion: cwl:draft-3
class: Workflow
requirements:
  - class: StepInputExpressionRequirement

inputs: 
  - id: reads1
    type: File
  - id: reads2
    type: File
  - id: slidingw
    type: string
    default: "30"
  - id: minl
    type: int
    default: 20
outputs:
  - id: trimmed_reads
    type: File
    source: "#trim_reads/output_read1_trimmed_file"

steps:
  - id: trim_reads
    run: { $import: "../../tools/trimmomatic/trimmomatic.cwl" }
    inputs:
      - id: input_read1_fastq_file
        source: "#reads1"
      - id: input_read2_fastq_file 
        source: "#reads2"
      - id: slidingwindow
        source: "#slidingw"
      - id: minlen
        source: "#minl"
      - id: end_mode
        valueFrom: "PE"
    outputs: 
      - id: output_read1_trimmed_file

[Peter vH]

For interest's sake, here is a single-step draft-3 workflow that works:

cwlVersion: cwl:draft-3
class: Workflow
requirements:
  - class: StepInputExpressionRequirement

inputs: 
  - id: reads1
    type: File
  - id: reads2
    type: File
  - id: slidingw
    type: string
    default: "4:30"
  - id: minl
    type: int
    default: 20
outputs:
  - id: trimmed_reads
    type: File
    source: "#trim_reads.output_read1_trimmed_file"

steps:
  - id: trim_reads
    run: { $import: "../../tools/trimmomatic/trimmomatic.cwl" }
    inputs:
      - id: input_read1_fastq_file
        source: "#reads1"
      - id: input_read2_fastq_file 
        source: "#reads2"
      - id: slidingwindow
        source: "#slidingw"
      - id: minlen
        source: "#minl"
      - id: end_mode
        valueFrom: "PE"
      - id: trimmomatic_jar_path
        valueFrom: "/cip0/software/java/Trimmomatic-0.33/trimmomatic-0.33.jar"
    outputs: 
      - id: "#trim_reads.output_read1_trimmed_file"

Answer 1

[Michael R. Crusoe]

The run: { $import ... } idiom is from the draft-2 era. draft-3 onwards you can just give the path to the CWL tool description.

cwlVersion: v1.0
[...]
steps:
  trim_reads:
    run: "../../tools/trimmomatic/trimmomatic.cwl"

[Peter vH]

Thanks! It works now, here's the final single-step v1.0 workflow:

cwlVersion: v1.0
class: Workflow
requirements:
  - class: StepInputExpressionRequirement
inputs:
  reads1: File
  reads2: File
  slidingw:
    type: string
    default: "4:30"
  minl:
    type: int
    default: 20
outputs:
  trimmed_reads:
    type: File
    outputSource: trim_reads/output_read1_trimmed_file

steps:
  trim_reads:
    run: "../../tools/trimmomatic/trimmomatic.cwl" 
    in:
      input_read1_fastq_file: reads1
      input_read2_fastq_file: reads2
      slidingwindow: slidingw
      minlen: minl
      end_mode:
        valueFrom: "PE"
      trimmomatic_jar_path:
        valueFrom: "/cip0/software/java/Trimmomatic-0.33/trimmomatic-0.33.jar"

    out: [output_read1_trimmed_file]