CWL: print out command lines?

Question

[sb10]

Is there a tool that, given a cwl file and its necessary inputs and options, will simply print out the actual commands that need to be run for that workflow, without running them?

Better yet, one that also indicates the dependencies between the commands it outputs? So it would indicate in some way if the 3rd command it output was dependent on the first 2 commands it output completing first (but the first 2 are independent)?

[karl.sebby]

I would like to know this as well. I'm digging through the cwltool code now to try and make it work. It shouldn't be too hard (famous last words) since the command is printed out when you run a job. It is not an option when running cwltool though. Will let you know if I figure it out. Hoping somebody beats me to it though.

[bruce.moran]

+1, would like to be able to summarise the full workflow without running samples. Thought perhaps the CWL viewer could be used as a basis for this?

Answer 1

[karl.sebby]

OK. So I figured a few things out. If you are using the cwltool to run a job e.g. $ cwltool echo.cwl echo.yml where these files are

echo.cwl

cwlVersion: v1.0
class: CommandLineTool
stdout: echoOutput.txt
inputs:
  message:
    type: string
    inputBinding:
      position: 1
  message2:
    type: string
    inputBinding:
      position: 2
outputs:
  output: 
    type: stdout

echo.yml

message: "Hello World"
message2: "you doing good?"

Part of your output will look something like this:

[job echo.cwl] /tmp/tmpxxxx$ echo \
    'Hello world' \
    'you doing good?' > /tmp/tmpxxxx/echoOutput.txt
[job echo.cwl] completed success
Final process status is success

So the command is being generated and output to the screen. This happens using the logging module on lines 180-187 of cwltool/run.py. The commands are stored in a list called command_line which is a JobBase class attribute. To just output the command without running the job, I did the following:

1) In cwltool/main.py I added the argument 'commandline' to the argument parser after the --validate argument like so (above and below lines added for context):

exgroup.add_argument("--validate", action="store_true", help="Validate CWL document only.")
exgroup.add_argument("--commandline", action="store_true", help="Print the command line to command only.")
exgroup.add_argument("--print-supported-versions", action="store_true", help="Print supported CWL specs.")

2) In the main function of cwltool/main.py I set the default commandline value to False (above and below lines added for context)

'validate': False,
'commandline': False,
'enable_ga4gh_tool_registry': False,

3) then in cwltool/run.py at line 309 I added an if statement to print out the command line if the --commandline argument was set to true (above and below lines added for context again):

relink_initialworkdir(self.generatemapper, self.outdir, self.builder.outdir, inplace_update=self.inplace_update)
        if kwargs['commandline']:
            print(" ".join(self.command_line))
            exit(0)
        self._execute([], env, rm_tmpdir=rm_tmpdir, move_outputs=move_outputs)

Now if I run $ cwltool --commandline echo.cwl echo.yml I get

Resolved 'echo.cwl' to 'file:///.../echo.cwl'
echo Hello world you doing good?

Process finished with exit code 0

Hopefully this helps. Haven't tried it with anything more complex than my echo job.

[sb10]

Thanks. Apparently it may also be possible by using custom executor and/or makeTool functions. I might give that a go at some point and report back here.

[karl.sebby]

Hi sb10, Have you made any headway on this? I'm just circling back to this problem.

[sb10]

Not yet had time to look at it. It's on the to-do list...