Troubleshoot command error/exit status 127

Question

[natural_decline]

Trying to run a pipeline in python2.7, executed by a simple bash command. I am using conda and have created a python 2.7 environment in my EC2 instance and I am stuck troubleshooting this error. I am reading typically this is related to directory/location issues but all of the files and programs appear to be in the right place, I am very green and clearly missing something! One thing- the process 'work dir' referred to here is not actually in the path it says and I am confused about this.

Error executing process > 'map_reads (1)'

Caused by: Process map_reads (1) terminated with an error exit status (127)

Command executed:

clockwork map_reads --threads 8 --unsorted_sam sample_name /data/TB/varpipe_wgs_211124/tools/clockwork/OUT/ref.fa contam_sam /data/TB/varpipe_wgs_211124/data/B18-CDC_S1_L001_R1_001.fastq.gz /data/TB/varpipe_wgs_211124/data/B18-CDC_S1_L001_R2_001.fastq.gz

Command exit status: 127

Command output: (empty)

Command error: .command.sh: line 2: clockwork: command not found

Work dir:
/data/TB/varpipe_wgs_211124/data/work/a0/c3e6d06956e32d14ddcdff3170b6ea

Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run

[Arup Ghosh]

Have you checked if the clockwork command is working?

[Pierre Lindenbaum]

clockwork: command not found

clockwork is not in your $PATH. set the $PATH in the workflow process, ( or invoke module, conda, etc...)

Answer 1

[Marcel Ribeiro-Dantas]

There is not much to be said apart from clockwork is not in your $PATH (or not installed). Can you share a minimal reproducible example of your nextflow pipeline?

[natural_decline]

The first thing I tried when I seeing this was installing with conda. I also read in the documentation that installation did not include the clockwork scripts (not sure why this would be?) so I cloned the github repo. I later realized nextflow/conda was already there, in the directory where the full pipeline is located. I tried adding both this directory and the one where I re-installed to $PATH, still not working. this is the nextflow command within the pipeline:

nextflow run /data/TB/varpipe_wgs_211124/tools/clockwork/nextflow/remove_contam.nf --ref_fasta /data/TB/varpipe_wgs_211124/tools/clockwork/OUT/ref.fa --ref_metadata_tsv /data/TB/varpipe_wgs_211124/tools/clockwork/OUT/remove_contam_metadata.tsv --reads_in1 B18-CDC_S1_L001_R1_001.fastq.gz --reads_in2 B18-CDC_S1_L001_R2_001.fastq.gz --outprefix Output_10_12_2022/B18-CDC_S1_L001/clockwork/B18-CDC_S1_L001 --mapping_threads 8

[Pierre Lindenbaum]

this is the nextflow command within the pipeline:

it's unrelated to nextflow.

[Pierre Lindenbaum]

I re-installed to $PATH, still not working

well, how did you set the path ? is there any executable named clockwork in the directory you added in the new PATH ?

[natural_decline]

when I echo $PATH there is a /clockwork directory containing:

Dockerfile LICENSE README.md Singularity.def db_schema_patches multipass nextflow python scripts vagrant

[natural_decline]

the nextflow folder contains the scripts called in the above command : fake_remove_contam.nf generic_pipeline.nf import.nf mykrobe_predict.nf qc.nf remove_contam.nf variant_call.nf

[natural_decline]

if it wasn't obvious, I am new to this.

[Marcel Ribeiro-Dantas]

Your script is being called by Nextflow, as expected, that's why it gets to the point where your shell complains that clockwork is not found. What you have to do is to find out why it is not found. Sharing the command line you run in the terminal is not really useful here. We need more information than you're sharing, and I understand it if you can't share, but we can't be helpful without more details.

[natural_decline]

Sorry! I am not sure what info you are asking for

[Marcel Ribeiro-Dantas]

nextflow.config, your conda YAML file (or command lines used to build the conda environment) and a reproducible minimal example of your pipeline.

[Arup Ghosh]

Download the docker/singularity image before running the pipeline.

Follow the installation instructions.