Chia-pipe cannot find the module pigz

Question

[bioinfo89]

Hi All,

I am using Chia-pipe to run chia-pet (cell line) data. I followed the installation process and modified the config file to run the tool. All the modules (pigz,perl,java,samtools,bedtools) are also installed in the chia-pipe folder. However, the program terminates with the following error:

Such error is repeated for all the modules mentioned in the config file:

Lmod has detected the following error: The following module(s) are unknown:
"pigz"

Please check the spelling or version number. Also try "module spider ..."
It is also possible your cache file is out-of-date; it may help to try:
  $ module --ignore-cache load "pigz"

Also make sure that all modulefiles written in TCL start with the string
#%Module

I also get the following errors:

/WGS_tools/ChIA-PIPE/0.chia_pipe_shell.sh: line 208: pigz: command not found
/WGS_tools/ChIA-PIPE/0.chia_pipe_shell.sh: line 307: pigz: command not found
/WGS_tools/ChIA-PIPE/0.chia_pipe_shell.sh: line 406: pigz: command not found
mv: cannot stat ‘rep1.e500.clusters.cis.chiasig.gz’: No such file or directory
mv: cannot stat ‘rep1.e500.clusters.trans.chiasig.gz’: No such file or directory
gzip: rep1.e500.clusters.cis.gz: No such file or directory
Can't open input bam file rep1.singlelinker.paired.UU.nr.bam
/WGS_tools/ChIA-PIPE/0.chia_pipe_shell.sh: line 498: java: command not found
/WGS_tools/ChIA-PIPE/0.chia_pipe_shell.sh: line 528: samtools: command not found
/WGS_tools/ChIA-PIPE/0.chia_pipe_shell.sh: line 530: samtools: command not found
/WGS_tools/ChIA-PIPE/0.chia_pipe_shell.sh: line 532: samtools: command not found
/WGS_tools/ChIA-PIPE/0.chia_pipe_shell.sh: line 535: samtools: command not found
/WGS_tools/ChIA-PIPE/0.chia_pipe_shell.sh: line 541: samtools: command not found
[E::hts_open_format_impl] Failed to open file rep1.for.BROWSER.bam
Failed to open BAM file rep1.for.BROWSER.bam
needLargeMem: trying to allocate 0 bytes (limit: 100000000000)
ls: cannot access rep1.singlelinker.paired.*.bam: No such file or directory
Traceback (most recent call last):
  File "/WGS_tools/ChIA-PIPE/util/scripts/annotate_loops_with_peak_support.py", line 2, in <module>
    import commands
ModuleNotFoundError: No module named 'commands'

It seems that the tool is not able to find the modules/tools from the config file, however I have mentioned the path. Any help for resolving this issue would be great. Thanks!

[GenoMax]

This appears to be case of $PATH not correctly set to account for the location of the programs you mention.

That said what do you mean by all the modules are present? modules is a software program that is used for managing version of programs on larger compute servers/clusters. Are you using that modules or something else?

[bioinfo89]

Yes, I did set the $PATH to the location of the programs.

I mean that the modules are present in the ChIA-PIPE folder as suggested by the installation guide. I am just running the shell script of the tool which is using these modules:

### 5) Uncomment below to module load dependencies
 module --ignore-cache load "pigz"
 module --ignore-cache load "java/1.7.0"
 module --ignore-cache load "perl/5.26.0"
 module --ignore-cache load "bedtools/2.26.0"
 module --ignore-cache load "samtools/1.5"
 module --ignore-cache load "R/3.2.1"
 module --ignore-cache load "MACS/2.1.0.20151222"

The above mentioned lines are provided in the config file which I am running.

[GenoMax]

I took a look at the config file they mention. I think the module commands are for Jackson Lab HPC. So if you are running this on a stand-alone machine they shouldn't apply.

Did the following part work without any errors?

dep_dir="dep_dir"
bash local_install_chia_pipe_dependencies.sh -i ${dep_dir}

[bioinfo89]

It did not throw any errors as such. I guess I should have mentioned the dir path for dep_dir, I think I missed that. Now it makes more sense why I am getting this error. I'll try to install it again.

Thanks @genomax!

[GenoMax]

Can you check the contents of echo $PATH and verify that all program directories appear there? It would help to confirm with which samtools, which pigz (you get the idea) once you verify $PATH.

[bioinfo89]

 echo $PATH
/opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4/bin:/opt/ohpc/pub/compiler/gcc/8.3.0/bin:/opt/ohpc/pub/utils/prun/1.3:/opt/ohpc/pub/utils/autotools/bin:/opt/ohpc/pub/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/krc190002/scratch/WGS_tools/bedtools2/bin:/home/krc190002/scratch/WGS_tools/root/bin:/home/krc190002/.local/bin:/home/krc190002/bin:/scratch/krc190002/WGS_tools/ChIA-PIPE/pigz:/scratch/krc190002/WGS_tools/ChIA-PIPE/bedtools:/scratch/krc190002/WGS_tools/ChIA-PIPE/samtools:/scratch/krc190002/WGS_tools/ChIA-PIPE/macs2:/scratch/krc190002/WGS_tools/ChIA-PIPE/java:/scratch/krc190002/WGS_tools/ChIA-PIPE/R-3.2.1/bin/:/scratch/krc190002/WGS_tools/ChIA-PIPE/bin/perl:/scratch/krc190002/WGS_tools/ChIA-PIPE/jre/bin/java

which samtools
/usr/bin/which: no samtools in (echo $PATH)

which pigz
 /usr/bin/pigz

which java
/usr/bin/java

which perl
/usr/bin/perl

which macs2
/usr/bin/which: no macs2 in (echo $PATH)

which bedtools
~/scratch/WGS_tools/bedtools2/bin/bedtools

It looks like only bedtools path is correct. Though the $PATH includes other tools, it does not match with which command output. What should I do to resolve this?

[GenoMax]

Even if you set $PATH to point to your local directories it looks like the system installed programs (that are in /use/bin) are ones found by your OS.

Can you check if there are other sub-dir under /scratch/krc190002/WGS_tools/ChIA-PIPE/samtools that contain the actual executable? You may need to add that full path added (e.g. /scratch/krc190002/WGS_tools/ChIA-PIPE/samtools/bin).

[bioinfo89]

Right ok. There are no sub dir under /scratch/krc190002/WGS_tools/ChIA-PIPE/samtools

[GenoMax]

Is that the actual executable? Does it have execute permissions?

[bioinfo89]

Yes it is the actual executable and it has execute permissions.

[GenoMax]

How are you launching the pipeline? As they show in on help page?

bash chia_pipe-master/0.chia_pipe_shell.sh -c chia_pipe-master/example_config_file.sh

If so you are starting a new bash shell which no longer has your personal $PATH settings but can still access programs that are installed in system default $PATH.

[lakhujanivijay]

Hi bioinfo89

/WGS_tools/ChIA-PIPE/0.chia_pipe_shell.sh: line 208: pigz: command not found

I could also see this error in your original post; were you able to resolve this?

[bioinfo89]

Yes I resolved it. Thanks!