Importing modules based on python in bash script wont work

Question

[mary.v.volkova]

Hi all, I am trying to play with creating a pipeline in a bash script (to then use it as a docker container), but am stuck already in the beginning. Here is my code:

#!/bin/bash 
set -e
set -u
set -o pipefail

source /home/maria/miniforge3/bin/conda
source /home/maria/miniforge3/bin/trimmomatic
source /home/maria/miniforge3/bin/bioawk

#iterating over files
echo "attempting loop"
for i in $(ls *.fastq.gz | rev | cut -c 17- | rev | uniq)
do
    echo "$i: " $(bioawk -c fastx "END {print NR}" $i)
    trimmomatic PE ${i}_R1_001.fastq.gz ${i}_R2_001.fastq.gz \
    trimmed-${i}_R1_001.fastq.gz unpaired-${i}_R1_001.fastq.gz \
    trimmed-${i}_R2_001.fastq.gz unpaired-${i}_R2_001.fastq.gz \
    ILLUMINACLIP:TruSeq3-PE.fa:2:30:10:2:True LEADING:3 TRAILING:3 MINLEN:36 HEADCROP:6
done

The reason Im using "source" here in the first place is because bash was giving me an error of not seeing trimmomatic and bioawk. Now that Im using source, it is giving me "import-im6.q16: attempt to perform an operation not allowed by the security policy PS @ error/constitute.c/IsCoderAuthorized/408." error. Googling it showed me that it is a problem is either in ImageMagick software, or in the fact that Im using a wrong shabang.

I don't want to rewrite this script for python and would prefer to stick to bash, while still using the programs (trimmomatic) I have in here. I know the problem is in shabang then but can't seems to understand what to do next. Does anyone know a way to do this? Thank you in advance.

Answer 1

[Pierre Lindenbaum]

nice try but this is an improper use of source which should be used to include BASH code, not a whole binary executable.

What you want is setting the PATH: https://opensource.com/article/17/6/set-path-linux

[mary.v.volkova]

Okay, I got it, PATH needs to be specified inside this script as well. Thank you so much!!

[mary.v.volkova]

I have /home/maria/miniforge3/bin in my PATH, but it still not seeing those programs. I also noticed that people usually include the .sh script, but I didn't manage to find those in my system, only the binary executables

Answer 2

[cfos4698]

If you have those programs installed in a conda environment, which is what it looks like, you could also just activate the environment in your script. There have traditionally been some headaches with making this work, but a common workaround is including the lines:

eval "$(conda shell.bash hook)"
conda activate <env-name>

So, assuming your tools are in your base conda environment, you could try the following:

#!/bin/bash 
set -e
set -u
set -o pipefail

# activate the env
eval "$(conda shell.bash hook)"
conda activate base

#iterating over files
echo "attempting loop"
for i in $(ls *.fastq.gz | rev | cut -c 17- | rev | uniq)
do
    echo "$i: " $(bioawk -c fastx "END {print NR}" $i)
    trimmomatic PE ${i}_R1_001.fastq.gz ${i}_R2_001.fastq.gz \
    trimmed-${i}_R1_001.fastq.gz unpaired-${i}_R1_001.fastq.gz \
    trimmed-${i}_R2_001.fastq.gz unpaired-${i}_R2_001.fastq.gz \
    ILLUMINACLIP:TruSeq3-PE.fa:2:30:10:2:True LEADING:3 TRAILING:3 MINLEN:36 HEADCROP:6
done