Using while loop to go over all files in a folder

Question

[catherine]

Hi,

I am currently running fastx_trimmer tool in order to trim all my paired end RNAseq input files using while loop.

The filenames are as follows:

1_1.fastq
1_2.fastq
2_1.fastq
2_2.fastq
.................
25_1.fastq
25_2.fastq

Below is the code that I use to run the while loop in a bash script using byobu screen

#!/usr/bin

i=1
while [ $i -le 25 ]
do
  cd fastq
  cd unzip_fastq
  fastx_trimmer -Q33 -f 1 -l 100 -i $i_1.fastq  -o  $i_1_trimmed.fastq
  mv $i_1_trimmed.fastq  ../trimmed_fastq
  fastx_trimmer -Q33 -f 1 -l 100 -i $i_2.fastq -o $i_2_trimmed.fastq
  mv $i_2_trimmed.fastq ../trimmed_fastq
  cd ~
  ((i++))
done

But, I got the follow error.

fastx_trimmer: failed to open input file '.fastq': No such file or directory

Any help is much appreciated. Thanks in advance

Catherine

[always_learning]

Seems that files are not in correct directories else code seems correct!!

[brentp]

_ is a valid character in a variable name. So it thinks you are looking for a variable names $i_1_trimmed.fastq. You should instead use ${i}_1_trimmed.fastq.

In addition, you can get more informative messages about errors if you put set -ex at the top of your script.

[Phil S.]

or you could escape it with: $i\_1_trimmed.fastq...

[catherine]

Thank you very much!! By using ${i}, it's working!!!

Answer 1

[Eric Normandeau]

You can try the following, which is slightly simpler and you can move your files after with mv

for i in folderWithFastqFiles/*.fastq
do
  fastx_trimmer -Q33 -f 1 -l 100 -i $i -o $i.trimmed.fastq
done

[dariober]

even simpler: for i in folderWithFastqFiles/*.fastq ...etc

[Eric Normandeau]

True. It's an old habit :) I edited my script to simplify it.

Answer 2

[Phil S.]

maybe try this one:

mkdir $2 #delete this if output directory already exists...
for f in $1/*
do
  if [ -f $f ]
  then
    fastx_trimmer -Q33 -f 1 -l 100 -i $f -o $2/$(basename $f .fastq)_trimmed.fastq
  fi
done

Call it with first argument as path to your fastq and second argumant as path where to store the trimmed files.

Answer 3

[5heikki]

cd into the fastq dir and copy&paste:

for f in *.fastq; do name=$(basename "$f" .fastq); fastx_trimmer -Q33 -f 1 -l 100 -i $f -o $name.trimmed.fastq; done