awk and sed in bash script and no output

Question

[bk11]

Hi

I wrote a bash script with awk and sed commands in it to parse text files in a directory. This script runs without giving any output. What is wrong with it?

#!/bin/bash

for i in *.txt; do
       awk -F'\t' '{print $16}' $i | sed '1d' | tr -d '\r' >$i_get_signals.sh
done

[Joe]

We need more information. What does your input data look like? What output do you expect? Are you running the script while you're in the directory that the .txt files are in? How are you invoking the script?

You will also need to tell us why this is a bioinformatics question and not just a programming question else we may close it as off topic.

[bk11]

Yes I am running the script in the same directory that has the text files. This is related to bioinformatics as it is used to get URL to download histone proteins binding signals from different cell types.

[WouterDeCoster]

It is related to bioinformatics, but only borderline. It is essentially just unix text processing, and there are more specialised forums for this. We usually tolerate posts like this as we understand that these are common problems in bioinformatics, but it's not pure bioinformatics.

[WouterDeCoster]

Why do you send the result to a .sh file?

[bk11]

I tried sending the result to .txt file but also it did not work.

[WouterDeCoster]

It shouldn't make a difference, but with the extension .sh you suggest this is a shell script.

[cpad0112]

That is because OP is using *.txt in loop and doesn't want output with .txt for obvious reasons. Instead OP could have used any other extension instead of sh (.out, .text, .file etc or without extension). WouterDeCoster

[WouterDeCoster]

Yeah, of course, and while file extensions have almost no meaning in unix it is quite confusing to use something unexpected.

Answer 1

[cpad0112]

try this:

for i in *.txt; do awk -F'\t' '{print $16}' $i | sed '1d' | tr -d '\r' > $i"_get_signals.sh" ; done

But whatever you are doing, this is long way of doing it. OP code is simply extracting 16th column, removing the header and removing carriage returns and writing it to a text file. This can be done better way.

Change the code to remove old file extension in new files:

for i in *.txt; do awk -F'\t' '{print $16}' $i | sed '1d' | tr -d '\r' > ${i%%.*}"_get_signals.sh" ; done

[bk11]

It worked very nicely though.

[cpad0112]

@ bk11

1. Make a new directory
2. Copy all the files in the new directory

3. Run following code:

   awk 'FNR!=1 {print $16 > FILENAME".out"}' *.txt

4. Compare the output from above bash script.