Bash loop using Edirect (esearch|efetch)
1
7
Entering edit mode
8.2 years ago
fhsantanna ▴ 620

I have a text file containing in each line an access number (file 16S_list). I tried to download sequences from genbank using this list through the following script:

#!/bin/bash
while read line; do
esearch -db nucleotide -query $line | efetch -db nucleotide -format fasta 
done < 16S_list

However, it did not worked, just the first sequence is downloaded. After I tried the following script, and this time, it worked:

#!/bin/bash
while read line; do
esearch -db nucleotide -query $line < /dev/null | efetch -db nucleotide -format fasta
done

I did not understand why the first one is not working. Could somebody explain why the first script failed?

edirect esearch while • 8.4k views
ADD COMMENT
0
Entering edit mode

do you have any empty line in 16S_list ? try to quote $line to "$line"

I don't get why the second script would work by reading /dev/null

ADD REPLY
2
Entering edit mode

Does the first one work if you set IFS to newline? IFS=$'\n'? Are your lines delimited by \n? I recall having problems with this too. AFAIR the esearch command consumed the entire file (stdin), hence I got output only for the first line. I think I ended up using IFS=$'\n'; for next in $(cat file); do ..; done

ADD REPLY
2
Entering edit mode

It worked. Here is the complete code:

IFS=$'\n'; for next in $(cat 16S_list); do esearch -db nucleotide -query $next | efetch -db nucleotide -format fasta; done

However, I still do not understand why my first script did not worked.

ADD REPLY
2
Entering edit mode

The shell splits the file in the cat command whereas in the while loop the esearch command consumes the entire file, i.e. it doesn't care about the linebreaks. It's basically like this

ADD REPLY
1
Entering edit mode

Great. So my fist code should be altered to the following one:

while read line <&3; do
esearch -db nucleotide -query "$line" | efetch -db nucleotide -format fasta
done 3<16S_list

It worked fine. Thank you very much.

ADD REPLY
0
Entering edit mode

There is no empty line. I quoted, nothing happened. Neither do I...

ADD REPLY
2
Entering edit mode

The second script works because the /dev/null redirect prevents esearch from consuming the entire file..

ADD REPLY
0
Entering edit mode

Why does esearch consume whole file? Is it a bug? Because when I echo the lines of the list using while loop, everything works fine.

ADD REPLY
0
Entering edit mode

It's not really a bug, but default behavior of the program..

ADD REPLY
1
Entering edit mode
2.8 years ago
Michael 55k

A bit late to the party here but the esearch behavior is still the same. When using esearch in a loop with read, < /dev/null is required as the behaviour of esearch is to "slurp" all input from the input stream to the end. When reading a file with while read .... done < file.txt, the input stream is set to file.txt, therefore read gets to read only the first line in the stream, while esearch reads the rest without doing anything with it.

ADD COMMENT

Login before adding your answer.

Traffic: 821 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6