bash loop to count variants using vcftools

Question

[bioguy24]

I can run the below vcftools command to count individual variant types in vcf.gz files

zcat /home/cmccabe/Desktop/vcf/file1.vcf.gz | vcf-annotate --fill-type | grep -oP "TYPE=\w+" | sort | uniq -c > /home/cmccabe/Desktop/vcf/file1_variant_counts.bed

However when I try a bash loop the command does not run at all and I can not seem to figure out why, I think it looks right? Thank you :).

for f in /home/cmccabe/Desktop/vcf/*.vcf.gz ; do
     bname=`basename $f`
     pref=${bname%%.vcf.gz
     zcat /home/cmccabe/Desktop/vcf/$f | vcf-annotate --fill-type | grep -oP "TYPE=\w+" | sort | uniq -c > /home/cmccabe/Desktop/vcf/${pref}_variant_counts.bed
done

I think I see it now, I missed a closing brace in the pref=.... Thank you :).

[Ram]

Exactly. The closing brace was the problem. Nice use of %% BTW :)

[michael.ante]

In your loop, f=/home/cmccabe/Desktop/vcf/*.vcf.gz; in your zcat command you open

"/home/cmccabe/Desktop/vcf/home/cmccabe/Desktop/vcf/*.vcf.gz"

[venu]

It must be zcat $f

Answer 1

[venu]

for f in /home/cmccabe/Desktop/vcf/*.vcf.gz
> do
> zcat $f | vcf-annotate --fill-type | grep -oP "TYPE=\w+" | sort | uniq -c > "$f"_variant_counts.bed
> done

Then change all the file names with rename command.

Ex:

rename "s/.vcf.gz_variant_counts.bed/.bed/" *.bed

[bioguy24]

Thank you all :).

Answer 2

[GouthamAtla]

Try to use GNU Parallel for these kind of tasks:

parallel --jobs <int> "zcat {} |  vcf-annotate --fill-type | grep -oP \"TYPE=\\w+\" | sort | uniq -c > {.}_variant_counts.bed" ::: /home/cmccabe/Desktop/vcf/*.vcf.gz