bcftools +fill-tag does not work in loop

Question

[brianaloredana]

Hello all!

I work with cattle genome. I am trying to update INFO of my vcf files by using +fill-tag. It works with one file, but it does not work in the loop.

module unload bcftools

bcftools +fill-tags file.vcf -Oz -o file.vcf_ok.gz -- -t AF,AN,AC, AC_Hom,AC_Het,MAF,NS

When I want to work my loop script in the slurm job. I used this:

module unload bcftools
    for file in ./*.vcf; do bcftools +fill-tags $file -Oz ${file%.*}.tagged_vcf.gz -- -t AF,AN,AC,AC_Hom,AC_Het,MAF,NS ; done

But it gave me this error message:

About: Set INFO tags AF, AC, AC_Hemi, AC_Hom, AC_Het, AN, ExcHet, HWE, MAF, NS.
Usage: bcftools +fill-tags [General Options] -- [Plugin Options]
Options:
   run "bcftools plugin" for a list of common options



Plugin options:
   -d, --drop-missing          do not count half-missing genotypes "./1" as hemizygous
   -l, --list-tags             list available tags with description
   -t, --tags LIST             list of output tags. By default, all tags are filled.
   -S, --samples-file FILE     list of samples (first column) and comma-separated list of populations (second column)



Example:
   bcftools +fill-tags in.bcf -Ob -o out.bcf
   bcftools +fill-tags in.bcf -Ob -o out.bcf -- -t AN,AC
   bcftools +fill-tags in.bcf -Ob -o out.bcf -- -d
   bcftools +fill-tags in.bcf -Ob -o out.bcf -- -S sample-group.txt -t HWE

I do not know how to fix this issue. Any help will be welcomed!

[williamsbrian5064]

What happens if you add quotes around $file like

for file in ./*.vcf; do bcftools +fill-tags "$file" -Oz "${file%.*}".tagged_vcf.gz -- -t AF,AN,AC,AC_Hom,AC_Het,MAF,NS; done

[brianaloredana]

Nothing changed! Same error output.

[GenoMax]

module unload bcftools

Why are you unloading bcftools module in your code? Do you want to use an older/newer version of bcftools that is present elsewhere and is in your default $PATH?

[brianaloredana]

I am currently using 1.9 version. The hpc has also older versions

[GenoMax]

Examples for bcftools +fill-tags show bcf format files as well as an output file name. Take a look at bcftools +fill-tags -h

[brianaloredana]

Shall I keep it?

module unload bcftools

[GenoMax]

By unloading the module we have no idea which version you are using. Normally people load a module they want to use before running some code.

[williamsbrian5064]

It sounds like you are using your own version of bcftools. I don't know it is just the HPC or Slurm itself but if you type in module unload it will look like a module is being unloaded even if there wasn't one already loaded. Could you use which bcftools? You might be able to see the path to where bcftools is installed. What will at least tell you if you're using your own version or the version your HPC probably has in some shared directory

[brianaloredana]

I was working on Conda. I tried it by deactivating conda. As John mentioned in the comment, I also missed -o option in the loop. I have also included the BCFTOOLS_PLUGINS plugin environment to my slurm script.

module load bcftools
BCFTOOLS_PLUGINS="/cm/shared/apps/bcftools/gcc/64/1.9/libexec/bcftools/"

for file in ./*.vcf; do bcftools +fill-tags "$file" -Oz -o "${file%.*}".tagged_vcf.gz -- -t AF,AN,AC,AC_Hom,AC_Het,MAF,NS ; done

Answer 1

[John Marshall]

Without loop:

bcftools +fill-tags file.vcf -Oz -o file.vcf_ok.gz -- -t AF,AN,AC, AC_Hom,AC_Het,MAF,NS

With loop:

for file in ./*.vcf; do
    bcftools +fill-tags $file -Oz ${file%.*}.tagged_vcf.gz -- -t AF,AN,AC,AC_Hom,AC_Het,MAF,NS ;
done

It's good to think about what the options and file arguments you're using actually mean. Hence:

1. The ${file%.*}.tagged_vcf.gz is intended to be the output filename.
2. This corresponds to file.vcf_ok.gz in the non-loop version.
3. Oh, the -o is missing.

[brianaloredana]

Thanks John for checking my script. As I shared it in the comments above, I have also deactivated conda and add bcftools_plugin environment variable

[GenoMax]

If this solved your problem then go ahead and accept the answer (green check mark) to provide closure to the thread.

[brianaloredana]

The answer of John is not a complete solution. But I am adding here the complete solution including accepting his answer

module load bcftools
BCFTOOLS_PLUGINS="/cm/shared/apps/bcftools/gcc/64/1.9/libexec/bcftools/"

for file in ./*.vcf; do bcftools +fill-tags "$file" -Oz -o "${file%.*}".tagged_vcf.gz -- -t AF,AN,AC,AC_Hom,AC_Het,MAF,NS ; done