Bcftools Back And Forth Conversion Error

Question

[skymningen]

I have been trying to convert bcf and vcf files back and forth with bcftools for testing purposes. So, first I ran a basic samtools mpileup on Galaxy, resulting in "test.bcf". After converting this file back to vcf in Galaxy, reconverting it to bcf did not work, so I downloaded the file and tried it "the basic way" on the command line.

Trying this works fine and produces a perfectly normal looking vcf file (it also works in Galaxy):

bcftools view test.bcf > test.vcf

##fileformat=VCFv4.1
##samtoolsVersion=0.1.18 (r982:295)
#CHROM    POS    ID    REF    ALT    QUAL    FILTER    INFO    FORMAT    path/to/test.dat
chrTest    420    .    N    T,X    0    .    DP=1;I16=0,0,1,0,0,0,71,5041,0,0,42,1764,0,0,0,0    PL    42,3,0,42,3,42
chrTest    421    .    N    C,X    0    .    DP=1;I16=0,0,1,0,0,0,71,5041,0,0,42,1764,0,0,1,1    PL    42,3,0,42,3,42
chrTest    422    .    N    T,X    0    .    DP=1;I16=0,0,1,0,0,0,71,5041,0,0,42,1764,0,0,2,4    PL    42,3,0,42,3,42
chrTest    423    .    N    A,X    0    .    DP=1;I16=0,0,1,0,0,0,71,5041,0,0,42,1764,0,0,3,9    PL    42,3,0,42,3,42

Then I went on converting test.vcf back to test.re.bcf.

bcftools view -b test.vcf > test.re.bcf
[bcf_sync] incorrect number of fields (0 != 5) at 0:0

I thought the error was caused by the header and produced a version of test.vcf without the first two and one without any header lines. Still the same error message.

I have seen the same error (incorrect number of fields) a lot in other forum posts and such, but what puzzles me here is the fact, that the first conversion from bcf to vcf is working and producing normal output, but obviously bcftools cannot cope with the type of vcf it has produced itself. Has anyone else experienced this and maybe found a reason and a solution for this problem?

Answer 1

[John Marshall]

Bcftools requires a BCF input file by default. To tell it that your input file is VCF, you'll need to add -S:

bcftools view -bS test.vcf > test.re.bcf

at which point you'll find that you need to add further options, but that'll get you started.

This is admittedly rather silly. Bcftools ought to be able to give a less incomprehensible error message for _this is not a BCF file_, and really it ought to be able to auto-detect what file format you've given it. We're building the infrastructure to do the latter at the moment, and future versions of bcftools will likely just read either BCF (v2) or VCF without needing to be explicitly told which they're getting.

[skymningen]

Ah, it is basically the same as for samtools. I could have thought of that. Edit: I just checked my Galaxy wrapper and realised that I already used -Sb there, and it still didn't work. Unfortunately I cannot test it right now on command line, due to some server problems.