How to filter out duplicate records in a vcf with bcftools?

Question

[vitor.aguiar]

Is there a bcftools command to remove duplicate records in a vcf file?

If a variant has duplicate records, I'd like to remove all entries of such variant in the vcf.

[Medhat]

beside unique can you take a look at this "Variant Normalization"

http://genome.sph.umich.edu/wiki/Variant_Normalization

http://blog.goldenhelix.com/grudy/variant-normalization-underappreciated-critical-infrastructure/

http://genome.sph.umich.edu/wiki/Vt

[cpad0112]

https://github.com/vcflib/vcflib#vcfuniq

Answer 1

[Brice Sarver]

Exactly duplicated records?

If the records are truly identical, sort the file and use uniq.

Make sure the vcf isn't compressed before you pass or pipe!

Edit: I should clarify that you don't need to use BCFtools for this. So, use good-old regular uniq not bcftools uniq; that doesn't exist.