Hello, I have low coverage (~1x) sequencing data from ~50 individuals from two populations and would like to calculate Fst for each biallelic variant. I can calculate allele frequencies based on the genotype likelihoods and estimate Fst following Wright 1951, but this assumes infinite populations which accurately sample allele frequencies. Alternately, I can use vcftools to estimate Fst following Weir & Cockerman 1984, but this implementation uses 0/1/2 genotypes while accounting for finite sample sizes (i.e. doesn't incorporate genotyping error). Is there a method that incorporates both genotype error and sample sizes? Alternately, does it seem kosher to use genotype likelihoods as the allele estimate per individual and the method of Weir & Cokerman?
Thank you in advance!
