Significant p values with non significant looking figures

Question

[mattescott44]

Hello I had a question about some results that I am seeing from my data comparing beta diversity of fish internal microbiomes. Looking at my PCoA there seems to be decent overlap between the two groups, but my permanova shoes that they are significantly different with a P value of .006. I was wondering if this is typical for microbiome comparisons or if it seems like I am doing something wrong.

Here are the results from my permanova: Permutation test for adonis under reduced model, Terms added sequentially (first to last), Permutation: free, Number of permutations: 999

adonis2(formula = GPdist ~ HostFamily, data = as(sample_data(phylum_ps_rare), "data.frame"))

HostFamily Df = 1, SumOfSqs = 0.2204, R2 = 0.03152, F = 5.2403, Pr(>F) = 0.006 **

Answer 1

[i.sudbery]

Overlapping does not mean insignificant. I don't have much familierity with microbiome studies in particular, but most tests can be significant, even when the distributions are highly overlapping, if your n is high enough. Your two distributions are overalling, but they are not identical. In particular the mean for your Lebiasinidae looks slighly higher on Axis 2 than the mean for your Characidae. This could just be random fluctuations in the sampling, and thats why we do statisitics. THe statistics are telling you that it is unlikely that the means are actaully the same.

[Philipp Bayer]

I think as a test, mattescott44 could remove a few of those (<5) outliers and rerun the test - I reckon they pull the group means out leading to that statistically significant difference, but might just be weird samples.