How to adjust p-values obtained by pairwise comparison of treatments, but I have multiple treatments?

Question

[dakterl79]

My experiment have several treatments (e.g. A, B, and C).

If I have interest in differentially expressed genes (DEGs) over all treatments, ANOVA will be good enough.

However, my major focus is identification of DEGs between two treatments, but came into trouble with adjusting p-values. What will be the best solution for adjusting p-values?

1. just perform multiple test correction (e.g. p.adjust function of R) using p-values from each pairwise comaprisons (e.g. A vs B)?

2. collect all p-values from all pairwise comparisons (A vs B, A vs C, B vs C) and run single multiple testing correction and report q-values?

3. any best solution for this?

Thank you in advance.

Answer 1

[dariober]

Hi- Loosely related, I wouldn't use a straight ANOVA applied to each gene independently but rather use the methods implemented packages like edgeR or DESeq since you get more power and better normalization.

If you perform several comparisons (A vs B, A vs C, B vs C) I think in theory you should correct for all the tests you have performed by collecting all the p values from A vs B, A vs C and B vs C and adjust on that long list (using FDR, probably). However, this may be overly stringent since some tests are not independent.

In practice, I think most of us in this situation correct p-values within each comparison, regardless of how many comparisons have been done. So +1 for taking care of this complication!