Doing multiple testing correction before or after GO analysis

Question

[c0tton]

Hello,

In a given RNA-Seq differential gene expression analysis, one might perform a gene ontology analysis by setting cutoffs to create a gene set (e.g. abs(Log2FC) > 1, p < 0.05). Here, it is possible to use raw p-values (includes more true positive and false-positive DEGs), or FDR-correct p-values (includes less true positives but a smaller-fraction of false positives). Of course, the GO analysis needs to be corrected with FDR as well.

My question is, what are your thoughts for creating gene sets using p-values or q-values? In my mind, any set of filters that are selected A priori should be fine to create a gene set to test for enrichment, and as long as the GO-enrichment tests are corrected using FDR or a similar method, the results are still statistically robust.

Thoughts?

[dsull]

Honestly speaking, your gene list is not "statistically robust" (it's arbitrary since you have no sense of the actual false discovery rate) and you're doing a downstream analysis on a non-statistically robust gene list. It's like designing a study that goes: "I set my type 1 error rate at 5% and my power at 80% but those values may actually be way off because I didn't do my calculations properly".

I've seen people do all sorts of arbitrary things with GO (I've been guilty of this earlier on too): Select top 100 genes, select top 250 genes, etc. -- while some of those papers are good papers with insightful findings, that sort of analysis is not something I'd call "statistically robust".

Is statistics perfect? No, assumptions are violated all the time by real data and we don't have all the time in the world to make things perfect and we have to use our intuition for a lot of things (e.g. why logFC > 1 rather than 0.5), but my philosophy is: we can at least try to do things the most correct way when possible. q-values are more correct than p-values, so use them! If you are ok with more false positives, loosen your q-value threshold before you begin your analysis rather than use unadjusted p-value (I can at least interpret what a q-value is; the p-value means very different things to me when you're testing one hypothesis vs. one million hypotheses).

Answer 1

[bigomics.team]

Many times a abs.logFC>1 and q-value<0.05 does not give any significant genes. I those cases (for the FIsher or Chi2 test) you need to lower logFC or raise the q-value thresholds. Going to p-values or top 100 genes can give you 'false positives' in the genes but because they are probably 'random', and the GO analysis should render the geneset insignificant, i.e. give a high p/q value. Therefore, I strongly prefer enrichment methods like GSEA (KS test) that work on the rank of all genes because there you do not need to set a cut-off for logFC or p/q value.