p-value/adjusted p-vlaue cutoff

Question

[alex.kang1014]

Hi,

I hope everyone reading this is having a great time with their research.

May I ask a question please?

I understand that conventionally, when selecting DEGs or other entities (such as pathways using the fgsea package), an adjusted p-value cutoff of 0.05 is typically used.

However, when I apply this cutoff, I sometimes end up with too few pathways in the fgsea results (as most pathways have an adjusted p-value greater than 0.05).

In such cases, is it acceptable to increase the p-value cutoff to 0.1, 0.2, or even 0.5 to include more pathways in the analysis?

Thank you in advance!

Answer 1

[ATpoint]

fgsea tends to produce small pvalues even when visual evidence based on enrichments plots is low, in my experience. If anything I would go far beyond the typical cutoff. Maybe 10^-5 or so but definitely not more lenient.

[alex.kang1014]

Thank you so much, Dr. ATpoint, for your insightful response.

I wasn’t aware that fgsea tends to produce small p-values even when the enrichment plots don’t show strong visual evidence. I’ll definitely try to keep this in mind moving forward.

I truly appreciate your valuable opinion again Dr. ATpoint!

Answer 2

[Jean-Karim Heriche]

The Benjamini-Hochberg adjusted p-value corresponds to the expected proportion of false positives among all the positives (i.e. those above the cut-off) so decide the cut-off based on how many false positives you find acceptable.

[alex.kang1014]

Thank you so much for your answer, Dr. Heriche.

I understand now that there isn't an absolute cutoff for the p-value or BH-adjusted p-value. Instead, it can be set based on the number of false positives I’m willing to accept among the significant results. I really appreciate the clarification, Dr. Heriche!

I'll keep the discussion open to hear other perspectives as well. Thanks again, Dr. Heriche!

[Mensur Dlakic]

so decide the cut-off based on how many false positives you find acceptable.

We don't do research in a vacuum, and it usually comes down to what reviewers, and readers in general, find acceptable.

[Jean-Karim Heriche]

Indeed but this is just one factor one can consider in deciding what is accpetable. There can also be other considerations depending on local context. For example, you may want to do follow-up experiments and it may be acceptable to have 10-20% false positives if that means catching more things of interest.

Answer 3

[Mensur Dlakic]

It is not just what cutoff you are willing to consider. What matters more is what your readers and your reviewers are willing to accept as a reasonable cutoff. Nobody will question if you pick values in the 0.01-0.05 range. I think you will need to show additional experimental evidence for selecting a cutoff > 0.1 and you may be fighting an uphill battle to make that stick.

[alex.kang1014]

Thank you so much, Dr. Dlakic, for your valuable insights.

I completely agree with your point about choosing p-value thresholds that will be acceptable to readers and reviewers, especially when using a cutoff greater than 0.1. I’ll be sure to keep your advice in mind for future analyses.

Thank you again for your valuable response, Dr. Dlakic!