Selecting between p.value, FDR, Bonferroni

Question

[niutster]

Hi,

I am using "seqlm" package to find DMRs. Seqlm returns statistical parameters such as p.value, Bonferroni, FDR, ...for each segment (deferentially methylated region) .In one of these analysis, most segments had meaningful p.value(<0.05) but their FDR are more than 0.5 and Bonferroni is 1 or near 1. How can i filter correct value based on statistic? Is there a problem with this run?

[Jean-Karim Heriche]

I think you should learn about multiple testing correction. It is perfectly possible that after correction, you don't get any significant test. This just means that at the level of confidence you selected, you can't reject the null hypothesis (usually of no difference between samples). However, don't make the mistake of thinking that statistical significance means biological relevance.

[ivivek_ngs]

Nice way to put it. It is important to understand how one performs the multiple testing. The line sentence speaks volumes about correlation of statistical testing with biological significance.

[Benn]

For omics data most people use FDR corrected p-values, Bonferroni is much too stringent. Don't use raw p-values with omics data.

Answer 1

[agata88]

I don't know seqlm package but If you have comparison between multiple segments you need to use multiple correction FDR or Bonferroni, Statistics says that 1 per every 20 segments in multiple comparison will always result with significant raw p-value (e.g. 5/100 t-tests will end up with p<0.05).

If you are checking for significance difference between two segments you can use raw p-value, when you have multiple segments - use p-value from FDR or Bonferroni.

Best,

Agata

[Benn]

I think factor is not the right term to use here, and is even confusing.

In this case there are probably many regions (segments) either methylated or not. That is just one factor, but many tests. There for you'll have to use a multiple testing correction.

[agata88]

Thanks, i am going to change that to make post more clear.