Entering edit mode
16 months ago
Francois Piumi
▴
70
Hi, I still don't understand the need for multiple testing correction in RNA-Seq.
I understand that we first test the probability of expression of a gene in two conditions (control vs treated), which the p.value, but why do we correct this test with the tests made on other genes (adjusted p.value)?
For me, expression of gene A and gene B are independent. What I intuitively get is multiple testing correction means that expression of gene A and B are linked, but I don't understand why.
It has nothing to do with the genes being linked. Rather it has to do with the structure of the experiment. In an RNA Seq experiment every gene is considered an experiment, thus every gene represents a hypothesis. Whenever you do thousands of experiments - even under the null hypothesis, you get a few differences between treatment and control (for instance, take the blood pressure of 10,000 people who DO NOT HAVE high blood pressure, and you will conclude at some p-value that some of them do because they measured high in your experiment). If instead of RNA Seq, you measured yeast gene expression by Northern blot, running gels and making probes for 6000 genes over the course of years or months, you would no doubt observe some differences in gene expression simply between control strains when there really is nothing there to observe. All adjustment does is account for the fact that a highly parallel experiment will generate spurious results with low p-values due to the nature of noise, by providing a way to pay attention to those results with the lowest p-values. (if you multiply all p-values by the number of experiments performed, things with really low p-values will still survive).
This is incorrect; rather the difference between major classes of multiple hypothesis testing control algorithms stratifies cleanly on how they address the degree of linkage between the observations. In some cases, the design of such a metric is even entirely predicated upon the degree of (non)independence between observations for that reason.
For instance, Bonferonni correction is the most conservative form of controlling for false positives precisely because it treats each hypothesis test performed as if it is entirely independent from the next, which is why the alpha level is divided by the number of tests performed. It is because this statement is nearly always untrue in biology that the use of bonferonni correction is considered conservative in biological systems.
Now lets take algorithms aimed specifically at linked data. The number of effective (Neff) markers in studies of DNA, for example. Calculation of Neff rely on precisely the degree of linkage between SNVs (or other variants being assayed), which is in turn a function of the degree of linkage disequilibrium between variants within each haploblock and the number of haploblocks per genome. In other words, exactly what is necessary to calculate Neff is the degree of 'linkage'.
Moving the domain of exploration to transcriptomics does not change this.
Is it really incorrect to say that the idea of multiple hypothesis testing as a necessity in a highly parallel experiment is independent of whether genes are linked? You raise Bonferonni as an example of a correction method where linkage is irrelevant. I think there are two things here. One is the necessity for correction when one is measuring many things at once. The lack of intuition around this for the OP is what I was referring to. The second is the method to use, once the necessity for a method has been realized. As you eloquently point out, some methods are designed with the idea of linkage in mind.
Regarding your second comment to Francois, I don't take his language quite as literally as you appear to, especially given the nature of the confusion in the original question. Rather I frame his statement as a first approximation of an experimental biologist, with the goal of simply detecting DE genes in an experiment. So his hypothesis about the expression of gene A is independent of gene B. Of course all genes are linked biologically at some level, but an experimental biologist would treat the idea of interdependence as a secondary question that can be addressed later, and is not directly related to or inferred from multiple hypothesis correction.
hi siedel,
Re: the first part, I understand what you mean, I just would rather have OP nuance the the statement to the degree they are able in this case.
I considered the possibility that he didnt mean it seriously - in actuality he probably doesn't. but I still wouldn't post anything like that on a science forum, under any circumstances, but particularly if the main question being asked is both A) beginner-level and B) the answer to it is related to the claim that is patently false.
Thank you very much LauferVA and seidel for these interesting and high-level exchanges (I'd rather say 'too high' sometimes for me ;-) ) So If I understand well, you'll correct me if I am wrong, A test is made for each gene. Some tests give p-values just below the significant level while related genes are actually false positives (incorrect rejecton of H0). This is due to experimental issues. What I still don't see is why we compare this result with other genes test results. What is allowing us to do that? If every gene is an experiment, how is it possible to compare experiments between them? I am maybe nitpicking but I truly want to understand. If I reverse the problem, is it possible to make a FDR correction only on one experiment (for example in a QPCR with only one gene)? Maybe this gene is also a false positive, but as we've done only one experiment, we will never have the answer.
I read your post and I understand the goal of it. However, I am going to respond to another part of the post, that I think is even more important to you as a biologist:
Francois - all of life from prokaryotes to fungi to metazoans is crucially dependent on this statement being very untrue.
In fact, the transcriptome is the most amazingly well-integrated and well-organized thing I know of rather for the opposite reason. When I explain transcriptomics to people without biological backgrounds, sometimes I explain it using an analogy like, "imagine that the transcriptome of a cell is like a large company":
The transcriptome is like that, but orders of magnitude richer and more complicated, and the reason why that is true is because of the marked non-independence of gene expression - that is, its incredible organization into coordinated programmes of expression regulated in space, time, quantity, etc., at the same time.
Thanks LauferVA. I understand your answer which is absolutely relevant. However I thought that FDR correction was also a method to correct mathermatical or probability calculation issues, and has nothing to do with biological interactions. I must confess that I am still missing something and I have great difficulties to solve this problem.
the idea is that different kinds of multiple testing correction address different scenarios.
bonferonni correction is most appropriate when the hypothesis tests are truly independent. controls that draw on FDR/FWER are more appropriate when the observations ARE related.
VAL