Entering edit mode
4.4 years ago
Thind amarinder
▴
340
I read in one article that "due to small sample size, p-values were not adjusted". What is the role of the sample size in adjusting the p-value? I am trying to understand this in more details. Author was referring to MutSigCV tool output.
Wow! This is a sentence that I could write! However, my friends that know more about statistics would translate it as:
(Sorry, I really had to tell this joke!)
I agree... I don't know why people are so afraid to say "We do not have power, so we will do a descriptive analysis of our data: these are the trends that we found". To me it is meaningless to try to categorize as "significant" the results when one changes so drastically the significance cutoff (such as not adjusting for multiple testing or using values such as FDR larger than 0.25). Moreover, these changes are often a bit "hidden" in the methodology (suspicious) and are easily overlooked.
Sample size influences your statistical power and the p-values which you can obtain in a comparison, because you have more or less information about your samples. Thus, a small sample size restricts the significance which you can obtain from your experiment. I'm guessing maybe they were referring to the fact that with a low number of samples it would be hard for them to observe low p-values in the first place, so they did not care for multiple-testing adjustment, because this procedure leads to making the adjusted p-values bigger (or requiring a threshold of p-values even lower/stricter than the "standard" 0.05) (I'm not judging if I believe this to be good practice, and I have also not used that tool you mention).
I understand the concept of p-value calculations but was confused that what they meant by "due to small sample size, p-values were not adjusted". I think you are right they skipped the multiple-testing adjustment for the reason you explained. Thanks.
My guess would be they mean low number of total tests conduced. E.g if there are fewer loci then the sum of tests done per loci aka total number of tests is few. For fewer number of total tests, multiple testing correction like Boneferroni would be too aggressive. And due to fewer number, the distribution of pvalues can not be determined so corrections like qvalue would be difficult to implement. Again it is my guess and not an authoritative statement.