Is it appropriate to apply quantile normalization on RNA-Seq before computing the Pearson correlation?

Question

[ATCG]

Is it appropriate to apply quantile normalization on RNA-Seq before computing the Pearson correlation? Specifically, RNA-Seq data displays a dense tail distribution, and Person's correlation (cor function in R) assumes a normal distribution. I tried this in R using the normalize.quantiles function discussed in the lecture, and my results make a lot of sense, but I want to make sure that this is an appropriate transformation before accepting these results. Thank you for your help!

[Asaf]

Of course you will get normal distributions out of normalize.quantiles but I wouldn't count on it. The main problem is the low counts, you will get random effects amplified. I would try spearman's correlation instead but make sure you normalize for library size first.

[ATCG]

Yes. My pipeline is rawcounts-->DESeq2-->normalized.quantiles Or do you suggest normalizing the rawcounts using a different method? Is there an R function to do the library normalization?

[Asaf]

DESeq2 normalization is great. Maybe try spearman's on the normalized count table. Make sure the r distribution looks good. You can use the normalize.quantiles but just make sure you're not getting artifacts from 0 counts.

[ATCG]

Yes. I am removing rowMeans<10.