How to decide the best threshold for filtering probes in limma

Question

[das2000sidd]

I am using an approach where I fit the model using all the probe sets and then subsequently at the eBayes step, set a filter and then on the eBayes object use "plotSA" to see if there is a monotonous pattern between the residual standard deviation and log average expression. Would this be a reasonable approach? Below is the code I am using:

no2_lm=lmFit(bryois_no2,no2_design_mat) 
no2_lmfit_eb=eBayes(no2_lm[no2_lm$Amean > 8,],trend = TRUE,robust = TRUE)
plotSA(no2_lmfit_eb,main="Plot of residual standard deviation vs Avg exp for fitted model using NO2")
summary(decideTests(no2_lmfit_eb))
no2.table <- topTable(no2_lmfit_eb, sort.by = "F", n = 8793)

Answer 1

[Gordon Smyth]

Yes, it is a reasonable approach, if you are analysing a data type where a decreasing variance trend is expected.

Your reference to "probe-sets" suggests that you might be analyzing Affymetrix microarray data. If you are analyzing Affymetrix microarray data with RMA normalization, then a quadratic trend (increasing then decreasing) is expected. In that case, the procedure you propose will work fine but will be conservative in the sense that you will filter some probe-sets that might be significantly DE.