PCA when the dimensionality is greater than the number of samples in R

Question

[LJ]

Dear all，

i have a dataset(n samples containing m dimensions,and n﹤m),and i would like to reduce the dimensionality of this data using PCA method("prcomp" function in R).However,it only returns a n-by-n matrix,So how to perform PCA when the dimensionality is greater than the number of samples in R? Thanks in advance.

[Devon Ryan]

PCA is working correctly, why would you expect more rows than that?

[LJ]

for example,n=10,m=1000,so the result returns a 10X10 matrix, and the 10 PCs explains 100% variance, but 10 components is far smaller than 1000 features,is it reasonable?

[Devon Ryan]

Yes, 10 components is reasonable, since it's impossible for there to be more. In practice, you want fewer than N, as dariober pointed out.

[dariober]

It's because PCs are orthogonal to each other and once you have as many dimensions as datapoints there is no more variation to explain.

[LJ]

Thank you guys for your replies.

Answer 1

[dariober]

If you want to reduce the dimensionality through PCA you should compute the principal components of your data matrix, as you did, and pick the largest n components. Usually the first 2 or 3 contain most of the information (plus if you choose 2 you can easily plot them). The plotting function screeplot can be useful to inspect the variance explained by each component.

Answer 2

[Carlo Yague]

From a matrix (genes X samples), you can make a PCA on either the samples or the genes :

• On the samples (as here), the maximal number of dimensions is the number of genes.
• On the genes (as here), the maximal number of dimensions is the number of samples.

Obviously those two PCA highlight very different things from your data. To switch from one to another, you usually have to transpose your matrix.