Entering edit mode
8.5 years ago
weixiaokuan
▴
140
Recently I ran a WGCNA network construction analysis. The array type has more than 50K genes but luckily I have a powerful cluster which has more than 600G memory, so I set the maxBlockSize=60000. But I got the error message: 'maxBlockSize must be less than 46340. Please decrease it and try again.' Where does this 46340 come frome? Can I use any size of maxBlockSize if I have enough memory? Thank you. -Xiaokuan
Answer by Peter Longfelder from bioconductor fourm:
"The limit is (the square root of) the maximum matrix size addressable by a 4-byte pointer. I will eventually update the WGCNA code to remove the limit, but for now it is what it is. The limit is independent of how much memory you have. Sorry! Peter
How much memory is on the current Node you are using? The tutorial says "32GB workstation should handle perhaps 30000." Are you multi-threading the WGCNA analysis?
At first I did enable multi-thread for WGCNA. After I found this error, I disabled multi-threading but this error still exists. -Xiaokuan
You didn't answer my question. How much memory is on the node you're using?
it has about 600G memory on the node I was using.
I think you should/could use a variance filter to remove non-variant genes before doing network construction.
I want to use all the genes not filtered ones. I guess that's one of the feature of WGCNA, it will identify eigengenes from matrix and no need to filter. my point is that i have large memory available but it seems I cannot take advantage of it. -Xiaokuan
I don't agree, genes with (nearly) no expression might show spurious correlation just by having small counts...
I believe OP is using micro-array data not RNAseq
yes, I used microarray.