Since not all small labs can afford access to a cluster or have the know-how to run their analysis on the cloud, I was wondering if people is looking at the current workstations in offer and how well they will perform for the most common NGS analysis these days.
I am presuming the biggest bottleneck is number of CPUs available, since disk space shouldn't be a problem these days. Something like HP or Dell's 12-core workstations?
Has anyone looked into workstations for NGS analysis?
1) you'd be surprised how quickly disk space can become a problem
2) With many common tasks, especially those that can utilize multi-core architectures (like short read alignment) disk IO is often limiting, as opposed to CPU time.
Memory and hard disk space are still issues depending on the task. What is it you want to do: reference mapping or denovo assembly? What size of genome?
e.g. denovo assembly using hiseq data from a large genome: 256GB of memory is not a luxury
For larger workflows (e.g. SNP association) with in-house database to increase time, 1TB per project before files are cleaned and compressed is not unusual.
There are ways to reduce memory footprint - e.g. for mapping, focus on 1 chromosome at a time and split files accordingly.
A workstation with the latest processor(s), filled with SATA disks and as much memory as you can afford will work (do not forget a backup solution though ...). Cheaper to build your own or find a specialised linux builder like DNUK.
Our main galaxy server for the centre cost about £7k: 20TB raid5,64GB ram and 12 cores (24 logical) Good for mapping NGS data pipelines and fast compared to waiting for queued jobs on the cluster.
Cost might be an issue, but if you lack the knowledge to run it on an existing (say Amazon) cloud, you should not even think about setting up your own desktop based system. The system does not necessarily get much simpler when you make it smaller.
On the other hand, yes it is doable. I heard a talk at NBIC Lunteren by a statistician from Groningen (wil try to find the name, but don't have it available here) who did run NGS analyses on a standard Windows (sic!) machine but he applied massive data reduction approaches based on his statistical ideas.
Our main galaxy server for the centre cost about £7k: 20TB raid5,64GB ram and 12 cores (24 logical) Good for mapping NGS data pipelines and fast compared to waiting for queued jobs on the cluster.
Why RAID5? It's not exactly the most reliable RAID config anymore: http://www.zdnet.com/blog/storage/why-raid-5-stops-working-in-2009/162
Quick check: I can get 12-cores, 2.4 TBs SATA and 54GB of RAM for about £5K. Add another £5K for external disk backups, not too bad. Thanks Alastair.
I meant 64GB of RAM
are raid disks easy to set up and maintain?
yes, most raid cards have a dedicated web interface