Scalemp Vsmp Or Physical Ram

Question

[Dave]

We are considering whether ScaleMP's vSMP is comparable to a single box with 512GB of RAM. Does anyone have any first hand experience with ScaleMP?

EDIT: It seems very few people have any experience with this technology. Anyone care to offer opinions one way or the other?

[Michael]

better ask that on serverfault.com

[Istvan Albert]

Interesting question but given how new these technologies are only a very small number of people have first hand experience with them.

[Dave]

Thanks, Michael. I'll ask over there.

Istvan, This company has been around a few years but seems to have a very low adoption rate. Most of the info on the web is company PR.

Answer 1

[Ketil]

I'm not familiar with ScaleMP as such, but you don't seem to get a load of other answers, so...

My opinion is that, for a lot of bioinformatics work, you really want an as big as possible, tightly coupled SMP system. Many applications can use multiple threads (blastall -a and so on), and can trivially take advantage of such systems.

Another class of bioinformatics solutions are scripts running various independent processes. These are often possible to parallelize into multiple processes using e.g. make -j or your shell's & and wait functionality.

AFAIU ScaleMP, it provides something like a SSI cluster, much like OpenMosix did all those years ago, and like Kerrighed and OpenSSI are doing now (or they were last time I checked). I've had some experience on OpenMosix, and it was okay for the second class, viz. parallel scripting. There were some interoperability problems, but mostly it worked well, and it sure was a lot easier than trying to convert scripts to the byzantine batch queuing language for running on a traditional cluster.

Threaded applications, on the other hand, are programmed with the viewpoint that memory is shared and "close". This is clearly not the case for a "virtual SMP", and even with Infiniband, I don't really see this working out. If you can split up your application, it's better to do it explicitly - either into separate processes (see above) or by using MPI. If you can't - and I contend that this is the case for much bioinformatics work - interconnect delays are going to kill performance, and you really need all those CPUs and all that RAM in the same box.

In general, I think HPC are doing the wrong thing for bioinformatics. It's okay to spend six weeks to rewrite your meteorology program to take advantage of the latest supercomputer (all of which tend to be just a huge stack of small PCs these days) if the program is going to run continously for the next three years. It is not okay to spend six weeks on a script that's going to run for a couple of days.

In short, I keep asking for a big PC with a bunch of the latest Intel or AMD core, and as much RAM as we can afford. (PS: Sorry about the rant)

[Dave]

Thanks for the response Ketil. We decided to purchase 4 128GB machines with intel procs and we are going to give scaleMP a try. We receive a very nice academic discount from Dell on the scaleMP software so the big cost is the yearly maintenance fee, which really isn't too expensive.

[Ketil]

Please report back on how it works out!

[Jimmy Chu]

Dave, how is your experience with ScaleMP vSMP so far? When it comes to cross-machines memory acccess, does the performance stand up to expectation? Any special tuning for the program and special method to start the app?

I am also learning about ScaleMP vSMP performance for bio-informatics app. Thank for your input!

[Jeremy Leipzig]

Ketil your rant was so good it made it into my blog.

[xrayp]

@Ketil Actually, the technology from ScaleMP is very different from Mosix/OpenSSI/etc. Without going into all the details, Mosix (and the others) provide a patch or module to run in/on your kernel. This means you run an operating system on each node. ScaleMP's technology, on the other hand, is a virtual machine monitor (a.k.a Hypervisor) - and there is only one (Linux) OS running on-top of the combined virtual system.

It seems like ScaleMP is actually putting a lot of focusing on the Bio-Informatics and Genomics spaces these days. They are publishing performance benchmarks for Assembly applications such as Velvet, as well as case studies (see this one posted by IBM: http://www-01.ibm.com/software/success/cssdb.nsf/CS/DLAS-8NNMVK -- seems like Brown University is very happy with the solution ;-)

Taking into consideration that with modern systems (Opteron 62xx and Xeon E5-46xx) you can only get to ~1TB RAM if not using uber-expensive 32GB DIMMs, one should actually give the technology from ScaleMP or look-into, especially if one is looking for >512GB RAM