Recent test results announced by SGI are touting the Altix UV 1000 as the world’s fastest Java application server. While I love the Altix UV and think it’s a cracking bit of kit, this result highlights how useless benchmarks are.

SGI claim:

• SGI Altix UV 1000, with 256 cores and 128 JVMs, beat ScaleMP, its nearest competitor, by 82 percent in Java performance with throughput of 12,665,917 BOPS using Oracle JRockit.
• SGI Altix UV 1000, with 256 cores, beat Fujitsu/Sun, its nearest competitor, by 60 percent with a single Oracle Java HotSpot JVM performance of 2,818,350 BOPS.

As Darth Vader says – “Impressive. Most impressive.” – but lets look at those figures in more detail. A 256 core Altix UV 1000 running 128 JVMs? Who on earth would buy a massive single system image machine, with massive shared memory performance, and then carve it up with hundreds of JVMs – which are pegged to cores?

This is nonsense. Far better to pick a smaller machine – say, an Altix UV 100 – which would much more realistically be used for this sort of task. Or an Altix XE cluster, which would give both good parallelism and also high availability.

Joerg Mollenkamp has even more details, comparing these results to those of a Sun T5440. The price difference is extreme, leading to some embarrassing price/performance comparisons, which also highlights how meaningless these sort of benchmark results really are.

There’s no doubt those are some cracking results from the Altix UV 1000. But they’re meaningless, pie in the sky figures, and I can safely say anyone who ordered one of those to run Java apps on would be laughed at, and then fired on the spot.

A far more meaningful result I’d like to see from SGI would be pitching the Altix UV where it really needs to go – into the corporate data centre. Let’s see some data warehouse figures using Oracle and Sybase for big data workloads that can take advantage of all that fast, shared memory.

SGI need to move out of the pure HPC play for their big kit, and add in more sales to big business. (They’ve need to do this since the first big Origins came out, but that’s a rant for another day). This means playing to the kits strength, and posting JVM benchmarks like this accomplishes nothing apart from opening them up to some – very valid – criticism.