I suppose I should be resigned to the whole benchmark silliness, where vendors highly tune their own systems and then run favourable benchmark tools – that have no real-world relevance – and compare the results against dissimilar and un-tuned gear from their rivals.

It’s the biggest ego match in the industry, and it’s all pretty sad. Are people really naive enough to buy kit based on these benchmarks, instead of trying out different vendors’ solutions in-house using their own real world workloads?

Regardless, I find the latest press release from Sun touting their ‘HPC leadership’ a bit much.

In it, Sun cites SGI beating FLUENT benchmarks. Yet, the press release doesn’t mention numbers at all. The link to the actual FLUENT benchmarks shows no values at all for the X2250 cluster Sun used, and the main Sun benchmarks website makes no mention of these ‘HPC dominating’ benchmark figures either. Surely if you’re going to trumpet the results, then you should also be making the actual figures available?

bmseer (a blogger from Sun who I have a huge respect for, and who regularly picks apart the outrageous benchmark figures from IBM) should be hanging his head in shame right about now ;-)

Regardless, the Sun benchmark site is well worth a visit at http://www.sun.com/benchmarks/

Silicon Graphics could take a leaf out of Sun’s book here and be far more pro-active and upfront with their current systems’ performance figures.

Meanwhile I live in hope for the day when we have a benchmark that measures the enormity of a vendor’s lies in their press releases. Like the Top 500, I’m sure IBM would be way out in front.