Catching up on some SGI news, and along with the details of some nifty new storage products (more on those soon) the really interesting news is that SGI have announced a new partnership with AMD.

Day-one support for the new AMD Opteron 6000 series platforms is offered across SGI’s entire design-to-order server portfolio, including CloudRack™ and Rackable™ scale-out servers and SGI® InfiniteStorage servers. The ICE Cube™ modular data center also supports AMD Opteron processors for the first time.

It’s not just the old Rackable gear which is getting some AMD love – the ‘proper’ SGI product line is also getting Opterons:

As part of SGI’s increased commitment to AMD processor support, SGI expects to release AMD Opteron processor-based configurations of its Altix® ICE high performance computing (HPC) clusters and Octane™ III personal supercomputer later this year. Similarly, the SGI HPC cluster software stack will also be available on the AMD Opteron platform for the first time.

No mention of AMD support for the Altix, which is odd. When Silicon Graphics first said they were dropping the MIPS Origins and moving to Intel processors, the first thought was – why not AMD? AMD had a credible NUMA connect – Hypertransport – whereas Intel’s x86 offerings were still stuck with legacy bus interconnects. Itanium was too much of a wild card – but SGI drank the cool aid and embarked on a painful path.

You just need to have a look at Cray, who have managed a successful transition to AMD cores, and done pretty well out of it, to see what might have been. Cray clearly had the better idea – migrate to AMD and Hypertransport, plugging it into their own NUMA interconnect, and then drop in Intel x86 chips when they finally mature.

Nehalem is the long overdue x86 with a sensible NUMA interconnect, and Cray are well positioned to take advantage of the manufacturing scale. SGI’s use of AMD Opterons seems long overdue, and the timing is odd now that AMD seem to be struggling to keep Opterons performing well against the new Nehalems.

Could SGI be hedging their bets, opening up customer choice with AMD at the low end, and seeing how things pan out before plugging Opteron in to the high end Altix? Or are they treading carefully with Intel to secure higher performing Nehalem Xeon chips for the high end?

Intel have announced that the massively delayed Tukwila Itanium processor is now available. HP is crowing about the performance advantages, but it’s not like they have much of a choice. Interestingly though, there have been no Altix related announcements from SGI.

The delays in Tukwila have hurt SGI’s NUMA sales, and the new Altix UV gives much better price/performance than Itanium could ever deliver. I predicted (Project Ultraviolet and the future of Itanium Altix) that we’d see a final Itanium Altix using Tukwila later this year, before the Itanium line was killed off.

With no product announcement from SGI to accompany the Intel fanfare, and with SGI’s Cyclone cloud offering accidentally offering a neat Itanium to x86 migration platform, it looks like we could finally be seeing the death of Itanium within SGI’s product line.

Silicon Graphics went there in the past with the R8000 MIPS CPU – it had the potential for massive performance, but only if you optimised the code and really knew what you were doing. That level of investment is always a niche game, and with the lower price and better performance for less effort from x86, Itanium was always going to struggle in the long term.

It’s just a shame SGI had to go bankrupt twice and shed a load of talented and skilled engineers to learn the lesson.

As expected, SGI have used the SC09 show to launch their latest single system image NUMA machine – the Altix UV. The specs are impressive – not only because SGI have dropped in Nehalem EX processors (as expected) – but the improvements in core density and NUMAlink bandwidth are also impressive.

As with the previous Origin and Altix machines, the Altix UV is available in two models. The Altix UV 100 is the familiar 3u ‘building block’ system, allowing you to scale up as needed. Altix UV 100 scales to 96 sockets (768 cores) and 6TB of shared memory in two racks for up to a claimed 7.0 Tflops of performance.

Altix UV 1000 is the big daddy, scaling up to 256 sockets (giving 2,048 processor cores) and 16TB of shared memory in four racks, for up to a claimed 18.6 Tflops of performance. Interestingly, the 16TB memory limit is imposed by the Nehalem EX architecture.

The next generation of NUMAlink offers a staggering 15 GB/sec transfer rate. The new hub chip has been designed to offload MPI communication. Instead of the CPUs having to handle the packaging and transmission of MPI messages, the UV hub now takes that load. This clears the CPUs to do pure number crunching, but still enabling the level of fast MPI communications that’s needed in such a large NUMA system.

Speaking of large systems, the UV design allows individual 4 rack systems to be hooked together in an 8×8 torus. The theoretical limit of the UV hub could provision over 32,000 cores. The UV hub also has some FB-DIMMs to cache directory information, which not only speeds up operations but also helps with the scalability of the solution.

The design of the processor board is interesting. SGI have used Intel’s Boxboro chipset to handle I/O, with the UV hub plugged directly into both Nehalem CPUs via the QPI interconnect.

The I/O risers mean that, since it’s a single system image, any processor core can access any I/O device anywhere in the system. With the potential for so much I/O throughput, it would be interesting to see what a large Altix UV system packed with Tesla GPUs could achieve.

The Altix UV is an evolution, rather than an evolution, of the flexible NUMA design that first appeared in the Origin 3000. Despite all the press about clusters, big single system image machines still remain the most efficient for many problems. The problems that needed solutions like the original Origin 2000 – pre- and post-processing tasks, very large data problems, I/O and memory intensive apps – have, if anything, gotten more complex and demanding over time, and SGI still have the technology to solve them.

The Nehalem EX won’t be formally launched by Intel until Q2 2010, so SGI aren’t releasing any performance figures. However SGI have announced four initial customers, who will be taking delivering once the processors start volume shipment.

The customers announced at launch are the University of Tennessee (1024 cores, 4TB memory), the North German Supercomputing Alliance (HLRN) (two systems totalling 4352 cores, 18TB of memory, to plug into their existing ICE installation), CALcul en MIdi-Pyrénées/Computations in Midi-Pyrénées (CALMIP) based at the University of Toulouse in France (128 cores and 1 TB of memory), and the Institute of Low Temperature Science at Hokkaido University in Japan (180 cores, 360 GB of memory).

With the Altix UV no longer requiring customers to recompile for Itanium, SGI now have a real chance to push these machines – not just for HPC, but also in business data centres, where Sun and HP have been very successful selling large machines like the F25k and Superdome.

Those paying attention to Silicon Graphics history will recall Dr. Eng Lim Goh back in 2004 talking about Project Ultraviolet – a supercomputer that used different types of processor types in a single frame. Although the original white paper has vanished from the SGI site, archive.org has a mirror available here.

The idea of HEC Architecture (High Efficiency Computing) and Multi-Paradigm Computing is a good one, and we’ve already been seeing the fruits in that, specifically Altix gear with FPGAs, and the rise of the GPGPU. Is the current Project Ultraviolet the same beast as the one outlined 5+ years ago?

I mentioned back in February the possibility that, with Tukwila delays and Nehalem processors having a NUMA interface, the time was ripe for SGI to put Nehalem Xeons into the Altix.

Now that Rackable have taken over the shell of SGI and have finalised their product offerings, we can see an interesting mix of processing power. Rackable have both Opteron and Xeon in their gear, whereas the (old) SGI had Xeon and Itanium.

With the Nehalem Xeons having the QuickPath interconnect, and Opteron always having had the NUMA HyperTransport, is now the time for Itanium to be shown the door?

HPCWire and InsideHPC (who know their stuff and are well worth a read) seem to think so. Even the eternally inaccurate TPM at The Register is agreeing (top tip, Timothy – NUMAlink is not a ‘cluster’ interconnect).

Xeons and Opterons inside the shared memory Altix make a lot of sense – immediate cost savings, the chips already have NUMA interconnects so not too much engineering required, and an instant boost in apps and developers.

Existing Itanium customers aren’t going to be too happy, but I’d point people at John Mashey’s excellent essay on the design behind the NUMAflex architecture. These machines were designed to be flexible, to swap CPUs without having to swap the frames, I/O subsystems, and everything else, thus protecting the customer’s investment in the technology, and lowering the price (and pain) of upgrades.

SGI CEO Mark Barranechea has re-confirmed the company’s commitment to Itanium on his blog – but with so many existing customers running on Altix gear, it would have been a PR disaster not to.

I think it’s clear that the next Altix cc:NUMA system will be based around Nehalem Xeon processors. The odds are good that we’ll be seeing an Opteron version as well, and it’ll be interesting to see how that stacks up against offerings from Cray.

As for Itanium? I think we will be seeing a Tukwila based Altix next year sometime. After that, I doubt very much we’ll see future Itanium kit from SGI.

Bad news – more layoffs at Silicon Graphics. This time 9% of the workforce – 120 people. Coming hard on the heels of the announcement that SGI have landed the DoD infrastructure refresh deal, this must be an especially bitter pill for those let go to have to swallow.

The Register has some more details, as well as more rumours and speculation that Silicon Graphics will finally ditch Itanium and stick with 64bit Xeon Nehalem chips. I’ve discussed their Quick Path Interconnect (QPI) before, and I reckon they’ll probably a much more favourable price point (for both SGI and it’s customers) in the big Altix gear.