Commodity Supercomputing - Networking

One of the main reasons that commodity supercomputing systems are becoming viable is the rapid advances in netowrking hardware. Fast Ethernet, rated at 100 MBits/s, is becoming the norm. This is now true commodity hardware, and the cost of network interface cards (NIC's) and, more importantly, switches is low - a little over 100 pounds sterling per node.

While fast Ethernet provides good performance for most networking requirements in an office environment, it is still significantly slower than proprietary supercomputer memory interconnects, such as those used in Cray and IBM systems. Some latency and asymptotic bandwidth figures are shown in the table I below. By investing significantly more money into the netowrking hardware, it is possible to reach and exceed the performance of man supercomputer systems with technologies such as Myrinet. New advances such as VIA hardware are also emerging, and may become commodity in the future.

Table I. Communications performance of commodity and proprietary memory interconnects using a unidirectional MPI pingpong benchmark.


Latency (microsecs)

Asymptotic Bandwidth (MBytes/sec)

Fast Ethernet (Linux/MPICH)



Myrinet (NT/Fast Messages)



SGI Origin 2000



SGI/Cray T3E-900



Meiko CS2



IBM SP2 (Thin2)



More network performance results are available on the GENESIS benchmarks web site.

It is important to realise the shortcomings of fast Ethernet and tune parallel programs appropriately. Clearly, tightly-coupled supercomputer systems will still dominate the high-end due to their excellent performance for all parallel applications. However, for medium- and coarse-grained problems commodity systems can provide extremely cost effective resources.

Details of the current state of this work are given on the status page. Please feel free to contact regarding this research.