HPC Architectures

Main Feature of HPC Systems

• Processors - which can cooperate concurrently.
• Memory - which is local to, or shared by the processors.
• Network - interconnect between processors, which allows data sharing and synchronisation.

A Taxonomy of HPC (Parallel) Architectures

• Single Instruction, Multiple Data (SIMD) Stream.
• Multiple Instruction, Multiple Data (MIMD) Stream.
  • Distributed memory (DM).
  • Shared memory (SM).

Single Instruction, Multiple Data (SIMD) Stream.

• Many, simple processing elements - 1000's.
• Each processor has its own local memory.
• Each processor runs the same program.
• Each processor processes different data streams.
• All processors work in lock-step (synchronously).

Single Instruction Multiple Data (SIMD) Stream SIMD Example

B(I) = A(I) * 4 => LOAD A(I), MULT 4, STORE B(I)

Multiple Instruction Multiple Data Stream (MIMD)

• Many processors (Alpha, SPARC, PowerPC, etc...)
• Each processor either has own, or accesses shared, memory.
• Each processor can run the same or different programs.
• Each processor processes different data streams.
• Processors can work synchronously or asynchronously.

Shared Memory MIMD

Distributed Memory MIMD

MIMD Example

HPC Systems.

Loosely Coupled (heterogenous networks of unix workstations):

IBM/SGI/HP/Sun/DEC/... workstations.

Interconnect by Ethernet (10 Mbit/s), FDDI (100 Mbit/s), ATM (155 - 622 Mbi$

Modestly Coupled (distributed memory, 1-1024 processors

Intel Paragon (i860), Meiko CS2 (SPARC), Cray T3D (Alpha)...

Interconnect by custom networking technology (400 - 1000 Mbit/s).

Tightly Coupled (shared memory, < 20 processors):

Silicon Graphics PowerChallenge (MIPS 8000), Sequent Symmetry (x486), Cray C90...

Interconnect by custom bus technology (200 - 400 Mbit/s).