SGI Origin 3400

The Silicon Graphics Origin 3000 series machines are modular single-system-image (distributed shared memory) supercomputers, released in the year 2000. They're scalable from 2 to 512 processors and from 1GB to 1TB of RAM. The interconnect between nodes is SGI's proprietary NUMAlink3, which offers very low latency (1 microsecond for MPI messages, 50 nanoseconds round-trip for raw NUMAlink packets) and high bandwidth (1.6 GBytes/s in one direction). The topology of the entire system is a fat-tree hypercube.

Table of contents

General description ^

Each Origin system consists of multiple different "bricks" (that is the official terminology), the following types are available:

NameDiagramBack panelDescription
C-brickC-brick diagramC-brick back panelContains up to 4 processors and 8GB of RAM. It's possible to mix different CPU types and speeds within the system, but not within a single C-brick.
I-brickI-brick diagramI-brick back panelProvides basic I/O: 10/100 ethernet, two fibre-channel disks, CD-ROM drive, five hot-pluggable PCI-X slots, two USB ports, and a FireWire (IEEE-1394) port.
R-brickR-brick diagramR-brick back panelProvides routing between C-bricks in systems with more than two C-bricks (there are 6- and 8-port versions of the router brick).
P-brickP-brick diagramP-brick back panelProvides an additional 12 hot-pluggable PCI-X slots (on 6 busses).
X-brickX-brick diagramX-brick back panelProvides four XIO expansion slots (XIO is SGI's proprietary expansion interface used in their previous line of Origin 2000, Onyx2, and Octane systems).
G-brickG-brick diagramG-brick back panelProvides graphics (this brick is the only difference between an Origin 3000 and an Onyx 3000). Each G-brick supports one or two InfiniteReality pipes (sets of boards).
V-brickV-brick diagramV-brick back panelProvides lower-end graphics (an Onyx 3000 can have either one or more G-bricks or one or more V-bricks, but not both). Each V-brick supports one or two InfinitePerformance pipes, each pipe consists of a VPro V12 graphics board (also available in workstations).
N-brickN-brick diagramN-brick back panelConnects four C-bricks to four G-bricks (useful if you don't need the extra I/O of I- and X-bricks that would have been used for this purpose instead).
D-brickD-brick diagramD-brick back panelProvides disk storage (twelve hot-pluggable fibre-channel drives, no RAID).
Power bayPower bay diagram (front)Power bay diagram (back)Provides 48V DC power to bricks within a rack. Contains 3 to 6 power supplies that are hot-swappable and N+1 redundant.

Each brick also contains massive hot-swappable fans in a front-to-back airflow configuration and an L1 controller with an illuminated LCD, showing the status of the brick.

In addition to bricks, each system contains one or more L2 controllers (one per rack) with optional monochrome touchscreen LCD (for turning the system and individual bricks on/off) and power bays (with hot-swappable redundant power supplies). The power supplies provide 48V DC to bricks within a rack.

The difference between the Origin 3200, 3400, and 3800 units is purely in the combination of bricks. The 3200 has a maximum of 8 processors and comes in a short rack with two C-bricks, one I-brick, and one power bay (one slot is free for an additional I-, P-, or X-brick). The 3400 has a maximum of 32 processors and comes in a tall rack with two to eight C-bricks, one I-brick, two R-bricks (6-port), and one power bay. The 3800 series is for larger installations and comes in multiple racks.

Block diagram of the Origin 3400 ^

Block diagram of the Origin 3400

My Origin 3400 ^

My Origin 3400 has 16 processors and 16GB of RAM (four C-bricks, one R-brick, one I-brick). Half the processors are 500MHz R14000 and half are 400MHz R12000, both types of processors have 8MB of L2 cache. The root filesystem is on two 15kRPM fibre-channel drives running in RAID1 (mirrored) mode. The machine is missing its rack, since it was way too heavy to ship, so compromises had to be made :)

The configuration I have weighs about 230kg (the bricks weigh about 200kg, the rest of the weight are cables). The power consumption of the entire system is about 1300W when idle, about 1400W at full load, and 300W in stand-by mode (system off, but L1 and L2 controllers on). The power factor is an almost-perfect 0.99!

Output of hinv

Processor 0: 500 MHZ IP35
CPU: MIPS R14000 Processor Chip Revision: 1.4
FPU: MIPS R14010 Floating Point Chip Revision: 1.4
Processor 1: 500 MHZ IP35
CPU: MIPS R14000 Processor Chip Revision: 1.4
FPU: MIPS R14010 Floating Point Chip Revision: 1.4
Processor 2: 500 MHZ IP35
CPU: MIPS R14000 Processor Chip Revision: 1.4
FPU: MIPS R14010 Floating Point Chip Revision: 1.4
Processor 3: 500 MHZ IP35
CPU: MIPS R14000 Processor Chip Revision: 1.4
FPU: MIPS R14010 Floating Point Chip Revision: 1.4
Processor 4: 500 MHZ IP35
CPU: MIPS R14000 Processor Chip Revision: 1.4
FPU: MIPS R14010 Floating Point Chip Revision: 1.4
Processor 5: 500 MHZ IP35
CPU: MIPS R14000 Processor Chip Revision: 1.4
FPU: MIPS R14010 Floating Point Chip Revision: 1.4
Processor 6: 500 MHZ IP35
CPU: MIPS R14000 Processor Chip Revision: 1.4
FPU: MIPS R14010 Floating Point Chip Revision: 1.4
Processor 7: 500 MHZ IP35
CPU: MIPS R14000 Processor Chip Revision: 1.4
FPU: MIPS R14010 Floating Point Chip Revision: 1.4
Processor 8: 400 MHZ IP35
CPU: MIPS R12000 Processor Chip Revision: 3.5
FPU: MIPS R12010 Floating Point Chip Revision: 3.5
Processor 9: 400 MHZ IP35
CPU: MIPS R12000 Processor Chip Revision: 3.5
FPU: MIPS R12010 Floating Point Chip Revision: 3.5
Processor 10: 400 MHZ IP35
CPU: MIPS R12000 Processor Chip Revision: 3.5
FPU: MIPS R12010 Floating Point Chip Revision: 3.5
Processor 11: 400 MHZ IP35
CPU: MIPS R12000 Processor Chip Revision: 3.5
FPU: MIPS R12010 Floating Point Chip Revision: 3.5
Processor 12: 400 MHZ IP35
CPU: MIPS R12000 Processor Chip Revision: 3.5
FPU: MIPS R12010 Floating Point Chip Revision: 3.5
Processor 13: 400 MHZ IP35
CPU: MIPS R12000 Processor Chip Revision: 3.5
FPU: MIPS R12010 Floating Point Chip Revision: 3.5
Processor 14: 400 MHZ IP35
CPU: MIPS R12000 Processor Chip Revision: 3.5
FPU: MIPS R12010 Floating Point Chip Revision: 3.5
Processor 15: 400 MHZ IP35
CPU: MIPS R12000 Processor Chip Revision: 3.5
FPU: MIPS R12010 Floating Point Chip Revision: 3.5
Main memory size: 16384 Mbytes
Instruction cache size: 32 Kbytes
Data cache size: 32 Kbytes
Secondary unified instruction/data cache size: 8 Mbytes
Integral SCSI controller 0: Version Fibre Channel QL2200A
  Disk drive: unit 1 on SCSI controller 0
  Disk drive: unit 2 on SCSI controller 0
Integral SCSI controller 5: Version IEEE1394 SBP2
  IEEE1394 CDROM: node 1010031001c187 port 0 on SCSI controller 5
IOC3/IOC4 serial port: tty3
Gigabit Ethernet: eg0, module 001c07, pci_bus 2, pci_slot 1, firmware version 12.4.10
Integral Fast Ethernet: ef0, version 1, module 001c07, pci 4
IOC3/IOC4 external interrupts: 1
IEEE 1394 High performance serial bus controller 0: Type: OHCI, Version 0 0
USB controller: type OHCI

Pictures ^

SGI Origin 3400 without a rack
NUMAlink cabling at the back
Inside of a C-brick
Inside of a C-brick (close-up)
Inside of a C-brick (3/4 view)
Close-up of the Dual 400MHz CPU module
Close-up of the L1 controller inside a C-brick
Inside the L2 controller
Power section of the L2 controller
Logic section of the L2 controller

More pictures coming soon!

Documents ^

Acknowledgements ^

All diagrams/drawings on this page were taken from SGI manuals and other documentation.

 

[ View other machines in my hardware collection ]