Performance Characteristics of Common Transports and Buses

Articles > Performance Characteristics of Common Transports and Buses

Memory

The following values are measured per CPU socket. They must be doubled or quadrupled to calculate the total memory bandwidth of a multiprocessor workstation or server. For dual-processor systems, multiply by two. For quad-processor systems, multiply by four.

Type # Channels Theoretical Bandwidth (unidirectional) Typical Bandwidth
(in Practice)
DDR4 2933MHz Six-Channel 140.8 GB/s TBD
DDR4 2666MHz Six-Channel 128 GB/s 90 GB/s
DDR4 2400MHz Quad-Channel 76.8 GB/s 64 GB/s
DDR4 2133MHz Quad-Channel 68.2 GB/s 55.5 GB/s
DDR3 1866MHz Quad-Channel 59.7 GB/s 42.8 GB/s
DDR3 1600MHz Quad-Channel 51.2 GB/s
DDR3 1333MHz Quad-Channel 42.7 GB/s
DDR3 1066MHz Quad-Channel 34.1 GB/s
DDR3 1333MHz Triple-Channel 32.0 GB/s
DDR3 1066MHz Triple-Channel 25.6 GB/s
DDR3 800MHz Triple-Channel 19.2 GB/s
DDR3 1866MHz Dual-Channel 29.9 GB/s
DDR3 1600MHz Dual-Channel 25.6 GB/s
DDR3 1333MHz Dual-Channel 21.3 GB/s
DDR3 1066MHz Dual-Channel 17.0 GB/s

Theoretical memory bandwidths are calculated with: 64 bits/transfer * DDR transfers/s * number of memory channels


PCI-Express

PCI-E Generation Lanes Theoretical Bandwidth (unidirectional) Typical Bandwidth
(in Practice)
Gen 1 x4 1,000 MB/s 880 MB/s
Gen 1 x8 2,000 MB/s 1,760 MB/s
Gen 1 x16 4,000 MB/s 3,520 MB/s
Gen 2 x4 2,000 MB/s 1,600 MB/s
Gen 2 x8 4,000 MB/s 3,200 MB/s
Gen 2 x16 8,000 MB/s 6,400 MB/s
Gen 3 x4 4,000 MB/s 2,800 MB/s
Gen 3 x8 8,000 MB/s 5,600 MB/s
Gen 3 x16 16,000 MB/s 12,100 MB/s

NVIDIA GPU NVLink

The NVLink connectivity on a GPU can be split different ways depending upon the system platform design. Most NVLink 1.0 configurations split the connectivity two ways or four ways (20GB/s on each of four links). NVLink 2.0 configurations can split connectivity two, three, or six ways (25GB/s on each of six links).

NVLink Generation Theoretical Bandwidth* (unidirectional) Typical Bandwidth
(in Practice)
NVLink 1.0 (4 bricks) 80 GB/s 73.4 GB/s
NVLink 2.0 (6 bricks) 150 GB/s 143.5 GB/s

SAS and SATA

Generation Theoretical Bandwidth (unidirectional)
4x wide port
Typical Bandwidth (in Practice)
SAS / SATA
1.5Gbps (SAS/SATA I) 600 MB/s 520 / 450 MB/s
3Gbps (SAS/SATA II) 1,200 MB/s 1,140 / 990 MB/s
6Gbps (SAS II/SATA III) 2,400 MB/s 2,280 / 1,975 MB/s
12Gbps SAS 4,800 MB/s 3,107 / — MB/s

Hard Drives and SSDs

Drive Type Random IOPS Sustained Sequential I/O
SAS/SATA 7,200RPM 70 – 175 100 – 230 MB/s
SAS 10,000RPM 275 – 300 125 – 200 MB/s
SAS 15,000RPM 350 – 450 125 – 200 MB/s
2.5″ Solid State (SSD) 15,000 – 100,000 110 – 500 MB/s
PCI-E Solid State (SSD) 70,000 – 625,000 1,100 – 3,200 MB/s

Intel QuickPath Interconnect (QPI) and UltraPath Interconnect (UPI)

The values listed below describe a single QPI/UPI link on an Intel Xeon processor. Note that the Xeon product lines are segmented. Within a given processor series (e.g., E5-2600), transfer speeds will vary from model to model. Additionally, the number of links between sockets will vary.

Interconnect Transfer Speed Theoretical Bandwidth (unidirectional)
QPI 4.8 GT/s 9.6 GB/s
QPI 5.6 GT/s 11.2 GB/s
QPI 6.4 GT/s 12.8 GB/s
QPI 7.2 GT/s 14.4 GB/s
QPI 8.0 GT/s 16.0 GB/s
QPI 9.6 GT/s 19.2 GB/s
UPI 10.4 GT/s 20.8 GB/s

AMD Infinity Fabric

The values listed below describe a single Infinity Fabric link on an AMD EPYC processor. Within a single-socket EPYC system, the only Infinity Fabric links are between chip dies within the CPU. In dual-socket EPYC systems, there are links within each CPU as well as four links between the CPU sockets. Infinity Fabric speeds are linked with memory frequency.

DDR4 Memory Speed Theoretical Bandwidth
DDR4-2666 (1333MHz) 42.667 GB/s
DDR4-2400 (1200MHz) 38.4 GB/s

Note that links between EPYC sockets include CRC overhead, which results in 8/9ths of the bandwidth values shown above (e.g., 37.9GB/s rather than 42.6GB/s).


AMD HyperTransport Link

The values listed below describe a single HyperTransport link on an AMD Opteron processor. In many of Microway’s systems, such as the Opteron WhisperStation, there are dual Hypertransport links between the CPUs.

Generation Transfers Theoretical Bandwidth (unidirectional)
3.1 (Socket G34) 6.4 GT/s (16-bit) 12.8 GB/s

Fibre Channel (FC)

FC Rate Theoretical Bandwidth (unidirectional)
2Gb 200 MB/s
4Gb 400 MB/s
8Gb 800 MB/s
16Gb 1600 MB/s
32Gb 3200 MB/s

See also: Performance Characteristics of Common Network Fabrics

Comments are closed.