Performance Characteristics of Common Transports and Buses

Articles > Performance Characteristics of Common Transports and Buses

Memory

The following values are measured per CPU socket. They must be doubled or quadrupled to calculate the total memory bandwidth of a multiprocessor workstation or server. For dual-processor systems, multiply by two. For quad-processor systems, multiply by four.

Type # Channels Theoretical Bandwidth (unidirectional) Typical Bandwidth
(in Practice)
DDR4 3200MHz Eight-Channel 204.8 GB/s 171.5 GB/s
DDR4 2933MHz Six-Channel 140.8 GB/s 98 GB/s
DDR4 2666MHz Six-Channel 128 GB/s 90 GB/s
DDR4 2400MHz Quad-Channel 76.8 GB/s 64 GB/s
DDR4 2133MHz Quad-Channel 68.2 GB/s 55.5 GB/s
DDR3 1866MHz Quad-Channel 59.7 GB/s 42.8 GB/s
DDR3 1600MHz Quad-Channel 51.2 GB/s
DDR3 1333MHz Quad-Channel 42.7 GB/s
DDR3 1066MHz Quad-Channel 34.1 GB/s
DDR3 1333MHz Triple-Channel 32.0 GB/s
DDR3 1066MHz Triple-Channel 25.6 GB/s
DDR3 800MHz Triple-Channel 19.2 GB/s
DDR3 1866MHz Dual-Channel 29.9 GB/s
DDR3 1600MHz Dual-Channel 25.6 GB/s
DDR3 1333MHz Dual-Channel 21.3 GB/s
DDR3 1066MHz Dual-Channel 17.0 GB/s

Theoretical memory bandwidths are calculated with: 64 bits/transfer * DDR transfers/s * number of memory channels


PCI-Express

PCI-E Generation Lanes Theoretical Bandwidth (unidirectional) Typical Bandwidth
(in Practice)
Gen 1 x4 1,000 MB/s 880 MB/s
Gen 1 x8 2,000 MB/s 1,760 MB/s
Gen 1 x16 4,000 MB/s 3,520 MB/s
Gen 2 x4 2,000 MB/s 1,600 MB/s
Gen 2 x8 4,000 MB/s 3,200 MB/s
Gen 2 x16 8,000 MB/s 6,400 MB/s
Gen 3 x4 4,000 MB/s 2,800 MB/s
Gen 3 x8 8,000 MB/s 5,600 MB/s
Gen 3 x16 16,000 MB/s 12,100 MB/s
Gen 4 x16 32,000 MB/s 26,200 MB/s

NVIDIA GPU NVLink

The NVLink connectivity on a GPU can be split different ways depending upon the system platform design. Most NVLink 1.0 configurations split the connectivity two ways or four ways (20GB/s on each of four links). NVLink 2.0 configurations can split connectivity two, three, or six ways (25GB/s on each of six links). NVLink 3.0 supports up to twelve links (25GB/s per link).

NVLink Generation Theoretical Bandwidth* (unidirectional) Typical Bandwidth
(in Practice)
NVLink 1.0 (4 bricks) 80 GB/s 73.4 GB/s
NVLink 2.0 (6 bricks) 150 GB/s 143.5 GB/s
NVLink 3.0 (12 bricks) 300 GB/s 276 GB/s

SAS and SATA

Generation Theoretical Bandwidth (unidirectional)
4x wide port
Typical Bandwidth (in Practice)
SAS / SATA
1.5Gbps (SAS/SATA I) 600 MB/s 520 / 450 MB/s
3Gbps (SAS/SATA II) 1,200 MB/s 1,140 / 990 MB/s
6Gbps (SAS II/SATA III) 2,400 MB/s 2,280 / 1,975 MB/s
12Gbps SAS 4,800 MB/s 3,107 / — MB/s

Hard Drives and SSDs

Drive Type Random IOPS Sustained Sequential I/O
SAS/SATA 7,200RPM 70 – 175 100 – 230 MB/s
SAS 10,000RPM 275 – 300 125 – 200 MB/s
SAS 15,000RPM 350 – 450 125 – 200 MB/s
SAS/SATA Solid State Drives (SSD) 15,000 – 100,000 110 – 500 MB/s
PCI-E Solid States (NVMe SSD) 70,000 – 625,000 1,100 – 3,200 MB/s

Intel QuickPath Interconnect (QPI) and UltraPath Interconnect (UPI)

The values listed below describe a single QPI/UPI link on an Intel Xeon processor. There are typically two to three UPI links between CPU sockets, but this will vary by platform. Note that the Xeon product lines are segmented. Within a given processor series (e.g., Xeon Scalable “Cascade Lake-SP”), transfer speeds will vary from model to model.

Interconnect Transfer Speed Theoretical Bandwidth (unidirectional)
QPI 4.8 GT/s 9.6 GB/s
QPI 5.6 GT/s 11.2 GB/s
QPI 6.4 GT/s 12.8 GB/s
QPI 7.2 GT/s 14.4 GB/s
QPI 8.0 GT/s 16.0 GB/s
QPI 9.6 GT/s 19.2 GB/s
UPI 10.4 GT/s 20.8 GB/s

AMD Infinity Fabric

The values listed below describe a single Infinity Fabric link on an AMD EPYC processor. In dual-socket EPYC systems, there are typically three or four links between the CPU sockets. Within each EPYC CPU, each of the eight dies on the chip is connected to the I/O die via one Infinity Fabric link.

DDR4 Memory Speed Theoretical Bandwidth (unidirectional)
Zen2/Zen3 18GT/s 72 GB/s
Zen1 10.6GT/s 42.667 GB/s

Note that links between EPYC sockets include CRC overhead, which results in 8/9ths of the bandwidth values shown above (e.g., 37.9GB/s rather than 42.6GB/s).


AMD HyperTransport Link

The values listed below describe a single HyperTransport link on an AMD Opteron processor. In many of systems, there were dual HyperTransport links between the CPUs.

Generation Transfers Theoretical Bandwidth (unidirectional)
3.1 (Socket G34) 6.4 GT/s (16-bit) 12.8 GB/s

Fibre Channel (FC)

FC Rate Theoretical Bandwidth (unidirectional)
2Gb 200 MB/s
4Gb 400 MB/s
8Gb 800 MB/s
16Gb 1600 MB/s
32Gb 3200 MB/s

See also: Performance Characteristics of Common Network Fabrics

Comments are closed.