Detailed Specifications of the Intel Xeon E5-2600v4 "Broadwell-EP" Processors

Home > Wikis > Detailed Specifications of the Intel Xeon E5-2600v4 “Broadwell-EP” Processors

Detailed Specifications of the Intel Xeon E5-2600v4 “Broadwell-EP” Processors

Edit

This article provides in-depth discussion and analysis of the 14nm Xeon E5-2600v4 series processors (formerly codenamed "Broadwell-EP"). "Broadwell" processors replace the previous 22nm "Haswell" microarchitecture and are available for sale as of March 31, 2016. For an introduction, read our blog post <a href="https://www.microway.com/hpc-tech-tips/intel-xeon-e5-2600-v4-broadwell-processor-review/" target="_blank">Intel Xeon E5-2600 v4 “Broadwell” Processor Review</a> Note: these have since been superceded by the <a href="https://www.microway.com/knowledge-center-articles/detailed-specifications-of-the-skylake-sp-intel-xeon-processor-scalable-family-cpus/" target="_blank">Intel Xeon Processor Scalable Family CPUs</a>.

<h2>Important changes available in E5-2600v4 "Broadwell-EP" include:</h2>
<ul>
	<li>Up to 22 processor cores per socket (with options for 4-, 6-, 8-, 10-, 12-, 14-, 16-, 18-, and 20-cores)</li>
	<li>Support for DDR4 memory speeds up to 2400MHz</li>
	<li>Floating Point Instruction performance improvements:
<ul>
	<li>Faster floating point multiplier completes operations in 3 cycles (down from 5 cycles)</li>
	<li>1024 Radix divider for reduced latency</li>
	<li>Split Scalar divides for increased parallelism/bandwidth</li>
	<li>Faster vector Gather</li>
	<li>As introduced with Haswell, Broadwell continues to support AVX2 and FMA3 instructions for significant speedups of floating-point multiplication and addition operations</li>
</ul></li>
	<li>Extract more parallelism in scheduling micro-operations:
<ul>
	<li>Reduced instruction latencies on ADC, CMOV and PCLMULQDQ</li>
	<li>Larger out-of-order scheduler, with 64 entries (up from 60 entries)</li>
	<li>Improved address prediction for branches and returns, with an expanded 10-way Branch Prediction Unit Target Array (up from 8-way)</li>
</ul></li>
	<li>Improved performance on large data sets:
<ul>
	<li>Larger L2 Translation Lookaside Buffer (TLB), with 1.5k entries (up from 1K entries)</li>
	<li>A new L2 TLB for 1GB pages (with 16 entries)</li>
	<li>Addition of a second TLB page miss handler for parallel page walks</li>
</ul></li>
</ul>

With a product this complex, it’s very difficult to cover every aspect of the design. Here, we concentrate primarily on the performance of the processors for HPC applications.

<a name="performance"></a>
<h2>Exceptional Computational Performance</h2>
The Xeon E5-2600v4 processors provide the highest performance available to date in a socketed CPU. Many of the higher-end models provide well over 500 GFLOPS (more than half a TFLOPS). Much of this performance is made possible through the use of AVX2 with FMA3 instructions. The plot below compares the peak performance of these CPUs with and without FMA instructions:

The colored bars indicate performance using only AVX instructions; the grey bars indicate theoretical peak performance when using AVX with FMA. Note that only a small set of codes will be capable of issuing almost exclusively FMA instructions (e.g., LINPACK). Most applications will issue a variety of instructions, which will result in lower than peak FLOPS. Expect the achieved performance for well-parallelized &amp; optimized applications to fall between the grey and colored bars.

<a name="specifications"></a>
<h2>Intel Xeon E5-2600v4 Series Specifications</h2>
The tabs below compare the features and specifications of the new model line. Intel has divided the CPUs into several groups:
<ul>
	<li>Standard: cost-effective CPUs with moderate performance</li>
	<li>Advanced: CPUs offering the highest performance for most applications</li>
	<li>High Core Count: ideal for highly multi-threaded applications; CPUs providing the highest number of processor cores (sometimes sacrificing clock frequency in favor of core count)</li>
	<li>Frequency Optimized: ideal for non-parallel/single-threaded applications; CPUs with the highest clock speeds (sacrificing number of cores in order to provide the highest frequencies)</li>
</ul>

Although these processors introduce significant performance increases, technical readers will see that many of the changes are incremental: increased core counts, improved DDR memory speed, etc. However, processor clock speeds/frequencies have not seen significant improvements.

In fact, in some cases the CPU frequency has been lowered from the previous models. Processor frequency and Turbo Boost behavior have changed fairly significantly in the last two CPU releases ("Haswell" and "Broadwell"). Those metrics are discussed in further detail in the next section.

[tabby title="CPU Cores"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_Number_of_Cores.png" alt="Chart of Xeon E5-2600v4 Number of CPU Cores" width="634" height="510" class="aligncenter size-full wp-image-7144" />
[tabby title="Memory Speed"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_Memory_Performance.png" alt="Chart of Xeon E5-2600v4 Memory Performance" width="634" height="600" class="aligncenter size-full wp-image-7145" />
[tabby title="L3 Cache"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_L3_Cache_Size.png" alt="Chart of Xeon E5-2600v4 CPU L3 Cache Size" width="634" height="600" class="aligncenter size-full wp-image-7146" />
[tabby title="QPI"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_QPI_Performance.png" alt="Chart of Xeon E5-2600v4 QPI Performance" width="634" height="600" class="aligncenter size-full wp-image-7147" />
[tabby title="TDP"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_TDP_Wattage.png" alt="Chart of Xeon E5-2600v4 CPU Wattage (TDP)" width="634" height="600" class="aligncenter size-full wp-image-7148" />
[tabby title="Specifications Table"]
<table>
<thead>
<tr>
<th>Model</th> <th>AVX Frequency</th> <th>AVX Turbo Boost</th> <th>Core Count</th> <th>Memory Speed</th> <th>L3 Cache</th> <th>QPI Speed</th> <th>TDP (Watts)</th>
</tr>
</thead>
<tbody>
<tr><td>E5-2699v4</td><td class="vcenter" rowspan="2">1.80 GHz</td><td class="vcenter" rowspan="2">3.60 GHz</td><td>22</td><td class="vcenter" rowspan="10">2400 Mhz</td><td>55 MB</td><td class="vcenter" rowspan="10">9.6 GT/s</td><td>145W</td></tr>
<tr><td>E5-2698v4</td><td>20</td><td>50 MB</td><td>135W</td></tr>
<tr><td>E5-2697Av4</td><td>2.20 GHz</td><td>3.10 GHz</td><td>16</td><td>40 MB</td><td class="vcenter" rowspan="2">145W</td></tr>
<tr><td>E5-2697v4</td><td>2.00 GHz</td><td>3.60 GHz</td><td class="vcenter" rowspan="2">18</td><td class="vcenter" rowspan="2">45 MB</td></tr>
<tr><td>E5-2695v4</td><td class="vcenter" rowspan="2">1.70 GHz</td><td>3.10 GHz</td><td class="vcenter" rowspan="2">120W</td></tr>
<tr><td>E5-2683v4</td><td>3.00 GHz</td><td>16</td><td>40 MB</td></tr>
<tr><td>E5-2690v4</td><td>2.10 GHz</td><td>3.50 GHz</td><td class="vcenter" rowspan="3">14</td><td class="vcenter" rowspan="3">35 MB</td><td>135W</td></tr>
<tr><td>E5-2680v4</td><td>1.90 GHz</td><td>3.30 GHz</td><td>120W</td></tr>
<tr><td>E5-2660v4</td><td>1.70 GHz</td><td>3.20 GHz</td><td class="vcenter" rowspan="2">105W</td></tr>
<tr><td>E5-2650v4</td><td>1.80 GHz</td><td>2.80 GHz</td><td>12</td><td>30 MB</td></tr>
<tr><td>E5-2640v4</td><td>2.00 GHz</td><td>3.40 GHz</td><td class="vcenter" rowspan="2">10</td><td class="vcenter" rowspan="3">2133 Mhz</td><td class="vcenter" rowspan="2">25 MB</td><td class="vcenter" rowspan="3">8 GT/s</td><td>90W</td></tr>
<tr><td>E5-2630v4</td><td class="vcenter" rowspan="2">1.80 GHz</td><td>3.10 GHz</td><td class="vcenter" rowspan="2">85W</td></tr>
<tr><td>E5-2620v4</td><td>3.00 GHz</td><td>8</td><td>20 MB</td></tr>
<tr><td>E5-2687Wv4</td><td class="vcenter" rowspan="2">2.60 GHz</td><td>3.40 GHz</td><td>12</td><td class="vcenter" rowspan="4">2400 Mhz</td><td>30 MB</td><td class="vcenter" rowspan="4">9.6 GT/s</td><td>160W</td></tr>
<tr><td>E5-2667v4</td><td>3.50 GHz</td><td>8</td><td>25 MB</td><td class="vcenter" rowspan="3">135W</td></tr>
<tr><td>E5-2643v4</td><td>2.80 GHz</td><td class="vcenter" rowspan="2">3.60 GHz</td><td>6</td><td>20 MB</td></tr>
<tr><td>E5-2637v4</td><td>3.20 GHz</td><td class="vcenter" rowspan="2">4</td><td>15 MB</td></tr>
<tr><td>E5-2623v4</td><td>2.20 GHz</td><td>3.20 GHz</td><td>2133 Mhz</td><td>10 MB</td><td>8 GT/s</td><td>85W</td></tr>
</tbody>
</table>
HPC groups do not typically choose Intel's "Basic" and "Low Power" models - those skus are not shown.

[tabbyending]

<a name="clockspeeds"></a>
<h2>Clock Speeds &amp; Turbo Boost in Xeon E5-2600v4 series "Broadwell" processors</h2>
With each new processor line, Intel introduces new architecture optimizations. The design of the "Broadwell" architecture acknowledges that highly-parallel/vectorized applications place the highest load on the processor cores (requiring more power and thus generating more heat). While a CPU core is executing intensive vector tasks (AVX instructions), the clock speed may be reduced to keep the processor within its power limits (TDP).

In effect, this may result in the processor running at a lower frequency than the "base" clock speed advertised for each model. For that reason, each "Broadwell" processor is assigned two "base" frequencies:
<ol>
	<li>AVX mode: due to the higher power requirements of AVX instructions, clock speeds may be somewhat lower while executing AVX instructions *</li>
	<li>Non-AVX mode: while not executing AVX instructions, the processor will operate at what would traditionally be considered the "stock" frequency</li>
</ol>
* a CPU core will return to Non-AVX mode 1 millisecond after AVX instructions complete

It is worth noting that these modes are isolated to each core. Within a given CPU, some cores may be operating in AVX mode while others are operating in Non-AVX mode. In the previous generation, AVX instructions running on a single core would cause all cores to run in AVX mode.

<h3>AVX and Non-AVX Turbo Boost</h3>
Just as in previous architectures, "Broadwell" CPUs include the Turbo Boost feature which allows each processor core to operate well above the "base" clock speed during most operations. The precise clock speed increase depends upon the number &amp; intensity of tasks running on each CPU. However, Turbo Boost speed increases also depend upon the types of instructions (AVX vs. Non-AVX).

The two plots below show that processor clock speeds can be categorized as:
<ol>
	<li>All cores on the CPU actively running Non-AVX instructions</li>
	<li>All cores on the CPU actively running AVX instructions</li>
	<li>A single active core running Non-AVX instructions (all other cores on the CPU must be idle)</li>
	<li>A single active core running AVX instructions (all other cores on the CPU must be idle)</li>
</ol>

[tabby title="Clock Speeds for All-Core Operation"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_Frequency_Speeds_AVX_and_Non-AVX_Instructions_All-Core.png" alt="Diagram of Xeon E5-2600v4 CPU Frequency Speeds (comparing AVX and Non-AVX Instructions) running on all CPU cores" width="634" height="600" class="aligncenter size-full wp-image-7149" />
[tabby title="Clock Speeds for Single-Core Operation"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_Frequency_Speeds_AVX_and_Non-AVX_Instructions_Single-Core.png" alt="Diagram of Xeon E5-2600v4 CPU Frequency Speeds (comparing AVX and Non-AVX Instructions) running on a single core" width="634" height="600" class="aligncenter size-full wp-image-7150" />
[tabbyending]

Note that despite the clear rules stated above, each value is still a range of clock speeds. Because workloads are so diverse, Intel is unable to guarantee one specific clock speed for AVX or Non-AVX instructions. Users are guaranteed that cores will run within a specific range, but each application will have to be benchmarked to determine which frequencies a CPU will operate at.

When examining the differences between AVX and Non-AVX instructions, notice that Non-AVX instructions typically result in no more than a 100MHz to 200MHz increase in the highest clock speed. However, AVX instructions may cause clock speeds to drop by 300MHz to 400MHz if they are particularly intensive.

Recall that AVX2 introduces support for both integer and floating-point instructions, which means any compute-intensive application will be using such instructions (if it has been properly designed and compiled). HPC users should expect their processors to be running in AVX mode most of the time.

<h3>Top Clock Speeds for Specific Core Counts</h3>
When workloads leave some CPU cores idle, the Xeon E5-2600v4 processors are able to use that headroom to increase the clock speed of the cores which are performing work. Just as with other Turbo Boost scenarios, the precise speed increase will depend upon the CPU model. It will also depend upon how many CPU cores are active.

We advise users to consider how many CPU cores their application is able to saturate. The tabs below detail the peak Turbo Boost frequencies for each CPU model, sorted by the number of active cores:

[tabby title="One to Two cores"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_Frequency_01-02_core_jobs.png" alt="Chart of Xeon E5-2600v4 CPU Frequency for single-core and dual-core applications" width="634" height="510" class="aligncenter size-full wp-image-7151" />
[tabby title="3"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_Frequency_03_core_jobs.png" alt="Chart of Xeon E5-2600v4 CPU Frequency for triple-core applications" width="634" height="510" class="aligncenter size-full wp-image-7152" />
[tabby title="4"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_Frequency_04_core_jobs.png" alt="Chart of Xeon E5-2600v4 CPU Frequency for quad-core applications" width="634" height="510" class="aligncenter size-full wp-image-7153" />
[tabby title="5"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_Frequency_05_core_jobs.png" alt="Chart of Xeon E5-2600v4 CPU Frequency for 5-core applications" width="634" height="510" class="aligncenter size-full wp-image-7154" />
[tabby title="6"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_Frequency_06_core_jobs.png" alt="Chart of Xeon E5-2600v4 CPU Frequency for 6-core applications" width="634" height="510" class="aligncenter size-full wp-image-7155" />
[tabby title="7"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_Frequency_07_core_jobs.png" alt="Chart of Xeon E5-2600v4 CPU Frequency for 7-core applications" width="634" height="510" class="aligncenter size-full wp-image-7156" />
[tabby title="8"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_Frequency_08_core_jobs.png" alt="Chart of Xeon E5-2600v4 CPU Frequency for 8-core applications" width="634" height="510" class="aligncenter size-full wp-image-7157" />
[tabby title="9"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_Frequency_09_core_jobs.png" alt="Chart of Xeon E5-2600v4 CPU Frequency for 9-core applications" width="634" height="510" class="aligncenter size-full wp-image-7158" />
[tabby title="10"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_Frequency_10_core_jobs.png" alt="Chart of Xeon E5-2600v4 CPU Frequency for 10-core applications" width="634" height="510" class="aligncenter size-full wp-image-7159" />
[tabby title="11+ cores"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_Frequency_11-22_core_jobs.png" alt="Chart of Xeon E5-2600v4 CPU Frequency for applications using 11 or more cores" width="634" height="510" class="aligncenter size-full wp-image-7160" />
[tabbyending]

All of the above plots show CPU frequencies for applications utilizing AVX instructions. The colored bars indicate the worst-case scenario - CPUs will run at least this fast. The grey bars indicate the expected clock speeds for most workloads.

<h2>Cost-Effectiveness and Power Efficiency of Xeon E5-2600v4 CPUs</h2>
The "Broadwell-EP" processors have nearly the same price structure and power requirements as earlier Xeon E5-2600 products, so their cost-effectiveness and power-efficiency should be quite attractive to HPC users. Savvy readers may find the following facts useful:
<ul>
	<li>HPC applications run best on the Advanced CPU models; they typically do not scale well on the High-Core-Count models.</li>
	<li>The High-Core-Count models are more common in Enterprise and Finance - these carry higher prices than other E5-2600 models.</li>
	<li>The following graphs depict the cost-effectiveness and power-efficiency of only the CPU itself. In many cases, HPC users will find that once they've taken the full platform and cluster design into account, the cost-effectiveness of an Advanced CPU may be higher than these plots demonstrate.</li>
</ul>

[tabby title="Performance vs. Price"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_Cost-Effectiveness.png" alt="Chart of Xeon E5-2600v4 Cost-Effectiveness (performance vs. price)" width="634" height="510" class="aligncenter size-full wp-image-7168" />
[tabby title="Performance vs. Power"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_Power-Efficiency.png" alt="Chart of Xeon E5-2600v4 Power-Efficiency" width="634" height="510" class="aligncenter size-full wp-image-7161" />
[tabby title="Processor Prices"]<img src="https://www.microway.com/wp-content/uploads/Xeon_E5-2600v4_CPU_Prices.png" alt="Chart of Xeon E5-2600v4 CPU Prices" width="634" height="510" class="aligncenter size-full wp-image-7169" />
[tabbyending]

<h2>Summary of features in Xeon E5-2600v4 "Broadwell-EP" processors</h2>
In addition to the capabilities mentioned at the top of this article, these processors include many of the successful features from earlier Xeon designs. The list below provides a summary of relevant technology features:
<ul>
	<li>Up to 22 processor cores per socket (with options for 4-, 6-, 8-, 10-, 12-, 14-, 16-, 18-, and 20-cores)</li>
	<li>Support for Quad-channel ECC DDR4 memory speeds up to 2400MHz</li>
	<li>Direct PCI-Express (generation 3.0) connections between each CPU and peripheral devices such as network adapters, GPUs and coprocessors (40 PCI-E lanes per socket)</li>
	<li>Floating Point Instruction performance improvements:
<ul>
	<li>Faster floating point multiplier completes operations in 3 cycles (down from 5 cycles)</li>
	<li>1024 Radix divider for reduced latency</li>
	<li>Split Scalar divides for increased parallelism/bandwidth</li>
	<li>Faster vector Gather</li>
</ul></li>
	<li>As introduced with "Haswell", "Broadwell" continues to supportAdvanced Vector Extensions (AVX 2.0):
	<ul>
		<li>effectively double the throughput of integer and floating-point operations with math units expanded from 128-bits to 256-bits</li>
		<li>introduce Fused Multiply Add (FMA3) instructions which allow a multiply and an accumulate instruction to be completed in a single cycle (effectively doubling the FLOPS/clock from 8 to 16 for each core of a CPU)</li>
		<li>add support for additional instructions, including Gather and vector shift</li>
		<li>F16C 16-bit Floating-Point conversion instructions accelerate data conversion between 16-bit and 32-bit floating point formats</li>
	</ul></li>
	<li>Turbo Boost technology improves performance under peak loads by increasing processor clock speeds. With version 2.0, (introduced in "Sandy Bridge") clock speeds are boosted more frequently, to higher speeds and for longer periods of time. With "Haswell" and "Broadwell", top clock speeds depend upon the type of instructions (AVX vs. Non-AVX).</li>
	<li>Extract more parallelism in scheduling micro-operations:
<ul>
	<li>Reduced instruction latencies on ADC, CMOV and PCLMULQDQ</li>
	<li>Larger out-of-order scheduler, with 64 entries (up from 60 entries)</li>
	<li>Introduction of the ADCX and ADOX instructions to speed up cryptography</li>
	<li>Improved address prediction for branches and returns, with an expanded 10-way Branch Prediction Unit Target Array (up from 8-way)</li>
</ul></li>
	<li>Improved performance on large data sets:
<ul>
	<li>Larger L2 Translation Lookaside Buffer (TLB), with 1.5k entries (up from 1K entries)</li>
	<li>A new L2 TLB for 1GB pages (with 16 entries)</li>
	<li>Addition of a second TLB page miss handler for parallel page walks</li>
</ul></li>
	<li>Dual Quick Path Interconnect (QPI) links between processor sockets improve communication speeds for multi-threaded applications</li>
	<li>Intel Data Direct I/O Technology increases performance and reduces latency by allowing Intel ethernet controllers and adapters to talk directly with the processor cache</li>
	<li>Transactional Synchronization Extensions (TSX) improve the parallelism of multi-threaded applications with synchronization locks</li>
	<li>Introduction of the RDSEED instruction for high-quality, non-deterministic, random seed values</li>
	<li>Advanced Encryption Standard New Instructions (AES-NI) accelerate encryption and decryption for fast, affordable data protection and security</li>
	<li>32-bit & 64-bit Intel Virtualization Technology (VT/VT-x) for Directed I/O (VT-d) and Connectivity (VT-c) deliver faster performance for core virtualization processes and provide built-in hardware support for I/O virtualization.</li>
	<li>Intel APIC Virtualization (APICv) provides increased virtualization performance</li>
	<li>Hyper-Threading technology allows two threads to "share" a processor core for improved resource usage. Although useful for some workloads, it is not recommended for HPC applications.</li>
	<li>Improved energy efficiency with Per Core P-States and independent uncore frequency control</li>
	<li>Hardware Controlled Power Management for more rapid and efficient decisions on optimal P- and C-State operating point</li>
	<li>DDR4 CRC provides better memory reliability and data integrity by detecting memory bus faults during write</li>
	<li>ECRC for PCI-Express provides optional data integrity protection for systems using PCI-Express switches or bridges</li>
</ul>

Cancel