Detailed Specifications of the “Ice Lake SP” Intel Xeon Processor Scalable Family CPUs

The "Ice Lake SP" CPUs are the 3rd generation of Intel’s Xeon Scalable Processor family. This generation brings new features, increased performance, and new server/workstation platforms. The Xeon ‘Ice Lake SP’ CPUs cannot be installed into previous-generation systems. Those considering a new deployment are encouraged to <a href="https://www.microway.com/contact/" rel="noopener noreferrer" target="_blank">review with one of our experts</a>.

<h2>Highlights of the features in Xeon Scalable Processor Family "Ice Lake SP" CPUs include:</h2>
<ul>
<li><strong>Up to 40 processor cores</strong> per socket (with options for 8-, 12-, 16-, 18-, 20-, 24-, 26-, 28-, 32-, 36-, and 38-cores)</li>
<li><strong>Up to 38% higher per-core performance</strong> through micro-architecture improvements (at same clock speed vs "Cascade Lake SP")</li>
<li>Significant memory performance &amp; capacity increases:
<ul>
<li><strong>Eight-channel memory controller</strong> on each CPU (up from six)</li>
<li>Support for DDR4 memory speeds <strong>up to 3200MHz</strong> (up from 2933MHz)</li>
<li>Large-memory capacity with <strong>Intel Optane Persistent Memory</strong></li>
<li>All CPU models support <strong>up to 6TB per socket</strong> (combined system memory and Optane persistent memory)</li>
</ul>
</li>
<li>Increased link speed between CPU sockets: <strong>11.2GT/s UPI links</strong> (up from 10.4GT/s)</li>
<li>I/O Performance Improvements – <strong>more than twice the throughput of "Cascade Lake SP"</strong>:
<ul>
<li><strong>PCI-Express generation 4.0</strong> doubles the throughput of each PCI-E lane (compared to gen 3.0)</li>
<li>Support for <strong>64 PCI-E lanes per CPU socket</strong> (up from 48 lanes)</li>
</ul>
</li>
<li>Continued high performance with the AVX-512 instruction capabilities of the previous generation:
<ul>
<li><strong>AVX-512 instructions</strong> (up to 16 double-precision FLOPS per cycle per AVX-512 FMA unit)</li>
<li><strong>Two AVX-512 FMA units per CPU core</strong> (available in all Ice Lake-SP CPU SKUs)</li>
</ul>
</li>
<li>Continued support for deep learning inference with <strong>AVX-512 VNNI instruction</strong>:
<ul>
<li>Intel Deep Learning Boost (VNNI) provides significant, more efficient deep learning inference acceleration</li>
<li>Combines three AVX-512 instructions (VPMADDUBSW, VPMADDWD, VPADDD) into a single VPDPBUSD operation</li>
</ul>
</li>
<li>Improvements to <strong>Intel Speed Select</strong> processor configurability:
<ul>
<li><strong>Performance Profiles</strong>: certain processors support three distinct core count/clock speed operating points</li>
<li><strong>Base Frequency</strong>: specific CPU cores are given higher base clock speeds; the remaining cores run at lower speeds</li>
<li><strong>Turbo Frequency</strong>: specific CPU cores are given higher turbo-boost speeds; the remaining cores run at lower speeds</li>
<li><strong>Core Power</strong>: each CPU core is prioritized; when surplus frequency is available, it is given to high-priority cores</li>
</ul>
</li>
<li>Integrated <strong>hardware-based security improvements</strong> and <strong>total memory encryption</strong></li>
</ul>
<em>With a product this complex, it’s very difficult to cover every aspect of the design. Here, we concentrate primarily on the performance of the processors for HPC &amp; AI applications.</em>

<a name="specialization"></a>
<h2>Continued Specialization of Xeon CPU SKUs</h2>
Those already familiar with Intel Xeon will see this processor family is divided into familiar tiers: <em>Silver, Gold,</em> and <em>Platinum</em>. The <em>Silver</em> and <em>Gold</em> models are in the price/performance range familiar to HPC/AI teams. <em>Platinum</em> models are in a higher price range. The low-end <em>Bronze</em> tier present in previous generations has been dropped.

Further, Intel continues to add new specialized CPU models that are optimized for particular workloads and environments. Many of these specialized SKUs are not relevant to readers here, but we summarize them briefly:
<ul>
<li><strong>N:</strong> network function virtualization (NFV) optimized</li>
<li><strong>P:</strong> virtualization-optimized (with a focus on clock frequency)</li>
<li><strong>S:</strong> max SGX enclave size</li>
<li><strong>T:</strong> designed for higher-temperature environments (NEBS)</li>
<li><strong>V:</strong> virtualization-optimized (with focus on high-density/low-power)</li>
</ul>
Targeting specific workloads and environments provides the best performance and efficiency for those use cases. However, using these CPUs for other workloads may reduce performance, as the CPU clock frequencies and Turbo Boost speeds are guaranteed only for those specific workloads. Running other workloads on these optimized CPUs will likely lead to CPU throttling, which would be undesirable. <em>Considering these limitations, the above workload-optimized models will not be included in our review.</em>

For those who prefer to see all details in one chart, Intel has provided a summary of all of the Xeon 3rd-generation CPUs (including both the ‘Ice Lake’ CPUs covered here, as well as the ‘Cooper Lake’ CPUs which provide 4-socket and 8-socket capability). The left-most column shows ‘Cooper Lake’ CPU SKUs. The middle and right columns summarize the ‘Ice Lake SP’ CPU SKUs covered in this article.
<a href="https://www.microway.com/wp-content/uploads/Intel-Xeon-3rd-generation-Scalable-Processor-lineup.png"><img src="https://www.microway.com/wp-content/uploads/Intel-Xeon-3rd-generation-Scalable-Processor-lineup-680×382.png" alt="Cheatsheet of all Intel Xeon 3rd-generation Scalable Processors" width="680" height="382" class="aligncenter size-large wp-image-13640" /></a>

<h3>Four Xeon CPU specializations relevant to HPC &amp; AI use cases</h3>
There are several specialized Xeon CPU options which are relevant to high performance computationally-intensive workloads. Each capability is summarized below and included in our analysis.
<ul>
<li><strong>Liquid-cooled – Xeon 8368Q CPU:</strong> optimized for liquid-cooled deployment, this CPU SKU offers high core counts along with higher CPU clock frequencies. The high clock frequencies are made possible only through the more effective cooling provided by liquid-cooled datacenters.</li>
<li><strong>Media, AI, and HPC – Xeon 8352M CPU:</strong> optimized for AVX-heavy vector instruction workloads as found in media processing, AI, and HPC; this CPU SKU offers improved performance per watt.</li>
<li><strong>Performance Profiles – Y:</strong> a set of CPU SKUs with support for Intel Speed Select Technology – Performance Profiles. These CPUs are indicated with a <em>Y suffix</em> in the model name (e.g., Xeon 8352Y) and provide flexibility for those with mixed workloads. Each CPU supports three different operating profiles with separate CPU core count, base clock and turbo boost frequencies, as well as operating wattages (TDP). In other words, each CPU could be thought of as three different CPUs. Administrators switch between profiles via system BIOS, or through Operating Systems with support for this capability (Intel SST-PP). <em>Note that several of the other specialized CPU SKUs also support multiple Performance Profiles (e.g., Xeon 8352M).</em></li>
<li><strong>Single Socket – U:</strong> single-socket optimized. The CPUs designed for a single socket are indicated with a <em>U suffix</em> in the model name (e.g., Xeon 6312U). These CPUs are more cost-effective. However, they do not include UPI links and thus can only be installed in systems with a single processor.</li>
</ul>

<h3>Summary of Xeon "Ice Lake-SP" CPU tiers</h3>
With the <em>Bronze</em> CPU tier no longer present, all models in this CPU family are well-suited to HPC and AI (though some will offer more performance than others). Before diving into the details, we provide a high-level summary of this Xeon processor family:
<ul>
<li><strong>Intel Xeon Silver – suitable for entry-level HPC</strong><br />The Xeon Silver 4300-series CPU models provide higher core counts and increased memory throughput compared to previous generations. However, their performance is limited compared to Gold and Platinum (particularly on Core Count, Clock Speed, Memory Performance, and UPI speed).</li>
<li><strong>Intel Xeon Gold – recommended for most HPC workloads</strong><br />Xeon Gold 5300- and 6300-series CPUs provide the best balance of performance and price. In particular, the 6300-series models should be preferred over the 5300-series models, because the 6300-series CPUs offer improved Clock Speeds and Memory Performance.</li>
<li><strong>Intel Xeon Platinum – only for specific HPC workloads</strong><br />Although 8300-series models provide the highest performance, their higher price makes them suitable only for particular workloads which require their specific capabilities (e.g., highest core count, large L3 cache).</li>
</ul>

<a name="performance"></a>
<h2>Xeon "Ice Lake SP" Computational Performance</h2>
With this new family of Xeon processors, Intel once again delivers unprecedented performance. Nearly every model provides <strong>over 1 TFLOPS</strong> (one teraflop of double-precision 64-bit performance per second), many models exceed 2 TFLOPS, and a few touch 3 TFLOPS. These performance levels are achieved through high core counts and AVX-512 instructions with FMA (as in the first and second Xeon Scalable generations). The plots in the tabs below compare the performance ranges for these new CPUs:

In the charts above, the <em>shaded/colored bars</em> indicate the expected performance range for each CPU model. The performance is a range rather than a specific value, because CPU clock frequencies scale up and down on a second-by-second basis. The precise achieved performance depends upon a variety of factors including temperature, power envelope, type of cooling technology, the load on each CPU core, and the type(s) of CPU instructions being issued to each core.

The first tab shows performance when using Intel’s AVX-512 instructions with FMA. Note that only a small set of codes will be capable of issuing exclusively AVX-512 FMA instructions (e.g., HPL LINPACK). Most applications issue a mix of instructions and will achieve lower than peak FLOPS. Further, applications which have not been re-compiled with an appropriate compiler will not include AVX-512 instructions and thus achieve lower performance. Computational applications which do not utilize AVX-512 instructions will most likely utilize AVX2 instructions (as shown in the second tab with <em>AVX2 Instruction performance</em>.

<a name="prices"></a>
<h1>Intel Xeon "Ice Lake SP" Price Ranges</h1>
The pricing of the 3rd-generation Xeon Processor Scalable Family spans a wide range, so budget must be kept in mind when selecting options. It would be frustrating to plan on 38-core processors when the budget cannot support a price of more than $10,000 per CPU. The plot below compares the prices of the Xeon "Ice Lake SP" processors:

<a href="https://www.microway.com/wp-content/uploads/Xeon_Ice-Lake-SP_CPU_Prices.png"><img src="https://www.microway.com/wp-content/uploads/Xeon_Ice-Lake-SP_CPU_Prices.png" alt="Comparison chart of Intel Xeon Ice Lake SP CPU prices" width="750" height="660" class="aligncenter size-full wp-image-13638" /></a>

As shown in the above plot, the CPUs in this article have been sorted by tier and by price. Most HPC users are expected to select CPU models from the Gold Xeon 6300-series. These models provide close to peak performance for a price around $3,000 per processor. Certain specialized applications will leverage the Platinum Xeon 8300-series

To ease comparisons, all of the plots in this article are ordered to match the above plot. Keep this pricing in mind as you review this article and plan your system architecture.

<a name="specifications"></a>
<h2>Intel "Ice Lake SP" Xeon Processor Scalable Family Specifications</h2>
The sets of tabs below compare the features and specifications of this new Xeon processor family. As you will see, the Silver (4300-series) and lower-end Gold (5300-series) CPU models offer fewer capabilities and lower performance. The higher-end Gold (6300-series) and Platinum (8300-series) offer more capabilities and higher performance. Additionally, certain CPU SKUs have special models integrating additional specializations:
<ul>
<li>Enabled for Intel Speed Select Performance Profiles <em>(indicated with a <strong>Y</strong> suffix on the part number)</em></li>
<li>Designed for single CPU socket systems <em>(indicated with a <strong>U</strong> suffix on the part number)</em></li>
<li>Requires a liquid-cooled thermal solution in the server <em>(indicated with a <strong>Q</strong> suffix on the part number)</em></li>
</ul>

In addition to the specifications called out above, technical readers should note that the "Ice Lake SP" CPU architecture inherits most of the architectural design of the previous "Cascade Lake-SP" architecture, including the mesh processor layout, redesigned L2/L3 caches, greater UPI connectivity between CPU sockets, and improvements to the processor frequency speeds/turbo.

<a name="recommended-cpus"></a>
<h1>Recommended Xeon CPU Models for HPC &amp; AI/Deep Learning</h1>
As stated at the top, most of this new CPU family offers excellent performance. However, it is common for HPC sites to set a minimum floor on CPU clock speeds (usually around 2.5GHz), with the intent that no workload suffers too low of a performance. While there are users who would demand even higher clock speeds, experience shows that most groups settle on a minimum clock speed in the 2.5GHz to 2.6GHz range. With that in mind, the comparisons below highlight only those CPU models which offer 2.5+GHz performance.