In-Depth Comparison of NVIDIA Tesla “Pascal” GPU Accelerators

<h2>Important changes available in the "Pascal" GPU architecture include:</h2>
<ul>
<li><strong>Exceptional performance</strong> with up to 5.3 TFLOPS double- and 10.6 TFLOPS single-precision floating-point performance.</li>
<li><strong>NVLink</strong> enables a 5X increase in bandwidth between Tesla Pascal GPUs and from GPUs to supported system CPUs (compared with PCI-E).</li>
<li><strong>High-bandwidth HBM2 memory</strong> provides a 3X improvement in memory performance compared to Kepler and Maxwell GPUs.</li>
<li><strong>Pascal Unified Memory</strong> allows GPU applications to directly access the memory of all GPUs as well as all of system memory (up to 512TB).</li>
<li><strong>Up to 4MB L2 caches</strong> are available on Pascal GPUs (compared to 1.5MB on Kepler and 3MB on Maxwell).</li>
<li><strong>Native ECC Memory</strong> detects and corrects memory errors without any capacity or performance overhead.</li>
<li><strong>Energy-efficiency</strong> – Pascal GPUs deliver nearly twice the FLOPS per Watt as Kepler GPUs.</li>
<li><strong>Efficient SM units</strong> – Pascal’s architecture doubles the number of registers per thread</li>
<li><strong>Improved atomics</strong> in Pascal allow for an atomic add instruction in global memory (previous GPUs supported only shared memory atomics). Atomics can also be performed within the memory of other GPUs in the system.</li>
<li><strong>Half-precision FP</strong> support improves performance for low-precision operations (frequently used in neural network training)</li>
<li><strong>INT8</strong> support improves performance for low-precision integer operations (frequently used in neural network inference)</li>
<li><strong>Compute Preemption</strong> allows higher-priority tasks to interrupt currently-running tasks.</li>
</ul>

<h2>Tesla "Pascal" GPU Specifications</h2>
The table below summarizes the features of the available Tesla Pascal GPU Accelerators. To learn more about any of these products, or to find out how best to leverage their capabilities, please speak with an <a href="https://www.microway.com/contact/" title="Talk to an Expert – Contact Microway">HPC expert</a>.

<h2>Comparison between "Kepler", "Maxwell", and "Pascal" GPU Architectures</h2>
<table>
<thead>
<tr>
<th>Feature</th>
<th>Kepler GK210</th>
<th>Maxwell GM200</th>
<th>Maxwell GM204</th>
<th>Pascal GP100</th>
<th>Pascal GP102</th>
</tr>
</thead>
<tbody>
<tr><td class="rowhead">Compute Capability</td><td>3.7</td><td colspan=2>5.2</td><td>6.0</td><td>6.1</td></tr>
<tr><td class="rowhead">Threads per Warp</td><td colspan=5>32</td></tr>
<tr><td class="rowhead">Max Warps per SM</td><td colspan=5>64</td></tr>
<tr><td class="rowhead">Max Threads per SM</td><td colspan=5>2048</td></tr>
<tr><td class="rowhead">Max Thread Blocks per SM</td><td>16</td><td colspan=4>32</td></tr>
<tr><td class="rowhead">Max Concurrent Kernels</td><td colspan=3>32</td><td>128</td><td>32</td></tr>
<tr><td class="rowhead">32-bit Registers per SM</td><td>128 K</td><td colspan=4>64 K</td></tr>
<tr><td class="rowhead">Max Registers per Thread Block</td><td colspan=5>64 K</td></tr>
<tr><td class="rowhead">Max Registers per Thread</td><td colspan=5>255</td></tr>
<tr><td class="rowhead">Max Threads per Thread Block</td><td colspan=5>1024</td></tr>
<tr><td class="rowhead">L1 Cache Configuration</td><td colspan=3>split with shared memory</td><td colspan=2>24KB dedicated L1 cache</td></tr>
<tr><td class="rowhead">Shared Memory Configurations</td><td>16KB + 112KB L1 Cache<br /><br />32KB + 96KB L1 Cache<br /><br />48KB + 80KB L1 Cache<br /><br /><em>(128KB total)</em></td><td colspan=2>96KB dedicated</td><td>64KB dedicated</td><td>96KB dedicated</td></tr>
<tr><td class="rowhead">Max Shared Memory per Thread Block</td><td colspan=5>48KB</td></tr>
<tr><td class="rowhead">Max X Grid Dimension</td><td colspan=5>2^32-1</td></tr>
<tr><td class="rowhead">Hyper-Q</td><td colspan=5>Yes</td></tr>
<tr><td class="rowhead">Dynamic Parallelism</td><td colspan=5>Yes</td></tr>
</tbody>
</table>
<em>For a complete listing of Compute Capabilities, reference the <a href="https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#features-and-technical-specifications__technical-specifications-per-compute-capability" target="_blank" rel="noopener noreferrer">NVIDIA CUDA Documentation</a></em>
<hr />

<h2>Additional Tesla "Pascal" GPU products</h2>
NVIDIA has also released Tesla P4 GPUs. These GPUs are primarily for embedded and hyperscale deployments, and are not expected to be used in the HPC space.

<h2>Hardware-accelerated video encoding and decoding</h2>
All NVIDIA "Pascal" GPUs include one or more hardware units for video encoding and decoding (NVENC / NVDEC). For complete hardware details, reference NVIDIA’s <a href="https://developer.nvidia.com/video-encode-decode-gpu-support-matrix" target="_blank" rel="noopener noreferrer">encoder/decoder support matrix</a>. To learn more about GPU-accelerated video encode/decode, see NVIDIA’s <a href="https://developer.nvidia.com/nvidia-video-codec-sdk" target="_blank" rel="noopener noreferrer">Video Codec SDK</a>.