In-Depth Comparison of NVIDIA “Ampere” GPU Accelerators

Edit

<em>This article provides details on the NVIDIA A-series GPUs (codenamed "Ampere"). "Ampere" GPUs improve upon the previous-generation <a href="https://www.microway.com/knowledge-center-articles/in-depth-comparison-of-nvidia-tesla-volta-gpu-accelerators/" rel="noopener noreferrer" target="_blank">"Volta"</a> and <a href="https://www.microway.com/knowledge-center-articles/in-depth-comparison-of-nvidia-quadro-turing-gpu-accelerators/" rel="noopener" target="_blank">"Turing"</a> architectures. Ampere A100 GPUs began shipping in May 2020 (with other variants shipping by end of 2020).</em>

Note that not all "Ampere" generation GPUs provide the same capabilities and feature sets. Broadly-speaking, there is one version dedicated solely to computation and a second version dedicated to a mixture of graphics/visualization and compute. The specifications of both versions are shown below - speak with one of our <a href="https://www.microway.com/contact/" rel="noopener" target="_blank">GPU experts</a> for a personalized summary of the options best suited to your needs.

<h2>Computational "Ampere" GPU architecture - important features and changes:</h2>
<ul>
	<li><strong>Exceptional HPC performance:</strong>
	<ul>
		<li>9.7 TFLOPS FP64 double-precision floating-point performance</li>
		<li>Up to 19.5 TFLOPS FP64 double-precision via Tensor Core FP64 instruction support</li>
		<li>19.5 TFLOPS FP32 single-precision floating-point performance</li>
	</ul></li>
	<li><strong>Exceptional AI deep learning training and inference performance:</strong>
	<ul>
		<li><strong>TensorFloat 32 (TF32)</strong> instructions improve performance without loss of accuracy</li>
		<li><strong>Sparse matrix optimizations</strong> potentially double training and inference performance</li>
		<li>Speedups of 3x~20x for network training, with sparse TF32 TensorCores (vs Tesla V100)</li>
		<li>Speedups of 7x~20x for inference, with sparse INT8 TensorCores (vs Tesla V100)</li>
		<li>Tensor Cores support many instruction types: FP64, TF32, BF16, FP16, I8, I4, B1</li>
	</ul></li>
	<li><strong>High-speed HBM2 Memory</strong> delivers 40GB or 80GB capacity at 1.6TB/s or 2TB/s throughput</li>
	<li><strong>Multi-Instance GPU</strong> allows each A100 GPU to run seven separate/isolated applications</li>
	<li><strong>3rd-generation NVLink</strong> doubles transfer speeds between GPUs</li>
	<li><strong>4th-generation PCI-Express</strong> doubles transfer speeds between the system and each GPU</li>
	<li><strong>Native ECC Memory</strong> detects and corrects memory errors without any capacity or performance overhead</li>
	<li><strong>Larger and Faster L1 Cache and Shared Memory</strong> for improved performance</li>
	<li><strong>Improved L2 Cache</strong> is twice as fast and nearly seven times as large as L2 on Tesla V100</li>
	<li><strong>Compute Data Compression</strong> accelerates compressible data patterns, resulting in up to 4x faster DRAM bandwidth, up to 4x faster L2 read bandwidth, and up to 2x increase in L2 capacity.</li>
</ul>

<h2>Visualization "Ampere" GPU architecture - important features and changes:</h2>
<ul>
	<li><strong>Double FP32 processing throughput</strong> with upgraded Streaming Multiprocessors (SM) that support FP32 computation on both datapaths<br />(previous generations provided one dedicated FP32 path and one dedicated Integer path)</li>
	<li><strong>2nd-generation RT cores</strong> provide up to a 2x increase in raytracing performance</li>
	<li><strong>3rd-generation Tensor Cores</strong> with TF32 and support for sparsity optimizations</li>
	<li><strong>3rd-generation NVLink</strong> provides up to 56.25 GB/sec bandwidth between pairs of GPUs in each direction</li>
	<li><strong>GDDR6X memory</strong> providing up to 768 GB/s of GPU memory throughput</li>
	<li><strong>4th-generation PCI-Express</strong> doubles transfer speeds between the system and each GPU</li>
</ul>

As stated above, the feature sets vary between the "computational" and the "visualization" GPU models. Additional details on each are shared in the tabs below, and the best choice will depend upon your mix of workloads. Please contact our team for additional review and discussion.

<h2>NVIDIA "Ampere" GPU Specifications</h2>

[tabby title="High Performance Computing &amp; Deep Learning GPUs"]
The table below summarizes the features of the NVIDIA Ampere GPU Accelerators designed for computation and deep learning/AI/ML. Note that the PCI-Express version of the NVIDIA A100 GPU features a much lower TDP than the SXM4 version of the A100 GPU (250W vs 400W). For this reason, the PCI-Express GPU is not able to sustain peak performance in the same way as the higher-power part. Thus, the performance values of the PCI-E A100 GPU are shown as a range and actual performance will vary by workload.

To learn more about these products, or to find out how best to leverage their capabilities, please speak with an <a href="https://www.microway.com/contact/" title="Talk to an Expert – Contact Microway">HPC/AI expert</a>.

<table>
  <thead>
    <tr>
      <th>Feature</th>
      <th>NVIDIA A30 PCI-E</th>
      <th>NVIDIA A100 40GB PCI-E</th>
      <th>NVIDIA A100 80GB PCI-E</th>
      <th>NVIDIA A100 SXM4</th>
    </tr>
  </thead>
  <tbody>
    <tr><td class="rowhead">GPU Chip</td><td colspan=4>Ampere GA100</td></tr>
    <tr><td class="rowhead">TensorCore Performance*</td><td class="table-in-table">
        <table><tr><td>10.3 TFLOPS</td><td>FP64</td></tr><tr><td>82 TFLOPS &dagger;</td><td>TF32</td></tr><tr><td>165 TFLOPS &dagger;</td><td>FP16/BF16</td></tr><tr><td>330 TOPS &dagger;</td><td>INT8</td></tr><tr><td>661 TOPS &dagger;</td><td>INT4</td></tr></table>
    </td>
    <td class="table-in-table" colspan=2>
        <table><tr><td>17.6 ~ 19.5 TFLOPS</td><td>FP64</td></tr><tr><td>140 ~ 156 TFLOPS &dagger;</td><td>TF32</td></tr><tr><td>281 ~ 312 TFLOPS &dagger;</td><td>FP16/BF16</td></tr><tr><td>562 ~ 624 TOPS &dagger;</td><td>INT8</td></tr><tr><td>1,123 ~ 1,248 TOPS &dagger;</td><td>INT4</td></tr></table>
    </td>
    <td class="table-in-table">
        <table><tr><td>19.5 TFLOPS</td><td>FP64</td></tr><tr><td>156 TFLOPS &dagger;</td><td>TF32</td></tr><tr><td>312 TFLOPS &dagger;</td><td>FP16/BF16</td></tr><tr><td>624 TOPS &dagger;</td><td>INT8</td></tr><tr><td>1,248 TOPS &dagger;</td><td>INT4</td></tr></table>
    </td>
    </tr>
    <tr><td class="rowhead">Double Precision (FP64) Performance*</td><td>5.2 TFLOPS</td><td colspan=2>8.7 ~ 9.7 TFLOPS</td><td>9.7 TFLOPS</td></tr>
    <tr><td class="rowhead">Single Precision (FP32) Performance*</td><td>10.3 TFLOPS</td><td colspan=2>17.6 ~ 19.5 TFLOPS</td><td>19.5 TFLOPS</td></tr>
    <tr><td class="rowhead">Half Precision (FP16) Performance*</td><td>41 TFLOPS</td><td colspan=2>70 ~ 78 TFLOPS</td><td>78 TFLOPS</td></tr>
    <tr><td class="rowhead">Brain Floating Point (BF16) Performance*</td><td>20 TFLOPS</td><td colspan=2>35 ~ 39 TFLOPS</td><td>39 TFLOPS</td></tr>
    <tr><td class="rowhead">On-die Memory</td><td>24GB HBM2</td><td>40GB HBM2</td><td>80GB HBM2</td><td>40GB HBM2 or 80GB HBM2e</td></tr>
    <tr><td class="rowhead">Memory Bandwidth</td><td>933 GB/s</td><td>1,555 GB/s</td><td>1,940 GB/s</td><td>1,555 GB/s for 40GB<br />2,039 GB/s for 80GB</td></tr>
    <tr><td class="rowhead">L2 Cache</td><td colspan=4>40MB</td></tr>
    <tr><td class="rowhead">Interconnect</td><td>NVLink 3.0 (4 bricks) + PCI-E 4.0<br /><em>NVLink is limited to pairs of directly-linked cards</em></td><td colspan=2>NVLink 3.0 (12 bricks) + PCI-E 4.0<br /><em>NVLink is limited to pairs of directly-linked cards</em></td><td>NVLink 3.0 (12 bricks) + PCI-E 4.0</td></tr>
    <tr><td class="rowhead">GPU-to-GPU transfer bandwidth (bidirectional)</td><td>200 GB/s</td><td colspan=3>600 GB/s</td></tr>
    <tr><td class="rowhead">Host-to-GPU transfer bandwidth (bidirectional)</td><td colspan=4>64 GB/s</td></tr>
    <tr><td class="rowhead"># of MIG instances supported</td><td>up to 4</td><td colspan=3>up to 7</td></tr>
    <tr><td class="rowhead"># of SM Units</td><td>56</td><td colspan=3>108</td></tr>
    <tr><td class="rowhead"># of Tensor Cores</td><td>224</td><td colspan=3>432</td></tr>
    <tr><td class="rowhead"># of integer INT32 CUDA Cores</td><td>3,584</td><td colspan=3>6,912</td></tr>
    <tr><td class="rowhead"># of single-precision FP32 CUDA Cores</td><td>3,584</td><td colspan=3>6,912</td></tr>
    <tr><td class="rowhead"># of double-precision FP64 CUDA Cores</td><td>1,792</td><td colspan=3>3,456</td></tr>
    <tr><td class="rowhead">GPU Base Clock</td><td>930 MHz</td><td>765 MHz</td><td>1065 MHz</td><td>1095 MHz</td></tr>
    <tr><td class="rowhead">GPU Boost Support</td><td colspan=4>Yes - Dynamic</td></tr>
    <tr><td class="rowhead">GPU Boost Clock</td><td>1440 MHz</td><td colspan=3>1410 MHz</td></tr>
    <tr><td class="rowhead">Compute Capability</td><td colspan=4>8.0</td></tr>
    <tr><td class="rowhead">Workstation Support</td><td colspan=4>no</td></tr>
    <tr><td class="rowhead">Server Support</td><td colspan=4>yes</td></tr>
    <tr><td class="rowhead">Cooling Type</td><td colspan=4>Passive</td></tr>
    <tr><td class="rowhead">Wattage (TDP)</td><td>165</td><td>250W</td><td>300W</td><td>400W</td></tr>
  </tbody>
</table>
<em>* theoretical peak performance based on GPU boost clock</em>
<em>&dagger; an additional 2X performance can be achieved via NVIDIA's new sparsity feature</em>

[tabby title="Visualization &amp; Ray Tracing GPUs"]
The table below summarizes the features of the NVIDIA Ampere GPU Accelerators designed for visualization and ray tracing. Note that these GPUs would not necessarily be connecting directly to a display device, but might be performing remote rendering from a datacenter.

To learn more about these GPUs and to review which are the best options for you, please speak with a <a href="https://www.microway.com/contact/" title="Talk to an Expert – Contact Microway">GPU expert</a>.

<table>
  <thead>
    <tr>
      <th>Feature</th>
      <th>NVIDIA RTX A5000</th>
      <th>NVIDIA RTX A6000</th>
      <th>NVIDIA A40</th>
    </tr>
  </thead>
  <tbody>
    <tr><td class="rowhead">GPU Chip</td><td colspan=3>Ampere GA102</td></tr>
    <tr><td class="rowhead">TensorCore Performance*</td><td class="table-in-table">
        <table><tr><td>55.6 TFLOPS &dagger;</td><td>TF32</td></tr><tr><td>111.1 TFLOPS &dagger;</td><td>FP16/BF16</td></tr><tr><td>222.2 TOPS &dagger;</td><td>INT8</td></tr><tr><td>444.4 TOPS &dagger;</td><td>INT4</td></tr></table>
    </td>
    <td class="table-in-table">
        <table><tr><td>77.4 TFLOPS &dagger;</td><td>TF32</td></tr><tr><td>154.8 TFLOPS &dagger;</td><td>FP16/BF16</td></tr><tr><td>309.7 TOPS &dagger;</td><td>INT8</td></tr><tr><td>619.3 TOPS &dagger;</td><td>INT4</td></tr></table>
    </td>
    <td class="table-in-table">
        <table><tr><td>74.8 TFLOPS &dagger;</td><td>TF32</td></tr><tr><td>149.7 TFLOPS &dagger;</td><td>FP16/BF16</td></tr><tr><td>299.3 TOPS &dagger;</td><td>INT8</td></tr><tr><td>598.7 TOPS &dagger;</td><td>INT4</td></tr></table>
    </td>
    </tr>
    <tr><td class="rowhead">Double Precision (FP64) Performance*</td><td>0.4 TFLOPS</td><td>0.6 TFLOPS</td><td>0.6 TFLOPS</td></tr>
    <tr><td class="rowhead">Single Precision (FP32) Performance*</td><td>27.8 TFLOPS</td><td>38.7 TFLOPS</td><td>37.4 TFLOPS</td></tr>
    <tr><td class="rowhead">Integer (INT32) Performance*</td><td>13.9 TOPS</td><td>19.4 TOPS</td><td>18.7 TOPS</td></tr>
    <tr><td class="rowhead">GPU Memory</td><td>24GB</td><td>48GB</td><td>48GB</td></tr>
    <tr><td class="rowhead">Memory Bandwidth</td><td>768 GB/s</td><td>768 GB/s</td><td>696 GB/s</td></tr>
    <tr><td class="rowhead">L2 Cache</td><td colspan=3>6MB</td></tr>
    <tr><td class="rowhead">Interconnect</td><td colspan=3>NVLink 3.0 + PCI-E 4.0<br /><em>NVLink is limited to pairs of directly-linked cards</em></td></tr>
    <tr><td class="rowhead">GPU-to-GPU transfer bandwidth (bidirectional)</td><td colspan=3>112.5 GB/s</td></tr>
    <tr><td class="rowhead">Host-to-GPU transfer bandwidth (bidirectional)</td><td colspan=3>64 GB/s</td></tr>
    <tr><td class="rowhead"># of MIG instances supported</td><td colspan=3>N/A</td></tr>
    <tr><td class="rowhead"># of SM Units</td><td>64</td><td colspan=2>84</td></tr>
    <tr><td class="rowhead"># of RT Cores</td><td>64</td><td colspan=2>84</td></tr>
    <tr><td class="rowhead"># of Tensor Cores</td><td>256</td><td colspan=2>336</td></tr>
    <tr><td class="rowhead"># of integer INT32 CUDA Cores</td><td>8,192</td><td colspan=2>10,752</td></tr>
    <tr><td class="rowhead"># of single-precision FP32 CUDA Cores</td><td>8,192</td><td colspan=2>10,752</td></tr>
    <tr><td class="rowhead"># of double-precision FP64 CUDA Cores</td><td>128</td><td colspan=2>168</td></tr>
    <tr><td class="rowhead">GPU Base Clock</td><td colspan=3>not published</td></tr>
    <tr><td class="rowhead">GPU Boost Support</td><td colspan=3>Yes - Dynamic</td></tr>
    <tr><td class="rowhead">GPU Boost Clock</td><td colspan=3>not published</td></tr>
    <tr><td class="rowhead">Compute Capability</td><td colspan=3>8.6</td></tr>
    <tr><td class="rowhead">Workstation Support</td><td colspan=2>yes</td><td>no</td></tr>
    <tr><td class="rowhead">Server Support</td><td colspan=2>no</td><td>yes</td></tr>
    <tr><td class="rowhead">Cooling Type</td><td colspan=2>Active</td><td>Passive</td></tr>
    <tr><td class="rowhead">Wattage (TDP)</td><td>230W</td><td>300W</td><td>300W</td></tr>
  </tbody>
</table>
<em>* theoretical peak performance based on GPU boost clock</em>
<em>&dagger; an additional 2X performance can be achieved via NVIDIA's new sparsity feature</em>

Several lower-end graphics cards and datacenter GPUs are also available including RTX A2000, RTX A4000, A10, and A16. These GPUs offer similar capabilities, but with lower levels of performance and available at lower price points.
[tabbyending]

<h2>Comparison between "Pascal", "Volta", and "Ampere" GPU Architectures</h2>
<table>
  <thead>
    <tr>
      <th>Feature</th>
      <th>Pascal GP100</th>
      <th>Volta GV100</th>
      <th>Ampere GA100</th>
    </tr>
  </thead>
  <tbody>
    <tr><td class="rowhead">Compute Capability*</td><td>6.0</td><td>7.0</td><td>8.0</td></tr>
    <tr><td class="rowhead">Threads per Warp</td><td colspan=3>32</td></tr>
    <tr><td class="rowhead">Max Warps per SM</td><td colspan=3>64</td></tr>
    <tr><td class="rowhead">Max Threads per SM</td><td colspan=3>2048</td></tr>
    <tr><td class="rowhead">Max Thread Blocks per SM</td><td colspan=3>32</td></tr>
    <tr><td class="rowhead">Max Concurrent Kernels</td><td colspan=3>128</td></tr>
    <tr><td class="rowhead">32-bit Registers per SM</td><td colspan=3>64 K</td></tr>
    <tr><td class="rowhead">Max Registers per Block</td><td colspan=3>64 K</td></tr>
    <tr><td class="rowhead">Max Registers per Thread</td><td colspan=3>255</td></tr>
    <tr><td class="rowhead">Max Threads per Block</td><td colspan=3>1024</td></tr>
    <tr><td class="rowhead">L1 Cache Configuration</td><td>24KB<br /><em>dedicated cache</em></td><td>32KB ~ 128KB<br /><em>dynamic with shared memory</em></td><td>28KB ~ 192KB<br /><em>dynamic with shared memory</em></td></tr>
    <tr><td class="rowhead">Shared Memory Configurations</td><td>64KB</td><td>configurable up to 96KB;<br />remainder for L1 Cache<br /><em>(128KB total)</em></td><td>configurable up to 164KB;<br />remainder for L1 Cache<br /><em>(192KB total)</em></td></tr>
    <tr><td class="rowhead">Max Shared Memory per SM</td><td>64KB</td><td>96KB</td><td>164KB</td></tr>
    <tr><td class="rowhead">Max Shared Memory per Thread Block</td><td>48KB</td><td>96KB</td><td>160KB</td></tr>
    <tr><td class="rowhead">Max X Grid Dimension</td><td colspan=3>2<sup>32-1</sup></td></tr>
    <tr><td class="rowhead">Tensor Cores</td><td>No</td><td colspan=2>Yes</td></tr>
    <tr><td class="rowhead">Mixed Precision Warp-Matrix Functions</td><td>No</td><td colspan=2>Yes</td></tr>
    <tr><td class="rowhead">Hardware-accelerated async-copy</td><td colspan=2>No</td><td>Yes</td></tr>
    <tr><td class="rowhead">L2 Cache Residency Management</td><td colspan=2>No</td><td>Yes</td></tr>
    <tr><td class="rowhead">Dynamic Parallelism</td><td colspan=3>Yes</td></tr>
    <tr><td class="rowhead">Unified Memory</td><td colspan=3>Yes</td></tr>
    <tr><td class="rowhead">Preemption</td><td colspan=3>Yes</td></tr>
  </tbody>
</table>
<em>* For a complete listing of Compute Capabilities, reference the <a href="https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#features-and-technical-specifications__technical-specifications-per-compute-capability" target="_blank" rel="noopener noreferrer">NVIDIA CUDA Documentation</a></em>

<h2>Hardware-accelerated raytracing, video encoding, video decoding, and image decoding</h2>
The NVIDIA "Ampere" Datacenter GPUs that are designed for computational workloads do not include graphics acceleration features such as RT cores and hardware-accelerated video encoders. For example, RT cores for accelerated raytracing are not included in the A30 and A100 GPUs. Similarly, video encoding units (NVENC) are not included in these GPUs.

To accelerate computational workloads that require processing of image or video files, five JPEG decoding (NVJPG) units and five video decoding units (NVDEC) are included in A100. Details are described on NVIDIA's <a href="https://devblogs.nvidia.com/improving-computer-vision-with-nvidia-a100-gpus/" rel="noopener noreferrer" target="_blank">A100 for computer vision</a> blog post.

For additional details on NVENC and NVDEC, reference NVIDIA's <a href="https://developer.nvidia.com/video-encode-decode-gpu-support-matrix" target="_blank" rel="noopener noreferrer">encoder/decoder support matrix</a>. To learn more about GPU-accelerated video encode/decode, see NVIDIA's <a href="https://developer.nvidia.com/nvidia-video-codec-sdk" target="_blank" rel="noopener noreferrer">Video Codec SDK</a>.

Cancel

Edit

Edit

In-Depth Comparison of NVIDIA “Ampere” GPU Accelerators

Edit

Archives

Meta

Technical Information

Talk to an Expert

Configure Your Solution

Schedule a Consultation

Knowledge Center Categories

Customer Testimonials

Technologies

Products

Knowledge Center

Pre-Configured Systems

NVIDIA DGX H100™

NVIDIA DGX POD™

EOL – NVIDIA DGX A100™

AI Anywhere Solution

Edit

Edit

In-Depth Comparison of NVIDIA “Ampere” GPU Accelerators

Edit

Archives

Meta

Technical Information

Talk to an Expert

Configure Your Solution

Schedule a Consultation

Knowledge Center Categories

Knowledge Center Tags

Customer Testimonials