As NVIDIA’s GPUs become increasingly vital to the fields of AI and intelligent machines, NVIDIA has produced GPU models specifically targeted to these applications. The new Tesla P40 GPU is NVIDIA’s premiere product for deep learning deployments. It is specifically designed for high-speed inference workloads, which means running data through pre-trained neural networks. However, it also offers significant processing performance for projects which do not require 64-bit double-precision floating point capability (many neural networks can be trained using the 32-bit single-precision floating point on the Tesla P40). For those cases, these GPUs can be used to accelerate both the neural network training and the inference.
Highlights of the new Tesla P40 GPU include:
- Up to 12 TFLOPS single-precision floating-point performance
- Support for INT8 operations with up to 47 TOPS (ideal for high-speed/high-volume inference)
- 24GB of GDDR5 GPU memory, with bandwidths up to 346GB/s
PCI-Express Data Transfer Speeds
The Tesla P40 GPUs use the same generation 3.0 PCI-E connectivity as other recent GPUs (such as the Maxwell generation), so you should expect to achieve similar transfer speeds. As shown below, we’re able to achieve transfers up to ~12.8GB/s between the host and the GPU:
[root@node4 ~]# ./bandwidthTest --memory=pinned --device=0 [CUDA Bandwidth Test] - Starting... Running on... Device 0: Tesla P40 Quick Mode Host to Device Bandwidth, 1 Device(s) PINNED Memory Transfers Transfer Size (Bytes) Bandwidth(MB/s) 33554432 11842.8 Device to Host Bandwidth, 1 Device(s) PINNED Memory Transfers Transfer Size (Bytes) Bandwidth(MB/s) 33554432 12899.9 Device to Device Bandwidth, 1 Device(s) PINNED Memory Transfers Transfer Size (Bytes) Bandwidth(MB/s) 33554432 240357.5 Result = PASS
Technical Details of the Tesla P40 GPU
Below are the technical details reported by nvidia-smi. Note that “Pascal” Tesla GPUs now include fully integrated memory ECC support that is always enabled (memory performance in previous generations could be improved by disabling ECC).
[root@node4 ~]# nvidia-smi -a -i 0
==============NVSMI LOG==============
Timestamp : Mon Feb 6 12:30:52 2017
Driver Version : 367.57
Attached GPUs : 4
GPU 0000:02:00.0
Product Name : Tesla P40
Product Brand : Tesla
Display Mode : Disabled
Display Active : Disabled
Persistence Mode : Enabled
Accounting Mode : Enabled
Accounting Mode Buffer Size : 1920
Driver Model
Current : N/A
Pending : N/A
Serial Number : 0324416xxxxxx
GPU UUID : GPU-16254654-0bd3-8d18-e8fe-d53865xxxxxx
Minor Number : 0
VBIOS Version : 86.02.23.00.01
MultiGPU Board : No
Board ID : 0x200
GPU Part Number : 900-2G610-0000-000
Inforom Version
Image Version : G610.0200.00.03
OEM Object : 1.1
ECC Object : 4.1
Power Management Object : N/A
GPU Operation Mode
Current : N/A
Pending : N/A
GPU Virtualization Mode
Virtualization mode : None
PCI
Bus : 0x02
Device : 0x00
Domain : 0x0000
Device Id : 0x1B3810DE
Bus Id : 0000:02:00.0
Sub System Id : 0x11D910DE
GPU Link Info
PCIe Generation
Max : 3
Current : 1
Link Width
Max : 16x
Current : 16x
Bridge Chip
Type : N/A
Firmware : N/A
Replays since reset : 0
Tx Throughput : 0 KB/s
Rx Throughput : 0 KB/s
Fan Speed : N/A
Performance State : P8
Clocks Throttle Reasons
Idle : Active
Applications Clocks Setting : Not Active
SW Power Cap : Not Active
HW Slowdown : Not Active
Sync Boost : Not Active
Unknown : Not Active
FB Memory Usage
Total : 22912 MiB
Used : 0 MiB
Free : 22912 MiB
BAR1 Memory Usage
Total : 32768 MiB
Used : 2 MiB
Free : 32766 MiB
Compute Mode : Default
Utilization
Gpu : 0 %
Memory : 0 %
Encoder : 0 %
Decoder : 0 %
Ecc Mode
Current : Enabled
Pending : Enabled
ECC Errors
Volatile
Single Bit
Device Memory : 0
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
Total : 0
Double Bit
Device Memory : 0
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
Total : 0
Aggregate
Single Bit
Device Memory : 0
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
Total : 0
Double Bit
Device Memory : 0
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
Total : 0
Retired Pages
Single Bit ECC : 0
Double Bit ECC : 0
Pending : No
Temperature
GPU Current Temp : 27 C
GPU Shutdown Temp : 95 C
GPU Slowdown Temp : 92 C
Power Readings
Power Management : Supported
Power Draw : 12.34 W
Power Limit : 250.00 W
Default Power Limit : 250.00 W
Enforced Power Limit : 250.00 W
Min Power Limit : 125.00 W
Max Power Limit : 250.00 W
Clocks
Graphics : 544 MHz
SM : 544 MHz
Memory : 405 MHz
Video : 544 MHz
Applications Clocks
Graphics : 1531 MHz
Memory : 3615 MHz
Default Applications Clocks
Graphics : 1303 MHz
Memory : 3615 MHz
Max Clocks
Graphics : 1531 MHz
SM : 1531 MHz
Memory : 3615 MHz
Video : 1379 MHz
Clock Policy
Auto Boost : N/A
Auto Boost Default : N/A
Processes : None
The latest NVIDIA GPU architectures support large numbers of clock speeds, as well as automated boosting of the clock speed (when power and thermals allow). Administrators can also set specific power consumption limits and monitor the clock speeds (including explanations for any reasons the clocks are running at a lower speed). The list below shows the available clock speeds for the Tesla P40 GPU:
[root@node4 ~]# nvidia-smi -q -d SUPPORTED_CLOCKS -i 0
==============NVSMI LOG==============
Timestamp : Mon Feb 6 12:31:56 2017
Driver Version : 367.57
Attached GPUs : 4
GPU 0000:02:00.0
Supported Clocks
Memory : 3615 MHz
Graphics : 1531 MHz
Graphics : 1518 MHz
Graphics : 1506 MHz
Graphics : 1493 MHz
Graphics : 1480 MHz
Graphics : 1468 MHz
Graphics : 1455 MHz
Graphics : 1442 MHz
Graphics : 1430 MHz
Graphics : 1417 MHz
Graphics : 1404 MHz
Graphics : 1392 MHz
Graphics : 1379 MHz
Graphics : 1366 MHz
Graphics : 1354 MHz
Graphics : 1341 MHz
Graphics : 1328 MHz
Graphics : 1316 MHz
Graphics : 1303 MHz
Graphics : 1290 MHz
Graphics : 1278 MHz
Graphics : 1265 MHz
Graphics : 1252 MHz
Graphics : 1240 MHz
Graphics : 1227 MHz
Graphics : 1215 MHz
Graphics : 1202 MHz
Graphics : 1189 MHz
Graphics : 1177 MHz
Graphics : 1164 MHz
Graphics : 1151 MHz
Graphics : 1139 MHz
Graphics : 1126 MHz
Graphics : 1113 MHz
Graphics : 1101 MHz
Graphics : 1088 MHz
Graphics : 1075 MHz
Graphics : 1063 MHz
Graphics : 1050 MHz
Graphics : 1037 MHz
Graphics : 1025 MHz
Graphics : 1012 MHz
NVIDIA deviceQuery on Tesla P40 GPU
Each new GPU generation brings tweaks to the design. The output below, from the CUDA 8.0 SDK samples, shows additional details of the architecture and capabilities of the “Pascal” Tesla P40 GPU accelerators. Take note of the new Compute Capability 6.1, which is what you’ll want to target if you’re compiling your own CUDA code.
./deviceQuery Starting...
CUDA Device Query (Runtime API) version (CUDART static linking)
Detected 4 CUDA Capable device(s)
Device 0: "Tesla P40"
CUDA Driver Version / Runtime Version 8.0 / 8.0
CUDA Capability Major/Minor version number: 6.1
Total amount of global memory: 22913 MBytes (24025956352 bytes)
(30) Multiprocessors, (128) CUDA Cores/MP: 3840 CUDA Cores
GPU Max Clock rate: 1531 MHz (1.53 GHz)
Memory Clock rate: 3615 Mhz
Memory Bus Width: 384-bit
L2 Cache Size: 3145728 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
Maximum Layered 1D Texture Size, (num) layers 1D=(32768), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(32768, 32768), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: No
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Enabled
Device supports Unified Addressing (UVA): Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 2 / 0
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 8.0, CUDA Runtime Version = 8.0, NumDevs = 4, Device0 = Tesla P40, Device1 = Tesla P40, Device2 = Tesla P40, Device3 = Tesla P40
Result = PASS
Additional Information on Tesla P40 GPUs
To learn more about the NVIDIA “Pascal” GPU architecture and to compare Tesla P40 with other models in the Tesla product line, read our “Pascal” Tesla GPU knowledge center article.
If you’re thinking about using GPUs for the first time, please consider getting in touch with us. We’ve been implementing GPU-accelerated systems for nearly a decade and have the expertise to help make your project a success!
If you’re an existing GPU user considering a new deployment, review or Tesla GPU clusters page and our list of GPU servers.
