Revision for “Solutions for Rackmounting Extra Depth Systems” created on September 13, 2019 @ 12:16:17
Solutions for Rackmounting Extra Depth Systems
Extra Depth Systems having socketed GPUs, such as the <strong>Tesla V100 SXM2</strong>, or the <strong>Tesla P100 SXM2</strong>, require an additional, or extended system board, onto which the socketed GPUs are seated. In this type of system, the presence of the additional or extended board requires a system chassis having more depth than most other rackmountable systems. The challenge that comes along with this is that most data centers have server cabinets, which in their most common configurations, cannot accommodate an extra depth chassis. For this reason, extra depth cabinets are usually required for rackmounting an extra depth system.
An extra depth chassis is not always required, though. Workarounds can be implemented in order to fit an extra depth chassis into a regular depth cabinet, but an uncommon cabinet configuration would be required.
<h3>Common Extra Depth Systems</h3>
Commonly used extra depth cabinets are described in <strong>Table 2</strong>, ranging in height from 42U to 48U. Extra depth cabinets are the easiest solution for rackmounting extra depth systems. There are workarounds, however, which can be implemented for instances where a customer already has a regular depth cabinet on-site, and would prefer to use the existing cabinet, due possibly, to scarcity of unused floor space.
<h3>Workaround Solution #1: Remove any vertical PDUs, and replace them with horizontal PDUs</h3>
Along with their unusually high power density, <strong>extra depth systems will require high airflow</strong>. Using the NVIDIA DGX-1 again as an example, four chassis fans will each produce a maximum of 340 CFM of air flow, for a total of 1,360 CFM. For groups of systems having high air flow requirements, the cabinet doors must be perforated.
Selecting a <strong>tri-phase horizontal PDU</strong> can be a challenge, since they do not present as many outlets as vertically mounted PDUs, and will sometimes present an outlet type which is not compatible with the inlet type on the system(s). If the entire cabinet will not be needed for mounting extra depth systems, then using a half height PDU, possibly in addition to a horizontal PDU, may be a good choice.
<strong>APC</strong> currently offers only one tri-phase horizontal PDU. <strong>Geist</strong> offers a wide variety of tri-phase horizontal PDU types. These can be searched using the <a href="http://www.geistglobal.com/pdu-search" rel="noopener noreferrer" target="_blank"><strong>Geist PDU finder</strong></a>. Geist does not offer a tri-phase PDU, for use with a 208V, 20Amp source. However, it offers a variety of horizontal, tri-phase PDUs, for lines carrying 30Amps, or more. <!– One example is the Geist <a href="http://www.geistglobal.com/power-distribution-unit/I10116L%20–%20MN01D1R1-06L198-3TL21A0H10-S" rel="noopener noreferrer" target="_blank"><strong>model no. I10116L — MN01D1R1-06L198-3TL21A0H10-S</strong></a>, used with tri-phase, 208V, 30Amp power sources, compatible with a <a href="https://www.microway.com/knowledge-center-articles/power-and-cooling-multiple-computer-installations/" rel="noopener noreferrer" target="_blank"><strong>NEMA L21-30P</strong></a> plug type (Wye circuit).–> Like Geist, <strong>Server Technology</strong> offers a range of tri-phase, horizontal PDUs. They offer a <a href="https://www.servertech.com/power-distribution-unit-pdu/switched-pops/2u-horizontal-2NH3" rel="noopener noreferrer" target="_blank">tri-phase PDU model</a> which can be used with a 208V, 20Amp power source, compatible with the <a href="https://www.microway.com/knowledge-center-articles/power-and-cooling-multiple-computer-installations/" rel="noopener noreferrer" target="_blank"><strong>NEMA L21-20P</strong></a> plug type.
<h3>Workaround Solution #2: Replace any full height vertical PDUs with half height PDUs</h3>
Half height vertical PDUs should only be used if the entire cabinet will not be needed for mounting extra depth systems. This is because half height vertical PDUs will still prevent installation of extra depth systems into approximately half of the cabinet’s rackspace. Some half height PDUs can be mounted on the exterior of the cabinet frame, so that they will not obstruct extra depth systems from being rackmounted.
<h3>Workaround Solution #3: Use an Extra Wide, Regular Depth Cabinet</h3>
<!– The DGX-1 will fit into AR3100 cabinet without right angle power connectors on the PDUs. Rear doors of the AR3100 do not have to be removed. The extra depth 1U systems described here, having socketed GPUs, will also fit into the AR3100 cabinet, without the need for right angle power connectors. –>
<h3>Workaround Solution #4: Mount Systems at Height Corresponding to Space between PDU Power Banks</h3>
For example, an IBM Power9 system (33.3" depth) will still fit into an APC 3100 regular depth cabinet, with four vertical AP7541 PDUs installed at the back of the cabinet, as long as it is installed at a height between the PDU power banks.