Can Your Racks Support NVIDIA DGX H100 Systems?

AI is booming.

The AI market is projected to grow 37.3% annually from 2023 to 2030.

With so many organizations adopting or considering AI applications, data centers need to be ready to support the new demand.

However, without the right tools and data, it is difficult to understand if your existing facilities have the capacity to support systems like the “gold standard for AI infrastructure,” the NVIDIA DGX H100.

The key considerations for deploying the NVIDIA DGX H100 system include:

• Power. The DGX H100 system’s power usage is 10.2 kW maximum. It contains six power supplies with balanced distribution of the power load. The specification for each power supply is 3300 W at 200-240 V, 16A, and 50-60Hz.
• Space. The system has an 8U rackmount form factor. It is 14 inches (356 mm) high, 19 inches (482.3 mm) wide, and 35.3 inches (897.1 mm) deep.
• Weight. The DGX H100 has a maximum weight of 287.7 lbs. (130.45 kg).
• Ports. The system has four OSFP ports serving 8 single-port NVIDIA ConnectX-7 VPI and two dual-port QSFP112 NVIDIA ConnectX-7 VPI.
• Environment. The DGX H100 system’s recommended operating environment is a temperature ranging from 41° F to 86° F (5° C to 30° C), relative humidity of 20% to 80%, and airflow of 1,105 CFM front-to-back at 80% fan pulse width modulation. The system’s heat output is 38,557 BTU per hour.

How to Know If Your Racks Can Support AI Workloads

Without the right tool, trying to figure out if you can deploy systems like the NVIDIA DGX H100 without introducing risk can be a nightmare. It can require gathering data from many disparate sources, manual math, and estimations that may or may not be accurate.

That’s where Data Center Infrastructure Management (DCIM) software comes in.

Modern DCIM software provides real-time power and environmental monitoring, accurate asset and circuit management, and intelligent capacity planning that enables you to know what resources are available to accommodate higher rack densities while mitigating the risk of downtime.

DCIM software enables more informed capacity planning to know if your racks can support the DGX H100 system with:

• Intelligent capacity search. Simply enter the make and model of the piece of equipment you are deploying and get a list of all the cabinets with enough available space, power, and ports to support it. You can even reserve all of those resources with a click.
• What-if analysis. Predict the impact your planned projects will have on your rack-level space and power utilization to understand if you have the capacity to support them and to manage the density of equipment across your racks.
• Automatic server power budgeting. Safely deploy more equipment in existing racks without introducing risk by using the “Auto Power Budget’ feature to automatically calculate power budget profiles for each server instance based on their actual trended power utilization. Comcast reported that this feature alone unlocked 40% more capacity in their existing facilities.
• Dynamic single-line power diagrams. Leverage built-in power circuit intelligence that lets you easily understand the power capacity and load at every hop in your power chain. With this information at your fingertips, you can ensure new loads won’t trip a breaker, know how to balance all three phases, and ensure redundancy. Single-line diagrams are automatically rendered and updated so you can ditch your static paper diagrams.
• Correlated capacity reporting. Visualize, correlate, and analyze multiple capacity parameters with a digital twin of your data center. Report simultaneously on multiple capacity constraints to see at a glance with red/yellow/green color-coding which racks can support new workloads. For example, weight is an often-overlooked consideration for high-density racks, but DCIM software with a comprehensive models library automatically calculates the total weight of your cabinets and shows you which cabinets you can deploy an NVIDIA DGX H100 system in without exceeding the weight capacity of your raised floor.

How to Monitor and Manage High-Density AI Infrastructure

If you are deploying the DGX H100 system or similar AI infrastructure, the job is not done once you have determined if and where you have capacity to install it. High-density infrastructure requires ongoing monitoring and management to ensure that uptime is maintained, capacity planning is not made more complex, and energy costs are contained.

DCIM software provides real-time monitoring of high-density power and environmental data to help you keep services and applications running smoothly.

Modern DCIM software is field-proven to monitor over 10 billion data points a day including power data from intelligent rack PDUs, floor PDUs, busways, branch circuits, RPPs, and UPSs and environmental data from temperature, humidity, airflow, and other sensors.

You can set warning and critical thresholds on power loads, three-phase balance, rack PDU circuit breaker state change, temperature, and humidity. Then, you will automatically receive alerts when thresholds are violated so you are the first to know of potential issues and can proactively investigate and resolve them before they become serious problems.

Maintain optimal temperatures by identifying the formation of hot spots and airflow patterns with thermal map time-lapse videos of your data center floor and see which cabinets are outside manufacturer- and industry-accepted thermal guidelines with patented ASHRAE psychrometric cooling charts.

Leverage out-of-the-box dashboard charts and reports to track high-density KPIs like capacity and utilization of power, space, cooling, and data/power port connections, delta T per cabinet, energy cost, and Power Usage Effectiveness (PUE) so you know the health and capacity of any site at a glance.

Finally, you can visualize your high-density racks and cabling on your 3D floor map with overlaid power and environmental data and see how everything connects with automatically updated network diagrams that include both active and passive components.

Bringing it All Together

As the AI market continues to surge with significant growth projected in the coming years, data centers must be prepared to support advanced systems like the NVIDIA DGX H100.

Understanding if your existing facilities can handle such high-demand equipment can be a daunting task without the right tools and data.

DCIM software has emerged as an important solution, offering real-time monitoring and capacity planning that simplifies high-density AI infrastructure management.

With DCIM software, you can not only determine if your racks can support AI workloads but also ensure the smooth operation of high-density infrastructure, maintain optimal conditions, and track the right KPIs, ultimately improving the efficiency and reliability of your data center.

Try DCIM For AI Infrastructure Management

Want to see for yourself how Sunbird’s second-generation DCIM software can help you plan and manage your high-density AI infrastructure? Test drive Sunbird’s DCIM today.