As we continue our series on Building to Suit in an AI World, let’s explore how AI deployments are denser than traditional cloud or enterprise deployments, and how they’re driving significant shifts in data center design. Between 2011 and 2020—before AI took off—average rack density rose from 2.4 kW to 8.4 kW. The current-generation NVIDIA GPU architecture, Blackwell, is designed for 120 kW per rack, up from 41 kW per rack one generation ago (Hopper). And architectures continue to densify; in March, NVIDIA CEO Jensen Huang announced a roadmap for 600 kW racks by the end of 2027.
AI deployments are so dense because the GPUs used to train AI models are much more power-intensive than the CPUs used for traditional workloads. Keeping the processors close together enables bigger cluster sizes, and bigger cluster sizes enable more powerful models. (Learn more in Building to Suit Location.)
At these kinds of densities, AI deployments generate too much heat for air cooling to handle; they need liquid cooling. At the same time, most data centers supporting AI today also support traditional workloads, which are much less dense and can be effectively cooled with air. So data centers need to be able to support both air and liquid cooling. Given that AI development and adoption are in the very early days, with lots of innovation to come, densities are likely to continue rising. Data centers need to be able to support today’s densities and be flexible to support future densities as well.
Building to Suit Density Requirements
An AI data center is not only designed differently; it’s operated differently. Delivering an AI-enabled data center solution is about the technology, the implementation, and the day-to-day execution. Our AI-enabled data center product offering, Ingenuity, has industry-leading SOPs, MOPs, and EOPs to ensure a best-in-class experience. As our Chief Marketing and Product Officer Phillip Marangella explained in an article about adapting data center infrastructure for the age of AI, “As we transition from air to liquid cooling, we’ve also completely reconfigured our operational procedures, training, and readiness to ensure a safe, secure, and sustainable environment for AI/HPC deployments.”
A modern data center needs to support both high-density AI workloads and lower-density traditional workloads. Ingenuity is flexible to support rack densities from 10 kW to over 500 kW. Accommodating a variety of power density and cooling requirements for GPU, CPU, storage, and networking, Ingenuity supports dedicated AI workloads as well as mixed workloads, all within the same data center. We have the flexibility to balance varying IT loads within same data hall, leveraging built-in thermal storage for continuous cooling capabilities.
Data centers built to support today’s AI workloads need to be flexible to support future AI workloads as well. As an NVIDIA DGX certified partner, we work closely with the chipmaker to ensure we have the technical solutions in place to support the power and cooling requirements for future generations of AI chips. Like a backplane in circuitry, our AI-enabled data center serves as a foundation upon which AI deployments can run. The design is flexible to support a range of cooling technologies and adaptable to support future densities in cost-effective manner as AI architectures evolve.
Data centers have to be sustainable, even as their power requirements grow. AI presents both challenges and opportunities. Challenges include the technological requirements for higher power density needs due to AI and HPC; opportunities include the rapid growth and adoption of AI technologies. As we address these challenges and opportunities, we remain dedicated to mitigating climate impacts through our sustainability strategy in both the short and long term. For example, our efficient cooling designs for high-density applications help optimize Power Usage Effectiveness (PUE).
Bottom Line
The rapidly rising densities associated with AI workloads are driving significant shifts in how data centers are designed. Success requires a developer committed to operational excellence and sustainability, with a design flexible enough to support both high-density AI workloads and lower-density traditional workloads—and future workloads as AI evolves. A developer like EdgeConneX.
Previously in the Building to Suit in an AI World Series: