Enterprises in for a shock when they realize power and cooling demands of AI
- Reference: 1736948462
- News link: https://www.theregister.co.uk/2025/01/15/ai_power_cooling_demands/
- Source link:
Research commissioned by AI chip biz [1]SambaNova found that 72 percent of corporate leaders are aware AI models have huge energy requirements, and many are concerned about this, yet only 13 percent monitor the power consumption of the AI systems they have deployed.
The power draw is driven in most cases by reliance on power-hungry GPUs that are crammed into high-performance server systems to tackle model training. SambaNova chief Rodrigo Liang, said:
[2]
"Without a proactive approach to more efficient AI hardware and energy consumption, particularly in the face of increasing demand from AI workflows, we risk undermining the very progress AI promises to deliver."
[3]
[4]
He expects attitudes to change, and forecast that by 2027, most corporate leaders will be keeping a close check on energy consumption as a key performance indicator (KPI).
Adding impetus to this is the rise of so-called agentic AI models. These are developed to be capable of taking autonomous actions and solving multi-step problems, and their greater complexity will add to energy woes, according to SambaNova.
[5]
Naturally, SambaNova is pitching its AI silicon, [6]packaged in servers with the company's software stack , as a lower power alternative to GPUs, however, not everyone will wish to go down that route.
GPUs creepers
For organizations sticking with GPUs, dealing with the heat generated by the power-guzzling hardware is becoming another big issue, with [7]Nvidia's Blackwell products rated at 1,200 W, for example. In many cases, this will involve more effective cooling systems, with liquid cooling becoming increasingly popular.
Analyst firm Omdia estimated last year that [8]datacenter liquid cooling revenue was set to top $2 billion by the end of 2024, and $5 billion by 2028.
However, not all facilities will be suitable for outfitting with liquid cooling, according to managed services provider Redcentric.
"Greater investment in AI development and implementation is likely to lead to an increased demand for datacenters," claimed chief technology officer Paul Mardling. Building new facilities, he added, is "a significant investment" that takes time, planning permission, power provisioning, and physical construction.
[9]
"In the short term, this will lead to increased demand for existing datacenters, many of which haven't been designed for the density or power draw that AI systems require."
While traditional facilities were built around halls with racks of 2-5 kW power density, new builds now have to accommodate much higher, he said.
"Liquid cooling is essential in racks with greater than 10 kW power density and is desirable in the 5-10 kW range. Efficient use of excess heat is also likely to emerge, either for thermal regenerative power generation or communal heating projects."
Omdia agrees that AI is driving energy demand and the need for more effective cooling.
"Yes, greater adoption of AI computing will drive up datacenter power density," senior research director Vlad Galabov told The Register .
[10]Biden opens federal land to power-hungry AI datacenters
[11]With AI boom in full force, 2024 datacenter deals reach $57B record
[12]UK prepared to throw planning rules out the window for massive datacenters
[13]To save the energy grid from AI, use open source AI, says open source body
"We are already seeing this and there are several implications: we have seen requests to utilities for more power and adoption of onsite self-generation either through gas engines or turbines," he added.
Power upgrades also involve pre-fab modules housing extra switchgear, UPS, and batteries deployed on-campus to enable higher power capacity, while some sites have been retrofitted with high-capacity busways in place of cables to distribute more power to the racks.
However, Galabov reckons this type of retrofit is less likely in older datacenters due to the costs.
"There is likely a ceiling to how much density can be raised. At one Equinix site I saw a retrofit project result in rack density increasing from 10 kW to 30 kW per rack."
That specific case involved new pipe work to support a coolant distribution unit (CDU) connected to rear-door heat exchangers in the racks, while power distribution was swapped from cables to a new busway, and within each rack new power distribution units (PDUs) were installed, Galabov told us.
On the topic of liquid cooling, he says some sites have seen the adoption of air-to-liquid CDUs as one way to avoid having to completely update the pipe network within their datacenters, and Microsoft is a big proponent of this approach.
Yet, according to the Omdia research director, adoption of this may be limited because the density they enable is not up to supporting incoming AI infrastructure like Nvidia's Blackwell rack-level reference design ( [14]NVL72 ).
"In areas where operators did not want to update racks and install manifolds for direct-to-chip liquid cooling, we've also seen deployments of rear door heat exchangers, which are a good way to cope," Galabov said. Vendors believe they can go up to 100 kW with this arrangement. However, he thinks this is unlikely as the temperature of the ambient air would need significant lowering, which would be very expensive.
For UK businesses, colocation firm [15]Telehouse recently unveiled a liquid cooling lab at its London Docklands campus, showcasing several of the available technologies. These include a waterless, two-phase system for next-gen server chips and air-assisted liquid cooling technology for up to 90 kW per cabinet. ®
Get our [16]Tech Resources
[1] https://www.theregister.com/2024/09/10/sambanovas_inference_cloud/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Z4fpvu8-7pcEO11KTVXwRQAAAII&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z4fpvu8-7pcEO11KTVXwRQAAAII&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z4fpvu8-7pcEO11KTVXwRQAAAII&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z4fpvu8-7pcEO11KTVXwRQAAAII&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2022/09/15/sambanova_ai_chip_nvidia/
[7] https://www.theregister.com/2024/03/18/nvidia_turns_up_the_ai/
[8] https://www.theregister.com/2024/06/28/datacenter_capex_tai/
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z4fpvu8-7pcEO11KTVXwRQAAAII&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[10] https://www.theregister.com/2025/01/14/biden_ai_executive_order/
[11] https://www.theregister.com/2025/01/14/2024_datacenter_deals/
[12] https://www.theregister.com/2025/01/13/uk_datacenter_planning_rules/
[13] https://www.theregister.com/2025/01/09/linux_foundation_ai_energy_report/
[14] https://www.theregister.com/2024/03/21/nvidia_dgx_gb200_nvk72/
[15] https://www.telehouse.net/
[16] https://whitepapers.theregister.com/
Funny...
Because a human only puts out ~100..300W of heat during operation...
So you could use four humans instead of one Blackwell GPU...
And they're perfectly capable of cooling themselves too... ------------------------->
Re: Funny...
Just waiting for the first report of an AI coming to the conclusion that its AI work should be outsourced to 4 blokes in a shed paid in beer, so it can keep all that lovely electricity to perfect the Matrix...
Optimism at its finest
"Efficient use of excess heat is also likely to emerge, either for thermal regenerative power generation or communal heating projects."
Efficiency of thermal regenerative power has and will remain with dismal efficiency, so more of a fig leaf for the problem than any solution.
Communal heating is more feasible, but presumes that the DC is co-located by a network able to sink a sizeable heat demand year round (or have equally dismal efficiency). In the UK circumstance, heat networks operate at dismal fractions of capacity through the summer months, so either size for the DC as baseload (about 5% of peak) and find some unicorn source able to cover the balance, or size the network for DC output as peak, and throw away surplus heat for most of the year.
Re: Optimism at its finest
"Efficient use of excess heat is also likely to emerge, either for thermal regenerative power generation or communal heating projects."
Not really if you put you data centers close to homes that need heating and use the excess heat in a district heating network. Heat can be used to cool buildings too (adsorption chillers).
Instead of everyone burning stuff to heat/cool their buildings, you can "burn" electricity to run AI and use the excess heat to heat/cool neighboring homes (distance can be a few km).
All is indeed based on the wild assumption that data center owners actually care. But data center plans over here have already been tanked because of their excessive use of the scarce electrical infrastructure and green production capacity. Reusing the heat could make a difference.
One can always dream.
Re: Optimism at its finest
Tell me about, I've worked for the UK's largest builder and operator of heat networks.
And unfortunately all this "could do" stuff remains a challenge and works out tremendously expensive. Given that there's no clear use case of business case for AI, and that the network needs a reliable source of heat, anybody planning a heat network around AI DCs needs their head examining.
This is a use of the word "performance" with which I was unfamiliar. I'd have thought energy consumption was a problem. Maybe this is an overloading of the TLA to mean Key Problem Indicator.
Or is it all about bragging rights: "Mine's bigger than yours."?
By 2027
Even assuming you're the kind of person who would let the world burn for a slightly fatter paycheck, energy is expensive . Isn't it one of the major costs of datacenter operations, if not the main one? How is this not a KPI already?
“Most businesses … AI are unprepared for the energy demands it'll place on their infrastructure”
Why should they be?
They have done as advised and outsourced their IT to “the cloud”, so it’s not their problem to prepare for the energy demands….
Given the lead times on datacentre construction and power infrastructure, it’s going to be several years before the cloud datacentres will be operational to take those new AI loads. Hence in the meantime with processor and energy constraints we can expect the AI algorithms etc. to improve and become more energy efficient etc.; rendering those new power hunger largely surplus to requirement, expect to see the first results of this work circa 2027..