Digital Realty CTO weighs in on AI's insatiable thirst for power
(2024/06/05)
- Reference: 1717541647
- News link: https://www.theregister.co.uk/2024/06/04/datacenter_power_ai/
- Source link:
Interview It's no secret the GPUs used to train and run generative AI models are power hungry little beasts.
So hungry in fact that by some estimates, datacenters in the US could use as much as [1]nine percent of the nation's electricity generation by the end of the decade. In light of this explosive growth, some have warned that AI infrastructure could [2]drain the grid .
Chris Sharp, CTO at colocation provider Digital Realty, is well aware of the challenges associated with accommodating these workloads.
[3]
Compared to the servers that run traditional workloads, such as virtual machines, containers, storage, and databases, hardware-accelerated AI is a different animal. A single rack of GPU servers today can easily consume 40 kW or more. Next-gen rack-scale systems from Nvidia and others will [4]require at least 100 kW.
[5]
According to Sharp, accommodating these demanding systems at scale isn't easy and requires a different way of thinking about datacenter power and cooling, which you can learn more about in this interview with The Register below.
[6]Youtube Video
[7]
It's possible datacenters could end up looking wildly different. Sharp suggests small nuclear reactors (SMRs) and other primary onsite power generation may play a role. ®
Get our [8]Tech Resources
[1] https://www.epri.com/about/media-resources/press-release/q5vU86fr8TKxATfX8IHf1U48Vw4r1DZF
[2] https://www.theregister.com/2024/03/27/ceo_of_uks_national_grid/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Zl-i5H2I2zNRPMmDxkQQqgAAANM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://www.theregister.com/2024/03/21/nvidia_dgx_gb200_nvk72/
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Zl-i5H2I2zNRPMmDxkQQqgAAANM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://www.youtube.com/watch?v=GgcHkC5LH-s
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Zl-i5H2I2zNRPMmDxkQQqgAAANM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] https://whitepapers.theregister.com/
So hungry in fact that by some estimates, datacenters in the US could use as much as [1]nine percent of the nation's electricity generation by the end of the decade. In light of this explosive growth, some have warned that AI infrastructure could [2]drain the grid .
Chris Sharp, CTO at colocation provider Digital Realty, is well aware of the challenges associated with accommodating these workloads.
[3]
Compared to the servers that run traditional workloads, such as virtual machines, containers, storage, and databases, hardware-accelerated AI is a different animal. A single rack of GPU servers today can easily consume 40 kW or more. Next-gen rack-scale systems from Nvidia and others will [4]require at least 100 kW.
[5]
According to Sharp, accommodating these demanding systems at scale isn't easy and requires a different way of thinking about datacenter power and cooling, which you can learn more about in this interview with The Register below.
[6]Youtube Video
[7]
It's possible datacenters could end up looking wildly different. Sharp suggests small nuclear reactors (SMRs) and other primary onsite power generation may play a role. ®
Get our [8]Tech Resources
[1] https://www.epri.com/about/media-resources/press-release/q5vU86fr8TKxATfX8IHf1U48Vw4r1DZF
[2] https://www.theregister.com/2024/03/27/ceo_of_uks_national_grid/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Zl-i5H2I2zNRPMmDxkQQqgAAANM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://www.theregister.com/2024/03/21/nvidia_dgx_gb200_nvk72/
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Zl-i5H2I2zNRPMmDxkQQqgAAANM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://www.youtube.com/watch?v=GgcHkC5LH-s
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Zl-i5H2I2zNRPMmDxkQQqgAAANM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] https://whitepapers.theregister.com/
TLDW:
Datacentres are going to slurp shit-tonnes of power and shit-tonnes of water in order to run completely useless stochastic bullshit generators that have become the latest fad.
The sector is "growing exponentially" without any real purpose, a bit like er, a tumour.
But it's OK because we will be using Renewables and Nuclear!
(Just don't mention that the Nuclear probably won't materialise, and when the renewables go behind a cloud we'll be gobbling up gas and Diesel, belching out emissions, and sending the price of energy through the roof - all so that your job can be replaced by a statistical model of your past actions and ideas, which is cheaper than you in the short term, but can't actually innovate in the long term)
Then there's the awkward question asked by the Reg hack about GPU financing and all of this being a potentially catastrophic bubble ...
I can imagine this guy selling railway bonds ...