AI's thirst for water is alarming, but may solve itself
(2024/09/05)
- Reference: 1725525187
- News link: https://www.theregister.co.uk/2024/09/05/ai_water_energy/
- Source link:
Comment Once an abstract subject of science fiction and academic research, the concept of artificial intelligence has become the topic of dinner table conversations over the past two years.
This shift has brought widespread awareness of the environmental implications of this technology, most prominently centered on the massive sums of power and water required to train and deploy these models. And it's understandable why.
A recent [1]report found that datacenter water consumption in Northern Virginia, the bit barn capital of the world, had increased by two-thirds over the past five years.
[2]
"ChatGPT needs to 'drink' a 500 ml bottle of water for a simple conversation of roughly 20-50 questions and answers, depending on when and where ChatGPT is deployed," the researchers estimated in a paper [3]published early last year.
[4]
[5]
To make matters worse, that was for a GPT-3-class model measuring roughly 175 billion parameters, a figure that feels positively tiny by today's standards. GPT-4 is estimated to be somewhere between 1.7 and 1.8 trillion parameters in size, and, as OpenAI's Trevor Cai put it in his Hot Chips keynote last week, these models are only going to get bigger.
While that doesn't bode well for datacenter power consumption, the same may not be true for its H 2 O addiction. At least, it doesn't have to be.
[6]
First, let's get something out of the way before the comments point out the obvious. Datacenters don't really consume water. The real problem is that water is being removed from the local environment rather than returned to its source. Second, the IT infrastructure, AI related or otherwise, isn't actually what's consuming the water.
Even when liquid cooled, these systems are usually closed loops that lose little if any appreciable quantity of fluids during normal operation. What's actually gobbling up all that H 2 O is the bit barn's air handlers, often called evaporative or swamp coolers, used to keep those systems from overheating.
However, it's important to note that this is a design decision and evaporative coolers aren't used in every facility. If Meta or Amazon are setting up an AI datacenter in your backyard, they will put a strain on your local power grid, but it doesn't necessarily mean they're going to [7]suck up a quarter of your town's water supply like Google does out in The Dalles, Oregon.
[8]
In colder climates, dry coolers and so-called "free cooling" are adequate, while in hotter, drought-prone regions, it's not uncommon to see DC operators opt for refrigerant-based systems. Last we heard, that's exactly what Microsoft is doing with its DC developments in Goodyear, Arizona, albeit only after a [9]wastewater dispute with the city.
Although there are alternatives to evaporative cooling, many come at the expense of higher power consumption, a commodity already in short supply, as CBRE recently [10]reported .
While there's only so much to be done about existing DC facilities, the decision to employ evaporative cooling in new builds ultimately comes down to greed, or, to use the politically correct parlance, capitalism.
For hyperscalers in particular, everything eventually comes down to margins. If you can do something 5 percent cheaper or more efficiently than the competition, you can make that much more in profits, or undercut them and win over their customer base. And water just happens to be incredibly efficient at stripping heat from the air compared to alternative technologies. That means lower electricity costs or the ability to build larger, denser facilities in power-constrained locales.
Even at industrial rates, electricity costs add up quickly. So, in markets where evaporative coolers are viable, the technology offers a competitive advantage.
The argument can also be made that the evaporative cooler's water consumption is a worthwhile trade-off if it means burning fewer fossil fuels to keep the lights on, but that's heavily dependent on location. The nature of evaporative cooling means that they're most efficient in arid climates where water is already a scarce resource.
Ultimately, it really boils down to this: you can either use more power or consume more water. If water is cheaper than power, and better yet, perceived to be plentiful such as around the Great Lakes, you can guess which operators are going to choose.
[11]Google's Irish bit barn plans denied over eco shortfall
[12]Meta digs deep to strike geothermal power deal for its US datacenters
[13]LiquidStack says its new CDU can chill more than 1MW of AI compute
[14]LG Electronics aims to become a datacenter cooling player, with aircon and immersion tech
However, this may be changing. The pace of AI innovation doesn't look like it's going to let up any time soon. In this climate, we've seen chips grow ever hotter, passing the one kilowatt mark, and driving a transition to liquid cooling.
Nvidia's Grace Blackwell Superchips that we [15]looked at back at GTC are rated for 2,700 W with two of them designed to fit into a single RU chassis. To accommodate this incredibly dense package, Nvidia unsurprisingly opted for direct liquid cooling (DLC).
While arguably better in terms of operation efficiency – DLC is substantially more [16]energy efficient than burning energy on fans – it also poses major headaches for datacenter operators as many older facilities can't be retrofitted to accommodate this technology easily.
While this is a headache for some, widespread adoption of liquid cooling, perhaps ironically, has the potential to cut water consumption in the long run. That's because the higher thermal coefficient of liquid cooling allows the use of dry coolers, which work a bit like a car's radiator but on an industrial scale.
There's also the potential for heat reuse in these scenarios. In one thought experiment [17]presented at SC23, it was estimated that training a GPT-3-sized model could generate enough heat to support roughly 4.6 greenhouses and grow over a million tomatoes. We've seen other [18]examples of datacenters contributing to district heating grids as well.
However, until a critical mass of liquid-cooled systems have been deployed, we're likely to continue seeing headlines about datacenter water consumption for better or worse. ®
Get our [19]Tech Resources
[1] https://www.theregister.com/2024/08/19/virginia_datacenter_water_consumption/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZtmBSConb2P5fVKwFPcSLgAAAdg&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://www.theregister.com/2023/05/15/ai_water_datacenter/
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZtmBSConb2P5fVKwFPcSLgAAAdg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZtmBSConb2P5fVKwFPcSLgAAAdg&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZtmBSConb2P5fVKwFPcSLgAAAdg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2022/12/19/google_datacenters_dalles/
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZtmBSConb2P5fVKwFPcSLgAAAdg&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[9] https://www.theregister.com/2023/04/11/microsoft_evaporative_coolers_arizona/
[10] https://www.theregister.com/2024/08/21/dc_na_boom/
[11] https://www.theregister.com/2024/08/27/google_data_center_ireland/
[12] https://www.theregister.com/2024/08/27/meta_geothermal_energy/
[13] https://www.theregister.com/2024/08/22/liquidstack_cdu_ai/
[14] https://www.theregister.com/2024/08/22/lg_electronics_datacenter_cooling_2030_vision/
[15] https://www.theregister.com/2024/03/18/nvidia_turns_up_the_ai/?td=rt-9c
[16] https://www.theregister.com/2023/12/26/thermal_management_is_changing/
[17] https://www.nextplatform.com/2023/11/17/pushing-the-limits-of-hpc-and-ai-is-becoming-a-sustainability-headache/
[18] https://www.theregister.com/2024/05/21/google_dc_finland_heating/
[19] https://whitepapers.theregister.com/
This shift has brought widespread awareness of the environmental implications of this technology, most prominently centered on the massive sums of power and water required to train and deploy these models. And it's understandable why.
A recent [1]report found that datacenter water consumption in Northern Virginia, the bit barn capital of the world, had increased by two-thirds over the past five years.
[2]
"ChatGPT needs to 'drink' a 500 ml bottle of water for a simple conversation of roughly 20-50 questions and answers, depending on when and where ChatGPT is deployed," the researchers estimated in a paper [3]published early last year.
[4]
[5]
To make matters worse, that was for a GPT-3-class model measuring roughly 175 billion parameters, a figure that feels positively tiny by today's standards. GPT-4 is estimated to be somewhere between 1.7 and 1.8 trillion parameters in size, and, as OpenAI's Trevor Cai put it in his Hot Chips keynote last week, these models are only going to get bigger.
While that doesn't bode well for datacenter power consumption, the same may not be true for its H 2 O addiction. At least, it doesn't have to be.
[6]
First, let's get something out of the way before the comments point out the obvious. Datacenters don't really consume water. The real problem is that water is being removed from the local environment rather than returned to its source. Second, the IT infrastructure, AI related or otherwise, isn't actually what's consuming the water.
Even when liquid cooled, these systems are usually closed loops that lose little if any appreciable quantity of fluids during normal operation. What's actually gobbling up all that H 2 O is the bit barn's air handlers, often called evaporative or swamp coolers, used to keep those systems from overheating.
However, it's important to note that this is a design decision and evaporative coolers aren't used in every facility. If Meta or Amazon are setting up an AI datacenter in your backyard, they will put a strain on your local power grid, but it doesn't necessarily mean they're going to [7]suck up a quarter of your town's water supply like Google does out in The Dalles, Oregon.
[8]
In colder climates, dry coolers and so-called "free cooling" are adequate, while in hotter, drought-prone regions, it's not uncommon to see DC operators opt for refrigerant-based systems. Last we heard, that's exactly what Microsoft is doing with its DC developments in Goodyear, Arizona, albeit only after a [9]wastewater dispute with the city.
Although there are alternatives to evaporative cooling, many come at the expense of higher power consumption, a commodity already in short supply, as CBRE recently [10]reported .
While there's only so much to be done about existing DC facilities, the decision to employ evaporative cooling in new builds ultimately comes down to greed, or, to use the politically correct parlance, capitalism.
For hyperscalers in particular, everything eventually comes down to margins. If you can do something 5 percent cheaper or more efficiently than the competition, you can make that much more in profits, or undercut them and win over their customer base. And water just happens to be incredibly efficient at stripping heat from the air compared to alternative technologies. That means lower electricity costs or the ability to build larger, denser facilities in power-constrained locales.
Even at industrial rates, electricity costs add up quickly. So, in markets where evaporative coolers are viable, the technology offers a competitive advantage.
The argument can also be made that the evaporative cooler's water consumption is a worthwhile trade-off if it means burning fewer fossil fuels to keep the lights on, but that's heavily dependent on location. The nature of evaporative cooling means that they're most efficient in arid climates where water is already a scarce resource.
Ultimately, it really boils down to this: you can either use more power or consume more water. If water is cheaper than power, and better yet, perceived to be plentiful such as around the Great Lakes, you can guess which operators are going to choose.
[11]Google's Irish bit barn plans denied over eco shortfall
[12]Meta digs deep to strike geothermal power deal for its US datacenters
[13]LiquidStack says its new CDU can chill more than 1MW of AI compute
[14]LG Electronics aims to become a datacenter cooling player, with aircon and immersion tech
However, this may be changing. The pace of AI innovation doesn't look like it's going to let up any time soon. In this climate, we've seen chips grow ever hotter, passing the one kilowatt mark, and driving a transition to liquid cooling.
Nvidia's Grace Blackwell Superchips that we [15]looked at back at GTC are rated for 2,700 W with two of them designed to fit into a single RU chassis. To accommodate this incredibly dense package, Nvidia unsurprisingly opted for direct liquid cooling (DLC).
While arguably better in terms of operation efficiency – DLC is substantially more [16]energy efficient than burning energy on fans – it also poses major headaches for datacenter operators as many older facilities can't be retrofitted to accommodate this technology easily.
While this is a headache for some, widespread adoption of liquid cooling, perhaps ironically, has the potential to cut water consumption in the long run. That's because the higher thermal coefficient of liquid cooling allows the use of dry coolers, which work a bit like a car's radiator but on an industrial scale.
There's also the potential for heat reuse in these scenarios. In one thought experiment [17]presented at SC23, it was estimated that training a GPT-3-sized model could generate enough heat to support roughly 4.6 greenhouses and grow over a million tomatoes. We've seen other [18]examples of datacenters contributing to district heating grids as well.
However, until a critical mass of liquid-cooled systems have been deployed, we're likely to continue seeing headlines about datacenter water consumption for better or worse. ®
Get our [19]Tech Resources
[1] https://www.theregister.com/2024/08/19/virginia_datacenter_water_consumption/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZtmBSConb2P5fVKwFPcSLgAAAdg&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://www.theregister.com/2023/05/15/ai_water_datacenter/
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZtmBSConb2P5fVKwFPcSLgAAAdg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZtmBSConb2P5fVKwFPcSLgAAAdg&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZtmBSConb2P5fVKwFPcSLgAAAdg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2022/12/19/google_datacenters_dalles/
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZtmBSConb2P5fVKwFPcSLgAAAdg&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[9] https://www.theregister.com/2023/04/11/microsoft_evaporative_coolers_arizona/
[10] https://www.theregister.com/2024/08/21/dc_na_boom/
[11] https://www.theregister.com/2024/08/27/google_data_center_ireland/
[12] https://www.theregister.com/2024/08/27/meta_geothermal_energy/
[13] https://www.theregister.com/2024/08/22/liquidstack_cdu_ai/
[14] https://www.theregister.com/2024/08/22/lg_electronics_datacenter_cooling_2030_vision/
[15] https://www.theregister.com/2024/03/18/nvidia_turns_up_the_ai/?td=rt-9c
[16] https://www.theregister.com/2023/12/26/thermal_management_is_changing/
[17] https://www.nextplatform.com/2023/11/17/pushing-the-limits-of-hpc-and-ai-is-becoming-a-sustainability-headache/
[18] https://www.theregister.com/2024/05/21/google_dc_finland_heating/
[19] https://whitepapers.theregister.com/
Re: Rain
cyberdemon
Well, it's not much good if you've drained the local aquifers for your datacentre and the rain falls elsewhere, or in the sea, is it
Look up the process of desertification, covered in GCSE Geography
Excessive draining of an aquifer means plants die out and no longer contribute to local rainfall, and then nothing grows in that area again
Plants (unlike datacentres) also have the ability to trap and store water. If rain falls heavily without them then it doesn't replenish the aquifers, it just flows out rapidly to sea
Re: Rain
elsergiovolador
So the nature will sort it out.
If desert gets created, datacentre no longer can cool itself off and burns down, then the nature comes back.
Re: Rain
Filippo
Causing damage does not become justifiable only because the damage is temporary. Time has intrinsic value.
Also, if "temporary" is in the same order of human lifespans or more, then it's actually permanent for all effects and purposes to all the people involved.
AI units: Greehouses and Tomatoes
b0llchit
...training a GPT-3-sized model could generate enough heat to support roughly 4.6 greenhouses and grow over a million tomatoes.
Refraining from training a GPT-3-sized model would enable you to support roughly 9.2 greenhouses and grow several million tomatoes.
The real question then is: What did we get for all the greenhouses and tomatoes? An artificial conversation on how many greenhouses were spared and tomatoes weren't spoiled?
Rain
Did the people concerned skip school?
If water evaporates, it doesn't get removed from environment - it is released back to the environment.
You know, water doesn't go to space, it comes back as something called "rain".
Rain is when you see droplets of water coming down from the sky and when you get wet if you don't get an umbrella -ella -ella