Racks of AI Chips Are Too Damn Heavy (theverge.com)
- Reference: 0180398601
- News link: https://tech.slashdot.org/story/25/12/16/198220/racks-of-ai-chips-are-too-damn-heavy
- Source link: https://www.theverge.com/ai-artificial-intelligence/844966/heavy-ai-data-center-buildout
AI racks are projected to reach 5,000 pounds compared to the 400 to 600 pounds that racks weighed three decades ago. The dramatic increase stems from hundreds to 1,000 GPUs packed densely into each rack alongside memory chips and liquid cooling systems that can add substantial weight. AI workloads now consume up to 350 kilowatts per rack, 35 times the 10 kilowatts that traditional computer chip workloads averaged a decade ago. Legacy data centers with raised floors typically max out at around 1,250 pounds per square foot for static loads.
Chris McLean, president of Critical Facility Group, said that rack heights have grown from 6 feet to 9 feet over nearly two decades, creating problems with doorframes and freight elevators in older buildings.
[1] https://www.theverge.com/ai-artificial-intelligence/844966/heavy-ai-data-center-buildout
And just think... (Score:3, Funny)
If they are able to achieve their goals with the AI, then in another seven or eight years they'll finally have figured out how to optimize personalized mattress sales.
ANGH (Score:2)
Absolutely not gonna' happen in reality since the people with the actual money are now realizing they're not going to get a return on what they thought was an investment.
just ask (Score:1)
I bet if they just ask ChatGPT it will have the answer.
Are there engineers working there? (Score:1, Interesting)
I'm not sure why there aren't competent engineers working at these places. They keep making the dumbest decisions possible, it seems.
Like, put the datacenter underground and it doesn't need as much cooling.
Put the datacenter in an underground chamber built like a yakchal and it needs even less cooling.
The problem is that the datacenter engineers don't seem to know very much about how to build things, as evidenced by their continued massive use of power and freshwater. And their validation of the classic "th
Re: (Score:1)
If you run computers underground they don't make heat? What if they are below sea level? Datacenters underground in Death Valley FTW!
Re: Are there engineers working there? (Score:5, Insightful)
What are you thinking? You can't just magic the heat away by building underground
Re: (Score:1)
Random internet commenter incorrectly assumes that they've discovered a silver bullet in their five-minute analysis that the experts have simply missed.
Must be a day ending in Y.
Re: (Score:2)
Underground? You'll have to pump air in there or your staff will die.
And you'll need to send the waste heat somewhere, maybe into the walls but the square-cube law indicates that this is not scalable. A really big data center will need to pump waste heat out through a medium, liquid or gas. Water is the obvious choice. So your underground data center ought to have a hyperboloid cooling tower sitting above it. Seems kind of pointless to have dug that big hole after all that.
Re: (Score:2)
I've never seen a more Dunning-Kruger post in my entire history on Slashdot, than what you just wrote there. You've in 4 sentences called engineers incompetent while demonstrating you don't know how cooling works, and proposed a solution that violates the basic physics behind conservation of energy.
Additionally you claim engineers forgot about weight when they literally didn't, and this article points out that weight very much has always been a fundamental consideration.
Please go home and sober up before po
Re: (Score:2)
How does putting it underground mean less heat? It gets hotter the lower you go, and on top of that the problem is heat generated by the servers themselves - where's it supposed to go in a room that's that well insulated?
If you'd had suggested building them atop mountains, you might have been on to something, but those places are typically hard to reach and thus construct upon, and you'd also still need to power them and, of course, connect them to the Internet.
Sounds like we need new data centers (Score:2)
Good thing they are building them
Re: (Score:2)
One could use heat exchangers such that the cooling water and the sea water never intermix.
Re: (Score:2)
What would you build the heat exchangers out of? Materials that don't corrode (like plastic) tend to not be very good conductors of heat.
Minor problems, from what I'm reading. (Score:4, Insightful)
> AI racks are projected to reach 5,000 pounds ...
... spread over probably 24" x 48", or 8 square feet, for a total of 625 pounds per square foot.
> Legacy data centers with raised floors typically max out at around 1,250 pounds per square foot for static loads.
Ignoring that the numbers above are probably less than half the static load limit, the static load limit is as low as it is because A. the raised floor has weight, and B. the raised floor likely has a much lower weight limit than the concrete slab under it.
Solution: Remove the raised floor.
> ... rack heights have grown from 6 feet to 9 feet over nearly two decades ...
Which means you can now run overhead cable trays above the height of your tallest employees. No need for the raised floors. Also, by ripping out the raised floors, you can have that extra height, so no need to rebuild the building.
> ... creating problems with doorframes and freight elevators in older buildings ...
Because people can't tip the racks up to get them through the doors, or lie them flat corner-to-corner in a freight elevator? This seems like a problem with movers not being creative enough when moving things, rather than a building problem.
This whole article reads like people looking for an excuse to spend more money and build monuments to themselves, rather than an actual problem. What am I missing?
The AI business is a grift (Score:2)
Consider it an investment in UFO science.
Re: (Score:1)
Usually the raised floor contains HVAC / cable runs. When I worked at rackshack back in the day, we had cooling towers that we could add that plugged into the floor panels. so... remove the floor and you would basically have to redo the entire datacenter.. cheaper to build a new one probablly. Still, I imagine they could find hardpoints and build up a frame, or use some kind of crane / suspension solution if they were locked into that location
This got me thinking (Score:3)
All those GPU chips, all those "AI factories"
I wonder what the invoice price for a 53' trailer of these GPU chips would be? Almost worth....a Netflix movie idea.
Missed Opportunity (Score:2)
> Chris McLean, president of Critical Facility Group, said that rack heights have grown from 6 feet to 9 feet over nearly two decades, creating problems with doorframes and freight elevators in older buildings.
Missed a perfect opportunity for a better headline: "The Rack is Too...Damn...High" [ [1]ref [youtube.com]]
[1] https://www.youtube.com/results?search_query=the+rent+is+too+damn+high
Damn... (Score:4, Interesting)
... and I thought the VAX6000 was a heavy power hog.
Yes, New Buildings (Score:5, Informative)
New buildings for AI datacenters are clearly the plan. The weight per rack and the rack height are minor considerations compared to the electricity and cooling requirements. This is obviously the approach Tesla is taking, having custom-built the Cortex datacenter, and now with construction well underway for Cortex 2.
With 350kW/rack, a large datacenter is looking at hundreds of MW. No wonder Tesla had to build a new substation for Cortex 2. I haven't looked at what the other companies are doing, but it can't be that different.
Re: (Score:3, Insightful)
> No wonder Tesla had to build a new substation for Cortex 2
They should be building entire power plants on-site. If they can't build a power plant at a given site, then build the whole thing, power plant and battery storage included, somewhere else.
An alternative to on-site power is a power plant that's close by (within tens of miles) then build dedicated power-transmission lines from the power plant to the data center.
Both solutions avoid the "we are going to bust the electrical grid with our power demands" problem.
Re: (Score:3)
I saw a video complaining they did just that. The problem was that the power plant was a bunch of mobile gas-powered generators and it played havoc with people living nearby, noise and air pollution.
If they'd declared it up front, it'd have been less of an issue, because they'd have been forced to build a little further from the city and probably given clean air mandates to adhere to. But Musk is cheap and figured out he could bypass that by pretending it was just a data center, and then bringing in mobile
Re: (Score:2)
> No wonder Tesla had to build a new substation for Cortex 2.
Err datacentres have literally always needed their own dedicated substations. The bar required for building substations is actually ludicrously low. Heck some commercial buildings without datacentres need their own substations. You pulling more than 1-2MW then having a dedicated substation is almost a necessity, unless you're feeding that power into a single load at the originally supplied voltage.
That's not to diminish your astonishment. No doubt Cortex 2 will draw a shitton of power, but the word substati