Dell exec reveals Nvidia has a 1,000-watt GPU in the works
(2024/03/05)
- Reference: 1709623627
- News link: https://www.theregister.co.uk/2024/03/05/nvidias_b100_gpu_1000w/
- Source link:
If you thought Nvidia's 700W H100s were hot and power-hungry machines, just wait until the GPU slinger's B100 arrives later this year.
According to Dell Technologies COO Jeff Clarke, Nvidia's latest AI accelerator will consume 1,000 watts – 42 percent more than its predecessor. But don't worry, he's pretty sure liquid cooling won’t be required to tame the beast.
"We're excited about what's happening with the H200 and its performance improvement," Clarke told investors on Dell's [1]earnings call [PDF] last week, before adding he feels the same emotion about Nvidia's forthcoming B100 accelerator and another he referred to as the B200.
[2]
He opined that direct liquid cooling won't be needed to handle GPUs that consume 1,000 watts apiece – a level he said "happens next year with the B200."
[3]
[4]
It's not entirely clear what card Clarke is referring to with the "B200," since no chip by that moniker appears on the roadmap Nvidia [5]shared with investors last fall. However, we suspect Clarke is actually referring to the GB200 Superchip which, like the [6]GH200 , is expected to combine Nvidia's Grace CPU with its B100 GPU.
[7]
According to an investor presentation released this month, Nvidia plans to shift to a one-year release cadence – Click to enlarge
Based on what we know of the Grace CPU in the GH200, and assuming no major changes in power consumption, that would put the GB200's thermal design power (TDP) somewhere in the neighborhood of 1,300 watts – 30 percent higher than its predecessor.
It's also possible that Nvidia has another card up its sleeve that we don't know about yet. Details of the GPU giant's next-gen Blackwell architecture remain scanty.
Nomenclature aside, Clarke suggested the forthcoming chip would provide an opportunity to showcase Dell's expertise in other forms of liquid cooling at scale. He referred to "things in fluid chemistry and performance, our interconnect work, the telemetry we are doing, the power management work we're doing" as steps toward alternatives to direct liquid cooling, even for very dense chips.
[8]
Nvidia declined to comment – as you'd expect, given its annual GTC conference is only a few weeks away. The Register will be onsite at the event to bring you all the details when they drop.
[9]HPE blames GPU shortage for contributing to unexpected sales slide
[10]Baidu admits it may never get leading-edge GPUs again
[11]Nvidia talks up local AI with RTX 500, 1000 Ada mobile GPUs
[12]Nvidia wants a piece of the custom silicon pie, reportedly forms unit to peddle IP
The B100 isn't expected to launch until late 2024 after Nvidia's [13]bandwidth juiced H200 GPUs debut in the first half of the year.
Announced in late 2023, the H200 is a refresh of the H100 with up to 141GB of HBM3e memory that's good for a whopping 4.8TB/sec of bandwidth. Nvidia [14]claims the device can double the performance of large language models including Llama 70B, thanks to the chip's HBM3e memory stacks.
Even with two new accelerators slated to hit the market this year, analysts warn Nvidia's supply of GPUs will remain supply constrained. That's despite [15]reports predicting Nvidia could move more than triple shipments of GPUs in 2024.
Beyond its new accelerators, Nvidia's [16]roadmap also calls for faster, more capable InfiniBand and Ethernet NICs and switches capable of 800Gb/sec of bandwidth per port before the year is out. ®
Get our [17]Tech Resources
[1] https://investors.delltechnologies.com/static-files/dcbb932e-8e25-49a9-a508-61e454f45ce5
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Zeb7XOkNA7D89yBABjugggAAAMY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Zeb7XOkNA7D89yBABjugggAAAMY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Zeb7XOkNA7D89yBABjugggAAAMY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2023/10/13/nvidias_accelerated_release_cadence_spells/
[6] https://www.theregister.com/2023/08/09/nvidia_gracehopper_hbm3e/
[7] https://regmedia.co.uk/2023/10/13/nvidia_accelerated_roadmap.jpg
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Zeb7XOkNA7D89yBABjugggAAAMY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://www.theregister.com/2024/03/01/hpe_q1_2024/
[10] https://www.theregister.com/2024/02/29/baidu_subpar_chips/
[11] https://www.theregister.com/2024/02/26/nvidia_ai_pc_gpus/
[12] https://www.theregister.com/2024/02/09/nvidia_custom_silicon/
[13] https://www.nextplatform.com/2023/11/13/nvidia-pushes-hopper-hbm-memory-and-that-lifts-gpu-performance/
[14] https://www.theregister.com/2023/12/21/nvidia_amd_benchmarks/
[15] https://www.nextplatform.com/2024/01/22/expect-datacenters-to-get-denser-hotter-and-smarter/
[16] https://www.theregister.com/2023/10/13/nvidias_accelerated_release_cadence_spells/
[17] https://whitepapers.theregister.com/
According to Dell Technologies COO Jeff Clarke, Nvidia's latest AI accelerator will consume 1,000 watts – 42 percent more than its predecessor. But don't worry, he's pretty sure liquid cooling won’t be required to tame the beast.
"We're excited about what's happening with the H200 and its performance improvement," Clarke told investors on Dell's [1]earnings call [PDF] last week, before adding he feels the same emotion about Nvidia's forthcoming B100 accelerator and another he referred to as the B200.
[2]
He opined that direct liquid cooling won't be needed to handle GPUs that consume 1,000 watts apiece – a level he said "happens next year with the B200."
[3]
[4]
It's not entirely clear what card Clarke is referring to with the "B200," since no chip by that moniker appears on the roadmap Nvidia [5]shared with investors last fall. However, we suspect Clarke is actually referring to the GB200 Superchip which, like the [6]GH200 , is expected to combine Nvidia's Grace CPU with its B100 GPU.
[7]
According to an investor presentation released this month, Nvidia plans to shift to a one-year release cadence – Click to enlarge
Based on what we know of the Grace CPU in the GH200, and assuming no major changes in power consumption, that would put the GB200's thermal design power (TDP) somewhere in the neighborhood of 1,300 watts – 30 percent higher than its predecessor.
It's also possible that Nvidia has another card up its sleeve that we don't know about yet. Details of the GPU giant's next-gen Blackwell architecture remain scanty.
Nomenclature aside, Clarke suggested the forthcoming chip would provide an opportunity to showcase Dell's expertise in other forms of liquid cooling at scale. He referred to "things in fluid chemistry and performance, our interconnect work, the telemetry we are doing, the power management work we're doing" as steps toward alternatives to direct liquid cooling, even for very dense chips.
[8]
Nvidia declined to comment – as you'd expect, given its annual GTC conference is only a few weeks away. The Register will be onsite at the event to bring you all the details when they drop.
[9]HPE blames GPU shortage for contributing to unexpected sales slide
[10]Baidu admits it may never get leading-edge GPUs again
[11]Nvidia talks up local AI with RTX 500, 1000 Ada mobile GPUs
[12]Nvidia wants a piece of the custom silicon pie, reportedly forms unit to peddle IP
The B100 isn't expected to launch until late 2024 after Nvidia's [13]bandwidth juiced H200 GPUs debut in the first half of the year.
Announced in late 2023, the H200 is a refresh of the H100 with up to 141GB of HBM3e memory that's good for a whopping 4.8TB/sec of bandwidth. Nvidia [14]claims the device can double the performance of large language models including Llama 70B, thanks to the chip's HBM3e memory stacks.
Even with two new accelerators slated to hit the market this year, analysts warn Nvidia's supply of GPUs will remain supply constrained. That's despite [15]reports predicting Nvidia could move more than triple shipments of GPUs in 2024.
Beyond its new accelerators, Nvidia's [16]roadmap also calls for faster, more capable InfiniBand and Ethernet NICs and switches capable of 800Gb/sec of bandwidth per port before the year is out. ®
Get our [17]Tech Resources
[1] https://investors.delltechnologies.com/static-files/dcbb932e-8e25-49a9-a508-61e454f45ce5
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Zeb7XOkNA7D89yBABjugggAAAMY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Zeb7XOkNA7D89yBABjugggAAAMY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Zeb7XOkNA7D89yBABjugggAAAMY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2023/10/13/nvidias_accelerated_release_cadence_spells/
[6] https://www.theregister.com/2023/08/09/nvidia_gracehopper_hbm3e/
[7] https://regmedia.co.uk/2023/10/13/nvidia_accelerated_roadmap.jpg
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Zeb7XOkNA7D89yBABjugggAAAMY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://www.theregister.com/2024/03/01/hpe_q1_2024/
[10] https://www.theregister.com/2024/02/29/baidu_subpar_chips/
[11] https://www.theregister.com/2024/02/26/nvidia_ai_pc_gpus/
[12] https://www.theregister.com/2024/02/09/nvidia_custom_silicon/
[13] https://www.nextplatform.com/2023/11/13/nvidia-pushes-hopper-hbm-memory-and-that-lifts-gpu-performance/
[14] https://www.theregister.com/2023/12/21/nvidia_amd_benchmarks/
[15] https://www.nextplatform.com/2024/01/22/expect-datacenters-to-get-denser-hotter-and-smarter/
[16] https://www.theregister.com/2023/10/13/nvidias_accelerated_release_cadence_spells/
[17] https://whitepapers.theregister.com/
Re: Bitcoin mining 2.0?
A Non e-mouse
We've been looking into some local commercial data centres. They're quoting us a maximum power budget of 6kW per rack. I know you wouldn't put these into a normal data centre, but 1kW for just one card is getting rather silly.
A Non e-mouse
Assuming the cards run at 5V, they're going to be pulling in the region of 200 amps. That's a crazy number. (even with 12V it's still a mind boggling 80 amps)
Turn it up to 11
m4r35n357
The US approach to everything.
An affront to the very concept of engineering.
Bitcoin mining 2.0?
I can't help but be reminded of how the power consumption really got out of hand on the mining front - only this time, it's Nvidia itself that is apparently fuelling the flames. Energy efficiency might not be what people are looking for in these accelerators, but this still sounds rather extreme.