Nvidia rival Cerebras says it's revived Moore's Law with third-gen waferscale chips
(2024/03/13)
- Reference: 1710334814
- News link: https://www.theregister.co.uk/2024/03/13/cerebras_claims_to_have_revived/
- Source link:
Cerebras revealed its latest dinner-plate sized AI chip on Wednesday, which it claims offers twice the performance per watt of its predecessor, alongside a collaboration with Qualcomm aimed at accelerating machine learning inferencing.
The chip, dubbed the WSE-3, is Cerebras' third-gen waferscale processor and measures in at a whopping 46,225mm 2 (that's about 71.6 inches 2 in freedom units.) The 4 trillion transistor part is fabbed on TSMC's 5nm process and is imprinted with 900,000 cores and 44GB of SRAM, good for 125 AI petaFLOPS of performance, which in this case refers to highly sparse FP16 — more on that in a minute.
[1]
Cerebras claims its CS-3 Systems are twice as fast as its predecessor (click to enlarge)
A single WSE-3 forms the basis of Cerebras' new CS-3 platform, which it claims offers 2x higher performance, while consuming the same 23kW as the older CS-2 platform. "So, this would be a true Moore's Law step," CEO Andrew Feldman boasted during a press briefing Tuesday. "We haven't seen that in a long time in our industry."
Compared to Nvidia's H100, the WSE-3 is roughly 57x larger and boasts roughly 62x the sparse FP16 performance. But considering the CS-3's size and power consumption, it might be more accurate to compare it to a [2]pair of 8U DGX systems [PDF] with a total of 16 H100s inside. In this comparison, the CS-3 is still about 4x faster, but that's only when looking at sparse FP16 performance.
The lead over the two DGX H100 systems is even smaller – at 2x – when you take into account that Nvidia's chips support FP8. Though this wouldn't exactly be an apples to apples comparison.
[3]
One major advantage Cerebras has is memory bandwidth. Thanks to the 44GB of onboard SRAM — yes, you read that correctly — Cerebras' latest accelerator boasts 21PBps of memory bandwidth, compared to the 3.9TBps the H100's HBM3 maxes out at.
[4]
[5]
That's not to say Cerebras' systems are faster in every scenario. The company's performance claims rely heavily on sparsity.
While Nvidia is able to achieve a doubling in floating point operations using sparsity, Cerebras claims to have achieved a roughly 8x improvement.
[6]
That means Cerebras' new CS-3 systems should be a little slower in dense FP16 workloads than a pair of DGX H100 servers consuming roughly the same amount of energy and space at somewhere around 15 petaFLOPS vs 15.8 petaFLOPS (16x H100s 989 teraFLOPS.) We've asked Cerebras for clarification on the CS-3's dense floating performance; we'll let you know if we hear anything back.
Considering the speed-up, we have a hard time imagining anyone would opt for Cerebras' infrastructure if they couldn't take advantage of sparsity, but even if you can't, it's pretty dang close.
Cerebras is already working to put its new systems to work in the third stage of its Condor Galaxy AI supercluster. Announced last year, Condor Galaxy is being [7]developed in collaboration with G42 and will eventually span nine sites around the globe.
[8]
Cerebras' Condor Galaxy 1 system installed at Colovore's Santa Clara datacenter (click to enlarge)
The first two systems — CG-1 and CG-2 — were installed last year and each featured 64 of Cerebras' CS-2 machines and were capable of 4 AI exaFLOPS a piece.
Wednesday, Cerebras revealed that CG-3 was destined for Dallas, Texas, and would implement the newer CS-3 platform boosting the sites performance to 8 AI exaFLOPS. Assuming that the remaining six sites also feature 64 CS-3s, the nine-site cluster would actually boast 64 AI exaFLOPS of collective compute rather than the 36 exaFLOPS of sparse FP16 initially promised.
[9]
However, it's worth noting that Cerebras' CS-3 isn't limited to clusters of 64. The company claims that it can now scale to up to 2,048 systems capable of pushing 256 AI exaFLOPS.
According to Feldman, such a system would be capable of training Meta's Llama 70B model in about a day.
[10]Cerebras's supercomputer has more cores than world's fastest iron – with a big catch
[11]What's going on with Eos, Nvidia's incredible shrinking supercomputer?
[12]2024 sure looks like an exciting year for datacenter silicon
[13]Cerebras's Condor Galaxy AI supercomputer takes flight carrying 36 exaFLOPS
Qualcomm, Cerebras collab on optimized inference
Alongside its next-gen accelerators, Cerebras also revealed it's working with Qualcomm to build optimized models for the Arm SoC giant's datacenter inference chips.
The two companies have been teasing the prospect of a collab going back to at least November. A [14]release revealing Qualcomm's Cloud AI100 Ultra accelerator included a rather peculiar quote by Feldman praising the chip.
If you missed its launch, the 140W single-slot accelerator boasts 64 AI cores and 128GB of LPDDR4x memory capable of pushing 870 TOPS at Int8 precision and 548GB/s of memory bandwidth.
A few months later, a Cerebras blog post [15]highlighted how Qualcomm was able to get a 10 billion parameter model running on a Snapdragon SoC.
The partnership, now official, will see the two companies work to optimize models for the AI 100 Ultra which take advantage of techniques like sparsity, speculative decoding, MX6, and network architecture search.
[16]
Under the partnership, Cerebras and Qualcomm will develop optimized models for the latter's AI 100 Ultra inference chips (click to enlarge)
As we've already established, sparsity, when properly implemented, has the potential to more than double an accelerator's performance. Speculative decoding, Feldman explains, is a process of improving the efficiency of the model in deployment by using a small, lightweight model to generate the initial response, and then using a larger model to check the accuracy of that response.
"It turns out, to generate text is more compute intensive than to check text," he said. "By using the big model to check it's faster and uses less compute."
The two companies are looking at MX6 to help reduce the memory footprint of models. MX6 is a form of quantization that can be used to shrink a model by compressing its weights to a lower precision. Meanwhile, network architecture search is a process of automating the design of neural networks for specific tasks in order to boost their performance.
Combined, Cerebras claims these techniques contribute to a 10x improvement in performance per dollar. ®
Get our [17]Tech Resources
[1] https://regmedia.co.uk/2024/03/13/cerebras_cs-3_system.jpg
[2] https://resources.nvidia.com/en-us-dgx-systems/ai-enterprise-dgx
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZfHbp4wHBaL4a122C7NiQgAAAMA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZfHbp4wHBaL4a122C7NiQgAAAMA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZfHbp4wHBaL4a122C7NiQgAAAMA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZfHbp4wHBaL4a122C7NiQgAAAMA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2023/07/20/cerebras_condor_galaxy_supercomputer/
[8] https://regmedia.co.uk/2023/07/18/cerebras_condor_galaxy_1.jpg
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZfHbp4wHBaL4a122C7NiQgAAAMA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[10] https://www.theregister.com/2022/11/15/cerebras_supercomputer_frontier/
[11] https://www.theregister.com/2024/02/19/eos_nvidia_supercomputer/
[12] https://www.theregister.com/2024/01/02/2024_dc_chip_preview/
[13] https://www.theregister.com/2023/07/20/cerebras_condor_galaxy_supercomputer/
[14] https://www.qualcomm.com/news/onq/2023/11/introducing-qualcomm-cloud-ai-100-ultra
[15] https://www.qualcomm.com/news/onq/2023/11/introducing-qualcomm-cloud-ai-100-ultra
[16] https://regmedia.co.uk/2024/03/13/cerebras_qualcomm_collab.jpg
[17] https://whitepapers.theregister.com/
The chip, dubbed the WSE-3, is Cerebras' third-gen waferscale processor and measures in at a whopping 46,225mm 2 (that's about 71.6 inches 2 in freedom units.) The 4 trillion transistor part is fabbed on TSMC's 5nm process and is imprinted with 900,000 cores and 44GB of SRAM, good for 125 AI petaFLOPS of performance, which in this case refers to highly sparse FP16 — more on that in a minute.
[1]
Cerebras claims its CS-3 Systems are twice as fast as its predecessor (click to enlarge)
A single WSE-3 forms the basis of Cerebras' new CS-3 platform, which it claims offers 2x higher performance, while consuming the same 23kW as the older CS-2 platform. "So, this would be a true Moore's Law step," CEO Andrew Feldman boasted during a press briefing Tuesday. "We haven't seen that in a long time in our industry."
Compared to Nvidia's H100, the WSE-3 is roughly 57x larger and boasts roughly 62x the sparse FP16 performance. But considering the CS-3's size and power consumption, it might be more accurate to compare it to a [2]pair of 8U DGX systems [PDF] with a total of 16 H100s inside. In this comparison, the CS-3 is still about 4x faster, but that's only when looking at sparse FP16 performance.
The lead over the two DGX H100 systems is even smaller – at 2x – when you take into account that Nvidia's chips support FP8. Though this wouldn't exactly be an apples to apples comparison.
[3]
One major advantage Cerebras has is memory bandwidth. Thanks to the 44GB of onboard SRAM — yes, you read that correctly — Cerebras' latest accelerator boasts 21PBps of memory bandwidth, compared to the 3.9TBps the H100's HBM3 maxes out at.
[4]
[5]
That's not to say Cerebras' systems are faster in every scenario. The company's performance claims rely heavily on sparsity.
While Nvidia is able to achieve a doubling in floating point operations using sparsity, Cerebras claims to have achieved a roughly 8x improvement.
[6]
That means Cerebras' new CS-3 systems should be a little slower in dense FP16 workloads than a pair of DGX H100 servers consuming roughly the same amount of energy and space at somewhere around 15 petaFLOPS vs 15.8 petaFLOPS (16x H100s 989 teraFLOPS.) We've asked Cerebras for clarification on the CS-3's dense floating performance; we'll let you know if we hear anything back.
Considering the speed-up, we have a hard time imagining anyone would opt for Cerebras' infrastructure if they couldn't take advantage of sparsity, but even if you can't, it's pretty dang close.
Cerebras is already working to put its new systems to work in the third stage of its Condor Galaxy AI supercluster. Announced last year, Condor Galaxy is being [7]developed in collaboration with G42 and will eventually span nine sites around the globe.
[8]
Cerebras' Condor Galaxy 1 system installed at Colovore's Santa Clara datacenter (click to enlarge)
The first two systems — CG-1 and CG-2 — were installed last year and each featured 64 of Cerebras' CS-2 machines and were capable of 4 AI exaFLOPS a piece.
Wednesday, Cerebras revealed that CG-3 was destined for Dallas, Texas, and would implement the newer CS-3 platform boosting the sites performance to 8 AI exaFLOPS. Assuming that the remaining six sites also feature 64 CS-3s, the nine-site cluster would actually boast 64 AI exaFLOPS of collective compute rather than the 36 exaFLOPS of sparse FP16 initially promised.
[9]
However, it's worth noting that Cerebras' CS-3 isn't limited to clusters of 64. The company claims that it can now scale to up to 2,048 systems capable of pushing 256 AI exaFLOPS.
According to Feldman, such a system would be capable of training Meta's Llama 70B model in about a day.
[10]Cerebras's supercomputer has more cores than world's fastest iron – with a big catch
[11]What's going on with Eos, Nvidia's incredible shrinking supercomputer?
[12]2024 sure looks like an exciting year for datacenter silicon
[13]Cerebras's Condor Galaxy AI supercomputer takes flight carrying 36 exaFLOPS
Qualcomm, Cerebras collab on optimized inference
Alongside its next-gen accelerators, Cerebras also revealed it's working with Qualcomm to build optimized models for the Arm SoC giant's datacenter inference chips.
The two companies have been teasing the prospect of a collab going back to at least November. A [14]release revealing Qualcomm's Cloud AI100 Ultra accelerator included a rather peculiar quote by Feldman praising the chip.
If you missed its launch, the 140W single-slot accelerator boasts 64 AI cores and 128GB of LPDDR4x memory capable of pushing 870 TOPS at Int8 precision and 548GB/s of memory bandwidth.
A few months later, a Cerebras blog post [15]highlighted how Qualcomm was able to get a 10 billion parameter model running on a Snapdragon SoC.
The partnership, now official, will see the two companies work to optimize models for the AI 100 Ultra which take advantage of techniques like sparsity, speculative decoding, MX6, and network architecture search.
[16]
Under the partnership, Cerebras and Qualcomm will develop optimized models for the latter's AI 100 Ultra inference chips (click to enlarge)
As we've already established, sparsity, when properly implemented, has the potential to more than double an accelerator's performance. Speculative decoding, Feldman explains, is a process of improving the efficiency of the model in deployment by using a small, lightweight model to generate the initial response, and then using a larger model to check the accuracy of that response.
"It turns out, to generate text is more compute intensive than to check text," he said. "By using the big model to check it's faster and uses less compute."
The two companies are looking at MX6 to help reduce the memory footprint of models. MX6 is a form of quantization that can be used to shrink a model by compressing its weights to a lower precision. Meanwhile, network architecture search is a process of automating the design of neural networks for specific tasks in order to boost their performance.
Combined, Cerebras claims these techniques contribute to a 10x improvement in performance per dollar. ®
Get our [17]Tech Resources
[1] https://regmedia.co.uk/2024/03/13/cerebras_cs-3_system.jpg
[2] https://resources.nvidia.com/en-us-dgx-systems/ai-enterprise-dgx
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZfHbp4wHBaL4a122C7NiQgAAAMA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZfHbp4wHBaL4a122C7NiQgAAAMA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZfHbp4wHBaL4a122C7NiQgAAAMA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZfHbp4wHBaL4a122C7NiQgAAAMA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2023/07/20/cerebras_condor_galaxy_supercomputer/
[8] https://regmedia.co.uk/2023/07/18/cerebras_condor_galaxy_1.jpg
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZfHbp4wHBaL4a122C7NiQgAAAMA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[10] https://www.theregister.com/2022/11/15/cerebras_supercomputer_frontier/
[11] https://www.theregister.com/2024/02/19/eos_nvidia_supercomputer/
[12] https://www.theregister.com/2024/01/02/2024_dc_chip_preview/
[13] https://www.theregister.com/2023/07/20/cerebras_condor_galaxy_supercomputer/
[14] https://www.qualcomm.com/news/onq/2023/11/introducing-qualcomm-cloud-ai-100-ultra
[15] https://www.qualcomm.com/news/onq/2023/11/introducing-qualcomm-cloud-ai-100-ultra
[16] https://regmedia.co.uk/2024/03/13/cerebras_qualcomm_collab.jpg
[17] https://whitepapers.theregister.com/
LogicGate
I believe the term should be stormtrooper units...
DAA-DAA-DAA - DA-DI-DAA - DA-DI-DAA
Yet Another Anonymous coward
We refer to them as "medieval" Allen keys
Ivor Catt
Mage
Ivor Catt developed and patented some ideas on Wafer Scale Integration (WSI) in 1972, and published his work in Wireless World in 1981, after his articles on the topic were rejected by academic journals.
I remember those articles. Also later ones about autorouting to avoid defects.
He's still alive.
Re: Ivor Catt
LogicGate
And that is the curse of being too far ahead. The Patents run off before the uptake takes place, leaving the inventor with no benefit.
Now, had he come up with a mouse with big round ears, then he would be a rich man today. (yes, I know that steamboat willie just exited protection)
Re: Ivor Catt
Yet Another Anonymous coward
The clever bit isn't saying: you should use all the chips on this wafer in one computer rather than chopping them up and re-assembling them on a PCB - it's making it actually work.
Otherwise I invented the intergalactic hyper death-ray battleship on my school book in infants
An unanswered question
Mike 137
" The 4 trillion transistor part is fabbed on TSMC's 5nm process and is imprinted with 900,000 cores and 44GB of SRAM "
That's an awful large target for production flaws. I wonder what the yield will be.
Re: An unanswered question
Yet Another Anonymous coward
|The engineering that makes this possible is mapping out and routing around the bad cores.
The problem has always been that this costs money and wafer area, so it was always more cost effective to chop the wafer up, test each chip and sell them separately
It's only with tasks that need a bazzillion cores and 44GB of ram for a single task that this thing starts to make sense
“Freedom units’ teehee