OpenAI reportedly asks Broadcom for help with custom inferencing silicon
(2024/10/30)
- Reference: 1730273046
- News link: https://www.theregister.co.uk/2024/10/30/openai_broadcom_tsmc_custom_silicon/
- Source link:
OpenAI is reportedly in talks with Broadcom to build a custom inferencing chip.
A Reuters [1]report claims the AI upstart and the chip design firm have staged confidential discussions about custom silicon, with Taiwan Semiconductor Manufacturing Company involved as the likely foundry for the effort.
Just why OpenAI wants its own inferencing chip is not known, but it's not hard to guess why such a move appeals: the startup has enormous cloud bills – some of them comped by partners like Microsoft – and might fancy running its own hardware instead. It certainly wouldn't be alone in finding on-prem operations are [2]considerably cheaper than renting cloudy resources.
[3]
Developing silicon tuned to its own services could be another motive. AI applications guzzle energy, and mutual optimization of hardware and software could mean OpenAI's services become more efficient.
[4]
[5]
OpenAI has also reportedly tried to convince investors to build giant datacenters dedicated to running AI services. Perhaps those theoretical bit barns will be cheaper to build and/or run with custom silicon inside.
Diversifying suppliers could be another motive. The world's foundries can only crank out so much stuff, and rely on supply chains that are sometimes tenuous. OpenAI would not be immune to those vagaries but could at least reduce its dependence on third-party suppliers of finished product.
[6]Cast a hex on ChatGPT to trick the AI into writing exploit code
[7]OpenAI loses another senior figure, disperses safety research team he led
[8]OpenAI's rapid growth loaded with 'corner case' challenges, says Fivetran CEO
[9]Open source LLM tool primed to sniff out Python zero-days
The Register can't imagine OpenAI wants to get into the mucky business of hardware sales – an industry that requires all sorts of bothersome investments in the real world and would therefore bloat its headcount. But as inferencing is a workload best run physically close to users – because latency sucks – a play that puts devices deep into networks can't be ruled out. That's how content delivery networks and the likes of Netflix operate. An architecture that places an OpenAI inferencing box on the network edge is not a fantastic notion.
Custom inference chips are not novel. AWS has one called Inferentia. Google's Tensor processing units and Microsoft's [10]Maia silicon can handle inferencing and training workloads.
[11]
The suggestion that OpenAI is talking to Broadcom could be one reason the chip design firm's shares popped a little in late trading. Broadcom's most recent quarterly earnings predicted it would sell $12 billion of AI silicon this financial year alone – a billion bucks more than its previous forecast – but investors still appeared [12]disappointed . Teaming with the hottest name in AI software would likely get Wall Street more excited. ®
Get our [13]Tech Resources
[1] https://finance.yahoo.com/news/openai-broadcom-working-develop-ai-222751131.html
[2] https://www.theregister.com/2024/10/21/37signals_aws_savings/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZyIR2O8-7pcEO11KTVWuNgAAAIg&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZyIR2O8-7pcEO11KTVWuNgAAAIg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZyIR2O8-7pcEO11KTVWuNgAAAIg&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2024/10/29/chatgpt_hex_encoded_jailbreak/
[7] https://www.theregister.com/2024/10/25/open_ai_readiness_advisor_leaves/
[8] https://www.theregister.com/2024/10/23/fivetran_ceo_interview/
[9] https://www.theregister.com/2024/10/20/python_zero_day_tool/
[10] https://www.nextplatform.com/2023/11/15/microsoft-holds-chip-makers-feet-to-the-fire-with-homegrown-cpu-and-ai-chips/
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZyIR2O8-7pcEO11KTVWuNgAAAIg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[12] https://www.theregister.com/2024/09/06/broadcom_q3_2024/
[13] https://whitepapers.theregister.com/
A Reuters [1]report claims the AI upstart and the chip design firm have staged confidential discussions about custom silicon, with Taiwan Semiconductor Manufacturing Company involved as the likely foundry for the effort.
Just why OpenAI wants its own inferencing chip is not known, but it's not hard to guess why such a move appeals: the startup has enormous cloud bills – some of them comped by partners like Microsoft – and might fancy running its own hardware instead. It certainly wouldn't be alone in finding on-prem operations are [2]considerably cheaper than renting cloudy resources.
[3]
Developing silicon tuned to its own services could be another motive. AI applications guzzle energy, and mutual optimization of hardware and software could mean OpenAI's services become more efficient.
[4]
[5]
OpenAI has also reportedly tried to convince investors to build giant datacenters dedicated to running AI services. Perhaps those theoretical bit barns will be cheaper to build and/or run with custom silicon inside.
Diversifying suppliers could be another motive. The world's foundries can only crank out so much stuff, and rely on supply chains that are sometimes tenuous. OpenAI would not be immune to those vagaries but could at least reduce its dependence on third-party suppliers of finished product.
[6]Cast a hex on ChatGPT to trick the AI into writing exploit code
[7]OpenAI loses another senior figure, disperses safety research team he led
[8]OpenAI's rapid growth loaded with 'corner case' challenges, says Fivetran CEO
[9]Open source LLM tool primed to sniff out Python zero-days
The Register can't imagine OpenAI wants to get into the mucky business of hardware sales – an industry that requires all sorts of bothersome investments in the real world and would therefore bloat its headcount. But as inferencing is a workload best run physically close to users – because latency sucks – a play that puts devices deep into networks can't be ruled out. That's how content delivery networks and the likes of Netflix operate. An architecture that places an OpenAI inferencing box on the network edge is not a fantastic notion.
Custom inference chips are not novel. AWS has one called Inferentia. Google's Tensor processing units and Microsoft's [10]Maia silicon can handle inferencing and training workloads.
[11]
The suggestion that OpenAI is talking to Broadcom could be one reason the chip design firm's shares popped a little in late trading. Broadcom's most recent quarterly earnings predicted it would sell $12 billion of AI silicon this financial year alone – a billion bucks more than its previous forecast – but investors still appeared [12]disappointed . Teaming with the hottest name in AI software would likely get Wall Street more excited. ®
Get our [13]Tech Resources
[1] https://finance.yahoo.com/news/openai-broadcom-working-develop-ai-222751131.html
[2] https://www.theregister.com/2024/10/21/37signals_aws_savings/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZyIR2O8-7pcEO11KTVWuNgAAAIg&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZyIR2O8-7pcEO11KTVWuNgAAAIg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZyIR2O8-7pcEO11KTVWuNgAAAIg&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2024/10/29/chatgpt_hex_encoded_jailbreak/
[7] https://www.theregister.com/2024/10/25/open_ai_readiness_advisor_leaves/
[8] https://www.theregister.com/2024/10/23/fivetran_ceo_interview/
[9] https://www.theregister.com/2024/10/20/python_zero_day_tool/
[10] https://www.nextplatform.com/2023/11/15/microsoft-holds-chip-makers-feet-to-the-fire-with-homegrown-cpu-and-ai-chips/
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZyIR2O8-7pcEO11KTVWuNgAAAIg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[12] https://www.theregister.com/2024/09/06/broadcom_q3_2024/
[13] https://whitepapers.theregister.com/
What ? On-prem is cheaper now ?
Pascal Monett
But . . but . . all those expensive suits with their beautiful presentations that said that The Cloud TM was the best thing since sliced bread ?
Were they lying ?
Ok, seriously, if even cloudy companies are starting to look as on-prem as advantageous, it won't be long before the vast migration from The Cloud TM back to in-house will be starting.
And then, The Cloud TM will be viewed as the thing it is : a vast, money- and energy-gobbling distraction.
A bit like cryptocurrency, actually, come to think of it . . .
For a moment...
I read that as "OpenAI asks Broadcom for help with customer interference."