China stops worrying about lack of GPUs and learns to love the supercomputer
- Reference: 1722409165
- News link: https://www.theregister.co.uk/2024/07/31/china_supercomputer_not_gpu_plan/
- Source link:
This was the belief expressed at the 2024 China Computing Power Development Expert Seminar – a conference co-organized by a Chinese industry alliance and national standardization body – and spread on [1]state-sponsored media on Sunday.
Zhang Yunquan, a researcher at the Institute of Computing Technology of the Chinese Academy of Sciences, reportedly lamented the US has taken action to "choke" China's AI development, "including banning the sale of high-end GPUs, terminating the sharing of source code for large models, and interrupting ecological cooperation."
[2]
"When large models require 10,000 to 100,000 GPUs, it is essential to overcome technical challenges like high energy consumption, reliability issues, and parallel processing limits by developing specialized supercomputers," he concluded.
[3]
[4]
Zhang suggested that the pathway for China to keep pace with global AI development and "break through the bottleneck of large model computing power in the short term" is to use its two decades of experience developing advanced supercomputing technology and create machines to crunch large models. Chinese supers appear to have [5]used locally designed chips .
Reflecting on the restrictions, Chinese Academy of Sciences academic Qian Depei mused that researchers in the Middle Kingdom "still have to conform to China's national conditions and cannot completely follow the Americans."
[6]AI models just love escalating conflict to all-out nuclear war
[7]Nvidia can't sell its best chips to China, but India is more than happy to take them
[8]Nvidia sees Huawei, Intel in rear mirror as it grapples with China ban
[9]Quantum computing next (very) cold war? US House reps want to blow billions to outrun China
The hope of some of researchers in attendance was that "super-intelligence fusion" – the integration of supercomputing and intelligent computing – will see China thrive.
"The combination of supercomputing and intelligent computing is inevitable, and there will be an organic integration, rather than simply putting them together," predicted Chen Runsheng, a Chinese Academy of Sciences boffin.
[10]
Zhang noted that China has invested huge amounts of money in the development of intelligent computing power in recent years.
Although China claims it's got the experience to rely on supercomputers, there's really no telling as it's [11]stopped reporting details of tech employed in the nation's mightiest machines – thereby missing out on spots on the famed TOP500 supercomputer scoreboard – alleging that disclosure would have negative effects.
"Setting aside China's progress in supercomputing, the key issue is that while the TOP500 list increasingly serves as a tool for the US Department of Commerce to sanction and suppress Chinese supercomputers, they still expect Chinese institutions to willingly participate and be targeted," complained a piece in Chinese state-sponsored organ [12]The Global Times last week.
[13]
"Entrapment is by no means an alarmist term," the author asserted.
The Global Times cited US sanctions on 20 Chinese entities – seven supercomputing business and 13 more for their potential to make weapons of mass destruction and other tech of national security concerns – as proof of "entrapment."
It then argued that TOP500 co-founder Jack Dongarra "repeatedly stated that China has faster supercomputers" and has speculated that "Chinese supercomputer power may exceed all other countries."
The Register met Dongarra at an event called the International Symposium On Blockchain Advancements in 2023, where he expressed regret that China no longer shares info with the TOP500 list.
"I like competition," explained Dongarra, He attributed the dilemma to "geopolitical issues."
According to Dongarra, in 2022, 162 Chinese machines were on the list, compared to 125 in the US. He added that rumours suggested there were two or three exascale machines in the Middle Kingdom.
China is already making progress with AI – with or without GPUs. On Monday, developers released what they claimed is the world's "first large-scale seismic data processing model with 100 million parameters," according to [14]state sponsored media . The model, called DiTing, was developed by the National Supercomputing Center in Chengdu in cooperation with the Institute of Geophysics of the China Earthquake Administration and Tsinghua University. ®
Get our [15]Tech Resources
[1] https://new.qq.com/rain/a/20240729A00O6Y00?suid=&media_id=
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/hpc&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZqoLSud2hNwme6BLhQQ1@gAAAMA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/hpc&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZqoLSud2hNwme6BLhQQ1@gAAAMA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/hpc&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZqoLSud2hNwme6BLhQQ1@gAAAMA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.nextplatform.com/2023/12/06/everything-we-know-about-chinas-new-tianhe-xingyi-supercomputer-at-guangzhou/
[6] https://www.theregister.com/2024/02/06/ai_models_warfare/
[7] https://www.theregister.com/2024/01/12/nvidia_india_gpu/
[8] https://www.theregister.com/2023/12/06/nvidia_sees_huawei_intel_in/
[9] https://www.theregister.com/2023/11/07/us_lawmakers_quantum/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/hpc&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZqoLSud2hNwme6BLhQQ1@gAAAMA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[11] https://www.nextplatform.com/2021/11/15/why-did-china-keep-its-exascale-supercomputers-quiet/
[12] https://www.globaltimes.cn/page/202407/1316680.shtml
[13] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/hpc&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZqoLSud2hNwme6BLhQQ1@gAAAMA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[14] https://www.globaltimes.cn/page/202407/1317009.shtml
[15] https://whitepapers.theregister.com/
Re: China can do what they want.
On the other hand, "the west" has basically no use for language models like chat GPT beyond it being an amusing little toy. Perhaps companies hope that they might be able to achieve efficiencies in the long term (ie; replace all of the minimum wage people in customer support etc) but there is no immediate use for it, and the long term use doesn't appear particularly socially useful.
The authoritarian axis meanwhile appears to wish to use language models to create bots to manipulate western public discourse and try (with some degree of success) to manipulate who gets elected in our democracies which is a goal that their governments are unlikely to abandon any time soon.
.. overcome in a short term
Very good solution indeed. Domestics GPUs & solutions will come in midtime. Every obstacle is for overcome.
"Chinese have been making huge leaps forward. They built their own spacestation, landed on the dark side of the moon, in one fell swoop on mars, and started fabbing 7nm chips. If I was the US I'd be fucking terrified of what happens in the next 5 - 10 years"
Peshman
What are they actually saying here? That they're going to use CPUs instead of GPUs? So instead of racks of highly specialised, densely packed, power efficient GPUs, they're going to use an equivalent number of cores in overspecified general purposes processors - which will be far less densely packed and likely use far more power and cost more?
If so, yes, you could. But fewer people will be able to do it. And you're tying up supercomputers doing AI instead of all the weapons simulations(?), etc... that you built the supercomputers for.
> "When large models require 10,000 to 100,000 GPUs, it is essential to overcome technical challenges like high energy consumption, reliability issues, and parallel processing limits by developing specialized supercomputers,"
Isn't "specialized supercomputer" a pretty decent description for [a bank of] GPUs?
China can do what they want.
...but at the end of the day, receiving a pallet fork of cash from the Government is not sustainable in the long term. People will turn up, take the money and leave when the money stops.
Realistically Microsoft/Google/Apple/etc combined are investing far more money anyway... ...and they have intentions of productising the results.