Ex-Googler Schmidt warns US: Try an AI 'Manhattan Project' and get MAIM'd
- Reference: 1741281251
- News link: https://www.theregister.co.uk/2025/03/06/schmidt_ai_superintelligence/
- Source link:
China would not sit idle waiting to be dictated to by the US once it achieves superintelligence, the authors warn. This assumes that rivals would accept a lasting imbalance of power rather than act to prevent it, thereby undermining the very stability the strategy purports to secure ...
Schmidt is one of three co-authors of a paper that likens artificial intelligence to nuclear weapons during the Cold War, and warns that the race to develop increasingly sophisticated AIs could disrupt the global balance of power and raise the odds of great power conflict.
The paper, " [1]Superintelligence Strategy ," posits that rapid advances in AI are poised to reshape nearly every aspect of society, but that governments see them as a means to military dominance, which will drive a "bitter race" to maximize AI capabilities.
The paper claims that the development of a "superintelligent" AI surpassing humans in nearly every domain would be the most precarious technological advancement since the atomic bomb.
The other authors are Dan Hendrycks, director of the [2]Center for AI Safety , and Alexandr Wang, founder and CEO of Scale AI.
[3]
The unstated assumption is that the US is the country that will lead the way in AI development, while China would be the fearful aggressor making some kind of preemptive strike. It doesn't seem to have occurred to the authors that China recently [4]surprised the world with AI capabilities that it was not thought to be capable of, or that most cyber threats come from Russia.
[5]
[6]
Another unstated assumption is that "superintelligence" is actually [7]possible at all , and not just some pipe dream.
Any state that succeeds in producing a superior AI poses a direct threat to the survival of its peers, and the paper authors assert that states seeking to secure their own survival will be forced to sabotage such destabilizing AI projects for deterrence. This might take the form of covert operations to degrade training runs to outright physical damage disabling AI infrastructure.
[8]
Thus, the paper states, we are already approaching a dynamic similar to nuclear Mutual Assured Destruction (MAD) – such as the state of détente that developed during the Cold War – in which no power dares attempt an outright grab for strategic monopoly, as any such effort would invite a "debilitating" response.
Schmidt and company christen this Mutual Assured AI Malfunction or (MAIM), a combination of words that seems likely to have been chosen for that acronym. Under MAIM, they posit that AI projects developed by states are constrained by mutual threats of sabotage.
Yet AI technology also has the potential to deliver benefits across numerous areas of society, from medical breakthroughs to automation. Embracing AI's benefits is important for economic growth and progress in the modern world, the authors believe.
[9]
States grappling with the challenges can follow one of three strategies, according to the paper. The first is a hands-off approach with no restrictions on AI developers, chips, or models. Proponents of this strategy insist that the US government impose no limitations on AI companies, lest they curtail innovation and give China an advantage.
[10]It begins: Pentagon to give AI agents a role in decision making, ops planning
[11]Worry not. China's on the line saying AGI still a long way off
[12]It's bad enough we have to turn on cams for meetings, now the person staring at you may be an AI deepfake
[13]Microsoft warns Trump: Where the US won't sell AI tech, China will
The second is a worldwide voluntary moratorium strategy to halt further AI advances, either immediately or once certain hazardous capabilities, such as hacking or autonomous operation, are detected.
Third is a monopoly strategy, where an international consortium along the lines of CERN in Europe would lead global AI development.
After outlining these three alternatives, the authors highlight a proposal from the US-China Economic and Security Review Commission ( [14]USCC ) to pour US government funding into a kind of Manhattan Project to build superintelligence. This would invoke the Defense Production Act to channel resources into a remote site dedicated to developing a super AI to gain a strategic monopoly.
Such a strategy would inevitably raise alarm, and China would not sit idle waiting to be dictated to by the US once it achieves superintelligence, the authors warn. This assumes that rivals would accept a lasting imbalance of power rather than act to prevent it, thereby undermining the very stability the strategy purports to secure.
The paper concludes that states should prioritize deterrence over winning the race for superintelligence. MAIM implies that any state seeking a strategic monopoly on AI power will face retaliatory responses from rivals, as well as non-proliferation agreements – similar to nuclear arms control – aimed at restricting AI chips and open-weight models to limit rogue actors.
"States that act with pragmatism instead of fatalism or denial may find themselves beneficiaries of a great surge in wealth. As AI diffuses across countless sectors, societies can raise living standards and individuals can improve their well-being however they see fit."
Fat chance of that happening. The US is far more likely to choose the hands-off strategy and let its tech sector do whatever it wants with no restrictions. At least we can console ourselves that any "superintelligence" is likely to be [15]a long way away from realization. ®
Get our [16]Tech Resources
[1] https://drive.google.com/file/d/1JVPc3ObMP1L2a53T5LA1xxKXM6DAwEiC/view
[2] https://www.safe.ai/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Z8opDIV9VxBt4bCF0GqaeAAAAJU&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://www.theregister.com/2025/01/27/tech_stocks_tank_as_us/
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z8opDIV9VxBt4bCF0GqaeAAAAJU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z8opDIV9VxBt4bCF0GqaeAAAAJU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://www.goldmansachs.com/insights/articles/will-ai-lead-to-superintelligence-or-just-super-automation
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z8opDIV9VxBt4bCF0GqaeAAAAJU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z8opDIV9VxBt4bCF0GqaeAAAAJU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[10] https://www.theregister.com/2025/03/05/dod_taps_scale_to_bring/
[11] https://www.theregister.com/2025/03/05/boffins_from_china_calculate_agi/
[12] https://www.theregister.com/2025/03/04/faceswapping_scams_2024/
[13] https://www.theregister.com/2025/02/28/microsoft_trump_ai_exports/
[14] https://www.uscc.gov/
[15] https://www.theregister.com/2025/03/05/boffins_from_china_calculate_agi/
[16] https://whitepapers.theregister.com/
> risk assessment ???
Pity that the thinkers failed to predict and counteract the disastrous impact of (a)social media. China had it right from the beginning, it seems. What if China suddenly becomes necessary to stand up against a democratically elected malware(*). Hitler was supposedly democratically elected.
Social media are the most urgent risk, not AI.
* Musk's Grok AI claims Trump is very likely a Russian asset: https://www.newsweek.com/elon-musks-chatbot-says-theres-strong-chance-trump-russian-asset-2040338
> Social media
Did you know that Russia does not have a TV license? And TV has been the main channel to spread propaganda mixed with well financed entertainment programs.
UK may have it wrong, when social media are a more popular alternative. This is an opportunity lost.
Re: James Bond Villains
Or more likely end up with a quasi-stable situation where we've always been at war with Eastasia.
And we'll have to forgo the internet because the AIs run by the thought police will be able to scrape our entire lives off it and deal with us as they see fit. "give me 6 things said by a man, and I'll find something to hang him with"
No wait... its eurasia we at war with .....sorry... no not the cage full of rats..... do it to her..... not me......
So he thinks China are not going to pursue AI "supremacy" if the US doesn't? How touchingly naive.
Fat chance that any country, or any large company, will follow guidance based on these authors
or even a state-issued edict.
Reminds me of what was happening in the beginning of the gene-editing days where quite a few cerebral organizations made pronouncements about how/when/where/etc. modifying the human germ genome should only be done with serious reflection and consultation with these learned organizations.
Then one scientist in China decided to do so anyway. Same will happen with AI - inside the USA (or USSR) and outside.
We had SEVERAL possibly world killing incidents
Computers, or the machines which preceded them, made the choice on both sides on the earth that the enemy had launched nuclear attacks. Some in space, some across the ocean, some under water, some flying towards us, and so on. We know of a few famous ones, but nobody sane would believe that those were the only ones.
Each time, EVERY time, humans were the one to prevent the catastrophe, simply by "na, that does not make sense". So little has changed over the last 100 years in that regard, even though the technology made leaps beyond imagination (and still lacking other aspects where imagination is still ahead).
He tried this before in that book with Kissinger but I guess it's worth another kick at the can. I mean, what IF we could just squash research to preserve our hegemony in the current crappy state of the art?
"We own Nvidia stock and those GPUs ain't gonna buy themselves!"(At least not without AI....)
"As AI diffuses across countless sectors, societies can raise living standards and individuals can improve their well-being however they see fit"
Sorry, best we can do is force more people into unemployment, have C-levels and shareholders pocket the short-term difference and let organizations drown in poorly guardrailed slop with indefinitely deferred responsibility.
FYI FTFY ... or maybe you doubt IT and AI, and think it cannot. Fear not, the ignorance passes
With particular and peculiar regard to .....
the US should refrain from pursuing a latter-day "Manhattan Project" to gain AI supremacy, as this will provoke preemptive cyber responses from rivals such as China that could lead to escalation.
.... is provocation of preemptive cyber responses from rivals the least of your worries methink whenever so much more beyond general knowledge has already been done to render to others, who may or may not be helping the Chinese, the clear advantage of an overwhelming gain of AI supremacy function.
GrahamC [2503061426] asks on [1]https://www.nationaldefensemagazine.org/articles/2025/3/5/b21-inherently-incumbent-on-nextgen-nuclear-communications-system
[Thank you. Your comment will may not be displayed soon after reviewing.]
"As the Air Force works to modernize the air and land legs of its nuclear triad with the B-21 stealth bomber and Sentinel intercontinental ballistic missile, their success will depend on the next generation of the United States’ nuclear command, control and communications systems, service officials said."
Does that next generation of the United States’ nuclear command, control and communications systems oppose and/or compete against and present itself as an alternative peer leader to C42 Quantum Communications Control Systems ...... AI@ITsWork ..... exercising NEUKlearer HyperRadioProACTive IT Protocols ...... Advanced CyberIntelAIgent Threat/Treat Programs ...... or is it perceived and believed whenever such as may be foreign abilities and possibilities for alien facilities and capable utilities which are not understood and misunderestimated to be dismissed as highly unlikely and quite improbable, that the one is not at risk and in clear and present danger of being overwhelmed and neutralised by the other, for some would posit such can all too easily be the case and therefore something gravely to be regarded and steered well clear of if in any sort of competitive adversarial opposition.
Whenever the whole nature of global defence and universal attack is fundamentally changed by unexpectedly novel and rapidly emerged and anomalous remotely controlled otherworldly virtual technologies exercising almighty vanquishing methodologies are all previous aggressive militarised means of commanding intimidation and international control rendered obsolete and a liability to retain and maintain even as a relic to remind future masses of the barbarism of the past that has been left behind for history to condemn as a collective folly born and borne of systemic madness and institutional mayhem, stoked by wilful human ignorance and encouraged by the arrogance displayed and supported in the hubris exhibited by worthless wannabe heroes and fiat paper tigers alike.
And the awkward question asked here, and further afield elsewhere too today, for more than just the United States and Departments of Defense to comment on and answer is ....... Is that not where y'all and IT and AI is all at nowadays, and blowin' in the wind?
[1] https://www.nationaldefensemagazine.org/articles/2025/3/5/b21-inherently-incumbent-on-nextgen-nuclear-communications-system
In which...
Squirrel boy tries to protect his nuts.
Not to worry
Chill, America. It's not like the CCP would pursue some big centralized program.
Where is Rosalind Russell now that we need her?
So this would be [1]Anti-MAIM ?
[1] https://www.youtube.com/watch?v=2gsRMtl70aQ
Covert operations to degrade training runs
Like suddenly opening up all the Chinese social media, accurately translated into English and cross-posting into Facebook, Reddit, 4chan etc etc. So that there is an even *larger* pile of codswallop for the training to scrape.
Hmm, if the [1]DoD's super mega AI turns out to be as prone as its progenitors to making things up, being "jail broken" and generally useless, do you think we can get the DoD to brand Facebook as unamerican, an obvious long-term deep sleeper agent strategy to destable the US defenses by ruining all the LLMs?
[1] https://www.theregister.com/2025/03/05/dod_taps_scale_to_bring/
Does anyone else wonder how much of what is going on in the world is just "distraction crap"? It makes sense when you think about it. We all have diplomatic channels and back channels with each other. These were put in place decades ago. AI as it is is just a distraction. It can't really do much. Ukraine was a blatant mineral grab (which I said 3 years ago) and don't even get me started on Gaza, China, Taiwan and the Islands of the Philippines.
What are we supposed to believe anymore?
The Party told you to reject the evidence of your eyes and ears. It was their final, most essential command.
AI, bah humbug.
James Bond Villains
Current world leaders look like Bond movie villains. They look like they actually want total world domination by any means possible.
That holds for all three (four?) of them.
I suspect they see this risk assessment as a road map, not as something to be avoided.
Or maybe they will simply divide the globe up into spheres of influence? Each can then be a tyrant in his own quarter. Not that that has ever been a stable strategy.