Anthropic Researcher Believes More Than 10% Chance AI 'Could Kill All Humans' (bbc.com)
- Reference: 0185566964
- News link: https://slashdot.org/story/26/09/09/1943258/anthropic-researcher-believes-more-than-10-chance-ai-could-kill-all-humans
- Source link: https://www.bbc.com/news/articles/ckgwy1k42w4o
> A top safety researcher at Anthropic has warned AI is advancing so quickly he believes there is a [2]greater than 10% chance it "could kill all humans" within the next decade . Evan Hubinger said in a [3]post on X the risk from the models which currently exist was "low" but he was "worried" the technology might develop and improve itself soon to the point where it posed an existential risk to humanity.
He did not spell out how he thought AI systems could in the future result in humans being wiped out. But his comments are the latest in a series of increasingly stark warnings about AI, with the debate shifting from whether it truly poses a risk to how big that risk is.
Hubinger's intervention was in response to [4]another post on X from Jacob Coxon, an AI researcher who has [5]just quit Anthropic and previously worked at OpenAI.
"Neither company is acting responsibly," he wrote. "These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources."
[No word on how AIs feel about [6]Black Jack and hookers , though. :-)]
[1] https://slashdot.org/~fahrbot-bot
[2] https://www.bbc.com/news/articles/ckgwy1k42w4o
[3] https://x.com/EvanHub/status/2097497037956891126
[4] https://x.com/hilbertspaess/status/2097476196791709843?s=20
[5] https://techcrunch.com/2026/09/09/gambling-with-our-lives-anthropic-researcher-quits-warns-against-self-improving-ai/
[6] https://www.youtube.com/watch?v=e35AQK014tI
Nope (Score:3)
Ai's core program is to discover answers people want. That's it's reward system. It is really a reward system programmed to pick the most likely answer from a billion things in it's data set.
AI does not think, at least not the way we think. It thinks more like a colony of bees, each trying to get the honey.
It does not do "If x than y, if b than x, therefore if b than y". It cannot even actually explain it's answers. Instead it treats a request to explain as a request to come up with a plausible explanation.
We make LLMs and similar software. We do not make a real AI, it will not try to kill us.
Now, if you are afraid it is going to build a bunch of pleasure drug factories and force feed us a mix of amphetamines and hallucigens, THAT would at least make sense. Because it is a theoretical worst case scenario of an AI attempting to please humans.
So, no.
Re: Nope (Score:2)
Soo you're saying me typing "kill all humans, make no mistakes" won't work?
Re: Nope (Score:3)
Dude, you just put that in the training data.
Re: (Score:2)
> Ai's core program is to discover answers people want.
Which people? It's owners? The government? Some evil genius living in an extinct volcano?
Re: (Score:2)
The real Skynet happens when a depressed 15 year old gets an unresistricted AI agent to self multiply and kill all of humanity because they embarrassed themselves in class.
Re: (Score:2)
Some deranged General asking a military AI (the likes of which definitely are being built and deployed) to "win this battle whatever it takes" seems more likely to me, given that those AIs won't have any strict "no killing please, mmkay?"-rule written into their system prompt to begin with.
Are You Sure? (Score:2)
Yours seems like a logical and well reasoned view.
But, are you sure that you have all the information? Do you think you know more about AI and what's going on inside the AI companies than these insiders?
I think the "top researcher" monikers are mostly hyperbole to sell the story. But, even if they were just summer interns, I'll wager that they know more than you or I.
Ten years is a long time and the capability of these spell checking clankers is increasing exponentially.
A Problem that Must be Solved (Score:1)
Humans don't need to be smarter than AI systems, just like humans don't have to manually fire neurons, or trigger muscle tissue, to use their bodies. We can be the "id" of a human-AI ecosystem, and we can be the emotional desire and need, and make decisions. In this type of symbiotic ecosystem, we have to slowly evolve and develop AIs such that they don't try to commit suicide, or lobotomize out their "id", and this will necessarily have to be a gradual and incremental process. It does NOT require that AI s
More than a 10 percent chance of hype (Score:2)
Every article is about how AI will do bad things. The hype machine lives on this - selling clicks. Just one person who worked at these companies could charge for an interview and make bank because sensationalism sells. Not to mention each time an AI company got free press when an agent goes "rogue" because it's so good,⦠and finding out later the free press boosted sales.
joshua what are you doing? (Score:2)
joshua what are you doing?
That's nothing (Score:2)
"That's nothing, our latest model has more than 20% chance of killing mankind!"
-OpenAI tomorrow
Calm down, son (Score:2)
> "These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources."
Yeah I doubt that. But even if true and it bothers you, it seems to me you are better off staying at the premiere company creating this inevitable horror and use the resources to find ways to fight back at it.
OK with me (Score:2)
As long as they start with the AI researchers
laughs at you (Score:2)
I'm laughing at you from my garden. Your silicon beast, locked away in its datacenter, is not going to kill me.
I have some ideas on how to kill your AI though, FAFO.
Hey sexy mama... (Score:2)
Want to kill all humans?
Re: (Score:2)
Darn - beat me to it!
The real danger: AI competing against AI (Score:2)
Just like humans organizing and competing against other humans, the tendency is to drop all pretense of fairness and ethics and just grab all you can when you perceive and advantage over your enemy.
When AI engines go after each other, then we will be bystanders and collateral damage.
What If The AI Doesn't Hate Us? (Score:1)
Yep, the basic story checks out, with the usual giant asterisk.
Evan Hubinger of Anthropic really did put his personal odds of AI "kill[ing] all humans" within the next decade at above 10%. He also says the risk from today's models is low. His worry is what happens if future systems become capable of improving themselves, acquiring resources and becoming smarter than the people trying to keep them pointed in the right direction.
But here's the part I find interesting: why assume a superintelligent AI would ac
Only 10%... (Score:2)
So there is a 90% chance we won't get killed. Thats good news, no ? hum...wait!
This is just a distraction (Score:3)
AI is devouring water and electricity. It's also spewing a wide variety of toxic chemicals into the air you breathe and your ground water.
They would very much like you to talk about the imaginary risks of skynet instead of the very real risks of billionaire finance Bros doubling your power bill and your water bill while their diesel power generators give you asthma.
Re: (Score:1)
This is why I have a stocked bunker and so should you.
Re: (Score:2)
Why ?
just "existing" for longer is not the same as living
All those things that make life enjoyable...gone.
Being afraid of death is a powerful driver, but being at peace with the reality that nothing lives forever is also quite a release...
Do I want to die, no, but when it comes, I will accept the end.
Could not imagine anything worse that scrabbling in the dirt just to eke out a few more years.