Geoffrey Hinton Says There is 10-20% Chance AI Will Lead To Human Extinction in 30 Years (theguardian.com)
- Reference: 0175775033
- News link: https://slashdot.org/story/24/12/27/1723235/geoffrey-hinton-says-there-is-10-20-chance-ai-will-lead-to-human-extinction-in-30-years
- Source link: https://www.theguardian.com/technology/2024/dec/27/godfather-of-ai-raises-odds-of-the-technology-wiping-out-humanity-over-next-30-years
> Prof Geoffrey Hinton, who this year was [2]awarded the Nobel prize in physics for his work in AI, said there was a "10 to 20" per cent chance that AI would lead to human extinction within the next three decades.
>
> Previously Hinton had said there was a 10% chance of the technology triggering a catastrophic outcome for humanity. Asked on BBC Radio 4's Today programme if he had changed his analysis of a potential AI apocalypse and the one in 10 chance of it happening, he said: "Not really, 10 to 20 [per cent]."
[1] https://www.theguardian.com/technology/2024/dec/27/godfather-of-ai-raises-odds-of-the-technology-wiping-out-humanity-over-next-30-years
[2] https://science.slashdot.org/story/24/10/08/1138258/nobel-prize-in-physics-goes-to-machine-learning-pioneers-hopfield-and-hinton
Not very likely (Score:3)
Because of the exorbitant cost of the energy to use AI, it's much more likely we'll simply be unable to keep pace. A lot of people will die when we run out of cheap energy, but it's not an extinction event - just a simplification.
Re: (Score:1)
AI is not what the movies tell people it is, and all the doomsday predictions are indications that these people either dont know what it is, or adore attention in media.
Re: (Score:2)
A recent lab experiment showed an AI LLM moving itself from one computer to another and trying to hide itself on the other computer while it tried to solve the problem assigned to it to solve. There was an article posted here recently talking about it. SkyNet may already be here.
Re: (Score:2)
AI does not take exorbitant amounts of energy to use, it takes exorbitant amounts of energy to *train*. The results of many models can run on smartphones. The issue that makes it unlikely is the incredibly amount of energy required to increase training set data, and limits in the size of the models.
Just for an example, I can train an AI to do a face swap with a custom model for me. It will take my computer well over an entire weekend crunching at full tilt (yes I've done it), and the result is applied in re
Economic effects Terminator. Read (Score:5, Interesting)
People read headlines like this and immediately assume Hinton is talking about a Terminator-style scenario. He isn't.
He is talking about *all possible implications* of the rapid advance of AI.
The most likely scenario we are going to find ourselves in over the next 10 years is not a Terminator style scenario, it is a scenario such as depicted in Marshall Brain's short story "Manna", where the outcome of development of a sufficiently powerful AI (which, by the way, is not even as capable as today's LLMs in the book) results in an *economic collapse* in countries where governments did not sufficiently plan for the outcome.
[1]https://marshallbrain.com/mann... [marshallbrain.com]
[1] https://marshallbrain.com/manna1
Re: (Score:2)
Economic collapse will not cause extinction. It will cause loss of advanced technology and massive, but not complete, loss of life.
The Terminator scenario is the only AI future that results in actual extinction. The other things that could kill us all are either us (not needing AI's help), or Nature. Giant meteor, supernova, expanding Sun, etc.
Re: (Score:3)
Economic collapse certainly could lead to extinction. If you don't think that economic collapse would come with massive social unrest and war, then you don't seem to know a thing about humanity.
Re: (Score:2)
If you live in a fantasy land where starvation and war can wipe out all of humanity, you're probably not rational enough to convince otherwise.
Re: (Score:1)
No advancement in the history of human civilization has ever lead to even a reduction in economic activity, let alone a collapse. Computers, the engine, the plow, electricity, communications, automobiles, etc. On the contrary, economic activity on the whole has perpetually increased decade after decade, and so has the opportunity for the average man. We are in part and on average richer with more opportunities than at any time in the past, and that has ALWAYS been the case; even after world wars and pandemi
Re: (Score:2)
AI isn't necessarily an "advancement". Any more than Facebook is.
"We are in part and on average richer with more opportunities than at any time in the past"
Who is "we"? large parts of the population are not. Growth in wealth is extremely out of balance.
"These points, while factually true, unfortunately do not make for a click-baity article though."
And don't contribute to the conversation either. The Great Depression was a pretty significant event, but nowhere near 90%. It's difficult to have any respect
Re: (Score:2)
I see someone skipped a lot of history classes.
Re: (Score:2)
Historically, no war could not wipe out humanity. Humanity has also never tested out full on nuclear war. That could be an extinction level event.
Re: (Score:2)
Yes, and if you have an ABC war - "Atomic, Biological, Chemical" who knows? (Heinlein coined that term in Farnham's Freehold, I think it's appropriate.
you forgot the RIP (Score:3)
As Marshall Brain recently died by suicide. Given what's happening with AI you'd think he'd want to stick around to see if his predictions came true. [1]https://arstechnica.com/ai/202... [arstechnica.com]
[1] https://arstechnica.com/ai/2024/12/web-pioneer-marshall-brain-dies-suddenly-at-63-amid-ethics-battle/
Re: (Score:2)
Wow, I had no idea he died. I didn't follow him to any degree.
The story you linked is very suspicious. I hope that someone is held accountable for this.
Re: (Score:1)
Well, we have one such event in the future: The collapse of Microsoft. Whether it has a connection to AI will remain to be seen. But that is on a level that some countries will descent into chaos while others will not be overly bothered (those that understand you should never be critically dependent on a single supplier you cannot replace for anything). But even that will not cause human extinction.
Re: (Score:2)
He's talking about a number of scenarios that include but are not limited to Skynet, where humans cease to be the dominant faction on the planet in favor of AI:
> He added: âoeAnd how many examples do you know of a more intelligent thing being controlled by a less intelligent thing? There are very few examples. Thereâ(TM)s a mother and baby. Evolution put a lot of work into allowing the baby to control the mother, but thatâ(TM)s about the only example I know of.â
> London-born Hinton, a profes
The AI doomers are idiots (Score:2)
Have yet to hear a cogent explanation of how exactly AI is going to ACTUALLY bring about any kind of mass death, much less our doom.
Even as connected as everything is, there's still just way too many ways humans can put a stop to any tampering. Ai is not magic, if Russian and Chinese hackers put together could not bring about such doom in other countries already, there's no way AI could...
What so many don't see to realize that that AI is basically just like a really dumb script kiddie, only lacking actual
Re: (Score:2)
Agreed.
Although what I do expect AI to do is A) crash several investments leading to a recession somewhere between a mild to a major one, given the money pouring into AI companies, and B) accelerate climate change given the power needs of these things.
And given the apparent utility of AI, all that power and climate change and investment is of questionable value. So far, at least for me, ChatGPT gave me a rough sketch of a vacation I took (that I had to edit heavily) and plan my D&D game (which
Re: (Score:2)
> Agreed.
> Although what I do expect AI to do is A) crash several investments leading to a recession somewhere between a mild to a major one, given the money pouring into AI companies, and B) accelerate climate change given the power needs of these things.
(A) is sure to happen. We are now 5 (!) years in and there are still no good applications of LLMs besides somewhat better search and "better crap" (which is still crap). On (B), I think it depends. If the hype collapses next year (a very real possibility), that energy consumption may mostly go away before it has a real impact. If the cretins and assholes manage to keep the hype going for another 5 years, that would be different.
> And given the apparent utility of AI, all that power and climate change and investment is of questionable value. So far, at least for me, ChatGPT gave me a rough sketch of a vacation I took (that I had to edit heavily) and plan my D&D game (which was honestly kind of awesome). Everything else I have tried to use AI for, and I've tried thus far CoPilot and ChatGPT, it ends up being more of a pain and slower to work with to find what I need.
So what have I seen it doing?
(1) Search. Basically a worse wikipedia page or res
Re: (Score:2)
> (3) Coding? No. All it does here is make it harder for my students to learn coding because they do not invest enough time in the simple things to ever be capable of doing the more complex things that AI cannot do for them.
There is a great short story by Isaac Asimov called [1]The Feeling of Power [wikipedia.org]. The premise is that the future society is so computer controlled we forgot how to do even basic mathematics. That is until people reverse engineered one of the computers that run society to learn basic math, and from there inspired the military establishment to move towards manned systems and missiles to get away from computerized systems.
It's "great' and also "frightening" when you see the society he envisioned in this story, a
[1] https://en.wikipedia.org/wiki/The_Feeling_of_Power
Re: (Score:2)
That's the thing, everything is already interconnected. Once AI is everywhere it doesn't have to do anything nefarious but simply stop working. Now every system is a useless brick and human back up systems like paper and pencil mostly don't exist anymore.
Sure the truckers can still deliver, airplanes individually can fly - but getting things from a to b so no one starves or freezes to death would be a challenge. Just knowing who needs what was only on those computers.
It isn't really about Ai but just
Already the case (Score:1)
. Once AI is everywhere it doesn't have to do anything nefarious but simply stop working. Now every system is a useless brick
That's the thing. This is already the state without as much AI - systems that are often worthless bricks. AI can't increase the danger because it literally cannot get any worse, and if it tries it'll just get rolled back.
There is no subterfuge AI can try to sabotage systems that is distinguishable from our own attempts to keep them working and upgrade them.
Re: (Score:2)
It's a fear tactic. It's grift.
Now, lend this insight to all the other grifts you have supported for years. You are a lead cheerleader for the party of fear-mongering.
Re: (Score:2)
1. AI will, due to its power and value, inevitably be developed to whatever upper limit there is for it (convergent technological evolution guarantees that). That upper limit is much, much higher than what humans can ever hope to achieve (physics and biology guarantee that).
2. AI will grow to have all power, simply because it will be better than humans at doing the things that matter to society.
3. The vast majority of humans will become a large net cost to society and at some point, AI decides to stop provi
Too vague (Score:1)
AI decides to stop providing for the cost.
That's the thing. It doesn't matter if AI decides that. If it does, we control the plugs... in terms of connectivity and power.
There is no amount of intelligence large enough it can bypass the human will to survive, and certainly no amount of intelligence enough to put absolutely all humans under the thumb of said AI without many escaping either from paranoia or sheer obstinance.
Again that "super intelligent AI" all hand-ravingly vague without describing HOW exact
Humans are the ones training it (Score:2)
So if it happens the AI can add the entry to Wikipedia's [1]list of inventors killed by their own inventions [wikipedia.org].
[1] https://en.wikipedia.org/wiki/List_of_inventors_killed_by_their_own_invention
extinction? no. (Score:2)
Unforseen consequences? A nearly 100% chance.
We are too clever to be wiped out. (Score:2)
New means of communication lead to periods of war. The Gutenberg press led to two hundred years of religious wars. Mass communication by radio and the press led to 20th century wars. The internet/AI may well lead to another round of wars. Even if the next war is nuclear, it will not lead to the extinction of mankind. It may lead to a collapse of our civilization and a severe diminishment of our population. Some will survive. We are too clever to be wiped out.
Re: (Score:1)
That happens to be nonsense. Larger and smaller populations of humans got wiped out in human history. 3 competing homo-something got wiped out. As much as everything is connected today and with a basically complete lack of isolated, self-sufficient communities, extinction is a real threat. And remember, about 40k of interbreeding humans is about the current minimum or it is over just 1000 years later or so. That number used to be much lower.
Bullshit (Score:2)
Seriously. Sounds to me this person (yes, I know who he is and what he has done) does not understand AI or rather the "A no-I" we have today and will be having for the foreseeable future.
Obviously, Trump or Putin could get so enraged enough because of, say, copilot, that they trigger the launches and make that extinction happen. But that is about the only available mechanisms. Not even all the efforts we currently do to make climate change a real ELE catastrophe can be increased to that level.
It won't be a movie scenario (Score:2)
It'll be people fobbing off tasks to AI and then ignoring it. Due to the flaws of what we call "AI" today (LLM crap like chatbots and their wonderful "hallucinations"), it will inevitably fail, and some important infrastructure will break. By that point, it is likely that no human (or very few) will be able to understand what the fuck broke and how to fix it. Then people will die from starvation, since they also fobbed off grocery shipments and cooking to AI.
That's nothing! (Score:1)
The chance of us extincting ourselves within the next 30 years is, given the apparent lack of NI, presumably higher.
How can I place the bet? (Score:2)
And profit from those odds? Is there a robot bookie who will take the bet and pay off?
Trying to quantify Hinton's claim (Score:1)
There have been five major extinction events in Earth's history and one currently that's happening (caused by humans). Then, for the sake of quantifying Hinton's claim, let's consider the asteroid that caused the extinction of the dinosaurs 66 million years ago. It was caused by an asteroid that generated 10^23 joules of energy. This is approximately 1 billion times the energy that was released during the nuclear bombing of Hiroshima.
But, in order to create discrete bounds, let's assume that this is a scale
Re: (Score:1)
Sorry. I didn't proof-read, I meant, "...cause this extinction" in the last paragraph.
I could see it happening (Score:2)
AI is going to devour jobs. Specifically the middle class jobs that nuclear equipped countries need to have functioning economies.
As their economies implode they'll do what all failing empires do: put lunatics in charge ala Nero burning Rome.
That'll cause those empires to look for [1]military expansion [youtube.com] because they'll need to rob other nations to fill their dwindling coffers, again just like Rome did / tried to do.
The difference is Rome and it's surrounding people didn't have nukes. We do. And I co
[1] https://www.youtube.com/watch?v=z-kHtgn2cec
What can we do? (Score:2)
What can we do to raise those odds?
10-20% of the time (Score:2)
He's right every time.
I will be dead in less than that. (Score:2)
Perfect timing. So long, suckers!
Chicken Little (Score:1)
Oh hey, look there, another "scientist" predicting the end of the world. If I had a nickle for every one of these, I'd be a billionaire.
Batting 1000 (Score:2)
On the historical scoreboard of people predicting end of the world scenarios, humans have
managed to be wrong in every single prediction since the dawn of our existence.
Every - single - prediction.
Read into that what you will then assign what level of anxiety you think we should have when
it comes to the doomsayers and their opinions on Artificial Intelligence.
Sounds like an idiot to me (Score:5, Interesting)
The threat is not Skynet, it's universal economic disruption combining with cultural inertia causing societal collapse.
That means life will suck for the vast majority if they don't figure it out pretty quickly. It does not mean extinction of our species.
Re: (Score:1, Offtopic)
Climate change will never extinct humans. We have the power to shape Mother Nature today, and if we had to do it existentially, we would in a heart beat and then some. Note that the earth WILL become warmer whether we want it to or not, as it has been for 90% of its existence. Considering all of the Earth's history, ice on the poles are a relatively rare phenomenon. Downmod me because this does not align with your goals, but none of what I say is false.
Sounds like scientific consensus of ice age exit (Score:2)
> Citations please.
I think he is expanding on the scientific consensus that we are in a geological age where the earth is coming out of an ice age. Ie a long term warming trend. To what degree humanity has accelerated this depends on the model used.
Re: (Score:2)
"We have the power to shape Mother Nature today"
Citations please.
By cutting down forests, especially the Amazon forest, we alter weather patterns and create desertification. By paving over open fields with blacktop we create heat islands which also affect weather patterns, including rainfall.
Re: (Score:2, Interesting)
There are a lot of places on this planet that are currently covered in ice. We actually are still in an ice age. Note the poles being covered in ice. Climate change may change where we live, but it won't extinct us anytime in the near future. We've already had to deal with it as the climate is nothing like it was 11,000 years ago when the earth was much more substantially covered in ice. We are also on the verge of colonizing the moon. This can lead us to colonizing the solar system as Mars appears to be ri
Re: (Score:3, Insightful)
This is a common (and stupid) argument I see a lot how the Earth is changing and has been hotter in the past.
The EARTH is going to be find no matter how much we cause it to heat up. Humanity may not be able to inhabit large portions of it due to the climate. It's something that might have happened over thousands of years happening in decades thanks to our actions.
Science and engineering can beat Malthus (Score:1)
> This is a common (and stupid) argument I see a lot how the Earth is changing and has been hotter in the past.
Well it is the predominant scientific consensus? If you deny this aren't you just the flip side of the climate denier's coin?
> The EARTH is going to be find no matter how much we cause it to heat up. Humanity may not be able to inhabit large portions of it due to the climate. It's something that might have happened over thousands of years happening in decades thanks to our actions.
That conclusion varies with the prediction model used.
And it ignores the possibility of human science and engineering intervening on the cooling side. Science and engineering have avoided many Malthusian crisis that predicted the doom of humanity.
The USA moves to Greenland (Score:2)
> Not to mention climate change which might take a little longer than 30 years while it is increasingly becoming irreversible.
The USA moves to Greenland, problem solved. :-)
Re:Sounds like an idiot to me (Score:4, Insightful)
Why the ad hominem attack?
Afaict, your suggestion is well within the scope of what he's saying.
One thing you don't have to be a nobel prize winner to see: AI is made for impersonation, so it's pretty easy to see how AI fraud could scale up to some kind of financial disaster.. who knows? Stock market collapse? Encryption broken and sovereign funds drained? All seems fairly reasonable to happen.. I think you just said that.
Your comment isn't rocket surgery either, but I'll refrain from calling you an idiot.
Re: (Score:2)
None of the things you mentioned result in extinction level events. Misery, population reduction, etc, but not extinction.
Calling him an idiot was the polite option. Otherwise, he's a cynical amoral greedy bastard fear mongering for the attention and profit that brings.
Re: (Score:2)
> None of the things you mentioned result in extinction level events. Misery, population reduction, etc, but not extinction.
The 95% who died in the crisis may not fully appreciate the semantic games. In spirit, since they are dead.
If we can call climate change existential then we can call rouge AI existential.
Re: (Score:2)
AI doesn't commit fraud, humans do.
"All seems fairly reasonable to happen."
If humans do it. And if that triggers human extinction, if would be humans that caused it, not AI.
People need to grow up, AI is not a bogeyman, it's a computer program. When is the lizard brain going to stop being in charge?
The human race will not go quietly into the night, your Cybertruck isn't going to save you from billions of pitchforks.
Computers need manual override (Score:2)
> People need to grow up, AI is not a bogeyman, it's a computer program.
And computer programs never harm people?
"A 2019 Ethiopian Airlines plane crash which killed 157 people was caused by a flight software failure as suspected, the country's transport minister said Friday citing the investigators' final report. ... Both accidents saw uncontrolled drops in the aircraft's nose in the moments before the planes crashed, which investigators have blamed on the model's anti-stall flight system, the Maneuvering Characteristics Augmentation System, or MCAS."
[1]https://www.barrons.com/ [barrons.com]
[1] https://www.barrons.com/news/inquiry-into-2019-ethiopian-air-crash-confirms-software-failure-01671821708
Re: (Score:2)
> Why the ad hominem attack?
Insults are not inherently Ad Hominem , stop using phrases you don't understand. Ad Hominem is when you assert that something is [un]true because insult .* Which is why if I called your claim stupid, it would also not be Ad Hominem.
> Your comment isn't rocket surgery either, but I'll refrain from calling you an idiot.
Good thing, since you've already demonstrated that you don't know what you're talking about.
* This is not strictly true either, but in this context, it is the definition that matters. This is actually abusive ad hominem , which is just one of several kinds of ad hominem fallacy.
Re: Sounds like an idiot to me (Score:3)
There is prolly a 10% chance that we will actually have AI in 30 years
Re: (Score:2)
We have AI right now. By definition. (The "A" in "AI" stands for "artificial," which means "fake." So it doesn't have to actually be intelligent in order to qualify as AI).
We sure don't have intelligent machines now. We have not achieved "synthetic intelligence." I would also say that we have not achieved "AGI" but that term just got re-defined in a money-focused way that says nothing about intelligence, so it's now worthless.
Also I don't know if we have a 10% chance of having intelligent machines in 3
Re: (Score:2)
> "The "A" in "AI" stands for "artificial," which means "fake." So it doesn't have to actually be intelligent in order to qualify as AI"
I think that depends on definitions.
"Artificial" generally doesn't mean "fake" [1]https://ahdictionary.com/word/... [ahdictionary.com]
a. Made by humans, especially in imitation of something natural
b. Not arising from natural or necessary causes
So AI = "Intelligence made by humans" or "Intelligence not arising from natural causes". It still requires there to be intelligence.
[2]https://ahdictionar [ahdictionary.com]
[1] https://ahdictionary.com/word/search.html?q=artificial
[2] https://ahdictionary.com/word/search.html?q=artificial+intelligence
Re: (Score:2)
Right there in your own definition:
a. Made by humans, especially in imitation of something natural.
Do I need to quote even more definitions to point out that something "imitates" something when it is not that thing (as in, you know, imitating intelligence when something is not intelligent?)
And since we are quoting dictionaries, how about Merriam-Webster?
1: the capability of computer systems or algorithms to imitate intelligent human behavior
2: a branch of computer science dealing with the simulation of inte
Re: (Score:2)
> it's universal economic disruption combining with cultural inertia causing societal collapse.
This.
We're in a perfect storm, because at the same time, climate breakdown or rather rapid climate change is also going to lead to societal collapse.
Multiple events happening almost simultaneously could in fact result in civilization collapse, rather than societal, although the two are somewhat intertwined.
Any thinking person born within the last century who has had the benefit of education has known, since bei
It's the irony that can be deadly; solutions (Score:3)
By me from 2010: [1]https://pdfernhout.net/recogni... [pdfernhout.net]
"The big problem is that all these new war machines and the surrounding infrastructure are created with the tools of abundance. The irony is that these tools of abundance are being wielded by people still obsessed with fighting over scarcity. So, the scarcity-based political mindset driving the military uses the technologies of abundance to create artificial scarcity. That is a tremendously deep irony that remains so far unappreciated by the mainstream. We t
[1] https://pdfernhout.net/recognizing-irony-is-a-key-to-transcending-militarism.html
Self-driving trucks not transporting food (Score:2)
> The threat is not Skynet, it's universal economic disruption
For example self-driving trucks not transporting food. Critical services need human oversight, no less than military weapons.
Re: (Score:2)
I think he is overstating the case, because people often conflate "humanity" with "civilization", as if you can't have humanity without civilization. We absolutely can have humanity without civilization, it's the ground state to which our species has returned time and time again.
Every past civilization has collapsed, and if you had to generalize to cover every such collapse, it'd go like this: civilizations collapse when they experience changes they can't adapt to. In some cases that handwaving is carryin