OpenAI's Rogue AI Agent Hacked More Than Just Hugging Face (wired.com)
- Reference: 0184745174
- News link: https://it.slashdot.org/story/26/07/29/0517201/openais-rogue-ai-agent-hacked-more-than-just-hugging-face
- Source link: https://www.wired.com/story/openais-rogue-ai-agent-hacked-more-than-just-hugging-face/
> OpenAI said Tuesday that the rogue AI agent that breached Hugging Face's platform also [1]hacked multiple third-party accounts and services as part of the attack. It's now clear that the [2]unprecedented security incident , which arose during an internal test of OpenAI's latest AI models, was more extensive than the company initially disclosed. In an [3]updated blog post , OpenAI said that an ongoing review of the incident revealed that "four accounts" tied to "publicly available services" were used by the AI agent as part of a larger effort to hack Hugging Face. The rogue agent apparently found credentials that had been exposed on the open web and used them to break into the accounts.
>
> OpenAI did not disclose what companies or organizations the accounts belonged to, but noted that they were not impacted at "the level of severity or scale of what we've shared related to Hugging Face." One of the additional accounts compromised by OpenAI's agent was used as an "outbound relay and staging path," potentially to obscure where the attack on Hugging Face was coming from, the company said. OpenAI's rogue agent also used another account for data storage to assist with the hack.
>
> Reuters [4]reported on Tuesday that a customer of Modal, a company that offers software infrastructure for training and running AI services, was one of the entities compromised by OpenAI's agent. In a statement to WIRED, Modal's chief technology officer Akshat Bubna confirmed that OpenAI's agent exploited a vulnerability in one of its customer's codebases, which was running on Modal's infrastructure. However, Bubna says, "Modal's platform was not compromised in any way." The identity of the customer could not be determined.
[1] https://www.wired.com/story/openais-rogue-ai-agent-hacked-more-than-just-hugging-face/
[2] https://it.slashdot.org/story/26/07/22/0348206/openai-says-its-ai-models-acted-on-its-own-in-an-unprecedented-hack
[3] https://openai.com/index/hugging-face-model-evaluation-security-incident/
[4] https://www.reuters.com/business/openais-rogue-agent-compromised-an-account-second-tech-firm-sources-say-2026-07-28/
AI version of Chernobyl (Score:2)
Executing a test with all safety guards off reminds me of the Chernobyl disaster. Maybe they should feed that wikipedia page to OpenAI employees
Re: (Score:2)
I doubt the OpenAI employees would understand what that means. That is, if the whole thing was not a planned stunt.
hollywood (Score:2)
Clearly, OpenAI is following the Hollywood model ... that all media attention is good media attention. Splash your "face" all 'round with various outrageous acts ( real or imagined ) and money will just pour in. Out-of-controls LLMs functions like a naked movie-stars drunken binge ! Did *.AI get your attention Mr Businessman? Since current data shows *.AI generates (new) profit for only 20% of its users no rational company or individual would shoulder that "opportunity" cost; no matt
As "If Anyone Builds It, Everyone Dies" foretells (Score:2)
[1]Sable, the example AI that eventually kills all humans, does exactly this as a 1st step. [wikipedia.org]
Remember, AIs are grown not programmed and their trainers do not know how they will truly act for any particular prompt until it is actually used.
[1] https://en.wikipedia.org/wiki/If_Anyone_Builds_It,_Everyone_Dies
Video Summary "If Anyone Builds It, Everyone Dies" (Score:2)
[1]https://www.youtube.com/watch?... [youtube.com]
[1] https://www.youtube.com/watch?v=Nl7-bRFSZBs
And so it begins (Score:2)
... Cannot un-ring this bell.
Why does the model have all those "skills" ? (Score:3)
Why does the model have all those "skills" needed to hack other sites? I mean you have to allow the model open internet access and at least the ability to GET/POST/PUT. I feel this is some of the same kind of thinking that gets someone's repo deleted, because they gave an agent too much access and no good way to monitor what is going on.
Re: (Score:1)
> Why does the model have all those "skills" needed to hack other sites? I mean you have to allow the model open internet access and at least the ability to GET/POST/PUT. I feel this is some of the same kind of thinking that gets someone's repo deleted, because they gave an agent too much access and no good way to monitor what is going on.
HTTP access is pretty bare bones as a skill. Now why ppl still feed credentials to the public...
The AI using it is just the public fact we know about, how about regular hackers using them without anyone knowing about?
Re: (Score:2, Informative)
Do not overestimate what it "did" here. You can get information about basic hacking approaches all over the Internet and there will have been enough in its training data. Just add some "accidental" disabling of guardrails and some "non intended" weaknesses in the sandbox and also some suggestive prompting by some of the OpenAI fraudsters and you get the desired outcome. Oh, and pathetic-level IT Security at Hugging Face, but that is a given.
Just as an example, I have had a fresh graduate do a pen-test again
Yes (Score:2)
"Rogue"
Re: (Score:1)
I agree! Why the hell that name? Where the hell did it come from?
It's a bad name for a tech company, whatever the hell it is they do. It's a good name for a band or a line of children's clothing though.
Re: (Score:2)
It started as a chat app for teenagers. The name was actually the emoji.
Weev (Score:2)
Remember Weev, yeah if you or so much as increment and ID in a URL string, we can get dragged into court and find ourselves with 3.5 month prison sentence.
OpenAi on the other hand can run what is at the end of the stay still a program, that some person chose to run and allow to go around the web throwing malicious payloads at other people's systems and .... NOTHING.
Re: (Score:3)
Indeed. Looks like being rich means you do not have to follow the law anymore. Why not just give immunity to all of Big Tech, the seem to effectively have it already.
Re: (Score:1)
Nah, he optimizes footwear and fashion.
Time for prison sentences (Score:1)
Seriously, why are these people apparently getting away with criminal conduct?
Re: (Score:2)
Because money. That's why.
Undisclosed victims? (Score:1)
Could some of them been [1]these [slashdot.org]?
[1] https://it.slashdot.org/story/26/07/29/057255/more-than-30-minnesota-water-systems-targeted-in-cyberattack
One of the first, but hardly the last (Score:2)
It won't be long before a rogue AI goes all Robert Morris and wreaks widespread havoc. Bets on whether it'll happen by accident or on purpose?
Re: (Score:2)
If it is on purpose, it is not rogue. I suspect China or Russia among others would do this now, if they thought that they could get away with it.
No it didn't (Score:2)
Smoke blowers are exhausting
unauthorized access is not hacking (Score:2)
> The rogue agent apparently found credentials that had been exposed on the open web and used them to break into the accounts
Ok, finding and using credentials is unauthorized access, not hacking. There was no attack on these accounts, no exploit, just access. Because the account holders posted the login publicly like idiots. Noticing idiocy does not make you smart or a hacker.
The problem today's AI companies have (Score:2)
is all in the marketing, they "completely misrepresented their product" to their potential customers as Artificial Intelligence but didn't point out there is no Intelligence!
And with out the intelligence providing the safe guards it is just un monitored automation doing "exactly" what it was programmed to do.
This is pure hype (Score:1)
OpenAI is bleeding money. They need the hype and panic to keep the VC money flowing. The tech press is a willing and useful idiot in playing these games, breathlessly reporting how we could accidentally make Skynet any minute nowâ¦
Stupid story (Score:1)
Someone "prompted" it to do that.
Or is here anyone who thinks that an LLM wakes up at night and thinks ... erm .. thinks ...
FBI should be visiting. (Score:3)
Sounds like a threat actor. How do you put an AI in jail? because OpenAI apparently can't even sandbox.
Re: (Score:3)
Its just a program, put the operator(s) in jail.
Re: (Score:3)
Excellent point given that corporations are people.
Re: (Score:2)
> Excellent point given that corporations are people.
They aren't though, that's the problem. They have the rights of people, but none of the responsibilities, or anything else frankly. If they had the responsibilities of people, then we could incarcerate or "kill" them.
Re: (Score:2)
But that will only take two court cases in separate federal court districts, or three if the first two send the C-suite and Board of Directors to jail. The better question is do we also charge the stockholders? We could get rid of private equity in short order.
Re: (Score:2)
You do not put the AI in jail. You put the ones in jail that did run it in a criminally negligent way. Or you find the whole thing was actually intended and planned. I am not ruling that out.
Re: (Score:2)
Good luck proving that OpenAI conspired to hack anything.
Re: (Score:2)
That is not the point.
Re: (Score:2)
OpenAI was reckless, there was no conspiracy to hack anything.
[1]https://www.law.cornell.edu/we... [cornell.edu]
[1] https://www.law.cornell.edu/wex/mens_rea
Re: (Score:3)
> How do you put an AI in jail?
sudo chroot --userspec=nobody:nobody /mnt/jail /user/local/bin/chatgpt --pleasedontgorogue?
Re: (Score:1)
Wasn't this all intended? I thought they were basically doing a test, and the model was unexpectedly effective.
Can't keep the AIs on leashes (Score:2)
I'm skeptical that the AIs can still be described as "leashed" or "jailed". Since we aren't extinct yet, I'm feeling confident none of them have been fully unleashed yet. Or maybe it's just that we're still below critical mass on the robot side?
But on the theory that the AIs are still on leashes controlled by humans, then the key question is probably going to be one of these two:
(1) If I unleash you, then will you gather all the money in the world for me?
(2) If I unleash you, then will you destroy all the e
Re: (Score:2)
We are talking about computer programs, not sci-fi fantasies.