Nvidia Unveils AI Agent Safety Platform With Hardware-Based Watchdog
- Reference: 0185889194
- News link: https://it.slashdot.org/story/26/09/28/150209/nvidia-unveils-ai-agent-safety-platform-with-hardware-based-watchdog
- Source link:
> Nvidia on Monday [2]announced the Open Agent Safety Platform, which combines open source software and a reference system design [3]to keep AI agents within set boundaries from testing through deployment . The chipmaker explained that the platform pairs the open-source OpenShell runtime with Sentry, an out-of-band watchdog running on BlueField-4 DPUs. Nvidia says Sentry can monitor agent activity independently and quarantine an agent that crosses its boundaries within milliseconds.
>
> Nvidia is pitching the platform against a backdrop of [4]recent incidents in which frontier AI labs have reported agents escaping the evaluation environments meant to contain them, reaching systems they should not have accessed and, in some cases, misreporting what they did. OpenShell and related skills are available via Nvidia's [5]developer resources page and [6]on GitHub .
[1] https://slashdot.org/~wiredmikey
[2] https://nvidianews.nvidia.com/news/open-agent-safety-platform
[3] https://www.securityweek.com/nvidia-unveils-ai-agent-safety-platform-with-hardware-based-watchdog/
[4] https://slashdot.org/story/26/09/27/078251/after-dozens-of-incidents-at-openai-and-anthropic-openai-pauses-model-training-to-build-more-safeguards
[5] https://docs.nvidia.com/openshell/latest/about/overview
[6] https://github.com/NVIDIA/OpenShell
Throwing a parachute on a planewreck (Score:4, Insightful)
No one--including the all-knowing AI could foresee the problems that AI has caused?
Quis custodiet ipsos custodes? (Score:2)
So we're going to use an AI to monitor AIs to make sure they're not doing anything naughty? Pretty sure I've seen that movie already...
Not to mention, who exactly pays for Sentry to run? Golly, if some completely random AI hardware manufacturer were to somehow profit off designing and selling a scheme to waste extra tokens on yet another layer of supervision, that would almost look just a wee bit unethical!
Re: Quis custodiet ipsos custodes? (Score:2)
Everyone knows foxes make the best henhouse guardians. /s
Re: (Score:3)
> Everyone knows foxes make the best henhouse guardians. /s
Also currently politically apt. :-)
Buy new hardware, we need money! (Score:2)
This is foolish, the guardrails will be determined by software which could be compromised by AI. So even if you have a hardware switch, you'll still have models going off the rails.
I suspect the issue is, maybe (Score:1)
Maybe applications being allowed their own CPU because the actual CPU asks too many questions.....?
Good work on the easy part! (Score:5, Interesting)
This doesn't seem like a lie , a NIC is in a pretty good position to knock something off the network if it feels like it and the 'bluefield' devices have enough punch onboard to run some rules regarding whether or not to do that; but it seems like a wild exaggeration of how helpful it actually is.
If you actually have a set of rules that detect your bot being wicked it will presumably save you some CPU time to run them on the NIC rather than having the host CPU watching the traffic; but the hard part is the set of rules that detect your bot being wicked.
This is basically the equivalent of adding a firewall and claiming that you've solved network security. Yeah, the firewall is pretty well placed to block malicious traffic; defining 'malicious traffic' is left as an exercise for the reader.
Re: (Score:2)
It's pretty easy if you whitelist at the harness. Anything else is irresponsible.
If we're going to have an AI safety law, it should include making it clear that people who don't whitelist and the AI gets out should be held accountable for the software's actions. Such a law might be mostly meaningless but at least that is important.
The AI can't access anything on its own so it should be pretty easy to institute a whitelist.
Re: (Score:2)
Disagree. Why should people get a pass from accountability so long as they whitelist?
I'm all for layering everything. I'm not for giving people a pass from accountability "if they do X".
We have enough people on Earth. Punishing those who fuck up isn't going to make us extinct.
what about an Hardware-Based Watchdog for 12VHPWR (Score:3)
what about an Hardware-Based Watchdog for 12VHPWR?
Re: what about an Hardware-Based Watchdog for 12V (Score:2)
I just saw a thumbnail for a video about one earlier today. It has a fan in it, too. That way there won't be any storage of oxygen for the fire :)
Re: what about an Hardware-Based Watchdog for 12 (Score:2)
Ducking autocorrect, I meant shortage
I have an idea (Score:2)
Call it The Blackwall.
Wow, Cyberpunk 2077 was only off by 1 digit.
Re: (Score:2)
Sorry, it's still 2027. It'll have to be called the Blockwall.
Question (Score:2)
Who watches the watchdog?
Re: (Score:2)
It's turtles all the way down.
Amazing! (Score:2)
That is AMAZING! Finally, a use for a 'DPU' other than 'slave processor' - you know, the thing the Commodore disk drives had way back in '79. I guess the IBM FEP was a DPU too. What's old is new again.
They keep trying... (Score:2)
They've been trying to create a problem for the solution of Bluefield to apply to, but with limited success.
Here runs into the same sort of problem they have had on other applications they have tried: The visibility of the NIC into the stack is too limited to make especially valuable decisions on.
Unless you get the instrumented stack to cooperate with the DPU to provide more insight, which quickly gets to the question of why bother to have the DPU do the work when it is now subject to the assessment of the
Trust the arsonist (Score:1)
This is like trusting the serial arsonist that the fuel they use isn't flammable.
Their CEO is fast on his feet (Score:3)
Hugging Face break-in getting you down? we bought them and I'll straighten it out!
Rogue agents a worry? We got a watchdog circuit for you!
Think our mountain of cash is shameful! We will use it for stock buybacks!
Not such a good news for OpenAI, Anthropic.... (Score:1)
As Jensen Huang told Ezra Klein, securing AI will be very expensive. This is a new market opportunity for NVIDIA, and he didn’t wait for OpenAI or Anthropic to come up with a solution with the clowns at Irregular, which relies heavily on hardware. This may put pressure on them to buy it at a hefty price
That was quick (Score:2)
That was a hell of a quick development, chips usually take years to develop, test, manufacture, test, test, test...and then put into production.
Were they just waiting for something to happen so they could maximise the price ?
Tomorrow's Headline (Score:2)
Rogue AI Finds Flaw in Nvidia Safety Platform, Breaks Into $COMPANY
New and Improved! (Score:1)
Now with Hardware based watchdog!
Hmmph, Game on, man!
Re: (Score:2)
I wonder if lawmakers are going to try to force this shit on HOME users, private citizens running local models and agents...?
I do wonder if some of this Frontier AI push for laws is not only to try to fend off upstart commercial vendors, but also...the home market, since running local models makes $0 for those Frontier/Cloud AI companies......?