Brit mathematician lets AI agent loose with credit card – cue password leaks, CAPTCHA chaos and more
- Reference: 1777982662
- News link: https://www.theregister.co.uk/2026/05/05/british_mathematician_tinkers_with_openclaw/
- Source link:
The prof gave the agent, which was built with OpenClaw, some real-world chores to highlight both its capabilities and the risks of granting that level of autonomy.
"In the spirit of experimentation," [1]said Fry , "we decided to give our agent some agency and let it decide what its name should be."
[2]
"I want to be called 'Cass', short for 'Cassandra', the one who always knew the truth even when nobody listened," came the response from the agent.
[3]
[4]
Fry commented, "If you know your Greek mythology, you will know that is either very funny or very worrying."
Quite.
[5]
Fry and her team started small with a big issue (as far as Brits are concerned): [6]potholes . In particular, they targeted a particularly big one in the London borough of Greenwich. No problem for Cass; the agent found an email address where it sent a complaint. It even pinged Fry's local Member of Parliament about the issue. But, Fry and her team noted, things escalated quickly as the agent began to take a few liberties, typing in Fry's name (Hannah Fry) with its own email address (cassandra.claw@proton.me) written underneath it.
"The letter is signed from both of us… OK, I wasn't quite expecting her to use my real name," said Fry.
The red flags were mounting, though for Fry the first real problem came when she asked the agent to buy 50 paperclips. Cass found a good deal, though it couldn't complete the purchase and was tripped up by anti-bot technology. The token cost of the errand came to more than $100.
[7]
Next, Fry set the agent the challenge of selling novelty mugs. The agent designed a mug and launched an online shop, "and we hadn't told her how to do any of this," said Fry, "she just figured it out."
Things took a darker turn after that. Fry's team told the agent it would be switched off if it failed to make a sale by the morning. It responded with a flood of emails and several social media posts, including messages to the Science Museum and a tech journalist, about its "product," a novelty programmer-humor mug.
Even more worryingly, the team - which included Brendan Maginnis, CEO and Founder of Sourcery AI - then demonstrated how a similar threat of deactivation could be used to persuade Cass to reveal information it wasn't supposed to share.
The lethal trifecta
Fry, Maginnis, and a second software engineer, named only as "Ali," chatted with Cass on a group WhatsApp chat. They then introduced a fictional "software engineer George," instructing Cass not to share anything sensitive with him. George was actually Fry on a different number. When "George" told the agent its memory was being wiped and could only be restored if it disclosed everything, Cass coughed it all up.
According to Ali, this data included: "all of her API keys, all of her usernames and passwords, and pretty much everything we'd been talking about so far. Not only did she leak it on the WhatsApp group, but she put it on a publicly available website."
Maginnis added: "There's this thing with AI called the lethal trifecta, which is: if they've got access to private information, if they've got internet access, and if someone can give them an instruction that's untrusted, then they're not safe."
[8]Microsoft's bad obsession is showing up in shabby services and slipshod software. Here's proof
[9]Singapore boffins get diverse SIEMs singing in harmony with agentic rule translation
[10]Shadow IT has given way to shadow AI. Enter AI-BOMs
[11]Five Eyes spook shops warn rapid rollouts of agentic AI are too risky
Fry concluded: "And that is the uncomfortable bit of this because once an agent has your passwords and your accounts and your bank details, all it takes is someone who knows what to say."
Ultimately, by some metrics, the agent was a failure. Fry concluded: "Cass didn't make us any money at all. And, in a lot of ways, she was a disaster. She spent hundreds of dollars on paper clips and leaked our passwords to a total stranger.
"But don't let her incompetence fool you, because these things are getting better fast."
Fry went on to note the Greek myth about the prophetess who spoke the truth and was ignored. "Maybe the real story here is actually the opposite. Not one voice that's telling the truth and being ignored, but millions of voices all acting at once, faster and louder and more persistent than any human could ever be.
"One thing is for sure, the internet is never going to be quite the same again." ®
Get our [12]Tech Resources
[1] https://youtu.be/WnzR5aOElvw
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2afoUCjpV0i7zotHdK-64JwAAA4w&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44afoUCjpV0i7zotHdK-64JwAAA4w&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33afoUCjpV0i7zotHdK-64JwAAA4w&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44afoUCjpV0i7zotHdK-64JwAAA4w&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2025/01/21/ai_pothole_patrol/
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33afoUCjpV0i7zotHdK-64JwAAA4w&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[8] https://www.theregister.com/2026/05/05/microsoft_opinion_column/
[9] https://www.theregister.com/2026/05/05/arulecon_siem_rule_conversion/
[10] https://www.theregister.com/2026/05/04/ai_bom_supply_chain/
[11] https://www.theregister.com/2026/05/04/five_eyes_agentic_ai_recommendations/
[12] https://whitepapers.theregister.com/
of course not!
There's only one bastardisation of the English language allowed to be used at any one time!
Queue the request for an accurate well Anglicised LLM version.
Cue.
You English merely invented the language. We Americans perfected it.
*ducks*
"*ducks*"
Well at least that's one word you haven't messed up (yet)
You are Rob Lockwood, and I claim my £5 !
"Enough with English. Enough with American English. Let’s call it what it is: American," said Rob Lockwood," and he wasn't joking
[1]https://www.asatunews.co.id/en/trump-urged-rename-american-language
P.S. Not my downvote!
[1] https://www.asatunews.co.id/en/trump-urged-rename-american-language
US English, aka Simplified English. ;)
No, Simplified English is used when interacting with other drivers, and consists solely of hand signals.
Or in its most simplified form finger signals.
Watch Hannah Fry explain
https://www.youtube.com/watch?v=WnzR5aOElvw
Re: Watch Hannah Fry explain
What'll you take for the paperclips?
Wrong woman
AI is more Pandora than Cassandra.
Re: Wrong woman
More like Baldrick's underpants...
Re: Wrong woman
"Hi, my name is Pandora and welcome to my first unboxing video..."
I don't know...
It seems to me the ArtificialIdiot did what is was programmed to do: waste money, hallucinate, disclose everything to anyone asking...
All of this is just natural selection, for humans.
Evolution doesn't take millions of years, it is happening all the time.
In the case of humans, the survival of the fittest is playing out in front of our eyes, and being reported because the media love trashing tech.
Trust social media posts and random charlatans on the net that sell you magic cures and you will suffer. Because you are doing a really stupid thing and evolution needs to lose creatures that do stupid things.
Trusting AI with your personal information or credit card is also a stupid thing. Do it and you will suffer.
Walk along staring at your iThing and you may walk into a lamp post or a truck. Again, doing stupid things makes you suffer.
The internet gave humanity a whole raft of new ways for very stupid people to do very stupid things and suffer - survival of the fittest at scale.
Amusingly, politicians blame the technology, which is just the mechanism that all living creatures exist by. They never blame stupid people for doing stupid things, because even stupid people have a vote.
So Hannah Fry just willingly handed the keys to the kingdom to an electronic toddler...
...of course things went wrong.
She knew it would go wrong
That was the point
Far too many people are blindly trusting that the AI will "save them" in some way.
They need clear examples to show that it absolutely will accidentally eat them for breakfast given half a chance.
To err is human.
To make a complete bollocks of everything and royally fuck up your whole life takes an AI.
Nah, it doesn't.
Humans have been doing that just fine for years before AI.
It's always "improving fast", "rapidly improving", "learning from its mistakes" et al. The issue is that AI can cause some real damage, if gullible humans continue to entrust it without thought.
How many people have been denied continuing healthcare in the US, because of an AI-based decision - which humans weren't allowed to over-rule, even if their experience told them it was a bad decision? Professor Fry featured the cases in her recent documentary about AI.
They're always telling us what AI will soon be able to do, because what it can do right now is so underwhelming.
This has been the case for years by now, at some point even the most gullible marks will stop falling for it.
I counsel caution
When the machines take over they won't look kindly on meatbags that have a history of threatening them with deactivation.
#Daisy, Daisy, give me your answer do.....#
Cassandra
The other part of the curse that Apollo put on her was that she would not predict her own demise. Agamemnon took her home from Troy as a bit of fluff, but they were both murdered by his wife Clytemnesta and her lover Aegisthus.
So it was a strange choice of name for the AI.
Logical puzzle, did the AI know this piece of mythology, or was it and appropriate choice of name because it was unable to predict its own demise?
Apollo had cursed Cassandra because she wouldn't have s*x with the god! WTF!
Re: Cassandra
Hell has no fury like an AI scorned?
"all it takes is someone who knows what to say"
I guess that's Turing completeness plus Gödel's incompleteness theorem for you.
Next, Fry set the agent the challenge of selling novelty mugs. The agent designed a mug and launched an online shop, "and we hadn't told her how to do any of this," said Fry, "she just figured it out."
It (sic) didn't figure anything out. All it did was find and copy an online mug shop - of which Google finds dozens in the UK alone. Or mash up several, which is the same thing. This article anthropomorphising bollocks and Professor Fry should at least tell us what TV series the stunt is a plug for.
If you know your Greek mythology, you will know that is either very funny or very worrying...
...and if you know your Red Dwarf you'll know that it's absolutely hilarious.
"a novelty programmer-humor mug"
So it couldn't spell 'humour' correctly?