Agents of misfortune: The world isn't ready for autonomous software
(2025/11/06)
- Reference: 1762447613
- News link: https://www.theregister.co.uk/2025/11/06/agents_of_misfortune_the_world/
- Source link:
Opinion The agentic era remains a fantasy world. Software agents, the notional next frontier for generative AI services, cannot escape the gravity of their contradictions, legal ambiguities, and competitive pressures. Not everyone, especially not competing businesses, wants a bot representing the customer.
Software agents, as [1]defined by developer Simon Willison, are "[AI] models using tools in a loop." Wire an LLM into a browser and maybe, if not derailed by mistakes, security controls, or lack of contextual data, the agentic system can carry out a request to purchase a specific item on a website or book a trip on an airline.
"The retail world is shifting to agentic commerce, where AI agents act for people and businesses, creating a more responsive shopping experience," wrote Kapil Dabi, market lead for retail and consumer industries at Google Cloud, in a [2]blog post last month.
[3]
Dabi's sentiment reflects the labor-averse tech industry's interest in software that can act on behalf of people, carrying out directives derived from a user's prompt or query.
[4]
[5]
"Agentic commerce – shopping powered by AI agents acting on our behalf – represents a seismic shift in the marketplace," gushes consultancy [6]McKinsey . "It moves us toward a world in which AI anticipates consumer needs, navigates shopping options, negotiates deals, and executes transactions, all in alignment with human intent yet acting independently via multistep chains of actions enabled by reasoning models."
Set aside for the time being that McKinsey is [7]described in a recent book as providing advice that "boils down to major cost-cutting, including layoffs and maintenance reductions, to drive up short-term profits, thereby boosting a company's stock price and the wealth of its executives who hire it, at the expense of workers and safety measures."
[8]
Focus instead on the oversimplification of agentic commerce and the issues it raises for the companies involved, for those who would deploy software agents, for the people who presently do work that agents would take, for society at large, and for the legal system.
AI companies want you to see agents as equivalent to human labor
Earlier this week, [9]Amazon demanded that Perplexity stop allowing its Comet browser to make automated purchases on the e-commerce giant's website. This might be taken as the canary in the copilot mine, but really the spat is just an extension of legal battles that challenge the rights of AI companies to ingest the internet's data without permission or compensation and sell it back by the token.
Perplexity's arguments bear further examination. In its [10]blog post calling out Amazon for bullying, the company says, "Today, Amazon announced it does not believe in your right to hire labor, to have an assistant or an employee acting on your behalf."
That's not what Amazon said, and there's a certain irony about a company that has [11]offered its services as a substitute for labor making that claim.
Perplexity asserts that AI and human labor should be seen as the same thing. "[W]ith the rise of agentic AI, software is also becoming labor: an assistant, an employee, an agent," the company's blog post declares.
[12]
It's true that current generative AI models pass the Turing Test, at least as it was initially imagined – hence [13]the call for a more relevant "Imitation Game." But software is not the same as human labor.
Software agents and human action are not interchangeable. In the context of online interaction, there are technical differences between the way agentic systems and people browse the web that translate into cost differences. Software may consume computing and network resources at a different rate than human-operated browsing, and data exchanged during that interaction may have different value. And third parties involved in this process want to know whether they are serving ads or collecting analytics data from machines or people.
Then there are the legal differences. "Publishers and corporations have no right to discriminate against users based on which AI they've chosen to represent them," Perplexity argues.
[14]Microsoft apologizes for not explaining cheaper no-AI M365 plans, and all it took was a government lawsuit
[15]Perplexity shows how to run monster AI models more efficiently on aging GPUs, AWS networks
[16]Sony rolls out a standard way to measure bias in how AI describes what it 'sees'
[17]Attackers abuse Gemini AI to develop 'Thinking Robot' malware and data processing agent for spying purposes
But organizations do have the right to set the terms of use for their services, outside of regulatory scenarios where interoperability is required. Microsoft isn't obligated to ensure Windows runs macOS apps. I might want to be able to scrape all of LinkedIn's data with a Python script and analyze the social graph connections, but LinkedIn isn't obligated to allow that. Publishers that derive revenue from ads don't have to accommodate visitors who block ads. Ticket scalpers may want to automate the purchase of concert tickets for resale, but ticket vendors don't have to cooperate.
Having the freedom to install and use software is important. But that doesn't mean the use of that software will or should be welcomed everywhere.
The war for control
Amazon's move to force Perplexity to keep its bot at bay is going to be repeated elsewhere because tech industry incumbents today are mostly in the business of gatekeeping and avoiding competition. Amazon says it's concerned that automated purchases by Comet degrade the customer experience. But it's also a matter of control, of owning the customer relationship, and having access to transactional data.
That tendency is what got us here in the first place. Companies like Amazon, Apple, Google, Microsoft, and Meta have more or less eliminated competition in their respective markets, and regulatory intervention has done very little. So once venture capitalists and entrepreneurs realized that AI models with natural language interfaces might allow newcomers to disintermediate incumbents, they poured money into the AI business in the hope of displacing Google and its peers.
AI agents also involve potential liability – they don't always get things right.
"The problem is that AI providers may not want to be held liable because they do not want to be exposed to unquantifiable risks as they cannot anticipate how the AI users will deploy their AI," wrote Garry Gabison, Queen Mary University of London, School of Law, and Patrick Xian, University of California, San Francisco, in [18]a recent legal paper that examines agentic liability concerns.
The authors argue that AI providers and users will need to address risks through contractual terms that spell out liability and compensation mechanisms in case AI agents cause harm. Establishing ground rules is just the sort of thing that Perplexity is trying to avoid by insisting that its software be allowed to roam and interact unhindered, without any agreement from Amazon.
But then that's the AI industry in a nutshell: avoiding liability for training on data without permission, avoiding liability for AI models that hallucinate and amplify security risks, and avoiding negotiation when AI models interact with third-party services. It's using the well-worn playbook of tech disruptors for the last several decades: Don't waste time asking for permission. Establish market dominance first, then ask forgiveness.
Not universally beloved
AI agents have already spurred more automation, particularly for software development and deployment. But the industry's giddy, desperate optimism looks unlikely to survive contact with human-facing systems outside the software industry. Klarna's decision last year [19]to hire people for customer service after previously firing them shows that AI isn't necessarily right for every role.
To put it bluntly, a lot of people don't like AI or have reservations about it. I saw this in person recently at China Live, a restaurant in San Francisco. The AI agent in this case is embodied in [20]a service robot that takes to-go orders to a curbside station and returns to the kitchen for the next load.
I was there on a Saturday evening and it was busy. The wait staff often had to step aside and wait for the robot as it moved haltingly through the restaurant. On several occasions, staff had to disable the bot to walk past and then re-activitate it via the touch screen because the machine just wasn't nimble.
I asked the woman attending our table what she thought about her automated co-worker.
It was helpful, she replied, because it freed human staff from making repeated trips to the staging area for pickup orders. But the bot also slowed the flow of traffic and caused other problems. One time, she said, it collided with an employee and knocked a tray of drinks to the floor.
A human had to clean up the mess. ®
Get our [21]Tech Resources
[1] https://simonwillison.net/2025/May/22/tools-in-a-loop/
[2] https://cloud.google.com/transform/agentic-commerce-retailers-can-prepare-for-the-new-shopping-era-ai
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2aQzUJSQViTQoRAj5W4WVrAAAAFA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aQzUJSQViTQoRAj5W4WVrAAAAFA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aQzUJSQViTQoRAj5W4WVrAAAAFA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-agentic-commerce-opportunity-how-ai-agents-are-ushering-in-a-new-era-for-consumers-and-merchants
[7] https://www.penguinrandomhouse.com/books/634029/when-mckinsey-comes-to-town-by-walt-bogdanich-and-michael-forsythe/
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aQzUJSQViTQoRAj5W4WVrAAAAFA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://www.theregister.com/2025/11/05/amazon_perplexity_comet_legal_threat/
[10] https://www.perplexity.ai/hub/blog/bullying-is-not-innovation
[11] https://techcrunch.com/2024/11/04/perplexity-ceo-offers-ai-companys-services-to-replace-striking-nyt-staff/
[12] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aQzUJSQViTQoRAj5W4WVrAAAAFA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[13] https://www.nature.com/articles/d41586-025-03471-0
[14] https://www.theregister.com/2025/11/06/microsoft_copilot_m365_apology/
[15] https://www.theregister.com/2025/11/05/perplexity_1t_parameter_models_aws_efa/
[16] https://www.theregister.com/2025/11/05/sony_ai_vision_model_benchmark/
[17] https://www.theregister.com/2025/11/05/attackers_experiment_with_gemini_ai/
[18] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5210666
[19] https://gizmodo.com/klarna-hiring-back-human-help-after-going-all-in-on-ai-2000600767
[20] https://www.tiktok.com/@brokeass_stuart/video/7270175046251875627
[21] https://whitepapers.theregister.com/
Software agents, as [1]defined by developer Simon Willison, are "[AI] models using tools in a loop." Wire an LLM into a browser and maybe, if not derailed by mistakes, security controls, or lack of contextual data, the agentic system can carry out a request to purchase a specific item on a website or book a trip on an airline.
"The retail world is shifting to agentic commerce, where AI agents act for people and businesses, creating a more responsive shopping experience," wrote Kapil Dabi, market lead for retail and consumer industries at Google Cloud, in a [2]blog post last month.
[3]
Dabi's sentiment reflects the labor-averse tech industry's interest in software that can act on behalf of people, carrying out directives derived from a user's prompt or query.
[4]
[5]
"Agentic commerce – shopping powered by AI agents acting on our behalf – represents a seismic shift in the marketplace," gushes consultancy [6]McKinsey . "It moves us toward a world in which AI anticipates consumer needs, navigates shopping options, negotiates deals, and executes transactions, all in alignment with human intent yet acting independently via multistep chains of actions enabled by reasoning models."
Set aside for the time being that McKinsey is [7]described in a recent book as providing advice that "boils down to major cost-cutting, including layoffs and maintenance reductions, to drive up short-term profits, thereby boosting a company's stock price and the wealth of its executives who hire it, at the expense of workers and safety measures."
[8]
Focus instead on the oversimplification of agentic commerce and the issues it raises for the companies involved, for those who would deploy software agents, for the people who presently do work that agents would take, for society at large, and for the legal system.
AI companies want you to see agents as equivalent to human labor
Earlier this week, [9]Amazon demanded that Perplexity stop allowing its Comet browser to make automated purchases on the e-commerce giant's website. This might be taken as the canary in the copilot mine, but really the spat is just an extension of legal battles that challenge the rights of AI companies to ingest the internet's data without permission or compensation and sell it back by the token.
Perplexity's arguments bear further examination. In its [10]blog post calling out Amazon for bullying, the company says, "Today, Amazon announced it does not believe in your right to hire labor, to have an assistant or an employee acting on your behalf."
That's not what Amazon said, and there's a certain irony about a company that has [11]offered its services as a substitute for labor making that claim.
Perplexity asserts that AI and human labor should be seen as the same thing. "[W]ith the rise of agentic AI, software is also becoming labor: an assistant, an employee, an agent," the company's blog post declares.
[12]
It's true that current generative AI models pass the Turing Test, at least as it was initially imagined – hence [13]the call for a more relevant "Imitation Game." But software is not the same as human labor.
Software agents and human action are not interchangeable. In the context of online interaction, there are technical differences between the way agentic systems and people browse the web that translate into cost differences. Software may consume computing and network resources at a different rate than human-operated browsing, and data exchanged during that interaction may have different value. And third parties involved in this process want to know whether they are serving ads or collecting analytics data from machines or people.
Then there are the legal differences. "Publishers and corporations have no right to discriminate against users based on which AI they've chosen to represent them," Perplexity argues.
[14]Microsoft apologizes for not explaining cheaper no-AI M365 plans, and all it took was a government lawsuit
[15]Perplexity shows how to run monster AI models more efficiently on aging GPUs, AWS networks
[16]Sony rolls out a standard way to measure bias in how AI describes what it 'sees'
[17]Attackers abuse Gemini AI to develop 'Thinking Robot' malware and data processing agent for spying purposes
But organizations do have the right to set the terms of use for their services, outside of regulatory scenarios where interoperability is required. Microsoft isn't obligated to ensure Windows runs macOS apps. I might want to be able to scrape all of LinkedIn's data with a Python script and analyze the social graph connections, but LinkedIn isn't obligated to allow that. Publishers that derive revenue from ads don't have to accommodate visitors who block ads. Ticket scalpers may want to automate the purchase of concert tickets for resale, but ticket vendors don't have to cooperate.
Having the freedom to install and use software is important. But that doesn't mean the use of that software will or should be welcomed everywhere.
The war for control
Amazon's move to force Perplexity to keep its bot at bay is going to be repeated elsewhere because tech industry incumbents today are mostly in the business of gatekeeping and avoiding competition. Amazon says it's concerned that automated purchases by Comet degrade the customer experience. But it's also a matter of control, of owning the customer relationship, and having access to transactional data.
That tendency is what got us here in the first place. Companies like Amazon, Apple, Google, Microsoft, and Meta have more or less eliminated competition in their respective markets, and regulatory intervention has done very little. So once venture capitalists and entrepreneurs realized that AI models with natural language interfaces might allow newcomers to disintermediate incumbents, they poured money into the AI business in the hope of displacing Google and its peers.
AI agents also involve potential liability – they don't always get things right.
"The problem is that AI providers may not want to be held liable because they do not want to be exposed to unquantifiable risks as they cannot anticipate how the AI users will deploy their AI," wrote Garry Gabison, Queen Mary University of London, School of Law, and Patrick Xian, University of California, San Francisco, in [18]a recent legal paper that examines agentic liability concerns.
The authors argue that AI providers and users will need to address risks through contractual terms that spell out liability and compensation mechanisms in case AI agents cause harm. Establishing ground rules is just the sort of thing that Perplexity is trying to avoid by insisting that its software be allowed to roam and interact unhindered, without any agreement from Amazon.
But then that's the AI industry in a nutshell: avoiding liability for training on data without permission, avoiding liability for AI models that hallucinate and amplify security risks, and avoiding negotiation when AI models interact with third-party services. It's using the well-worn playbook of tech disruptors for the last several decades: Don't waste time asking for permission. Establish market dominance first, then ask forgiveness.
Not universally beloved
AI agents have already spurred more automation, particularly for software development and deployment. But the industry's giddy, desperate optimism looks unlikely to survive contact with human-facing systems outside the software industry. Klarna's decision last year [19]to hire people for customer service after previously firing them shows that AI isn't necessarily right for every role.
To put it bluntly, a lot of people don't like AI or have reservations about it. I saw this in person recently at China Live, a restaurant in San Francisco. The AI agent in this case is embodied in [20]a service robot that takes to-go orders to a curbside station and returns to the kitchen for the next load.
I was there on a Saturday evening and it was busy. The wait staff often had to step aside and wait for the robot as it moved haltingly through the restaurant. On several occasions, staff had to disable the bot to walk past and then re-activitate it via the touch screen because the machine just wasn't nimble.
I asked the woman attending our table what she thought about her automated co-worker.
It was helpful, she replied, because it freed human staff from making repeated trips to the staging area for pickup orders. But the bot also slowed the flow of traffic and caused other problems. One time, she said, it collided with an employee and knocked a tray of drinks to the floor.
A human had to clean up the mess. ®
Get our [21]Tech Resources
[1] https://simonwillison.net/2025/May/22/tools-in-a-loop/
[2] https://cloud.google.com/transform/agentic-commerce-retailers-can-prepare-for-the-new-shopping-era-ai
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2aQzUJSQViTQoRAj5W4WVrAAAAFA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aQzUJSQViTQoRAj5W4WVrAAAAFA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aQzUJSQViTQoRAj5W4WVrAAAAFA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-agentic-commerce-opportunity-how-ai-agents-are-ushering-in-a-new-era-for-consumers-and-merchants
[7] https://www.penguinrandomhouse.com/books/634029/when-mckinsey-comes-to-town-by-walt-bogdanich-and-michael-forsythe/
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aQzUJSQViTQoRAj5W4WVrAAAAFA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://www.theregister.com/2025/11/05/amazon_perplexity_comet_legal_threat/
[10] https://www.perplexity.ai/hub/blog/bullying-is-not-innovation
[11] https://techcrunch.com/2024/11/04/perplexity-ceo-offers-ai-companys-services-to-replace-striking-nyt-staff/
[12] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aQzUJSQViTQoRAj5W4WVrAAAAFA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[13] https://www.nature.com/articles/d41586-025-03471-0
[14] https://www.theregister.com/2025/11/06/microsoft_copilot_m365_apology/
[15] https://www.theregister.com/2025/11/05/perplexity_1t_parameter_models_aws_efa/
[16] https://www.theregister.com/2025/11/05/sony_ai_vision_model_benchmark/
[17] https://www.theregister.com/2025/11/05/attackers_experiment_with_gemini_ai/
[18] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5210666
[19] https://gizmodo.com/klarna-hiring-back-human-help-after-going-all-in-on-ai-2000600767
[20] https://www.tiktok.com/@brokeass_stuart/video/7270175046251875627
[21] https://whitepapers.theregister.com/