OpenAI's Operator agent wants to tackle your online chores – just don’t expect it to nail every task
(2025/01/23)
- Reference: 1737667583
- News link: https://www.theregister.co.uk/2025/01/23/openai_unveils_operator_agent/
- Source link:
OpenAI on Thursday launched a human-directed AI agent called Operator that can use a web browser by itself to accomplish various online tasks, or at least try to do so.
As [1]demonstrated by OpenAI CEO Sam Altman, software engineer Yash Kumar, researcher Casey Chu, and technical staff member Reiichiro Nakano, the Operator agent can perform online activities that require multiple steps and have specified parameters, such as booking a restaurant reservation through OpenTable within a certain time window or finding concert tickets for a specified performer within a given price range.
Just like you feed queries into OpenAI's ChatGPT to answer or respond to, users can give Operator instructions to carry out on the web as their personal assistant.
[2]
While individuals can perform such tasks on their own time at no extra cost, Operator can do so less reliably for US-based ChatGPT Pro subscribers, who pay [3]$200 per month . OpenAI subscribers to Plus, Team, and Enterprise tiers can expect access once the rough spots get ironed out.
[4]
[5]
[6]Operator is similar to Anthropic's [7]computer use API in that it combines the sort of browser automation enabled by software frameworks like Playwright and Selenium with text-based machine learning models and computer vision models for evaluating online words and images presented by browsing websites.
The overall aim is to automate web-based tasks to free humans from dull work ... or from employment all together.
[8]
"Operator can be asked to handle a wide variety of repetitive browser tasks such as filling out forms, ordering groceries, and even creating memes," OpenAI explains in a [9]write-up . "The ability to use the same interfaces and tools that humans interact with on a daily basis broadens the utility of AI, helping people save time on everyday tasks while opening up new engagement opportunities for businesses."
Those engagement opportunities presently involve negotiation with OpenAI. The biz said it is working with firms "like DoorDash, Instacart, OpenTable, Priceline, StubHub, Thumbtack, Uber, and others to ensure Operator addresses real-world needs while respecting established norms."
In other words, OpenAI's Operator may not interoperate well with web services that aren't expecting frequent automated contact. But to the extent agent-based interaction becomes popular, OpenAI and like-minded agent purveyors may devalue search as a marketing and sales channel, since automated connections to services – and partner preferencing paved by APIs – have the potential to reduce the need for human-driven queries.
[10]
OpenAI's agent is based on a model called [11]Computer-Using Agent (CUA), which combines GPT-4o's computer vision capabilities with training about how to deal with graphical user interfaces (GUIs). TikTok parent ByteDance recently released a similar open source project for automating GUI interactions, [12]UI-TARS .
[13]Google Gemini 2.0 Flash comes out with real-time conversation, image analysis
[14]Tool touted as 'first AI software engineer' is bad at its job, testers claim
[15]ChatGPT has a Thursday lie down
[16]Why is Big Tech hellbent on making AI opt-out?
According to OpenAI, CUA has achieved a 38.1 percent success rate on the OSWorld benchmark test for full computer use tasks, a 58.1 percent success rate on WebArena, and an 87 percent success rate on WebVoyager for web-based tasks. So use Operator when you're open to the possibility of not having your restaurant reservation booked or your groceries ordered.
CUA's computer vision modality works by capturing and storing screenshots, which it uses to perform chain-of-thought "reasoning" to perform the requested task. Those familiar with the controversy surrounding Microsoft's screen capturing [17]Recall feature in the latest version of Windows may have some concerns about how OpenAI handles screenshot data.
The Register inquired to OpenAI seeking clarification, and we've not heard back. The biz says disabling the "Improve the model for everyone" in ChatGPT settings – on by default – will prevent data in Operator from being used to train its models.
We know bad actors may try to misuse this technology
As mentioned above, users of Operator enter the task as a text prompt and the AI agent is expected to attempt to accomplish that task, breaking it down into a series of steps and awaiting user intervention when the user is required to log in, provide payment details or solve CAPTCHAs – something current computer vision models can do quite effectively, if allowed.
"We know bad actors may try to misuse this technology," OpenAI said. "That’s why we’ve designed Operator to refuse harmful requests and block disallowed content. Our moderation systems can issue warnings or even revoke access for repeated violations, and we’ve integrated additional review processes to detect and address misuse."
According to the ChatGPT maker, Operator has been designed to defend against adversarial websites that might try to lead the AI agent astray through hidden prompts, malicious code, or phishing attempts. The AI agent supposedly has been designed to detect and ignore prompt injection attacks. And it's said to operate under the supervision of a "monitor model" that watches for dubious behavior, augmented by anomaly detection processes involving human review and automated processes.
Nonetheless, OpenAI acknowledges, "no system is flawless and this is still a research preview."
Operator arrives amid what AI industry leaders have heralded as " [18]the agentic era ," a time when generative AI models apply multimodal text, audio, and vision capabilities to interact with other computing systems in order to tackle multi-step tasks that require some form of reasoning and progress assessment.
While AI agents may sound promising in theory, they've been something of a letdown in practice – possibly because every step in a complex task adds another opportunity for failure. A [19]recent evaluation of AI code helper Devin, for example, suggests further work will need to be done to make these systems reliable. ®
In other AI news...
US President Donald Trump [20]has made an [21]executive order calling for the development of AI systems "free from ideological bias or engineered social agendas,” and undoing Biden-era policies that "act as barriers" to progress.
Anthropic has added a [22]Citations feature to its Claude API. "Claude can now provide detailed references to the exact sentences and passages it uses to generate responses, leading to more verifiable, trustworthy outputs," the lab, a rival to OpenAI, announced.
Get our [23]Tech Resources
[1] https://www.youtube.com/watch?v=CSE77wAdDLg
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Z5LKBv9jyF4FcyWCI7VqHgAAAFY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://openai.com/chatgpt/pricing/
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z5LKBv9jyF4FcyWCI7VqHgAAAFY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z5LKBv9jyF4FcyWCI7VqHgAAAFY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://operator.chatgpt.com/
[7] https://www.anthropic.com/news/3-5-models-and-computer-use
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z5LKBv9jyF4FcyWCI7VqHgAAAFY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://openai.com/index/introducing-operator/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z5LKBv9jyF4FcyWCI7VqHgAAAFY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[11] https://openai.com/index/computer-using-agent/
[12] https://github.com/bytedance/UI-TARS
[13] https://www.theregister.com/2024/12/11/google_gemini_20_flash_shines/
[14] https://www.theregister.com/2025/01/23/ai_developer_devin_poor_reviews/
[15] https://www.theregister.com/2025/01/23/chatgpt_has_a_thursday_lie/
[16] https://www.theregister.com/2025/01/23/why_is_ai_optout/
[17] https://www.theregister.com/2024/12/09/windows_recall_intel_amd/
[18] https://www.theregister.com/2024/05/30/nvidia_ai_conference/
[19] https://www.theregister.com/2025/01/23/ai_developer_devin_poor_reviews/
[20] https://apnews.com/article/trump-ai-artificial-intelligence-executive-order-eef1e5b9bec861eaf9b36217d547929c
[21] https://www.whitehouse.gov/presidential-actions/2025/01/removing-barriers-to-american-leadership-in-artificial-intelligence/
[22] https://www.anthropic.com/news/introducing-citations-api
[23] https://whitepapers.theregister.com/
As [1]demonstrated by OpenAI CEO Sam Altman, software engineer Yash Kumar, researcher Casey Chu, and technical staff member Reiichiro Nakano, the Operator agent can perform online activities that require multiple steps and have specified parameters, such as booking a restaurant reservation through OpenTable within a certain time window or finding concert tickets for a specified performer within a given price range.
Just like you feed queries into OpenAI's ChatGPT to answer or respond to, users can give Operator instructions to carry out on the web as their personal assistant.
[2]
While individuals can perform such tasks on their own time at no extra cost, Operator can do so less reliably for US-based ChatGPT Pro subscribers, who pay [3]$200 per month . OpenAI subscribers to Plus, Team, and Enterprise tiers can expect access once the rough spots get ironed out.
[4]
[5]
[6]Operator is similar to Anthropic's [7]computer use API in that it combines the sort of browser automation enabled by software frameworks like Playwright and Selenium with text-based machine learning models and computer vision models for evaluating online words and images presented by browsing websites.
The overall aim is to automate web-based tasks to free humans from dull work ... or from employment all together.
[8]
"Operator can be asked to handle a wide variety of repetitive browser tasks such as filling out forms, ordering groceries, and even creating memes," OpenAI explains in a [9]write-up . "The ability to use the same interfaces and tools that humans interact with on a daily basis broadens the utility of AI, helping people save time on everyday tasks while opening up new engagement opportunities for businesses."
Those engagement opportunities presently involve negotiation with OpenAI. The biz said it is working with firms "like DoorDash, Instacart, OpenTable, Priceline, StubHub, Thumbtack, Uber, and others to ensure Operator addresses real-world needs while respecting established norms."
In other words, OpenAI's Operator may not interoperate well with web services that aren't expecting frequent automated contact. But to the extent agent-based interaction becomes popular, OpenAI and like-minded agent purveyors may devalue search as a marketing and sales channel, since automated connections to services – and partner preferencing paved by APIs – have the potential to reduce the need for human-driven queries.
[10]
OpenAI's agent is based on a model called [11]Computer-Using Agent (CUA), which combines GPT-4o's computer vision capabilities with training about how to deal with graphical user interfaces (GUIs). TikTok parent ByteDance recently released a similar open source project for automating GUI interactions, [12]UI-TARS .
[13]Google Gemini 2.0 Flash comes out with real-time conversation, image analysis
[14]Tool touted as 'first AI software engineer' is bad at its job, testers claim
[15]ChatGPT has a Thursday lie down
[16]Why is Big Tech hellbent on making AI opt-out?
According to OpenAI, CUA has achieved a 38.1 percent success rate on the OSWorld benchmark test for full computer use tasks, a 58.1 percent success rate on WebArena, and an 87 percent success rate on WebVoyager for web-based tasks. So use Operator when you're open to the possibility of not having your restaurant reservation booked or your groceries ordered.
CUA's computer vision modality works by capturing and storing screenshots, which it uses to perform chain-of-thought "reasoning" to perform the requested task. Those familiar with the controversy surrounding Microsoft's screen capturing [17]Recall feature in the latest version of Windows may have some concerns about how OpenAI handles screenshot data.
The Register inquired to OpenAI seeking clarification, and we've not heard back. The biz says disabling the "Improve the model for everyone" in ChatGPT settings – on by default – will prevent data in Operator from being used to train its models.
We know bad actors may try to misuse this technology
As mentioned above, users of Operator enter the task as a text prompt and the AI agent is expected to attempt to accomplish that task, breaking it down into a series of steps and awaiting user intervention when the user is required to log in, provide payment details or solve CAPTCHAs – something current computer vision models can do quite effectively, if allowed.
"We know bad actors may try to misuse this technology," OpenAI said. "That’s why we’ve designed Operator to refuse harmful requests and block disallowed content. Our moderation systems can issue warnings or even revoke access for repeated violations, and we’ve integrated additional review processes to detect and address misuse."
According to the ChatGPT maker, Operator has been designed to defend against adversarial websites that might try to lead the AI agent astray through hidden prompts, malicious code, or phishing attempts. The AI agent supposedly has been designed to detect and ignore prompt injection attacks. And it's said to operate under the supervision of a "monitor model" that watches for dubious behavior, augmented by anomaly detection processes involving human review and automated processes.
Nonetheless, OpenAI acknowledges, "no system is flawless and this is still a research preview."
Operator arrives amid what AI industry leaders have heralded as " [18]the agentic era ," a time when generative AI models apply multimodal text, audio, and vision capabilities to interact with other computing systems in order to tackle multi-step tasks that require some form of reasoning and progress assessment.
While AI agents may sound promising in theory, they've been something of a letdown in practice – possibly because every step in a complex task adds another opportunity for failure. A [19]recent evaluation of AI code helper Devin, for example, suggests further work will need to be done to make these systems reliable. ®
In other AI news...
US President Donald Trump [20]has made an [21]executive order calling for the development of AI systems "free from ideological bias or engineered social agendas,” and undoing Biden-era policies that "act as barriers" to progress.
Anthropic has added a [22]Citations feature to its Claude API. "Claude can now provide detailed references to the exact sentences and passages it uses to generate responses, leading to more verifiable, trustworthy outputs," the lab, a rival to OpenAI, announced.
Get our [23]Tech Resources
[1] https://www.youtube.com/watch?v=CSE77wAdDLg
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Z5LKBv9jyF4FcyWCI7VqHgAAAFY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://openai.com/chatgpt/pricing/
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z5LKBv9jyF4FcyWCI7VqHgAAAFY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z5LKBv9jyF4FcyWCI7VqHgAAAFY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://operator.chatgpt.com/
[7] https://www.anthropic.com/news/3-5-models-and-computer-use
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z5LKBv9jyF4FcyWCI7VqHgAAAFY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://openai.com/index/introducing-operator/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z5LKBv9jyF4FcyWCI7VqHgAAAFY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[11] https://openai.com/index/computer-using-agent/
[12] https://github.com/bytedance/UI-TARS
[13] https://www.theregister.com/2024/12/11/google_gemini_20_flash_shines/
[14] https://www.theregister.com/2025/01/23/ai_developer_devin_poor_reviews/
[15] https://www.theregister.com/2025/01/23/chatgpt_has_a_thursday_lie/
[16] https://www.theregister.com/2025/01/23/why_is_ai_optout/
[17] https://www.theregister.com/2024/12/09/windows_recall_intel_amd/
[18] https://www.theregister.com/2024/05/30/nvidia_ai_conference/
[19] https://www.theregister.com/2025/01/23/ai_developer_devin_poor_reviews/
[20] https://apnews.com/article/trump-ai-artificial-intelligence-executive-order-eef1e5b9bec861eaf9b36217d547929c
[21] https://www.whitehouse.gov/presidential-actions/2025/01/removing-barriers-to-american-leadership-in-artificial-intelligence/
[22] https://www.anthropic.com/news/introducing-citations-api
[23] https://whitepapers.theregister.com/
Re: I've long thought "personal assistant" is the huge opportunity for Siri, Gemini etc.
Inventor of the Marmite Laser
£10/£20 a month? I suppose I'd give it a go if they paid me that much.
Anonymous Coward
So an AI agent decides to place an order. What's the contractual relationship between you, the supplier, the AI, whoever provides the AI and the person or company on whom the order has been placed?
Who sues whom if things go titsup / payment fails / warranty claim / recurring payments / minors buying age restricted items?
Arriving at the restaurant...
Andy Non
Sorry sir, your table for 12 is booked for 2 pm.
But I asked the agent to book a table for 2 at 12.
Thank you..
IGotOut
....for solving a problem I don't actually have.
Let's see, so I type into a website some text to say I want to book a table at Chain Restaurant at 8pm.
Off it goes and the following happens
It finds the table, but is unable to put a deposit because, sure I'll give that info to chatGPT
It is unable to find a table, so books it at a completely different time.
It books a table with same name in a different country.
This is what they don't get.
If I have to type into chatgpt to go off and do a booking, I may as well type in the fucking website and know it's been done.
Do these people actually ASK real people what they want?
No need, to answer I already know.
No. Not now, not ever.
Tron
I'm so glad you have to pay for this crap, as it offers an easy way of avoiding it.
quidquid Latine dictum sit altum videtur
Steve Aubrey
'And it's said to operate under the supervision of a "monitor model" that watches for dubious behavior'
Quis custodiet ipsos custodes?
I've long thought "personal assistant" is the huge opportunity for Siri, Gemini etc.
If it works it would really help your daily life. But I'm going to be letting others beta test it for a few years before I risk letting it go wild with my credit card based giving it instructions I hope it understands and follows.
I wouldn't pay $200/month for it or anything like that either. But I could see $10 or $20 a month, depending on how much time it actually saved me per week.