Workplaces Look For Cheaper AI As 'Tokenmaxxing' Fades As a Corporate Fad (apnews.com)
- Reference: 0184741326
- News link: https://news.slashdot.org/story/26/07/28/2242250/workplaces-look-for-cheaper-ai-as-tokenmaxxing-fades-as-a-corporate-fad
- Source link: https://apnews.com/article/ai-token-openai-anthropic-corporate-31bb80ac1cd7862d05f6397177d826b1
> A corporate fad of "tokenmaxxing" on artificial intelligence technology is [1]hitting its limits as workplaces throwing AI at everything are seeing the costs rise without a similar spike in productivity. What started as tech industry-fueled springtime hype over squeezing as much AI-generated work as possible out of products like OpenAI's ChatGPT and Anthropic's Claude has shifted to a summertime backlash. [...] Just a few months ago, Silicon Valley executives were [2]promoting high token consumption as a signal of high-performing employees. The stereotypical tokenmaxxer was staying up late -- perhaps ignoring their significant other -- while orchestrating an army of 24-hour AI agents performing work on their behalf. [...] The trend boosted revenue for leading AI large language model developers like Anthropic and OpenAI, but it fizzled as it became apparent it wasn't necessarily the best strategy for everyone else.
>
> [...] Bain & Company management consultant Jue Wang said many of the big businesses her firm advises have been taking a closer look at returns on their AI investments. "The token cost for them has been doubling, almost every other month," she said. "Let's say $200 per developer per month. Multiply that by 20,000 developers, which is often what we're dealing with at these companies, and that quickly [3]gets you to a number that is not a line item that any general manager has planned for." Sometimes that just means not using the AI equivalent of a sledgehammer to crack a nut. "Not everything needs a Claude Opus 4.6," she said of one of Anthropic's more capable models suited to software engineering or deep research. "And yet you see so many companies, so many users, default to using Opus for everything, including generating emails." That's led to a search for tools that do AI "model routing" -- in which easier queries get automatically sent to cheaper and more efficient AI systems and more complex tasks go to more powerful models.
>
> [...] At the same time, those who favor racking up as many tokens as possible are [4]having a field day with new open-source models from Chinese startups like Moonshot's Kimi or Zhipu's GLM , which nearly match the capabilities of top U.S. models at a [5]fraction of the price . "There is some validity to the theory that this could push tokenmaxxing a little bit further," said Raffi Krikorian, the chief technology officer at Mozilla. "But if we look at the industry overall, I think it's realizing that tokenmaxxing is a dumb thing." It's similar, Krikorian said, to how software companies once considered how many lines of code a programmer wrote to be a good metric of productivity. That later fell out of favor. "I think tokenmaxxing is moving through the exact same pattern," he said. "I think this is going to be an interesting blip that we're all going to look back to laugh at in a year."
[1] https://tech.slashdot.org/story/26/07/22/179241/the-army-is-burning-through-its-ai-tokens
[2] https://slashdot.org/story/26/05/12/1724231/amazon-employees-are-tokenmaxxing-due-to-pressure-to-use-ai-tools
[3] https://it.slashdot.org/story/26/06/28/0519223/developer-ai-token-costs-could-exceed-their-salaries-in-two-years
[4] https://apnews.com/article/ai-token-openai-anthropic-corporate-31bb80ac1cd7862d05f6397177d826b1
[5] https://slashdot.org/story/26/07/17/1820200/china-just-erased-americas-ai-lead
I'm shocked! (Score:2)
Shocked, I tell you! I keep thinking this whole AI thing must be modeled on a pump 'n' dump scheme.
This is a good sign (Score:2)
Maybe they hype bubble is starting to burst.
Yes, AI is useful for many things. I genuinely *like* having AI write most of my code. But the pipe dream of orchestrating a bunch of AI bots doing the work of a whole team, is just a pipe dream. Humans can't manage that much work all at once, and neither can AI.
The point though is this: If companies are finally starting to scale back on AI expenditures, it will force a reckoning in the AI world. The AI companies won't any longer be able to just charge whatever th
CEO brain (Score:2)
2 weeks ago my large company you've heard of had a townhall meeting where the CEO, who had hitherto been quite levelheaded and sensible about the AI stuff, finally threw all-in with the AI boosters and encouraged everyone to use as much AI as we could, for every possibly use we could think of.
3 days later we were out of tokens and even the useful stuff like support ticket summaries were unavailable.
The only thing I find it genuinely useful for (Score:2)
Is that Google search has gotten so crap-tastic that I can use AI to cut through the crap.
From what I understand if you put a bunch of time into learning how to prompt it it can generate mostly workable code. I find when I ask it to generate code though the code inevitably has a few mistakes and bugs. And with some of the Python code I've worked on porting from version 2 to 3 it gives you some of the most bizarre code I've ever seen. Like a mishmash between the two versions. The default python conversio