Microsoft Tells Engineers 'Tokenmaxxing Is Not What We Are Optimizing For'
- Reference: 0184853728
- News link: https://slashdot.org/story/26/08/04/1833219/microsoft-tells-engineers-tokenmaxxing-is-not-what-we-are-optimizing-for
- Source link:
> "As we accelerate our use of GitHub Copilot to deliver on our goals, we all need to be aware of how we consume tokens," Jay Parikh, an executive vice president at Microsoft said in an email to Microsoft employees. GitHub is owned by Microsoft, and GitHub Copilot is an AI coding tool. "Tokenmaxxing is not what we are optimizing for. I want all of us focused on maximizing outcomes that move the needle for our customers and our business." "As such, we are updating our internal guidance and managing token spend with the same discipline we apply to every other critical resource," Parikh said in the email.
>
> Parikh's email says that in an effort to "get greater value from our token investment" Microsoft is making OpenAI GPT-5.6, which is cheaper to use than other models, the default model for internal use. His email also links to updated internal Copilot guidelines stating that, as of July 2026, Microsoft divisions will have an "AI token budget target," and that employees can track their individual AI spending. "While there is no target spend value being shared at this time. The data shows that many engineers spend in the range of hundreds of dollars a month to a few thousand dollars in tokens," the guidelines say. They also say that some decisions may place further restrictions as they monitor spend.
>
> [...] Parikh's email said Microsoft will keep learning and adjusting its AI policies as models and products evolve, and stressed that he doesn't want to slow down the company's progress towards becoming "AI-first." "We are not optimizing for fewer tokens," he said. "We are optimizing for more impact per token.
[1] https://www.404media.co/microsoft-tells-engineers-tokenmaxxing-is-not-what-we-are-optimizing-for/
But how can line go up- (Score:2)
-if customers of Microsoft like Microsoft don't pay to make line go up?
Re: (Score:2)
Can I subscribe to your newsletter?
Re: (Score:1)
Translation?
Focus on customers? (Score:2)
When has Microsoft, at least in the last couple of decades, ever considered customers? Microsoft is forcing AI into everything, even if you tell him to stop, they'll proclaim you're wrong, and actually want the AI.
Re: Focus on customers? (Score:2)
They mean they're adjusting the focus on the sniper scope
Re: (Score:2)
That's hilarious!
WTAF?? (Score:2)
"telling engineers to focus on business results rather than maximizing AI usage"
why does that even need to be said?
especially for a longstanding for-profit, publicly traded corporation
Re: (Score:2)
As they try to cram copilot down the gullet of all their enterprise customers. It's laughable.
Re:WTAF?? (Score:5, Insightful)
Because a lot of corps are saying the opposite right now, saying you'll be judge on your use of AI, and punished for your lack of use of it.
Which is why you get insane shit like people punching bullet points into ChatGPT telling it to expand it into a full email, and the recipient receiving it and putting that into ChatGPT to summarize it as a set of bullet points.
I'd say I'm glad the companies who have been doing this are the ones who'll eventually go tits up when the bill comes in (which is more than the cost of tokens, it's the lack of control over their own businesses), but the reality is that management isn't the group that'll suffer. The people fired for non-compliance with this stupidity are merely the first in a long line of people who'll get hurt despite never doing anything more than what their managers said they had to do.
This timeline absolutely sucks.
This is pretty much the beginning of the crash (Score:3)
They have dropped the narrative about replacing people because it became totally unsustainable. But that narrative was the only thing that could have made the large LLMs profitable and it was what brought the employee-hating CEOs on board.
I do not think the hype will even make it to the end of the year.
Re:This is pretty much the beginning of the crash (Score:4, Insightful)
> But that narrative was the only thing that could have made the large LLMs profitable and it was what brought the employee-hating CEOs on board.
Doesn't have to be sustainable, just needs to keep businesses using them for long enough that critical business knowledge is lost among employees who become reliant upon the LLM providers for everything.
The question was never "Is this unsustainable if taken at face value", because the plot was never to sell useful services, it was to take over most US businesses by making them unable to run themselves. The question now is whether they managed to keep it going for long enough that this will happen, or whether the crash will happen before that to happen.
People often describe BigAI as a con. But BigAI is better described as a heist.
Duh! (Score:2)
> ... telling engineers to focus on business results rather than maximizing AI usage ...
The fact that this needed to be said tells us everything we need to know about corporate culture and managerial incompetence at Microsoft.
Re: (Score:2)
Oops! I somehow missed haruchai's prior comment saying pretty much what I just said. My bad.
Damn dyslexia (Score:3)
> Tokenmaxxing Is Not What We Are Optimizing For
I was half way through re-reading [1]The Two Tower [wikipedia.org] - (*sigh*)
[1] https://en.wikipedia.org/wiki/The_Two_Towers
Re: (Score:2)
Tolkienmaxxing! Now that's an idea I like!
Managers maximize metrics, news at 11 (Score:2)
Managers are always saying they want end results, but insist on those idiotic day to day metrics to measure it.
Who is "we"? (Score:2)
It's very important to recognize that what one executive says is rarely reflective of how middle-managers behave. The bottom line is that nobody should trust Microsoft, especially not it's employees. [1]#GetTheFacts [wikipedia.org]
[1] https://en.wikipedia.org/wiki/Criticism_of_Linux#Criticism_by_Microsoft
The metrics you chose create the behavior you see (Score:2)
If you're measuring token use, you're encouraging token use to game the stats.
Re: (Score:2)
Stupid metrics produce stupid results. And stupid people love stupid metrics. What else is new.
Re: The metrics you chose create the behavior you (Score:2)
Maybe they can measure token use and punish X number of standard deviations away from the norm?
They probably don't want people not using any, but also not people using too many?
Still probably not a great metric, but there's ways measuring it don't mean maximizing use, and it sounds like MS is saying that's what's happening.
Re: (Score:2)
> Maybe they can measure token use and punish X number of standard deviations away from the norm?
That just encourages everyone to conspire together to use more, and anybody not on board is in trouble for not using enough.
Re: (Score:1)
Well, maybe you get rid of the AI junk and go back to looking at what a person's real-world productivity is.
And, maybe MS or whoever shouldn't be using someone else's AI and have their own in-house AI that doesn't cost them an arm-and-leg (or anything at all) to have their own employees use.
Sure, the AI might generate the code you ask for in 30 minutes (what would've taken you 3 days)... how much staring at code and test runs and fixing stuff did that AI code need? Can that code be trusted to _only_ balanc
Re: (Score:2)
> If you're measuring token use, you're encouraging token use to game the stats.
Reminds of the Dilbert cartoon where the PHB announces they will reward staff for each bug they find and Wally announces "I'm off to write a mini-van..."