'Results Were Fudged': Departing Meta AI Chief Confirms Llama 4 Benchmark Manipulation (ft.com)
(Friday January 02, 2026 @11:00AM (msmash)
from the move-fast-and-fudge-things dept.)
- Reference: 0180502235
- News link: https://tech.slashdot.org/story/26/01/02/1449227/results-were-fudged-departing-meta-ai-chief-confirms-llama-4-benchmark-manipulation
- Source link: https://www.ft.com/content/e3c4c2f6-4ea7-4adf-b945-e58495f836c2
Yann LeCun, Meta's outgoing chief AI scientist and one of the pioneers credited with laying the groundwork for modern AI, has acknowledged that the company's Llama 4 language model had its benchmark results manipulated before its April 2025 release. In [1]an interview with the Financial Times , LeCun said the "results were fudged a little bit" and that the team "used different models for different benchmarks to give better results."
Llama 4 was widely criticized as a flop at launch, and the company faced accusations of gaming benchmarks to make the model appear more capable than it was. LeCun said CEO Mark Zuckerberg was "really upset and basically lost confidence in everyone who was involved" in the release.
Zuckerberg subsequently "sidelined the entire GenAI organisation," according to LeCun. "A lot of people have left, a lot of people who haven't yet left will leave." LeCun himself is departing Meta after more than a decade to start a new AI research venture called Advanced Machine Intelligence Labs. He described the new hires brought in for Meta's superintelligence efforts as "completely LLM-pilled" -- a technology LeCun has repeatedly called "a dead end when it comes to superintelligence."
[1] https://www.ft.com/content/e3c4c2f6-4ea7-4adf-b945-e58495f836c2
Llama 4 was widely criticized as a flop at launch, and the company faced accusations of gaming benchmarks to make the model appear more capable than it was. LeCun said CEO Mark Zuckerberg was "really upset and basically lost confidence in everyone who was involved" in the release.
Zuckerberg subsequently "sidelined the entire GenAI organisation," according to LeCun. "A lot of people have left, a lot of people who haven't yet left will leave." LeCun himself is departing Meta after more than a decade to start a new AI research venture called Advanced Machine Intelligence Labs. He described the new hires brought in for Meta's superintelligence efforts as "completely LLM-pilled" -- a technology LeCun has repeatedly called "a dead end when it comes to superintelligence."
[1] https://www.ft.com/content/e3c4c2f6-4ea7-4adf-b945-e58495f836c2
LLM pilled? (Score:3)
by liqu1d ( 4349325 )
Ahh so moron then.
Oh dear (Score:2)
by Viol8 ( 599362 )
Sounds like Llama is going to be about as successful as the "metaverse". I guess this is what happens to a company whose foundations are built on the sands of IP theft and in the right place at the right time luck.
Re: (Score:2)
by Plugh ( 27537 )
You could say by llama4 the genAI division had lost its legs. But the Metaverse division was way ahead of them...
Not fudged! (Score:3)
by RitchCraft ( 6454710 )
The results were hallucinated. Get your corporate team speak terms correct people. The founders of AI would never intentionally lie. They are not sociopaths in any sense of the meaning. They are the benevolent holders of the newly born AI's hand. Llama states this explicitly.
Okay, this is Meta (Score:3)
Frankly, is there anyone who didn't already assume they weren't being honest? Lying is "in their DNA", as the saying goes.
Re: (Score:2)
> LeCun said CEO Mark Zuckerberg was "really upset and basically lost confidence in everyone who was involved" in the release.
Only those that can prevaricate believably will please the Zuk. Confidence at Meta is built on a tissue of lies.
Re: (Score:2)
I suspect that the team was given impossible goals to hit. Probably aggressive deadlines too. The same story everywhere.
I am not saying that makes it ok to cheat. I am just saying that problems like this start at the top, so being "really upset and losing confidence" is no evidence that leadership stands blameless for the team's failure.