MIT Report: 95% of Generative AI Pilots at Companies Are Failing (fortune.com)
- Reference: 0178764990
- News link: https://slashdot.org/story/25/08/19/146205/mit-report-95-of-generative-ai-pilots-at-companies-are-failing
- Source link: https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/
> Despite the rush to integrate powerful new models, about 5% of AI pilot programs achieve rapid revenue acceleration; the vast majority stall, delivering little to no measurable impact on P&L. The research -- based on 150 interviews with leaders, a survey of 350 employees, and an analysis of 300 public AI deployments -- paints a clear divide between success stories and stalled projects.
>
> To unpack these findings, I spoke with Aditya Challapally, the lead author of the report, and a research contributor to project NANDA at MIT. "Some large companies' pilots and younger startups are really excelling with generative AI," Challapally said. Startups led by 19- or 20-year-olds, for example, "have seen revenues jump from zero to $20 million in a year," he said. "It's because they pick one pain point, execute well, and partner smartly with companies who use their tools," he added.
>
> But for 95% of companies in the dataset, generative AI implementation is falling short. The core issue? Not the quality of the AI models, but the "learning gap" for both tools and organizations. While executives often blame regulation or model performance, MIT's research points to flawed enterprise integration. Generic tools like ChatGPT excel for individuals because of their flexibility, but they stall in enterprise use since they don't learn from or adapt to workflows, Challapally explained.
[1] https://docs.google.com/forms/d/e/1FAIpQLSc8rU8OpQWU44gYDeZyINUZjBFwu--1uTbxixK_PRSVrfaH8Q/viewform
[2] https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/
Startup vs established corp (Score:4, Insightful)
Startup: Hey, let us use AI to solve a specific problem.
Established corp: Hey, let us use AI to resolve a problem we created by our bloated processes and decisions, and while at there, to reduce workforce.
No wonder why it is working for a few of them.
Re: (Score:2)
This might also have something to do with the fact that AIs still hallucinate a lot and are fundamentally unreliable.
Individuals using AIs casually can easily check their facts. But when actual responsibility is accorded to an AI, as in a corporate setting, it is going to fail.
Re: (Score:2)
> when actual responsibility is accorded to an AI, as in a corporate setting, it is going to fail
This. And for a corporation, the whole point is that the AI's output has to be used without having humans look at it, because humans cost [more] money. This is a fundamental, well-discussed issue with many LLM use cases that live in important spaces - to act on the output, you need human invigilators, and once you add enough of those to restore safety to the system, the AI becomes irrelevant.
Re: Startup vs established corp (Score:2)
Why assume humans don't make mistakes, hallucinate, lie, cheat, steal etc.?
What measurable impact on profit and loss do you have?
Claude 4.0 can't match braces!!! (Score:2)
> Startup: Hey, let us use AI to solve a specific problem. Established corp: Hey, let us use AI to resolve a problem we created by our bloated processes and decisions, and while at there, to reduce workforce.
> No wonder why it is working for a few of them.
The number is likely greater than 95%. I will wager those 5% who claim a gain are using misleading math and staked their careers on the company's LLM future. Remember, in order to be more than a fun toy to play with, it has to bring in more revenue or reduce headcount. If you're paying a human being to do it now, it requires a lot more accuracy than today's LLMs offer.
EVERYONE in the tech world is pushing LLMs to boost developer productivity. Claude 4.0 generates Java that compiles maybe 50% of the t
Why not include fundraising as revenue? (Score:2)
How much of a factor is the revenue from a potential IPO? Or getting bought out? Why does MIT ignore the reality of stock markets as revenue generators that aren't necessarily reported in P&L but likely dwarf real revenue?
Re: (Score:2)
Just remember to devalue the potential revenue by the percentage of companies that actually gain long term value from an IPO or buy out.
Re: Why not include fundraising as revenue? (Score:2)
If you bet on enough, if only one hits, will it be worth it?
Re: (Score:2)
> Just remember to devalue the potential revenue by the percentage of companies that actually gain long term value
Why? Whose "success" are we talking about here? I would argue that if you're an entrepreneur and you build a hallucination, and it sells to someone so you, personally, make out like a bandit - that's success.
Which means 5% are a success! (Score:3)
AI was all worth it.
Have they tried having more faith? (Score:2)
The AI prophets have promised that if you just put your faith, your trust, and your livelihood in the hands of the AI gods, all will be well. Clearly, the 95% didn't have enough faith, and didn't invest heavily enough in the AI. If they had, it would have worked out for them.
Re: Have they tried having more faith? (Score:2)
Have you considered that AI could have written that report to lull you into a false sense of security?
And there is a good reason... (Score:4, Interesting)
"they stall in enterprise use since they don't learn from or adapt to workflows"
Unlikely that many organizations will permit their AI model to 'learn' from workflows that are proprietary and considered trade secrets. Especially if their AI models need to talk to the mothership at all.
Re: (Score:2)
I am sure there is a classic try to force new technology onto the old process problem. However I would also guess that the larger the enterprise the more corner cases there are, and GenAI just isn't good at corner cases.
Re: And there is a good reason... (Score:2)
How many corner cases are there in spelling and grammar that AI gets spectacularly right?
Re: And there is a good reason... (Score:1)
When the case occurs tens of millions of times within the corpus that is used to train the model, itâ(TM)s no longer a corner case.
Re: And there is a good reason... (Score:2)
How many times was the corner case screwed up by fallible humans?
Re: (Score:2)
Equally, work that is transactional in nature, such as much customer support, lends itself to automation. AI is the latest tool to use to build those automations.
My last 3 roles were transactional, they just needed large knowledge sets to accomplish. And storage is cheap... Cheaper than my team.
No - GenAI shines where errors are acceptable... (Score:2)
And in enterpresi in 99% of places errors are not acceptable...
It is fine if the picture with your cat is a litte off... it is not if your enterprise serve has a huge security hole... or messes customers data...
Re: No - GenAI shines where errors are acceptable. (Score:2)
What is the Enterprise doing with your data, selling it to the government for tracking?
Terrible headline (Score:2)
The headline assumes that "no measurable impact on P&L" = failure
This is a symptom of a bigger problem, the idea that everything about a business can be reduced to measurable numbers
I suspect that AI tools will help some workers a bit and help others a lot, while annoying or impeding others
As for me, I find AI tools to be very useful in the work I do. It doesn't write my code or design my circuit, but it helps me find answers in difficult documentation
Re:Terrible headline FTFY (Score:1)
The headline assumes that "no measurable impact on P&L" = failure The headline assumes that "no measurable impact on P&L" == failure
Bullshit (Score:2)
"little to no value" is actually lost opportunity to do real business
Citation Needed (Score:3)
> Startups led by 19- or 20-year-olds, for example, "have seen revenues jump from zero to $20 million in a year," he said. "It's because they pick one pain point, execute well, and partner smartly with companies who use their tools
Citations needed. Which startups are implementing AI that results in 20 million dollar businesses? These stats are quite astonishing. But without any specifics whatsoever, it impossible to understand what is actually going on.
Which one pain point(s) have startups been able to address with AI that generated so much revenue?
Also, where is the actual report? One link is a Fortune fluff piece and the other is a link to a very vague Google forms survey.
Re: (Score:1)
> Which one pain point(s) have startups been able to address with AI that generated so much revenue?
I can think of only two: loneliness and inadequate access to AI. I'm pretty sure the only profitable uses for AI so far involve virtual girlfriends, porn, and selling AI to other companies.
startups (Score:4, Informative)
Isn't this true of most startups?