AI coding hype overblown, Bain shrugs
- Reference: 1758652300
- News link: https://www.theregister.co.uk/2025/09/23/developers_genai_little_productivity_gains/
- Source link:
As things stand, generative AI in software development has failed to live up to the hype, the wide-ranging [1]Technology Report 2025 from management consultants Bain & Company says. Two-thirds of software firms have rolled out GenAI tools, but developer adoption is low among those, and teams using AI assistants report a productivity boost of perhaps 10 to 15 percent.
Meanwhile, another recent study from nonprofit research group Model Evaluation & Threat Research (METR) found that [2]AI coding tools actually made software developers slower , despite expectations to the contrary, because they had to spend time checking for and correcting errors made by the AI.
[3]
This is perhaps what Bain & Co means when it notes that the time saved often isn't redirected toward higher-value work, so even the modest gains that have been made have not translated into positive returns.
[4]
[5]
Early initiatives focused on using generative AI to produce code faster, but writing and testing code typically accounts for about 25 to 35 percent of the total development process, the report states, so speeding up this stage alone is not going to be effective at reducing time to market. Perhaps greater value will be found from applying generative AI across the entire development life cycle?
Nearly every phase ought to benefit, the report authors posit, from the discovery and requirements stages, through planning and design, to testing, deployment, and maintenance. This will call for process changes as well, since code review, integration, and release must keep pace with AI-powered coding, the thinking goes.
[6]
At this point, we come to the latest trendy buzzword, "agentic AI." Until now, generative AI has served as a smart assistant, a copilot with a human in control, the report says, but agentic AI will usher in more autonomous versions that can manage multiple steps of the development process with minimal human intervention.
Bain points to Cognition's Devin, an AI "software engineer" unveiled last year and touted as capable of building whole applications from natural-language prompts.
However, as The Register has [7]reported , Devin proved to be far from satisfactory at its job, completing just three out of 20 tasks successfully in tests conducted by a group of data scientists earlier this year, and often "getting stuck in technical dead-ends or producing overly complex, unusable solutions."
[8]
Research biz Gartner forecasts that more than 40 percent of agentic AI projects will be cancelled by the end of 2027. And a benchmarking study by Carnegie Mellon finds that for multi-step office tasks, [9]AI agents fail roughly 70 percent of the time .
The Bain report also points to a number of roadblocks that stand in the way of broader generative AI adoption in development.
First is a lack of executive direction, whereby any project is likely to run out of steam if senior leadership doesn't set clear objectives.
But another factor is resistance. Some engineers distrust AI (we can't imagine why) or worry that it will undermine their role, the report states. Three-quarters of companies say the hardest part of adoption is getting people to change how they work, and overcoming this requires strong change management.
The report flags an inevitable skills gap in areas such as writing prompts and reviewing AI output. Many firms have not bothered with training, the report claims.
A lack of adequate performance tracking is additionally blamed. Without clear key performance indicators, you cannot realistically prove generative AI's value, the report authors say, and even real productivity gains won't show up in business terms.
[10]GenAI FOMO has spurred businesses to light nearly $40 billion on fire
[11]Generative AI isn't just a matter of life and death. It's far more important than that
[12]AI coding tools make developers slower but they think they're faster, study finds
[13]AI agents get office tasks wrong around 70% of the time, and a lot of them aren't AI at all
However, tech leaders at a recent [14]Wall Street Journal Leadership Institute Technology Council Summit claimed that it's nearly impossible to measure general productivity gains from using AI tools. This raises the question of why they are bothering to invest so much money in it.
Bain's report asserts that to break out of "pilot mode" and get real returns from generative AI, firms must be radical and frame their roadmap as an AI-native reinvention of the software life cycle, integrating it seamlessly into every phase of development.
In other words, corporate leadership needs to be bold with their AI vision, then back it up with clear goals and measurable outcomes to ensure that investment pays off. Some companies already report 25 to 30 percent productivity boosts by pairing generative AI with end-to-end process transformation, the report claims. But that's a tough call for a manager to make if a pilot project is just not showing the expected benefits. ®
Get our [15]Tech Resources
[1] https://www.bain.com/insights/topics/technology-report/
[2] https://www.theregister.com/2025/07/11/ai_code_tools_slow_down/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2aNMYdcks6tCCJumNrlWKCQAAAEI&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aNMYdcks6tCCJumNrlWKCQAAAEI&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aNMYdcks6tCCJumNrlWKCQAAAEI&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aNMYdcks6tCCJumNrlWKCQAAAEI&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2025/01/23/ai_developer_devin_poor_reviews/
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aNMYdcks6tCCJumNrlWKCQAAAEI&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[9] https://www.theregister.com/2025/06/29/ai_agents_fail_a_lot/
[10] https://www.theregister.com/2025/08/18/generative_ai_zero_return_95_percent/
[11] https://www.theregister.com/2025/08/18/opinion_column_gen_ai/
[12] https://www.theregister.com/2025/07/11/ai_code_tools_slow_down/
[13] https://www.theregister.com/2025/06/29/ai_agents_fail_a_lot/
[14] https://www.wsj.com/articles/stop-worrying-about-ais-return-on-investment-d5cbc822
[15] https://whitepapers.theregister.com/
Ah, you're missing the key detail. it doesn't work for small tasks because people have to spend so much time fixing its mistakes. So the best thing to do is make it do even more with even less supervision. If no humans are there meddling with their "corrections" the AI will be free to be truly revolutionary.
Innovation!
It kind of reminds me of the NRA 'solution' to school shootings, which is to arm the teachers.
Sounds familiar
Meanwhile, another recent study from nonprofit research group Model Evaluation & Threat Research (METR) found that AI coding tools actually made software developers slower, despite expectations to the contrary, because they had to spend time checking for and correcting errors made by the AI.
Now do offshoring development to 50-packs of cheap developers.
It's fab at directing money to big tech pockets though
The 5 year plan - fate or folly?
The US economy has degenerated into committee driven centrally planned 5-year planning.
McKinsey: Scaling bigger, faster, cheaper data centers with smarter designs - Powering AI and new computation technologies can accelerate data center project delivery and optimize spending to reduce the projected $1.7 trillion global spend through 2030 by up to $250 billion.
There is a real end goal - real time monitoring and central control of all behavior.
Re: a real end goal - real time monitoring and central control
Doesn't Peter Thiel own the Eye of Sauron?
I hope
It will be good if the AI bubble bursts sooner than later. I wonder how much stuff that does work (no doubt not using LLM / Generative AI) that's labelled AI will be still labelled AI afterwards?
Dot Com bubble v.2.0
10/10 for hype.
Unfortunately, it doesn't work, has no ROI, and the putative figures are actually largely that than the amount of money available in the system.
It will find a niche, with tweaking, but those who have embedded it in their stuff may need to extract it or add 'off' buttons.
Microsoft added AI to Windows just as Thames Water added organic nutrients to English rivers. Neither were welcome.
Where are the provable success stories?
Where are the people outside of the AI industry who can show solid and provable results for doing anything with LLM AI? I'm not seeing them. What I'm seeing is people in the LLM AI business trying to sell their LLM service to other people, and AI dilettantes who say they are getting productivity improvements based on their "feelings" rather than on any measurable results.
My own experiments with it and the experience of other people that I've talked to has only resulted in massive disappointment in terms of how the hype has failed to live up to the reality
i come from a background of automating things in factories. You calculate the cost of having someone do a job, you calculate the cost of building and running a machine to do the same job, and if the second comes out cheaper than the first then the project gets done. If it doesn't come out provably and measurably cheaper, then the project gets cancelled before any capital gets spent. There's no screwing about with the sort of touchy-feely stuff we are seeing from AI LLM promoters.
If Bain & Co think that LLM AI is so great, why don't they tell us all about how they are using it in their company, and what unsupervised tasks they have successfully handed over to it? Surely they must have some.
Oh fuck off!
"Bain points to Cognition's Devin, an AI "software engineer" unveiled last year and touted as capable of building whole applications from natural-language prompts."
Go on then, do it. Prove what you are spewing out of you pathetic mouth or, I don't know, remove yourself from the gene pool by seeing how high you can bounce after leaping from an aircraft at 10'000ft without a parachute.
So, if it isn't working for small tasks, make people use it for everything? Yes that will fix it...