Does AI Really Make Coders Faster? (technologyreview.com)
- Reference: 0180431425
- News link: https://developers.slashdot.org/story/25/12/20/2335253/does-ai-really-make-coders-faster
- Source link: https://www.technologyreview.com/2025/12/15/1128352/rise-of-ai-coding-developers-2026/
But is AI making coders faster? "After speaking to more than 30 developers, technology executives, analysts, and researchers, MIT Technology Review [1]found that the picture is not as straightforward as it might seem ..."
> For some developers on the front lines, initial enthusiasm is waning as they bump up against the technology's limitations. And as a growing body of research suggests that the claimed productivity gains may be illusory, some are questioning whether the emperor is wearing any clothes.... Data from the developer analytics firm GitClear shows that most engineers are producing roughly 10% more durable code — code that isn't deleted or rewritten within weeks — since 2022, likely thanks to AI. But that gain has come with sharp declines in several measures of code quality. Stack Overflow's survey also found trust and positive sentiment toward AI tools falling significantly for the first time. And most provocatively, [2]a July study by the nonprofit research organization Model Evaluation & Threat Research (METR) showed that while experienced developers believed AI made them 20% faster, objective tests showed they were actually 19% slower...
>
> Developers interviewed by MIT Technology Review generally agree on where AI tools excel: producing "boilerplate code" (reusable chunks of code repeated in multiple places with little modification), writing tests, fixing bugs, and explaining unfamiliar code to new developers. Several noted that AI helps overcome the "blank page problem" by offering an imperfect first stab to get a developer's creative juices flowing. It can also let nontechnical colleagues quickly prototype software features, easing the load on already overworked engineers. These tasks can be tedious, and developers are typically glad to hand them off. But they represent only a small part of an experienced engineer's workload. For the more complex problems where engineers really earn their bread, many developers told MIT Technology Review, the tools face significant hurdles...
>
> The models also just get things wrong. Like all LLMs, coding models are prone to "hallucinating" — it's an issue built into how they work. But because the code they output looks so polished, errors can be difficult to detect, says James Liu, director of software engineering at the advertising technology company Mediaocean. Put all these flaws together, and using these tools can feel a lot like pulling a lever on a one-armed bandit. "Some projects you get a 20x improvement in terms of speed or efficiency," says Liu. "On other things, it just falls flat on its face, and you spend all this time trying to coax it into granting you the wish that you wanted and it's just not going to..." There are also more specific security concerns, she says. Researchers have discovered a worrying [3]class of hallucinations where models reference nonexistent software packages in their code. Attackers can exploit this by creating packages with those names that harbor vulnerabilities, which the model or developer may then unwittingly incorporate into software.
Other key points from the article:
LLMs can only hold limited amounts of information in context windows, so "they struggle to parse large code bases and are prone to forgetting what they're doing on longer tasks."
"While an LLM-generated response to a problem may work in isolation, software is made up of hundreds of interconnected modules. If these aren't built with consideration for other parts of the software, it can quickly lead to a tangled, inconsistent code base that's hard for humans to parse and, more important, to maintain."
"Accumulating technical debt is inevitable in most projects, but AI tools make it much easier for time-pressured engineers to cut corners, says GitClear's Harding. And GitClear's data suggests this is happening at scale..."
"As models improve, the code they produce is becoming increasingly verbose and complex, says Tariq Shaukat, CEO of Sonar, which makes tools for checking code quality. This is driving down the number of obvious bugs and security vulnerabilities, he says, but at the cost of increasing the number of ' [4]code smells ' — harder-to-pinpoint flaws that lead to maintenance problems and technical debt."
Yet the article cites a [5]recent Stanford University study that found employment among software developers aged 22 to 25 dropped nearly 20% between 2022 and 2025, "coinciding with the rise of AI-powered coding tools."
The story is part of MIT Technology Review's new [6]Hype Correction series of articles about AI.
[1] https://www.technologyreview.com/2025/12/15/1128352/rise-of-ai-coding-developers-2026/
[2] https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
[3] https://arxiv.org/pdf/2406.10279
[4] https://en.wikipedia.org/wiki/Code_smell
[5] https://digitaleconomy.stanford.edu/wp-content/uploads/2025/08/Canaries_BrynjolfssonChandarChen.pdf
[6] https://www.technologyreview.com/supertopic/hype-correction/
Here's What Happens To Me (Score:2)
Here is what keeps happening to me. I keep falling into the same trap.
I throw various simple things at the AI, sort of a Google replacement, and it gives me what I want. I'll have several successes. And then it will lead me down the wrong rabbit hole, misdirect me, refuse to break out of it's mistaken path, and waste way way way too much of my time chasing my tail.
Eventually, I'll arrive at my destination and I'll ask it why the fuck it took such a circuitous route full of errors and straight up lies to get
Re: (Score:1)
Yep, this is when the context is full. Nuke the chat and start again.
My current favourite is "Oh, now I understand completely what's happening" (for the seventeenth time in a row - all of which were too-hasty.
Re: (Score:2)
I call it the coding LLM Doom Loop.
A good bit of my effort with using LLMs has been in trying to avoid and correct it.
I've found it gets easier when you start to treat the LLM and its entire context window as a single unit rather than thinking about prompts.
Coding agents are variably successful at this.
For my own agentic tests, I've had good results "context engineering" the LLM to solve tasks reliably that it previously couldn't.
In the end- I'm not sure it's worth the effort, but hey, it keeps me ent
Re: (Score:2)
The key here is that it helps, but it can't replace you. Not that I care whether you get replaced, but there are a couple trillion bubble bux riding on whether you can be replaced, so it's a big deal.
Quick! (Score:3)
AI companies should pivot to predicting Anthropogenic Global Warming, I'm sure it will be perfect for that.
Re: (Score:1)
All that matters is they have everyone's money, can influence elections and are too big to fail. Give them a break - AI is hard.
At first (Score:3)
Professional dev in my third decade of experience speaking here. At first, these products really did assist quite a bit. In 2023 and 2024, I found the tools to be pretty decent at offering suggestions for small to medium snippets of code.
Something changed late last year. It may just be that the shine is wearing off, but I find most of the AI products producing less quality results than they did previously.
I rarely ever reach for them anymore. I sure would not rely on them over even an inexperienced junior dev, either.
Re: (Score:2)
> Professional dev in my third decade of experience speaking here.
Only second decade, here.
> I rarely ever reach for them anymore. I sure would not rely on them over even an inexperienced junior dev, either.
I find them comparable, unfortunately. But my new hires may not be as good as yours.
It has its uses (Score:3)
Asking natural language questions is a fantastic way to search documentation.
If you can describe exactly what you want, it can do a fine job accelerating that.
If you are stuck, asking it to try to solve your problem can at least be entertaining.
Today's AI may suffer from a critical flaw! (Score:2)
The fact is! Some guesses are wrong! The question is can they recover. by post guess, guessing and conditioning output.
Of course it does. (Score:3)
Those bots that auto-expire valid bug reports after only a week of inactivity cuts way down on wasting time fixing bugs and lets me review^^^^^^H use and fix^^^H release all the bad code the other AI bots are creating for me.
Only if your a true believer! (Score:3)
And don't bother to double check things. And if you do check their goes the speed gain.
Re: Only if your a true believer! (Score:2)
Only if P = NP.
Nope! (Score:2)
Hear me out, the way all companies are pushing AI makes for additional, unwanted distraction. And i mean both - my company insisting on using the new fad just to stay hip, and all the service and tool providers pushing the product. It all adds overhead to daily work. I'm fine with AI being an improvement on search engine, but having to tell stuff to an AI agent is like babysitting an intern. An intern who majored in human scince at that.
Re: Nope! (Score:2)
oh well, i exaggerated that last bit. AI agent at least will try to correct and knows where to look for answers when i point the mistakes it made. Still, it's the same as leading a junior. The only benefit is the hope of the company to have less payed employees i guess
Bloat Industrial Complex (Score:2)
AI seems to be feeding the bloat habit instead of trimming it. It's becoming an auto-bloater.
Very few in the industry are interested in parsimony. Devs would rather collect buzzwords for their resume rather than try to trim out layers and eye-candy toys. It's kind of like letting surgeons also be your general doctor, they'd recommend surgery more often than you really need it.
The principles of typical biz/admin CRUD haven't really changed much since client/server came on the scene in the early 90's. Yet the
Re: Bloat Industrial Complex (Score:2)
You gotta be cynical. I agree this is a dog chasing its tail. Increasing consumption is the purpose of the machine. I do nearly the same things online as I did 10 years ago, but my bandwidth consumption is up by probably 3x. It's a self fulfilling prophecy until it's a bursting bubble.
Re: (Score:2)
> Very few in the industry are interested in parsimony.
I've come to accept that this as true, and further conjecture that bloat is often a corporate/institutional goal.
[1]This seems to be a joke [zerobugsan...faster.net], but in reality corporate incentives are aligned to make things more bloated. If you're a manager, then the more people you have under you, the more power you have. This means you want your people to go slower so you have to hire more of them.
I don't have a solution but there must be one.
[1] http://www.zerobugsandprogramfaster.net/essays/2.html
It's not about being faster (Score:4, Insightful)
It's about lowering the skill ceiling so that they can pay substantially less. If there are productivity gains that's just a bonus.
The goal here for AI is to eliminate wages. There is more than one way to skin a cat.
Remember good enough is always good enough especially when monopolies exist and we refuse to vote for politicians that will enforce antitrust law because we're busy freaking out about whatever Petty moral panic or culture War bullshit the boob tube tells us to today
Re: (Score:2)
Most of the Indian H1B visa holders I've worked with have been pretty good at their job.
Re: (Score:2)
> good enough is always good enough
Yeah, the only outcome to a constantly lowering average standard is a downward spiral.
Trying to achieve high quality is too much effort, so let's set aside pride in whatever's done.
Re: (Score:2)
Eliminate wages, assuming the result still works, would be classed as a productivity gain.
"i need that small annoying snippet that does..." (Score:2)
In this, it is decent, and need to be something you actually understand as a result so you can proof read it.
But letting it loose on the big code at large is pretty dumb.
Re: (Score:2)
> But letting it loose on the big code at large is pretty dumb.
I do this frequently, with both packaged agents and my modifications to them, just to see what comes out the other side.
Sometimes it's great, sometimes it's pretty bad.
I do it as a side job, not my regular work, so the consequences of the failure are minimal- I just throw it away and try again with another modification.
If it were my actual main workflow... I think that would stress me the fuck out- each failure being significant wasted time and money.
Re: (Score:2)
Side job meaning "occupying one of my side monitors", not "side income"
Re: "i need that small annoying snippet that does. (Score:2)
But if the net effect is "costs more", why would you actually ever use it at all?
\o/ (Score:2)
No (but also yes).
If you let it write code freely, you can spend hours or days trying to figure out what the f**k it's done and why it's not done what you asked or why it randomly introduces a complete re-write for a single-line fix.
Heavily constrained yes.
It complements me perfectly. (Score:2)
It's really good at things I'm bad at. Making things look nice mostly. I can agonize over padding and fonts for hours and end up with something that looks terrible.
Hidden cost (Score:2)
Do the productivity estimates include the tedious additional overhead at the start of every meeting talking about AI meeting summaries, during every meeting talking about AI, after the meeting talking about the AI-generated meeting summary notes, in workplace chat groups talking about the efficiency of otherwise of AI, etc?
The Flow State (Score:3)
Quite frankly, I don't want to talk to anyone all day, including an agent. And it's not really about being anti-social. As most people know, we have two primary brains - logical and emotional. I find that trying to explain what needs to be done the model engages my emotional/social brain too much, and the logical part can't focus on getting the job done. In order to do quality work, I need to completely ignore the emotional side of my brain and let the logical brain get into the state of flow, where I can accurate track the state of what I'm working on and follow logical workflows only. Talking to an agent all day engages my social side too much to do quality work.
Re: The Flow State (Score:2)
This is a very refreshing new aspect, never thought of it!
Yeah... (Score:2)
Tried to use Gemini to just hack my way through using a scientific Python package recently. In the end ended up almost all being worthless as a requirement couldn't be installed working at all (yay Assimulo) and no amount of AI was helping, had to scrap multiple days worth of relying partially on AI and just macguyver a different way around getting the results I wanted, if I hadn't had AI at all I might've started there and ended up going faster.
I'm sure this is all worth trillions of dollars or whatever
A simple, but lame, example. (Score:2)
Write a C++ program which calculates a "stardate" where each date is a 24 hour Earth day, and the origin (0) is January 1st, 2001 at midnight.
Finished
🌐
🌐
🌐
To calculate a "stardate" based on Earth days since January 1, 2001 at midnight, you can use the C++
Here's a complete C++ program that calculates the stardate:
#include
#include
Re: A simple, but lame, example. (Score:2)
What exactly does this tell us? That the LLM was trained on this example / code fed by tens of thousands of trekkies and was therefore able to regurgitate it. Would be interesting to see where the code snippets are originating from
It helped research some 25-year-old code (Score:4, Insightful)
I came across some Emacs elisp code I'd written about 25 years ago, and it looked pretty useful. Emacs didn't like it. I researched the functions and variables and they apparently had been rejiggered about 5 years later. I said to myself, Self, sez I, this could be an interesting AI test. I could probably make this do what I want in a few minutes now if I did it from scratch, but that wouldn't help me understand why it was written that way 25 years ago.
So I asked Grok. I was pleasantly surprised to find it understood 25 year old elisp code just fine, explained when and how they had been rejiggered, and rewrite it for the current standards. That was more than I had expected and well worth the time invested.
One other time Grok surprised me was asking how much of FDR's New Deal legislation would have passed if it had required 2/3 passage instead of just 1/2. Not only did it name the legislation which would not have passed, it also named all the legislation which had passed by voice vote and there was no way to know if 2/3 had voted for it. The couple of bills I checked did match and were not hallucinations. The voice vote business was a nice surprise.
I program now for fun, not professionally. The idea of "offshoring" the fun to AI doesn't interest me. But trying to find 25-year-old documentation and when it changed doesn't sound like fun, and I'm glad to know I can offshore at least some of the dreary parts.
Didn't see that one coming (Score:2)
Huh, what are the odds that MIT releases yet another paper with subjective contrarian views on productivity with AI?
There is a MASSIVE conflict of interest with these MIT papers here, and nobody's calling it out.
So yeah, okay, sure, MIT thinks:
- AI makes you dumber (with methodology nobody without a dedicated lab can duplicate)
- 95% of ai projects fail (using extremely rigid metrics and ignoring norms in the larger industry to reach conclusions, while including prototypes and showboat projec
Re: Didn't see that one coming (Score:2)
It seems that the experiences shared here largely confirm the findings
Gardening time (Score:2)
I've worked for myself as an independent developer for more than a decade now.
Apps and websites and I do well working on my own.
I'm getting old though that the saying "can't teach an old dog new tricks", is starting to make sense.
AI couldn't have come at a better time in my life.
As I've always warned youth thinking of getting into tech at higher education, the older you are the less valuable you become. The complete opposite of the other white collar grad workers. You want the old experienced doctor, lawyer
Faster, no. Multi-tasking yes. (Score:2)
As a developer, AI workflows still rub me the wrong way. If I was dedicated to the task, I'd produce better code.
As a human, AI workflows let me have a life. I can let the agents knock out the easy things while I'm working on other tasks. I still need design out what's to be worked on, review the code, fix bone mistakes they make, etc. It's basically like having a junior developer assigned to you.
Which brings up an important point. Junior developers need clear instructions/requirements and so do AIs
Re: Faster, no. Multi-tasking yes. (Score:2)
Yeah, only in the one case, you're helping a fellow human being learn something and become good in the profession. In the other, you're blowing money and resources away for absolutely no other gain but to increase bottom-line of a company. If you kill all junior devs because "you can replace them with the AI", who are going to be the senior developers of tomorrow?
One thing is faster - increase of technical debt (Score:1)
I really do think coding using AI tools is a bit faster, at least it seems that way to me. As most of the morning but lengthy work can be done faster by AI.
But I am also pretty sure it's VERY easy to rapidly incur technical debt, especially if you are telling AI to review its own work. Yeah it will do some stuff but who is to say post review fixes it's really better?
More than ever I think the right approach to coding with AI is to build up carefully crafted frameworks that are solid (maybe use AI to help
"Coding" is not software development (Score:2)
AI might make newbies faster at producing... something. Probably something full of bugs and security holes.
But it won't help non-newbies with software development, of which "coding" is a relatively minor part.
Brittle tech (Score:2)
I've been playing with these genAI system both as code producer but as helper on various tasks.
And overall, I find the models quite brittle unless they are fine tuned on the precise task that you want.
The main problem that I see is that the tool is fundamentally a string in string out. But the strings could be absolutely anything including completely insane things without proper fine tuning.
Today, I am writing a simple automatic typo correction tool. The difficult bits are making sure that the tool didn't c
Make up your minds (Score:2)
These wild swings between AI thinning out the workforce and making all our jerbs obsolete to not being sure if AI is even useful is giving me a headache.
Re: (Score:2)
It'll do both... dumb management and bean-counter types will replace people with AI, and the AI will suck at actually getting work done.
Lose-lose!
Re: Make up your minds (Score:2)
They can both be true. A lot of companies have short sighted leaders. The LLM may produce sub standard code that will be a nightmare down the road but who cares I've saved the costs of 10 developers. But when non programming people are expected to judge the quality of LLM code it's no wonder "works on my machine" is the new baseline of quality.
It depends on your skills level (Score:4, Insightful)
It depends on your skills level. For trivial beginner stuff, it's OK but then again.
For anything out of mainstream which no or very few examples are available for the model to train, it's pretty much useless.
Re: (Score:2)
Watch for the AI bubble crash in 2026.
Re: (Score:2)
As much as I hate seeing a brute-force approach burn huge amounts of electricity to enable morally dubious applications based on inputs that are often corrupt or illegal, I think the AI bubble is as likely to pop as the Bitcoin bubble.
(You might ask: "Do you mean that AI is a brute-force approach that burns huge amounts of electricity, etc., or that Bitcoin is?" To which I answer: "Yes.")
Re: (Score:2)
Right now we are in the “first hit is free” phase of trying to get everyone hooked on this AI crap. All these upstarts are trying to get marketshare and get some spaghetti to stick to the wall and the usage is heavily subsidized by all the startup money gushing in. Once the influx peters out and these places have to pay their own rent we will see the reality of which companies are able to actually survive and which of the horde are houses of cards.
I fully expect there to be plenty of actual ap
Re: (Score:2)
That's just what the LLM is trained on, rather than anyone's skill at using the LLM.