The Great Software Quality Collapse (substack.com)
- Reference: 0179776992
- News link: https://developers.slashdot.org/story/25/10/14/0826220/the-great-software-quality-collapse
- Source link: https://techtrenches.substack.com/p/the-great-software-quality-collapse
> The Apple Calculator leaked 32GB of RAM. Not used. Not allocated. Leaked. A basic calculator app is hemorrhaging more memory than most computers had a decade ago. Twenty years ago, this would have triggered emergency patches and post-mortems. Today, it's just another bug report in the queue. We've normalized software catastrophes to the point where a Calculator leaking 32GB of RAM barely makes the news. This isn't about AI. The quality crisis started years before ChatGPT existed. AI just weaponized existing incompetence.
>
> [...] Here's what engineering leaders don't want to acknowledge: software has physical constraints, and we're hitting all of them simultaneously. Modern software is built on towers of abstractions, each one making development "easier" while adding overhead: Today's real chain: React > Electron > Chromium > Docker > Kubernetes > VM > managed DB > API gateways. Each layer adds "only 20-30%." Compound a handful and you're at 2-6x overhead for the same behavior. That's how a Calculator ends up leaking 32GB. Not because someone wanted it to -- but because nobody noticed the cumulative cost until users started complaining.
>
> [...] We're living through the greatest software quality crisis in computing history. A Calculator leaks 32GB of RAM. AI assistants delete production databases. Companies spend $364 billion to avoid fixing fundamental problems. This isn't sustainable. Physics doesn't negotiate. Energy is finite. Hardware has limits. The companies that survive won't be those who can outspend the crisis. There'll be those who remember how to engineer.
[1] https://techtrenches.substack.com/p/the-great-software-quality-collapse
What is the effect of the leak? (Score:2)
On iOS, malloc (1ull 35) will succeed. It will reserver 32 GB in the address space but nothing else. As long as you don't write to the memory, nothing happens at all. So you are right to ask "WTF is going on here", but it won't hurt any user.
Re: (Score:2)
I think you'll find that leaking means written to. And the SSD is swapping like crazy. Hence the bug complaints.
Re: (Score:2)
But the OP says the 32GB was "Not used. Not allocated. Leaked." It's a little hard to parse, but if true, then maybe the actual effect is truly negligible.
Re: (Score:2)
Dame on Linux. This is really bad software engineering, but it gets made worse by a really bad OS (Windows).
Re: (Score:2)
Or even written to once then paged out, would have minimal consequences. Yes, it's a bug, but virtual memory is cheap in a 64 bit architecture. One bug that has low severity does not make a great crisis.
Re: (Score:2)
I bet the problem is that parts of original calculator app is written in Objective-C from before there was automatic reference counting, and someone messed with the legacy code and didn't free memory properly - a good old fashioned dangling pointer leak.
It should not be possible for this sort of bug to occur in a modern language with garbage collection (or even ARC unless you're going crazy with your program structure).
Re: (Score:2)
> On iOS, malloc (1ull 35) will succeed. It will reserve 32 GB in the address space but nothing else.
Well, at least some page table entries were created. Free memory block list in heap was extended as well (though this could be avoided if only brk or mmap were used). The operation has some overhead but it is very little if the allocated memory is never used.
Answer is simple: (Score:2)
Software development needs to slow the hell down, to allow development to happen properly - resources need o be given so developers can take a machine and do what it needs to - no more, or less.
We need to stop shifting the blamer on costs (it's a bullshit excuse IMO; not WANTING to spend the resources, time and money to allow quality to shine =/= being unable to).
We need to stop blaming complexity when we don't give developers the time and space needed to explore that complexity and work with it.
Compan
Re:Answer is simple: (Score:5, Insightful)
Actually we need to bring down complexity. No amount of time and money will make the current complexity in the IT space sustainable.
And there is another thing: Disregard for KISS and high complexity is amateur-hour. Actually competent engineers avoid complexity like the plague it is.
Re: (Score:2)
> Actually competent engineers avoid complexity like the plague it is.
Yeah but avoiding complexity takes a lot of time. It's like the saying 'I didn't have time to write a short letter...'. Especially with powerful high level languages, I can just bash out something that works extremely quickly. Things like JavaScript where what they call 'dynamic programming features' could also be construed as 'no respect for any sort of useful scoping rules' means you can pull assets around your project like globals on steroids. This is damn fast if you just want to mock something, but it'
Re: (Score:2)
There's a saying that goes "A complex software project that works started from a simple project that worked". For instance I have a 40k lines program that has been working non-stop for years without any memory leak or crash, and it started from a half-page of specs "to do a test" and then they kept asking me to add thing... Granted it now looks like a monstrous pile of kludges, but still...
Re: (Score:2)
"If you are not embarrassed by the first version of your product, you've launched too late"
Reid Hoffman originally coined this sentence when discussing startup culture and the launch experience of LinkedIn.
This is the reality of the market. People will jump on the first service available, be it bad or otherwise. The competitor that release its stuff 1 year later has already lost the race.
So the bad technical decision makes market sense. You can either make money with fast crap or go bankrupt with a well eng
Percentages. (Score:2)
20 years ago a calculator leaking 32 GB would have been physically impossible. The hardware that it ran on didn't have 32 GB. So leaking even 1GB would have been a major issue.
Now we have Terrabytes, so a Gig is NOT THAT BIG A DEAL.
As our memory size has increased, the acceptable losses have also gone up. Similarly, the developers have shifted their requirements. More important to get things done faster and add features than to be a memory miser.
Is the leak good? No. But it is not a catastrophe nor a
Re: (Score:2)
You are the problem.
Re: (Score:2)
No common devices have 1 TB of memory, nor would taking 32 GB of storage for a calculator be reasonable, either.
Re: (Score:2)
Why are you running a calculator app on the cloud instead of locally?
Re: (Score:3)
> As our memory size has increased, the acceptable losses have also gone up. Similarly, the developers have shifted their requirements. More important to get things done faster and add features than to be a memory miser.
This attitude, IMO, is a huge part of the fucking problem in the first place.
Re: (Score:2)
I built a top line gaming PC late last year. It has 64 GB of RAM. If a fucking CALCULATOR took up half of that I don't even want to know what a game with the complexity of Solitaire would do to it!
Re: (Score:2)
> 20 years ago a calculator leaking 32 GB would have been physically impossible. The hardware that it ran on didn't have 32 GB. So leaking even 1GB would have been a major issue.
> Now we have Terrabytes, so a Gig is NOT THAT BIG A DEAL.
> As our memory size has increased, the acceptable losses have also gone up. Similarly, the developers have shifted their requirements. More important to get things done faster and add features than to be a memory miser.
> Is the leak good? No. But it is not a catastrophe nor a sign of bad times. It's just a shifting of priorities due to a shifting of resources.
> Also, this is an AI issue because our AI are being trained on the existing software. That is more of a problem than the current memory surpluses.
You know the feeling of not knowing whether someone is being serious? I have a similar feeling now except I don't know whether everyone from TFA on down is posting AI slop.
Streaming Apps (Score:2)
OK. Old man rant. But I've been saying this for years about streaming apps. How is the overall quality so awful?
It is *stunning* how bad they are and people just put up with them. If you have two or three streaming apps, you will be constantly playing whac-a-mole with the same basic set of player bugs all the time. And somehow, even trillion dollar companies can't handle syncing where I am watching a show across multiple devices, it's the software challenge of our time.
When I press the "next show" button in
Re: (Score:1)
I agree with these comments. You know it's a bad situation when the Netflix app is the best one out there. I find the Netflix app to at least be usable but some of the other big ones (HBO-Max, Prime Video, Apple) suck big time. I used to think the versions I used on an old Roku were the problem but the ones I see on new smart TV's are no better.
Don't get me started on minimalist remote controls...
I agree (Score:2)
Software is typically being done incompetently and that seems to slowly be getting worse. This is mostly in the Windows-space, including the OS. For example Win 11 crashed on me 2 times in the first hour of using it, while Win10 was rock-solid on the same hardware. And the GUI is infantilized and decidedly has worse usability. What can you expect from application development, if this is the culture being set?
At the same time, requirements increase. Software now runs everything, attackers have solid business
Re: (Score:2)
I think that "AI programming" vibe will only make this worse.
Re: (Score:2)
Oh, definitely. It makes the defenders weaker and the attackers stronger, because attack software does not have to be reliable or secure. And I think we may see a real software maintenance catastrophe from that stupidity in a few years as well.
The great writing quality collapse (Score:3)
Most of the "sentences," as defined by punctuation, are really just phrases. The article was published on Substack, so not only are there no editors, but apparently no standards. Please take time to edit your work, otherwise it's illegible. I'll take a stab at repeating what I just said, but in that same terse style:
The author used sentence fragments. Hardly 5 words. Barely even phrases. What the hell? Twenty years ago, this would have triggered editorial review. Not now. Nobody cares! It's awful. Here's what bloggers don't want to acknowledge: writing takes time and effort. It's a skill. And guess what? If you write well, people will have an easier time understanding your point. If there ever was one.
Re: (Score:2)
Seriously. I agree with the author, but the writing was atrocious. Also, at least one of his examples of problems was with beta software. It's built to have problems like this, not really an issue for me if my known beta software doesn't work right. I'm glad someone else noticed this too.
Not surprising (Score:2)
"Move Fast and Break Things" inspired a generation of incompetence. Note that actually fixing the things you broke gets swept under the rug.
It seemed to be an almost orgasmic revelation to the business-types when they realized they can continually ship broken software and nobody cares.
Re:Not surprising (Score:4, Interesting)
> "Move Fast and Break Things" inspired a generation of incompetence.
I think there were three elements of this mindset that were assumed knowledge on the part of the person who said it:
1. "Move fast and break things...in a development environment where possible".
2. "Move fast and break things...in a way that is easily reversible." (see #1)
3. "Move fast and break things...and assume they will break, so assume you'll be fixing what broke" (see #2 and #1).
I can appreciate that Facebook can have this mindset, and in the case of a social network, there *is* an element of wisdom in not treating it like the IBM-of-old that overengineered EVERYTHING, making it super-reliable, but also making development very slow and very expensive. Facebook's focus on agility makes perfect sense for the nature of the work.
This doesn't work in every field, though. From finance to medicine to engineering, the costs are much, much greater than the loss of cat videos. Just because something makes sense in one field, doesn't mean it makes sense in EVERY field...and unfortunately, there are very, very few MBAs who understand the one thing that is more valuable than money: wisdom. Wisdom can earn money, but money can't buy wisdom.
Drama (Score:2)
Such a dramatic statement... a calculator app leaked 32 GB of RAM, and now the world is ending. I don't think the root cause was that it was using this ridiculous tool chain: "React > Electron > Chromium > Docker > Kubernetes > VM > managed DB > API gateways". It was likely because the developer made one dumb mistake in how they wrote it.
I assure you we had the same hand-wringing about too much abstraction over 20 years ago, and that was just because of the JVM or .NET bytecode.
Nope, software's been crap for years (Score:2)
And in 2005, if the Apple Calculator had leaked arbitrary amounts of memory... it just would have been a bug report in a queue. Because it's just a little utility, not an emergency.
He inadvertently disproves his own thesis by linking to the Spotify bug. It's from 2020, when his chart was still in the green, though it's likely there are multiple different memory leak bugs discussed in the thread.
It's true that in the much-further distant past there were fewer bugs of this nature. Or at least fewer that ma
This really, REALLY isn't new (Score:2)
I remember in the last 90's / early 2000's discussing with a colleague how it was possible that X allocated 2 megabytes for an empty window, just for sitting there on the screen.
Resources allow the incompetent to make products (Score:2)
When you have 32 kilobytes of RAM and a 1 MHz processor, you need all the programming talent you can get to squeeze the most performances out of them.
When you have 32 gigabytes and dozens of cores, any incompetent code monkey can churn out the same application in Visual Basic or Python.
Resources don't make your computer faster. They empower incompetent and sloppy developers, who crucially are paid less than good ones, so their boss can make more money.
Perspective from an old timer (Score:2)
I learned software engineering in the 70s. Back then, there weren't very many of us and even fewer who were really good at it. As a result, we were paid really well.
Then the news spread, software is the key to riches.
Pundits, advisors and politicians told kids that everybody needs to learn to code.
This resulted in a flood of CS students of varying talent. The talented ones worked hard, the not-so-talented ones somehow ended up with diplomas and entered the market.
At the same time, the beancounters wanted to
DHH spoke about the same thing recently (Score:1)
Not evangelizing Ruby, but the start of the keynote made a lot of sense.
[1]https://www.youtube.com/watch?... [youtube.com]
[1] https://www.youtube.com/watch?v=gcwzWzC7gUA
Re: (Score:2)
They guy is a racist tool that is responsible for this anti pattern as much as anyone.
It isn't just software (Score:1)
It is likely true that software quality is dropping. But the important point I would like to make is that quality elsewhere is horrible too. Our relatively new house is on its third bathroom sink faucet in about 12 years total time. I cannot fathom how this could be so bad. Car quality, parts, engines, transmissions all of it is worse. Worse parts, worse designs, it all is bad and so much more expensive. A twenty year old car w/ only front wheel drive, a four speed transmission, a reasonable power v6 in com
Dependence vs dependability (Score:1)
If you agree that every year dependence on software / systems to perform everyday tasks increases, and that every year software / systems dependability decreases, the outlook is discouraging.
Marketing/MBA driven slop... (Score:3)
If I had to pin on why software quality is so crappy, it is because of a few factors:
* Companies don't have to do better. They can ship something that doesn't even run, and make money. At most, they can ship something barely functional so implied running is taken care of. However, even that can be stomped in the TOS/EULA agreement.
* Devs are viewed as fungible. They are not getting what marketing wants, even if it is impossible? Offshore and outsource until management is happy.
* The entire Agile/Scrum system had well intentions, but has become nothing except beating on devs until they quit. I worked at one software place that had daily 4-6 hour standups, where every deliverable was asked for, by every dev, and the Scrum master personally insulting the devs every day. Then people would whine that they are blocked, point fingers, the other person pointed at would start cursing at them. After that exhausting kangaroo court, not much got done after lunch. Even worse, I was in Ops, and was dragged into the meetings, and often considered the cause of deadlines slipping... because I refused to put things into production unless they went through some type of testing first. This was after a dev used a root PW on a production machine, threw his code in, and caused an insanely expensive outage. Code quality was absolute garbage. Security? Hah!
Security was out the window, because the devs knew that if the company had software that caused a major breach, there were many layers between the legal team and them, while not getting a deliverable out the door meant the boss saying tata to their job. So, on install, SELinux was disabled, firewalld was masked out, and the program would quietly exit with an error code zero if not run by root. All DB access was done by the SYSTEM or sa user, and the program would just exit out, error code 0, if that wasn't present. The devs didn't care... to them and the Scrum master, security has no ROI.
Want something done right, don't do permanent sprints, and allow for times where the entire code base goes through a complete refactor, perhaps every 6-12 months. Get that technical debt paid off.
Overall, this makes me glad I work with embedded programming. Stuff that is acceptable for web pages isn't going to cut it when at most, you are programming on the resources of an Apple ][+. You are not going to vibe program stuff in that environment.
That's unfortunate (Score:2)
Gartner predicted this when they critiqued Microsoft's new philosophy of releasing when the software is "just good enough." Inherent flaws and known bugs were ignored.
Now that that's what everyone does, expect it to get worse.
Let's all get nack to basics... (Score:2)
He makes a bit of a valid - if poorly articulated - point.
We should be using assembly language! More efficient!
Re: (Score:2)
The author forgot the most important thing: You can't make iPhone apps on an iPhone (or Android apps on an Android phone, now that Google will lock sideloading). You can't mod iPhone or Android apps (on an un-jailbroken/un-rooted device). You can't even view the files that make up an app on an iPhone or an Android phone. Anything that could motivate the "smartphone generation" (Gen-Z) to try coding is not there. Gen-Z are "digital natives" but mostly clueless about what an app really is. In fact, some of th
Part of this decline is all MBA-driven (Score:3)
Part of this decline is MBA-driven.
Beancounters ruin every shop they run, whether it be Boeing, or any ones that I've worked at where Mr. Money Bags controls IT.
In shops that Mr. Moneybags (CFO) do not control IT I've noticed things are much better for IT.
Re: (Score:1)
Is it a decline? 32GB of RAM means nothing - from the dawn of computing when dinosaurs roamed the Earth and 8 words of RAM was plenty, programs have got into resource consuming loops. Programmers have made typos that allocated 65536x as much memory as they intended to.
The main difference was that back then the computer would crash and all data would be lost. Now the OS copes with the massive RAM requirements put on it, or at most kills the offending task. Less catastrophic, but the underlying bug is the sam
Re: (Score:2)
> 32GB of RAM means nothing
Just add more swap space. Thank goodness that's free.*
*An actual comment I found in some crappy code. Most likely put there by someone attempting to debug it.
Re: (Score:2)
All that does is collapse into crazy levels of swapping. The whole system bogs down. Which will likely be why the bug reports were happening in the first place.
Re: (Score:2)
Right now, the OS's OOM killer will do it... but until then, it causes extreme memory pressure on a system, forcing swap, which will nosedive performance on the machine until that process is killed. It may cause other processes to see that the machine has a lot memory pressure and quietly freeze or exit.
Historically, failing to keep track of RAM was a bad thing, especially pre OS X on Macs, before the OS had "hard" memory protection, and a memory like like what is mentioned would cause a complete system cr
Re: (Score:3)
> 32GB of RAM means nothing
Well, it's about double the amount of RAM most modern computers come with, so I'm not sure it's nothing. Are you sure you're not confusing Gb with Mb?
That said, I'm curious to know where the figure comes from. Modern operating systems are extremely unreliable at reporting memory usage figures. According to 'top' right now on my PC, Pluma, a very basic text editor shipped with MATE is using either 15.5g or 8g. There's a Java app that's either using 23.1g (Virt) or 2.1g (Res). Fi
This. (Score:2)
IT, whether it be software, hardware or the people that tie it together, is seen by all-too-many CxOs as a commodity cost to be minimised, rather than an asset and competitive advantage to invest in and nurture.
Of course, those same CxOs don't take any responsibility for the failures they create either...
Re: (Score:2)
Might you be seeing that small companies with focused development teams (perhaps it's just a development team) are more efficient that large companies, which need bean counters?
Re: (Score:2, Insightful)
Short-term planning and profit is everything to these cretins. Everyone of them is essentially a get-rich-quick preacher. That destroys things reliably. In some sense this is a variant of the "Tragedy of the Commons": Nobody makes things sustainable, nobody gets things to a stable and dependable state, nobody looks at the longer-term effects.
I mean Microsoft is making record profits while their cloud gets hacked again and again and their OS is crumbling. Same thing for Boeing with their planes falling out o
Re: (Score:2)
Honestly, I just had a "I have thoughts, let me spit them out without thinking" moment with this post - unnecessary complexity (piles of dependencies, frameworks, etc) absolutely IS a big part of the problem.
Re: (Score:1)
CFO's often simply weigh the short-term ahead of the long-term, and engineering quality is very leveraged on unwise shortcuts. One can over-milk the cow early, but that makes for a sick cow later.
The CFO's perhaps plan to retire or move on to a different company beyond say 4 years. Or they want their beemer now so they can get laid now because they are thinking with the wrong head.
"The future is somebody else's problem."