Rubbish IT Systems Cost the US At Least $40 Billion During Covid (ft.com)
- Reference: 0179848000
- News link: https://it.slashdot.org/story/25/10/22/047219/rubbish-it-systems-cost-the-us-at-least-40-billion-during-covid
- Source link: https://www.ft.com/content/7610a755-27e3-4792-a337-27d9b301000b
> A lot of critical financial and government infrastructure runs on Cobol. The more-than-60-year-old mainframe coding language is embedded into payments and transaction rails, even though there are very few Cobol-literate coders available to maintain them. The big argument in favor of sticking with Cobol systems is that they work. The catch is that, whenever they stop working, it is difficult to figure out why. That's not good in a crisis, which is exactly when they're most likely to break. Covid-19 [1]put a lot of strain the US state benefit systems .
>
> The ones that used Cobol for processing unemployment claims failed spectacularly, according to a new [2]working paper from The Atlanta Fed: "States that used an antiquated [unemployment insurance]-benefit system experienced a 2.8 percentage point decline in total credit and debit card consumption relative to card consumption in states with more modern UI benefit systems. [...] Using this estimate in a back-of-the-envelope calculation, I find that the lack of investment in updating UI-benefit systems in COBOL states was associated with a reduction in real GDP of at least $40 billion (in 2019 dollars) lower during this [March 13 2020 to year-end] period
>
> The paper uses Cobol as a proxy for old and inefficient IT, not the direct cause of failure. Claimants faced much longer delays in the 28 states that still used Cobol in 2020, both because of the unprecedented volume of claims and the difficulty updating systems with new eligibility rules, author Michael Navarrete finds. [...] As an aside, one oddity of the data is that Republican-controlled states were more likely to have replaced old IT systems, even though their standard unemployment insurance payments are lower on average. Why? Absolutely no idea, but [3]here are [4]the maps . And, once adjusted for state politics, here's the [5]key finding .
[1] https://www.ft.com/content/7610a755-27e3-4792-a337-27d9b301000b
[2] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5621276
[3] https://images.ft.com/v3/image/raw/ftcms%3A478fb2af-e8ac-47d1-a234-c93d83f2fc23?source=next-article&fit=scale-down&quality=highest&width=451&dpr=1
[4] https://images.ft.com/v3/image/raw/ftcms%3A1e3edf8f-247f-4d32-a7bc-430d0216135c?source=next-article&fit=scale-down&quality=highest&width=456&dpr=1
[5] https://images.ft.com/v3/image/raw/ftcms%3Ab1abb20b-cec8-4e4c-bd04-11e52807e805?source=next-article&fit=scale-down&quality=highest&width=700&dpr=1
Why? (Score:2)
The catch is that, whenever they stop working, it is difficult to figure out why.
Becauuuse... Why?
Is Cobol specifically more difficult to debug?
Is there really a deficit in Cobol literate programmers?
Isn't Cobol actually spelled COBOL?
I don't buy the implicit implication that because it's old it's not as good as new, for certain new would be worse given the AI riddled times we vibe in.
Is the real reason it's "difficult to figure out why" because ... ok, probably because the most knowledgeable COBOL pro
Re: (Score:2)
I took COBOL in college and it wasn't rocket science. It is quite a bit different than today's languages, but any programmer worth their salary should be able to figure it out relatively easily.
Re: (Score:2)
My guess is that "programmers worth their salary" (and it would need to be a higher one here) are very, very rare these days.
Re: (Score:2)
I also took a course in COBOL way back when. It was the wordiest damn thing I ever encountered. An RPG course was also offered but only because the local amusement park (Cedar Point) relied on it heavily for coaster functions back then, or so I was told.
Re:Why? (Score:4, Insightful)
> Is Cobol specifically more difficult to debug?
Yes. Much Cobol code was written before structured programming was standard. There are no unit tests. There are no classes to encapsulate complexity.
> Is there really a deficit in Cobol literate programmers?
No, not really. A programmer can learn Cobol well enough to get work done in about a week. The problem isn't "learning the language", but learning about the legacy application you need to maintain. For instance, why is a discount applied to every invoice for customer #478324? Well, because he was the CEO's college roommate, but you won't learn that from the manual.
> Isn't Cobol actually spelled COBOL?
Only if you want to be pedantic.
> I don't buy the implicit implication that because it's old it's not as good as new
There's been a lot of progress in programming languages in the last 60 years.
> If it ain't broke, don't fix it ...
It is broke.
Re: (Score:2)
> It is broke.
So it does not work? It is impossible to fix? Explain, because that is what broke is - not even being pedantic.
Re: (Score:2)
COBOL is easy.
Easy to learn
Easy to program with
Easy to read.
It is very simple. Which is both a strength and its biggest weakness.
The problem is that programs written are NOT structured except the way the guy who wrote the code thought it should be ... if he even thought about it at all.
I was once upon a time hired to convert a COBOL programmed system into an SQL database. The example I use is there was this one proceedure done in COBOL ( take data, modify it this way, output accordingly), literally the same
Re: (Score:2)
No. It's because you were figuring things out and making terrible decisions 50 years ago. None of you really knew what you were doing. You just called IBM support and had them do things for you, or did what the manual said.
The fact that systems can't be upgraded and have to run in layer after layer of emulation is proof that you did a poor job building a maintainable system. You never changed the program to run on a new system. You always had IBM to save you from doing it by having companies pay them more a
Re: (Score:2)
> I don't buy the implicit implication that because it's old it's not as good as new, for certain new would be worse given the AI riddled times we vibe in.
I agree, it ain't broke, so it don't need "fixed".
Maybe teach COBOL again? Talk about a job with gainful opportunities. In similar fashion, while I'm not a COBOL programmer, I've come back from retirement because after all, everyone simply knows analog and radio is dead so get into digital, it's the only thing we'll use in the brave new world of the future. So well versed electromagnetic people became kinda rare.
Turns out if you have a solid electromagnetic background and understand digital you name y
Just since covid? (Score:1)
It is not financially viable to write good software.
Re: (Score:2)
That was always the case. The problem is lack of liability and lack of regulation. Hence the whole business of writing software is a persistent race to the bottom and solid engineering does not even enter the picture. Try building a bridge the way "modern" software gets written and you will end up in prison.
$40 billion? (Score:1)
Why does that number sound familiar? Oh right, that's the amount of U.S. taxpayer money dear leader is handing over to a country whose leader ran the country into the ground and will now use that money to prop up his campaign for re-election.
In other words, in one fell swoop, the U.S. will lose another $40 billion in a matter of weeks compared to the first few years of covid.
Talk about efficiency!
Re: (Score:2)
It gets better. The soy bean farmers have lost their markets in China due to la Presidenta. China decided to buy from Brazil and Argentina because of the new tariffs from la Presidenta. Milei of Argentina took the initial $20 Billion (with $20 Billion more promised) and gave a hearty thank you to la Presidenta, and then turned around and cut Argentina's export tariff on soy beans. Chine gave a hearty thank you to Milei for this. Now, la Presidenta is promising to bail out the soy bean farmers in America due
In my experience... (Score:1)
I worked for a state govt for the better part of 15 years or so and had a lot of exposure to a variety of the ins and outs of how govt works with respect to IT systems, especially the older mainframe stuff, even though my team worked with more modern stuff.
Based on that experience....there is just no way a typical state govt is going to spend anywhere near the money required to replace a "perfectly functioning" system, no matter how old it is, no matter how much sense it makes, simply to "head off at the pa
Pournelle's Iron Rule of Bureaucracy (Score:4, Insightful)
I think it fits here:
First, there will be those who are devoted to the goals of the organization. Examples are dedicated classroom teachers in an educational bureaucracy, many of the engineers and launch technicians and scientists at NASA, even some agricultural scientists and advisors in the former Soviet Union collective farming administration.
Secondly, there will be those dedicated to the organization itself. Examples are many of the administrators in the education system, many professors of education, many teachers union officials, much of the NASA headquarters staff, etc.
The Iron Law states that in every case the second group will gain and keep control of the organization. It will write the rules, and control promotions within the organization.
Taken from: [1]https://www.jerrypournelle.com... [jerrypournelle.com]
[1] https://www.jerrypournelle.com/reports/jerryp/iron.html
I don't think that's an iron law (Score:2)
With that theory ignores is the third group who is actively trying to take control of the organization to twist it to their personal benefit.
That theory is basically just a round about way to say we shouldn't have government. It's literally saying that every organization, and every organization is some form of government, is bad and will always be bad.
What I have noticed about people who despise every kind of organization is a falling to a few groups. You have the true believing libertarians/anarchis
Re: (Score:2)
> With that theory ignores is the third group who is actively trying to take control of the organization to twist it to their personal benefit.
That's what the second group is. The first group is trying to do work. The second group is trying to amass power for personal benefit.
There's a corollary argument he would later make that as an organization grows, the second group will outnumber the first group. It's not a polemic against government. It's a warning against any organization getting too large.
Re: (Score:1)
In my experience, that just doesn't hold true.
Wait... (Score:3)
We count tax dollars spent through entitlement programs as GDP?
Re:Wait... (Score:5, Insightful)
Everything increases GDP. Increase in the number of car accidents? GDP increases. Companies suing each other over stupid bullshit? GDP increases. Recovering from massive weather events that just destroyed half your city? GDP increases. Cleaning up that oil spill? GDP increases.
All those circular investments by "AI" companies? GDP increases.
It's a bullshit number/concept.
Re:Wait... (Score:5, Funny)
Two economists were walking down the street when they saw dog poop on the sidewalk.
The first economist said to the second, "I'll pay you $100 to eat a spoonful of that dog poop."
The second economist figured it was an easy $100, so ate a spoonful of poop. Then he said to the first, "I'll pay you $100 to eat a spoonful."
The first economist ate a spoonful and collected his $100.
As they walked further down the street, the second economist said to the first, "We both ate dog poop for nothing."
The first economist replied, "Not true. We just added $200 to the GDP."
Re: (Score:2)
No, but when the recipient spends it, we might.
Re: (Score:2)
Yeah, and Healthcare is 20% of GDP.
According to Keynesian economists, if we were all much healthier the economy would be worse off.
I'm not sure how much more evidence you need that the entire economic school is a bunch of self-styled money-priests making excuses for government spending.
Keynes did some really good early work but then he got caught diddling kids and after that the King's spending was all the best thing anybody could do.
An early version of "trust the experts".
Re: (Score:2)
Money gets moved around a lot. The more it moves, the more people benefit from it. Every dollar spent becomes someone else's paycheck, someone else's income tax, etc. It's true that if you look at the number and try to move that needle independently, you wreck the meaning of the numbers. They are only symbolic numbers. But they represent something real.
It matters less and less (Score:3)
> The ones that used Cobol for processing unemployment claims failed spectacularly ...
Any payments from the US government to individual citizens - such as unemployment, Medicare, Medicaid, tax refunds, etc. - are well on their way to being a tiny fraction of what they were in previous administrations. And somehow I doubt that bailouts to oligarchs, and to countries such as Israel and Argentina, will be affected by legacy COBOL systems.
OTOH, a shortage of brown paper bags might put a serious crimp in the current administration's style.
Maintenance (Score:2)
> Why? Absolutely no idea
This isn't surprising to anybody who's studied the psychology of political science.
Those who identify as 'conservative' value maintenance much higher than those who identify as 'progressive'. You're more likely to see them in their driveway changing their oil and measuring their tire tread depth. It's just different kinds of people with different time-preference mindsets.
Note that with a limited budget maintenance spending is money that cannot be spent on immediate benefits.
You
Careful what you wish for (Score:2)
Have you seen the horror stories about the "upgrades" to government systems, I'm thinking [1] this one [slashdot.org] as recent example ?
The specs seem to be written by Steven King or the Brothers Grimm. If I were in charge of updating systems I'd take these as cautionary tales and maybe cling onto my old, working, system and add new functionality around the edges in a more modern way
[1] https://developers.slashdot.org/story/25/09/26/013211/disastrous-oracle-implementation-at-europes-largest-city-council
Rubbish Report from Atlanta Fed (Score:3)
This is completely incorrect. It was not the COBOL systems that failed, it was the "modern" front ends to them that failed.
COBOL ain't the problem (Score:2)
IMHO, the problem is at least twofold. 1) The problem domain of maintaining systems with changing bureaucratic / legal requirements occurs in all similar systems. That's deeply embedded organizational knowledge which is often upheld by the proud verbal tradition. 2) Again, similar systems have the challenge of fitting new interfaces onto existing systems. This isn't quite as challenging as 1), but the intricacies of fitting the bespoke glue and gaskets to the old pipes just isn't available from cut 'n
fraud (Score:2)
"States that used an antiquated [unemployment insurance]-benefit system experienced a 2.8 percentage point decline in total credit and debit card consumption"
Any chance that 2.8% 'consumption' difference is due to fraud built-into modern non-COBOL software? I mean, a time existed when thieving would embarrass ... thieves. But, post-modern business feels no such moderating constraint and corrupted software ( browsers are the trivial example ) is one of the expressive venue
Covid? (Score:2)
" The big argument in favor of sticking with Cobol systems is that they work. The catch is that, whenever they stop working, it is difficult to figure out why. That's not good in a crisis, which is exactly when they're most likely to break."
There's a crisis as soon as the old dude maintaining the code dies, from Covid or run over by a truck.
AI makes this problem moot (Score:2)
a good general software engineer with an agentic-ish AI IDE like Cursor, will have no problem fixing cobol. I don't know php, or wordpress, or SQL, or how to mitigate a ddos, or how to write an MCP plugin for Qt Creator, or how so use cloudflare to do the five separate things i need it to do, or any of the other DOZEN things that i accomplished last week, but with claude and several "make sure it passes the pinning and unit tests" prompts, I got it all working.
Like debugging Java or C# is any easier (Score:2)
Would the debugging been easier in Java, Rust, Javascript, or C#?
This isn't a case were they can just jump frameworks every couple of years to whatever is trendy...
Re: (Score:2)
Yes. It will be easier.
Rust and C# are easier to debug than Java. Not sure about Javascript but all 4 are easier to debug than Cobol...
Re:Like debugging Java or C# is any easier (Score:5, Insightful)
Yes. Definitely. Without a doubt.
The problem with these old COBOL systems is that they have decades of patches one on top of another, and very little formal testing. These systems were made in a time long before "modern good practices" were established. They work because the business requirements are straightforward and change very little. And the things they do are relatively simple. The barrier to entry is extremely high. COBOL is not taught anymore, and even if you learn COBOL on your own in Linux, in real life it won't be a Linux OS. It'll probably be several layers of proprietary IBM VM emulation, with Linux running AS/400 running AIX. And on top of that, you have whatever customizations this particular user made. You're a slave of what someone that wasn't necessarily a "wizard" decided 40 years ago.
With a more "modern" language, COBOL can make use of modern "good practices", especially automated testing and such.
the "jump frameworks every couple of years to whatever is trendy" is out of place when you are mentioning Java and C#. Both are well-established languages and have been stable for literally decades now. Java and C# (actually .NET) people are not in the same game as JS developers.
The problem isn't the language, but all of the things that come around it. Using a modern language would, if anything, let you ditch the expensive IBM support contracts for mainframe hardware (and maybe switch to slightly less expensive support contracts for regular hardware)
Re: (Score:2)
Not really. Mainframes have [1]come a long ways [youtu.be] from the 360, and can have a reliability regular hardware doesn't.
[1] https://youtu.be/I5tpoD4tCAg
Re: (Score:2)
> The problem with these old COBOL systems is that they have decades of patches one on top of another, and very little formal testing
That's true of modern systems too. Good practices should include highly modularized interconnected systems. A ground-up rewrite is impossible for monolithic software that's been around very long. That's why you need small enough pieces that you can actually take them on.
Re: (Score:2)
Indeed. But that would be good engineering. Modern enterprises do not do good engineering in the IT space. It is expensive and requires competent and experienced people. You cannot get-rich-quick that way and that is all that counts in today's greed-dominated IT industry.
Re: (Score:2)
Back in the '90s my instructor for BASIC programming told us, "If you learn COBOL next you'll never lack for a job for the rest of your life."
I studied RPG instead. Oh, well.