Dev's last-day-of-contract code helped to crash app used by 350,000 people
- Reference: 1763966090
- News link: https://www.theregister.co.uk/2025/11/24/who_me/
- Source link:
This week, meet a reader we'll Regomize as "Ray" who reached the end of a temporary contract working at what he described as "a major corporation."
Ray was ready to move on to whatever came next, when the mega-corp asked if he could do just one more day of work to troubleshoot a production issue. He kindly agreed and showed up again.
[1]
To address the issue, Ray needed a production database connection string – code that includes the name and address of a database, plus credentials to log onto it. He therefore copied the string into his configuration file and spent a few hours fixing the problem.
[2]
[3]
"I quickly checked in my changes and headed out," he told Who, Me? Checking in those changes meant the major corporation's source control tools added his bug fix to its code base and default base configuration.
Ray had therefore made it possible for anyone else at the major corporation who could use that code to access his production database connection string.
[4]
He thought nothing of it until a few days later a former colleague from the major corporation called. Someone had used Ray's code and mistakenly used it to delete a database table, crippling an app that relied on it.
[5]Developer made one wrong click and sent his AWS bill into the stratosphere
[6]Techie ran up $40,000 bill trying to download a driver
[7]‘ERP down for emergency maintenance’ was code for ‘You deleted what?’
Readers may be wondering why we used the epithet "major corporation" so many times in this story.
We did to prepare you for the impact of that table deletion: 350,000 users could not access an app they relied on.
"They were down for pretty much a full workday," he told Who, Me?
Fortunately, the major corporation had good backups. And of course Ray didn't work there any more, so this incident was not his problem!
[8]
Have you made mistakes that caused damage long after you left a job? If so, [9]click here to send email to Who, Me? We won't make the mistake of ignoring your story. ®
Get our [10]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/columnists&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2aSQ6yfXfVVPzBb30tLxQrQAAAJU&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/columnists&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aSQ6yfXfVVPzBb30tLxQrQAAAJU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/columnists&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aSQ6yfXfVVPzBb30tLxQrQAAAJU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/columnists&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aSQ6yfXfVVPzBb30tLxQrQAAAJU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2025/11/17/who_me/
[6] https://www.theregister.com/2025/11/10/who_me/
[7] https://www.theregister.com/2025/11/03/who_me/
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/columnists&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aSQ6yfXfVVPzBb30tLxQrQAAAJU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[9] mailto:whome@theregister.com
[10] https://whitepapers.theregister.com/
Yeah, it doesn't Rayn, but it pours
The fact Rayn caused pain.
Was mainly able to be explained.
I wonder if he lived on the plain ..... in Spain ..... and if he ever had any falls?
" Fortunately, the major corporation had good backups. And of course Ray didn't work there any more, so this incident was not his problem! "
I'd like to think that Ray got a few quid for giving Major Corp. Inc. a good real world disaster test of their back ups, and any issues found on the back of that were duly fixed.
Have a pint Ray.
I hope he had a good Rayte there
Reading between the lines ...
It looks like Ray committed a file containing production database login credentials into the company's source code repository. That is shoddy practice on so many levels. If Ray had worked in my organisation, he'd be getting a very severe bollocking from our cybersecurity team *and* our DBA team.
Re: Reading between the lines ...
I'm still a bit fuzzy on that next bit of the story - someone finds that conn str in the repository and decides to use it to access access a test database ?
The clue should be in the server name in the string .
And thats not a random user , thats a developer with access to the repo
Re: Reading between the lines ...
The clue is probably in "default base configuration" and "anyone else at the major corporation who could use that code" and if that code was now part of the applications used by the 350,000 users not just "a developer with access to the repo". OTOH if that code also allowed a user to issue a "DROP TABLE" there were more bugs than the one Ray fixed.
Re: Reading between the lines ...
A contractor who did that should be told they're not working for the major company again.
For a junior employee, a formal disciplinary would be appropriate, for anyone at all senior there is no excuse.
This might seem harsh, but remember this would, amongst everything else, be a GDPR violation.
It's always a good idea to check the backups actually work, isn't it?
As a long time consultant/contractor
I knew well that permanent employees will always blame us for stuff after we're gone. Usually to cover their own ass, but sometimes it is even our fault. I always figured that's part of the reason we're getting paid a lot more than they are, we give them a convenient target to point the finger at. Both because we're no longer there to defend ourselves, and because they don't like higher ups bringing us in to "touch their stuff". Goes with the job, sort of like how a good manager will take the bullets for his employees.
Re: As a long time consultant/contractor
Oh, [1]PLEASE .
[1] https://how-i-met-your-mother.fandom.com/wiki/Provide_Legal_Exculpation_and_Sign_Everything
> Ray didn't work there any more
I should hope not.