Tired techie botched preventative maintenance he soon learned wasn't needed
- Reference: 1737966609
- News link: https://www.theregister.co.uk/2025/01/27/who_me/
- Source link:
This week, meet a reader we don't need to Regomize because his name is Dave and he is happy for us to use it.
Dave sent a tale of his time working for a company that operated a PC he was told simply had to run 24x7. No ifs, no buts. No change windows allowed. This PC would always run.
[1]
The machine ran SCO Linux (which later became the subject of [2]perplexingly lengthy litigation ) and two workloads. One was a reporting app. The other was a program that took the horrible format of the reporting app's output and translated it into something other software could consume.
[3]
[4]
If those reports weren't reformatted, they couldn't be processed and people would get mad.
This being the 1990s, the essential machine was equipped with a hard disk drive that was making the sort of noises that suggested it would be prudent to replace it sooner rather than later.
[5]
Dave learned of the disk's declining health late on Friday after what he described as "a brain-melting 80-hour week with a lot of European travel involved." He was therefore more than a little fatigued but still felt he could replace the disk, and that a brief outage on Friday night was tolerable and preferable to dealing with a longer and unplanned problem in the future.
To get the PC back into robust condition, he decided to clone its hard drive onto fresh hardware, then swap the old disk for the new.
Did we mention Dave was really tired?
[6]
Turns out he was so tired he flubbed the job and overwrote the source drive instead of cloning it.
The essential PC was now dead.
But all was not lost! Dave knew that another PC in a different office did the same job and was therefore configured in identical fashion. If he could get to that office and clone that drive, he could save the day.
Which was a great plan except for the fact that the other office was four hours away by car.
Dave stocked up on energy drinks, hit the road, made it to the other office, convinced security to let him in, and cloned the drive properly.
[7]Developers feared large chaps carrying baseball bats could come to kneecap their ... test account?
[8]Life lesson: Don't delete millions of accounts on the same day you go to the dentist
[9]Brackets go there ? Oops. That's not where I used them and now things are broken
[10]Coder wrote a bug so bad security guards wanted a word when he arrived at work
He left that office at around 9PM, reached the first office and its essential PC at 1AM, plugged in the drive and watched it come to life.
At around 2AM he reached home and told us he "slept the sleep of kings."
On Monday morning, Dave was summoned to a meeting at which, thankfully, the essential PC's downtime was not on the agenda.
But its ultimate demise was.
"We were told about plans to replace the PC and the reporting application," he told Who, Me?
The move happened three weeks later, meaning Dave realized the drive he fixed would probably have lasted and his Friday night trek was futile. And even if the box had broken down, its absence would probably have been tolerated given the imminent migration.
Ain't IT grand?
Have you tried to maintain tech but instead made a problem worse? Or been needlessly proactive? [11]Click here to send us an email with your story so we can share it on a future Monday. ®
Get our [12]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Z5dnXjfmiQq7f-id6OC0TwAAAQs&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://www.theregister.com/2021/08/30/sco_tsg_vs_ibm_settlement/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z5dnXjfmiQq7f-id6OC0TwAAAQs&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z5dnXjfmiQq7f-id6OC0TwAAAQs&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z5dnXjfmiQq7f-id6OC0TwAAAQs&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z5dnXjfmiQq7f-id6OC0TwAAAQs&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2025/01/20/who_me/
[8] https://www.theregister.com/2025/01/13/who_me/
[9] https://www.theregister.com/2025/01/06/who_me/
[10] https://www.theregister.com/2024/12/16/who_me/
[11] mailto:whome@theregister.com
[12] https://whitepapers.theregister.com/
Re: Ah, hardware replacement...
If you keep the old kit running, your hardware replacement budget will be zero. You need a bit of downtime to justify replacing it.
Re: Ah, hardware replacement...
This is so true it's painful.
SCO Linux?
Being the 1990s I am guessing SCO Unix which vaguely recall might have been a System V.3 + Xenix product.
Re: SCO Linux?
The only SCO Linux that I recall was Caldera OpenLinux which was renamed "SCO Linux powered by UnitedLinux", but that was in 2002 or thereabouts.
Insane litigation soon followed.
Dave stocked up on energy drinks, hit the road, made it to the other office, convinced security to let him in, and cloned the drive properly.
He left that office at around 9PM, reached the first office and its essential PC at 1AM, plugged in the drive and watched it come to life.
I bet he was quite SCSI by the end of that...
A good fix, ATA boy...
At least he didn't become a Serial killer.
He was a SCSI Terminator though
Interleave at 9pm for the other office.
You're on the right track.
Nearly, but not quite...
A PC was shipped from the US to the UK - it held a setup to be used for training on a new product. Problem was, the ride across the pond was not a smooth one. When I unboxed the system to set it up, it rattled. I had to reseat a couple of cards that had come lose. Powered it up, and the disk gave some errors. So I ran a check disk (can't remember which one) and it reported a number of bad sectors. I ran it again and the number increased, and the disk sounded increasingly unhealthy - until I put the system on its side. It lasted long enough for the training course.
I remember shipping a Dell across the US and getting a system in the shape of a parallelogram, and not of the square variety.
It was a rather beefy server machine, so I've always wondered just how much force that was.
…in the shape of a parallelogram, and not of the square variety. Well, that's not normal, then.
As Murphy's Laws tell us
No good deed ever goes unpunished.
Though in this case, it merely went unnoticed and unrewarded
The drive would have failed before the system would have been replaced, because Universe is a b***h.
one way of getting the budget for an upgrade
One guy told me he was fed up justifying hardware upgrades, and being told no money available - while people were getting modern laptops.
He solved this by shutting down one of the key systems and restarting it during a busy time. This caused a mini outage, and so there was a task force to look into it.
He said the hardware might not last a month. If it failed it would be a day's outage; and the amount of money being spent on this task force would pay for the hardware upgrade 5 times!
Fortunately people saw sense and the upgrade was approved before he got back to the office.
One of the senior manager walked past later and said "good job - well done - don't do it again"
Re: one way of getting the budget for an upgrade
Thumbs up for the manager, for (a) understanding what had happened and (b) for not dobbing the guy in.
Ah, hardware replacement...
I'm talking from experience here: The hardware replacement budget is directly correlated to the amount of dark incantations you just had to perform to keep the old junk running.