Field support chap got married – which took down a mainframe
- Reference: 1749454215
- News link: https://www.theregister.co.uk/2025/06/09/who_me/
- Source link:
This week, meet "Zeke" who told us about his time as a field engineer for a data communications company.
I am still embarrassed 45 years later because someone else had to clean up my mess
"The job involved installing and repairing business modems and other comms gear in large banks, insurance companies, etc," Zeke told Who, Me?
His career was moving along nicely. "I had worked my way up to district tech support and was on my way to a regional position," he wrote.
One day, Zeke's employer dispatched him to a major department store operator.
[1]
"Their comms system had lost communications with their mainframe," he told Who, Me? "To isolate the problem, which was not easily identifiable, I had to remove and replace each of five comms circuit cards."
[2]
[3]
The retailer would not let Zeke conduct that investigation with the power off, because it needed various systems to continue operating.
"I had removed and replaced the suspect cards one by one without success, and was now looking for a problem on the backplane – perhaps the bent or broken pins that were not uncommon in this system – when I noticed a very slight spark," Zeke told Who, Me?
[4]
The brief and small discharge had no apparent effect on the machine, so Zeke ignored it in the hope it was nothing to worry about.
He was wrong.
"After replacing some cards, and reseating them all, the entire system had come down," he confessed to Who, Me?
[5]
He soon figured out what had happened.
"While perusing the machine, my recently acquired wedding ring had hit some pins and short-circuited the backplane, destroying almost all of the cards."
[6]US govt login portal could be one cyberattack away from collapse, say auditors
[7]Teens maintained a mainframe and it went about as well as you'd imagine
[8]Users hated a new app – maybe so much they filed a fake support call
[9]Untrained techie botched a big hardware sale by breaking client's ERP
Zeke didn't have the parts needed to repair the system and told us he was "so embarrassed about what I had done that I couldn't bring myself to tell the customer what had happened."
He instead contacted a colleague who came and fixed things up.
"Forty-five years later, I am still embarrassed about that job; not only about trashing the customer's system, but also because someone else had to clean up my mess," he told Who, Me? "I never wore my ring at work again," he added.
Has your jewellery or clothing caused a computational or comms crisis? If so, [10]click here to send us an email so we can share your story in a future instalment of Who, Me? ®
Get our [11]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2aEaws7mg8AEuYzOUtI2EpgAAAso&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aEaws7mg8AEuYzOUtI2EpgAAAso&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aEaws7mg8AEuYzOUtI2EpgAAAso&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aEaws7mg8AEuYzOUtI2EpgAAAso&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aEaws7mg8AEuYzOUtI2EpgAAAso&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2025/06/04/login_gov_backup_testing_insufficient_gao/
[7] https://www.theregister.com/2025/05/05/who_me/
[8] https://www.theregister.com/2025/04/11/on_call/
[9] https://www.theregister.com/2025/02/24/who_me/
[10] mailto:whome@theregister.com
[11] https://whitepapers.theregister.com/
Back in my uni days we had a lecturer who'd spent time working with mech. engineers on ships. It didn't take much to get him to go off topic and recount some past adventures. I can vividly remember him, completely unaware he was doing it, automatically feeding his tie between two shirt buttons to keep it out of trouble. Muscle memory.
VAX field service engineers
I can still remember, in the early 80's DEC VAX field service engineers removing their wedding ring before working anywhere near the computer's backplane. Apparently the current available from the power supply could heat the ring to red heat in only a short contact.
And I can still remember FIELD / SERVICE and SYSTEST / UETP too. The good old days of insecurity.
Re: VAX field service engineers
My father was a motor mechanic (and, back in my younger days, I did most of my car maintenance) he was strict about removing all jewellery when working on a car (in fact, doing almost any DIY work). He told me, quite graphically, of a fellow mechanic whose wedding ring shorted a battery terminal - not pretty. In my first job I was working with metal forming equipment that used high current to melt metal between dies - reinforced the warning. When I visited offshore North Sea oil installations I didn't need to be reminded of no-jewellery rules when "on-plant". It's not just electricity - just getting a ring caught on something and it's quite painful. And the comedy routine where someone gets their shirt tie caught in something is no laughing matter.
Re: VAX field service engineers
It's not just electricity - just getting a ring caught on something and it's quite painful
My mother was in the WAAF during the war, some of the time she was servicing airplanes. Lots of stories to tell - the technicians were "encouraged" to go up in the plane on the first flight after servicing. Winding up the undercarriage on an Avro Anson. Pilots showing off with aerobatics and trying to impress the attractive young woman sitting next to them. But what sticks in the mind most is the scar from a ring that got caught on the fuselage as she got out.
Re: VAX field service engineers
There is a footballer who lost a finger, due to a ring when he climbed a fence
Re: VAX field service engineers
This is actually a significant part of the reason I took to wearing bow ties, which don't suffer from that problem. As an added bonus, when I was riding my motorbike, it was much less inclined to wrap itself around my neck and strangle me.
Dammit, why are none of the icons obviously wearing bow ties? Well, pretend these two swords make for a lovely bow tie. --->
Re: VAX field service engineers
BT Technicians in the days of the strowger telephone exchanges were also not allowed to wear rings and bracelets for the same reason.
The telephone exchanges were powered by a massive 50 volt battery, which could heat a ring to a high temperature very quickly.
Re: VAX field service engineers
A friend of mine reached behind a large bank of relay racks and managed to get his Rolex watchband across the large 48V supply ... The resulting loud "CRACK!" and fans spinning down, coupled with the smell of roasting/burning pork, were rather disturbing. To say nothing of the screaming. I managed to calm him down & get him to the ER ... Xrays showed little balls of gold melted into his wrist behind the 3rd degree charring. The surgeons later told him he was lucky to still have full use of his hand. Today, 40 years later, the scarring is still impressive, despite skin grafts. He got a new band for the watch, and now wears it on his other wrist. It still works.
And people wonder why I always take off my wedding ring when working on electrical stuff. Yes, that includes cars, trucks, bikes, boats, etc.
For the record, the power supply was no worse for the wear, and the equipment in the relay rack automatically powered back up as if nothing had happened ... Thank you, Lorain.
Oddly enough, we were both working for DEC at the time. Beers all 'round for the memories :-)
Note that the "watch pocket" in a pair of Levis 501s is the perfect place to park a ring for the duration ...
Re: VAX field service engineers
SYSTEST / UETP
Surprising how many people forgot that one when "securing" their systems. I remember wandering semi-randomly around JANET back in the 80s, and finding many accessible systems. That was in the days when the worst that would happen was a polite "who are you?" request.
Schedule a effing downtime!
And don't skimp on the over hours payment.
It is literally as simple as that, as long as you treat your staff with the respect they deserve and make your company a place they like to be at. If I hate the place (first, why would I still be there) I won't really enjoy doing maintenance at night, and no small amount of money will motivate me.
Working on an open powered up machine is high risk, I had a mate drop a network card onto his machine's mainboard once, grateful it was not a production environment and he was just showing off hotplugging.
The card was fried, the rest of the system survived, surprisingly.
Yeah, try to avoid it.
Re: Schedule a effing downtime!
"The RETAILER would not let Zeke conduct that investigation with the power off, because it needed various systems to continue operating."
Now do you see why Zeke couldnt' do it?
Of course it didn't work out for them but even so decisions like that aren't necessarily in IT's control, let alone a field tech's.
Re: Schedule a effing downtime!
As the experts, and the people they've hired to fix the problem, you tell them what you need. If they'd said he couldn't touch the servers they'd have said "we can't do it" so why is needing the power to be off any different?
Re: Schedule a effing downtime!
Agreed. I would simply say the risk of damage or serious injury was too great. The choices are: shut down for a short while, or short out something and have a much longer outage.
This should also teach the customer a valuable lesson on the need for redundancy in critical systems.
My brother's wedding ring saved his fingers, literally.
We was a truck mechanic back then, and was working on an engine with the cab jacked up, when the hydraulics holding the cab up suddenly failed, the cab fell down onto his hand. Luckily for Peter, his newly acquired wedding ring took the brunt of the force.
The ring needed cutting off, whcih was a much simpler task than sewing all his fingers back on.
Cue the pressure door scene from The Abyss .
When I got married
I was called to see the company owner when I came back from honeymoon. Was it a congratulatory pay rise, or redundancy, or what?
He showed me a photograph of a very burnt finger from where a wedding ring heated up almost instantaneously after shorting some backplane power pins.
He didn't need to say anything - the ring was already off after seeing that!
And it only comes out of its original presentation box for special occasions, as does the metal banded watch that is was presented on the 21st birthday!
Was it a Token Ring network?
Maybe he married Sauron and it was a Tolkien Ring?
OK, I'm sorry already! Please don't hurt me!
We'll let you off .... but don't make a hobbit of it!
Not don't be Nazgul about it
I bet that ring cost a packet
Errr, embarassingly similar with my (much) younger self, but nobody to see it happen.
Working on kit with a 5V 200A power supply to power a rack that I had buit from the boards upwards, and hence written the safety and instruciton manual.
"Remove jewellery".
I thought I knew better for a brief job inside the rack: no need to remove the wrist watch with stainless steel strap, I am nowhere near the bus bars.
Bzzzt..... LEDs dim, and two brown marks appear in the steel.
Fortunately the PSU had an active overload detector, so the only other damage was to my pride, but it enhanced my wisdom.