How to destroy expensive test kit: What does that button do?
- Reference: 1638779410
- News link: https://www.theregister.co.uk/2021/12/06/who_me/
- Source link:
Today's tale comes from a reader Regomised as "Trev" and has a slightly naval tinge to it.
"I was involved in installing a system in a corvette for a Middle Eastern navy," he told us. "Our customer was equal parts naive, hopeful, and bloody difficult."
[2]
Petrolheads should pipe down at this point, for the corvette in question was not of the four-wheeled variety, but instead a small warship. Not quite a frigate (although some have been redesignated as such), but a step up from a patrol boat. Corvettes tend be armed with missiles and medium to small calibre guns.
[3]
[4]
The system concerned was a Vertical Launch System (VLS), which does exactly what the name suggests. A VLS is a system for holding and firing missiles from naval platforms (such as a corvette). Each VLS consists of a number of cells and each cell can hold different types of missile. Someone presses a button, a missile is fired vertically, clears the cell and then heads off toward the unlucky recipient of its explosive largesse.
The most commonly used example of the breed is the [5]Mark 41 , of which over 11,000 cells have been delivered or ordered. The devices are used by navies around the world.
[6]
As for Trev, his team was in the installation phase. The customer was onboard to observe the goings-on and hopefully sign off on the paperwork. And, as is so often the case, it was taking a while and attention was starting to drift.
"A significant amount of test equipment," said Trev, "was located in the VLS missile compartment during the acceptance trials of the interface to the VLS."
Not cheap, we suspect, considering government money was probably involved.
[7]
"The customer reps were getting bored..."
One was idly looking at the control panel while Trev's team busied themselves with trial preparations.
In hindsight, what happened next was obvious to anyone who has ever had to deal with a customer who hasn't been securely handcuffed to an office chair.
"What does the button marked 'Pre wet' do?" came the question.
We imagine time slowed down a bit at that point, as the response "Don't touch that button!" came just a millisecond too late.
[8]Firewalls? Pfft – it's no match for my mighty spares-bin PC
[9]Hacking the computer with wirewraps and soldering irons: Just fix the issues as they come up, right?
[10]That thing you were utterly sure would never happen? Yeah, well, guess what …
[11]Quality control, Soviet style: Here's another fine message you've gotten me into
"Our budding rocket scientist presses the button," sighed Trev, "The VLS space is sprayed down with water, including a lot of (expensive) test equipment."
Unsurprisingly, the trial had to be called off. The test engineer had to start looking for new test equipment.
And the customer? Looking for lunch, we expect. Or, as Trev put it, "bugger[ed] off [on] an undeserved rest."
Harsh but, let's face it, where missile launch system acceptance testing is concerned, it could have been oh so much worse.
We've all had customers do some astonishingly silly things over the years. Ever had one get bored and start pressing buttons that maybe they shouldn't? Tell your story with an email to [12]Who, Me? ®
Get our [13]Tech Resources
[1] https://www.theregister.com/Tag/who-me
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Ya3tbyv73Z2ZMr8mRzuylQAAAUc&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Ya3tbyv73Z2ZMr8mRzuylQAAAUc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Ya3tbyv73Z2ZMr8mRzuylQAAAUc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.seaforces.org/wpnsys/SURFACE/Mk-41-missile-launcher.htm
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Ya3tbyv73Z2ZMr8mRzuylQAAAUc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Ya3tbyv73Z2ZMr8mRzuylQAAAUc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[8] https://www.theregister.com/2021/10/04/who_me/
[9] https://www.theregister.com/2021/08/23/who_me/
[10] https://www.theregister.com/2021/06/09/who_me/
[11] https://www.theregister.com/2021/04/12/who_me/
[12] mailto:whome@theregister.com
[13] https://whitepapers.theregister.com/
On my first weekend job as an extremely untrained PFY, I was told to just do what I normally did for the same client during the week - look after the Mac kit on a bus/ring network during the day, make sure the OS X server was happy, then switch off and lock up when I leave. With a line of sight to the doors, I could live the life of Larry sitting in the server room doing whatever I liked, like fixing up an old Mac II to play Risk clones on or messing around with the likes of Photoshop and Captivate that I would never afford.
If someone quiet came in, and I was otherwise engaged, I wouldn't see or hear anything different if I missed them coming in the door. The server room was out of sight of the main room, and they hadn't told me anyone else was in - I couldn't see anyone else on Sunday night and wasn't expecting anyone. However, there was indeed someone in doing a sensitive sequential backup. As I left for the night, sending the shutdown command (which had been set to be ignored this time) and flicked the switches, I'll never forget the blood-curdling "NOOOOOOO!" my long haired silent colleague screamed as I wiped out his day's work.
Oddly enough I didn't lose my PFY status, at least until the long haired silent colleague became the long haired silent boss, but I never got asked to do weekends again.
Silent Bob, by any chance?
"Do not press" button
Many years ago (1997) I designed and programmed a database system to record incoming discharge prescriptions to a pharmacy in a large UK teaching hospital (1000 beds).
It also recorded time to reach various staes in dispensing process (to help tweak processes)
This last bit is irrelevant to the story.
As a bit if fun, I put a button on main menu marked "Do Not Press" complete with "Stop" symbol...
Pressing opened a borderless window dialog asking if you wanted to format HDD..with "OK" and "Cancel". Of course "cancel" started "format" anyway complete with faux Windows progress ending in full screen DOS style "reboot" "screen" and "No operating system" message...
This stayed on screen for 30 secs for effect before "fooled you!" message.
System went live and I waited...unfortunately I missed it's "use" (out on wards) but heard ALL about it.
Our deputy chief pharmacist ran the VAX JAC based stock control/labelling system and managed its use with fearsome SOPs and weekly meetings at recent "errors" by unfortunate staff.
You guessed it, the ine person who should have known better...
He pressed it....deliberately....apparently his went bright red and he went rigid at the "reboot"
much to the delight and mirth of the surrounding staff.
He never told me he had pressed it...staff used to 'dare' new staff members to press it for a laugh with identical effect in the unfortunate.
I retired in 2017 but the system is still in use and ex-colleagues tell me the button is still in regular use
From the late, great Sir Terry
“Some humans would do anything to see if it was possible to do it. If you put a large switch in some cave somewhere, with a sign on it saying 'End-of-the-World Switch. PLEASE DO NOT TOUCH', the paint wouldn't even have time to dry.”
From "Thief of Time
The more senior the person the more likely they will press a big red button
Early 1990's had a High Availability VaxCluster system with 4 nodes, this system was very robust and basically acted like a fault tolerant system with no unplanned downtime. The financial institution that owned the system was very proud of its record of not having service interruptions (As these were very costly, hundreds of thousands in a few hours type.) and for reasons that are lost to the midst of time the CIO wanted to show the board the very expensive cluster system that was making them more money and enhanced their customer service experience etc.
The CIO said how in this single facility they had duplicate systems, power, comms etc and they were investing in a second site to provide improve resilience etc, the Board Chairman seem to have taken the CIO at his word that nothing would take the system down and wandered over to the big red emergency power off button and pressed it - This brought the tour to a immediate end, cost an eye watering amount of money in lost business, permanently ended DC tours (Even for the most senior of executives) and of course the Chairman was exonerated as how was he to know that the Big Red Emergency Power Off labelled button would actually do what it said.
But on the bright side, the CIO got promoted out of the company and the financial institution 2 years later replaced the VAX'es with a Pyramid Unix system (From Pyramid Technology which was eventually bought by Siemens) and although powerful and fast for the time the Pyramid system was not super reliable and the service interruptions became almost a weekly event.
The VLS space is sprayed down with water
Fresh or sea water?
the story's about a navy in the middle east. it would have been fresh water bottled from a glacier in the himalayas.
Anyone got any suggestions what the purpose of the "Pre wet" button might be , aside from ruining test equipment?
Expensive test equipment
Once you get beyond the ordinary multimeters, just about all test equipment is expensive. It is a relatively small market so the margins have to be somewhat higher than consumer kit.
A new oscilloscope / analyser I specified 8 years ago to test multi-gigabit links and DDRx cost about £150k including the probes.