Turning a computer off, then on again, never goes wrong. Right?
- Reference: 1688974025
- News link: https://www.theregister.co.uk/2023/07/10/who_me/
- Source link:
This week meet a reader we'll Regomize as "Arnie" who encountered a problem that required finesse and a delicate approach, but decided to brute-force it and hope.
Arnie worked for Britain's National Health Service (NHS) – a noble and fine institution that deserves much respect, yet has featured in this column rather more than you'd like a health service to feature in recent times. As Arnie and the NHS [1]point out , it is "a complex system ... made up of a wide range of different organisations with different roles, responsibilities and specialities", some of which often operate quite independently of each other.
[2]
In the county where Arnie worked, communication between those different orgs was sometimes not perfect.
[3]
[4]
On one occasion, Arnie was sent to an NHS facility to check on a printer problem. The bit of the NHS Arnie worked for had provided both the printer and an HP box for the facility to use.
When he arrived, though, he did not find the HP box that had been provided. It had, at some point, been replaced by a Lenovo box – presumably by some other bit of the NHS. What's more, the Lenovo box was not running vanilla Windows, as the supplied HP box had been, but Windows Server. And an unfamiliar configuration at that.
[5]
Now, you may say at this point that Arnie should have put a call in to someone. Should have told his superiors that this was not his system, and whoever had configured it should come and fix it. You clearly do not know Arnie.
Arnie had been around a while, and knew that some NHS facilities had some quirky requirements, so these changes should simply be taken in stride. Sure, Windows Server isn't Windows, but it's still Windows, right? How different can it be?
So he set about troubleshooting, but was getting nowhere with his familiar bag of tricks. The printer simply would not respond.
[6]
When in doubt, of course, what do we do? We switch it off and then on again. Boom. Old faithful, never fails.
[7]Hacking a Foosball table scored an own goal for naughty engineers
[8]Security? Working servers? Who needs those when you can have a shiny floor?
[9]Data cleanser did its job, but – oopsie! – also doubled customers' bills
[10]A toast to being in the right place at the right time
Except, remember how this thing was running Windows Server? And you know how servers are quite often … what's the word … serving things?
Well it turned out this server had been serving facilities distributed around half the county. And what's more, whoever had configured it had not configured the database and core applications to restart when the machine was rebooted.
Chaos, as you might imagine, ensued. At the inevitable "what went wrong and who can we blame" meeting that followed, it transpired that the server had been configured by the son of a senior manager at the facility, which goes some way to explaining the not-quite-NHS way it was being managed. ("Yay for nepotism!" says Arnie.)
As for the original printer problem? It was never fixed.
The missing HP box? It was never found.
And Arnie? He will not be back.
If you've ever pushed the wrong button, restarted the wrong box or troubleshot the wrong trouble, The Reg wants to hear about it. [11]Click here to send an email to Who, Me? and we'll share your tales with the world.
The Who, Me mailbox has seen better days so please don't be shy about sending your stories! ®
Get our [12]Tech Resources
[1] https://www.england.nhs.uk/get-involved/nhs/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZKvWw6jwH@vj9SQDbkPH1wAAAcI&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZKvWw6jwH@vj9SQDbkPH1wAAAcI&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZKvWw6jwH@vj9SQDbkPH1wAAAcI&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZKvWw6jwH@vj9SQDbkPH1wAAAcI&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZKvWw6jwH@vj9SQDbkPH1wAAAcI&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2023/07/03/who_me/
[8] https://www.theregister.com/2023/06/26/who_me/
[9] https://www.theregister.com/2023/06/19/who_me/
[10] https://www.theregister.com/2023/06/12/who_me/
[11] mailto:whome@theregister.com
[12] https://whitepapers.theregister.com/
Re: Reminds me of an old (early '80s) AI koan ...
Oh Jake you sweet, summer child...
Your inexperience with Windows is showing here. Power-cycling with no understanding of what is going wrong is how 90% of the issues I have with my computer are fixed!
Although I am reminded of all the times that stuff starts magically working as soon as the relevant expert is watching from over somebody's shoulder.
Re: Reminds me of an old (early '80s) AI koan ...
Happy memories of when I did desktop support & would shlep across site or to a different site only to hear a variation of:
Me: "Can you show me the problem please?"
User: "I open this & select here & then click this button and then... Oh, it's working now"
Re: Reminds me of an old (early '80s) AI koan ...
The big problem there is usually that previously they were either opening the wrong thing, selecting the wrong thing or not selecting anything, clicking the wrong button or all of the above.
Users are idiots. (And I can say that confidently because most of the time, I'm a (L)user)
Re: Reminds me of an old (early '80s) AI koan ...
In my experience these 'issues' are normally down to 10 problems.
P.I.C.N.I.C errors
1 D 10 T problems.
In about 1% of cases, I actually had to fix something instead of provide implicit instructions on how to use something properly.... for the 50th time to the same dept.
PC Engineers...
I've probably recounted the tale previously of a 486 based HP SCO Unix box I managed for a while. I was on holiday but had taken the company "mobile" (a car phone attached to a blooming great battery pack) just in case. Half way through I got a call, the backup tape wasn't ejecting, what should they do?
I balanced the risks and decided the best thing was to wait until my return a few days later and I'd sort it.
That apparently wasn't good enough, they summoned an engineer, against my instructions and without telling me, from the parent company's PC support people. He travelled the 150 miles or so from the South Coast, turned the box off without any thought of a controlled shutdown, manually ejected the tape then turned the box back on. When it wouldn't boot again he ran away after saying, "Oh shit!" or words to that effect.
It took a few days to completely wipe the file system, reinstall and reconfigure Unix then restore the last good backup tape which by then was over a week old. Had it been my own fault I'd have pulled an overnighter and got it back within 24 hours but as it was down to senior management I wasn't going to rush (you can always find an excuse to go home* such as running a disk check that will take several hours).
*A DEC engineer once used a thunderstorm as his reason for not continuing to fix a fault, I didn't argue as I wanted to go home too.
Re: PC Engineers...
Arnie reminds me of one or two PC engineers I've had the misfortune to encounter in the past. They can't be bargained with. They can't be reasoned with. They don't feel pity, or remorse, or fear. And they absolutely will not stop... ever, until the server is dead...
Re: PC Engineers...
A DEC engineer once used a thunderstorm as his reason for not continuing to fix a fault
Well trying to fix faults when there is a thunderstorm lurking about can be pretty shocking.
Re: PC Engineers...
Love the description of a thunderstorm lurking, as if they hang about on street corners like Teenagers or something..:)
Re: PC Engineers...
>Love the description of a thunderstorm lurking, as if they hang about on street corners like Teenagers or something..:)
This last weekend they certainly seemed to. We had at least 3 including one really spectacular one on Saturday night.
And Arnie? He will not be back.
Ah, too bad. Seems the "blame meeting" converged rapidly onto him ...
Re: And Arnie? He will not be back.
Probably correctly from my reading of it?
Maybe a bit harsh.
No, not really.
Doing anything to a box you know nothing of, especially in a server room, is a big no-no.
Re: And Arnie? He will not be back.
No way of knowing if the manager came back, or brought his nephew in again.
Network problem
I was expecting a network problem for Arnie because it hadn't been terminated properly...
Hmm ...
Let's see: unexpected hardware, unfamiliar software, not the system you installed ...
Use the pink paint, deploy the SEP field and GTFO as rapidly as possible.
Re: Hmm ...
Unfortunately, they didn't apply BistroMath to settle the account.
Ah yes, the joy of software that runs as an application on a server. Frequently we'd have to support customers who had installed updates (go them!) but failed to complete the process by logging the server back on to the required account. As regulated customers the lack of records could prove expensive..
It's never been installed
Someone told me that they worked on a project where the development was done in one country and the testing was done in a different country. Due to the executive announcing the customer date before development was quarter done, it was pretty chaotic and corners were cut.
The installation team followed the process of getting a clean machine and trying to install the product. It always failed. They executive blew his top and sent his top expert to "SHOW THE INSTALLATION TEAM HOW TO DO IT". The guy got off the plane, went directly to the lab and tried to install it. It didn't install. He spoke to the development team - they hadn't installed it either - they just replaced the binaries every day and were careful not to turn the machine off. I think no one had actually been given the job of writing the installation process, people just used some scrappy notes.
In the development team, because the project was running late, they were asked to resize the work. Now they knew what they were working on, instead of some chart-ware design, they resized it properly - and said it had gone up from 3 months work to 9 months work. The executive blew his top as this was not the answer he wanted to hear. He wanted them to resize it again! but someone told him - every time they resize it - it costs 2 weeks work. Do you really want to ask them?
Re: resize it again!
Yes, because obviously if it didn't give the proper result the first time, Shirly it will the next time, right ?
Ah, manglement. They deserve every meme they get.
Re: resize it again!
It's not about results, it's about power.
(And money. Money is power.)
6 weeks
I was working on an installation in Germany of a product from the US of Texas.
We could not get it to correctly configure for our environment. I edited, tweaked, wiped, reinstalled.
Customer was getting angry at delays, Vendor was showing how it worked on their Dev environment and blaming me.
We had a marathon session where I reinstalled and configured from scratch with their team remotely watching.
They agreed I had done everything right.
3 days week later they rebuild their code, issued a patch release and it worked.
Many drinks later with a friend in the EU branch I found out why - they'd HARDCODED the path to the config file in their dev environment.
development team - they hadn't installed it either - they just replaced the binaries every day and were careful not to turn the machine off.
Oh yes, we've all been there.
That's why our QA engineer always started a new test run of a new version by installing the product exactly as the documentation said, no matter how well she knew the product by then. The result was often bug reports filled against product and/or docs, but we rarely had issues from customers.
At two or three companies I managed to get a process for installing a development environment for new developers.
It started with a printed sheet of paper. The paper said “follow the instructions on this paper. If they don’t work, then ask for help, and change the instructions so they work”. That was needed because what’s on a brand new machine would change over time.
And one part of the instructions was where to find the instructions as an editable document so the new guy could update them.
Printer Problems
I was sent to a remote site to install some new software many years ago. The senior engineer had mentioned (with a grin) that although he had never been to the site, the younger lads in IT were always popping in there to fix 'printer problems'. The printer seemed absolutely fine, but oddly enough the young lady who worked there was absolutely stunning.
Reminds me of an old (early '80s) AI koan ...
A novice was trying to fix a broken Lisp machine by turning the power off and on.
Knight, seeing what the student was doing, spoke sternly: “You cannot fix a machine by just power-cycling it with no understanding of what is going wrong.”
Knight turned the machine off and on.
The machine worked.