Job 1: Get the boss on the network. Job 2: Figure out why Job 1 broke the network for everyone else
- Reference: 1669010468
- News link: https://www.theregister.co.uk/2022/11/21/who_me/
- Source link:
This week meet a reader we'll Regomize as "Walt" who found himself working in technical support at, of all things, a theme park. His role was to solve the myriad computer problems of the various staff including engineers, cleaning staff, and park management, who used a range of iDevices to coordinate their work days. Life on a roller coaster is not just about the ups and downs, you know – there's technology involved, too.
Well, this particular day Walt was on his lonesome at the Help Desk when word came down that the personal fondleslab of the chairman was refusing to connect to the network. The device was brought down and Walt discovered, lo, that it was so.
[1]
In that moment Walt had what he generously describes as a caffeine-deprived brainfart, and decided the solution was to create a rule that meant the chairman's iPad would always be allowed to connect – essentially forcing the network to see it. It worked, and Walt then escorted the functional slab to the chairman.
[2]Just follow the instructions … no wait, not that instruction to lock everyone out of everything
[3]Run a demo on live data? Sure! What could possibly go wrong? Hang on. Are you sure that's not working?
[4]The boss worked in a fishbowl, so office tricks were a treat
[5]Data loss prevention emergency tactic: keep your finger on the power button for the foreseeable future
Upon his return to the office, he was confronted by a crowd of disgruntled engineers and cleaning staff bigger than the line for the river splash ride. Their devices could not connect to the network. None of them could.
It transpired – as Walt realized even before inspecting a device – that by creating an allow-list just for the boss he had unintentionally created a deny-list for every other device on the network.
[6]
What's worse, he had disconnected hundreds of park visitors from the free guest Wi-Fi.
You know that feeling in the pit of your stomach just after the first big drop on a roller coaster, as it goes back up again and you realize it's going to get worse before it gets better and you begin reevaluating your life choices? That's where Walt was at that moment.
[7]
He reversed the special rule for the chair's iPad immediately, thus restoring connectivity to the park and its many employees and guests. But of course that meant the chairman's iPad disconnected again, and that problem would need a less flatulent solution – as well as requiring Walt to tell the boss what had happened.
Have you ever found yourself plummeting from the summit of victory to the pit of despair because of a single slip-up? Brought a business to its knees with an idea that seemed clever at the time? Tell us all about it in an email to [8]Who, Me?
Get our [9]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Y3taSlCXhPP0NRAzmzqEYAAAABg&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://www.theregister.com/2022/11/14/who_me/
[3] https://www.theregister.com/2022/11/07/who_me/
[4] https://www.theregister.com/2022/10/31/who_me/
[5] https://www.theregister.com/2022/10/24/saved_by_the_analog/
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y3taSlCXhPP0NRAzmzqEYAAAABg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y3taSlCXhPP0NRAzmzqEYAAAABg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] mailto:whome@theregister.com
[9] https://whitepapers.theregister.com/
Re: What's the password?
Man I'm slow some days. It was only when I read this comment that the penny dropped as to which theme park it was.
Re: What's the password?
Here to meet Jessica, eh?
I hope it was only the WiFi
As the park would be very dark, very quiet and very still if the wired network went down too.
Re: I hope it was only the WiFi
...apart from the screaming as all the rides span faster and faster out of control(!)
But anyway - Why were the public and staff on the same WiFi network?
Re: I hope it was only the WiFi
"Why were the public and staff on the same WiFi network?"
I've noticed you'll often get that with Apple geniuses in charge of iDevices.
Re: I hope it was only the WiFi
Almost certainly on different SSIDs and VLANs but the same physical access points and cabling. Because it's expensive to double-up everything over a whole park when VLANs will do the job just as well.
But the MAC allow/deny list may not be VLAN specific.
Re: I hope it was only the WiFi
Because the safest approach is to consider EVERY WiFi network hostile and thus only allow access via VPN? That way you don't have to worry about staff using airport WiFi (which is *always* intercepted) or falling victim to a proxied network.
In addition, just because you hide the SSID and use a complex password doesn't mean it cannot be accessed. Even an "internal" WiFi network should not be able to reach anything critical without at least a DMZ or better in the way.
That's why you use a VPN.
But ...
... what was the underlying reason for the Chairman's problem? Particularly if an Allow rule for a given MAC fixed it, even though there was apparently an implicit allow-all beforehand.
I always find it irritating when these stories don't give the full explanation.
Re: But ...
I'm gonna go out on a limb and guess there were no available IPs in the DHCP pool when he tried to connect. If so, the solution would be to assign a static IP for VIP devices - and make them know if they replace the device they need to contact him to make that change if they want to insure this doesn't happen again.
The allow list wasn't a fix for the problem, other than for the fact that it caused everyone else to be kicked off, which voila freed up plenty of IPs :)
Re: But ...
Well, you have to admit he fixed the initial problem..
:)
The situation after this
The Chairman's iPad works here, but Walt disnae.
Banyan Vins network
I once was the network guy for an entertainment company. This was in the 90s in all its win95 glory, with the cursed NetBUI protocol by which every connected PC was telling the whole world it was here every single bloody second, via broadcast. That was annoying.
Since we'd just replaced the network legacy hubs by brand new switches, I began to explore the new possibilities and Oh, I found our switches could rate limit broadcasts !
So, I went the following morning to set this up and remove 90% of the broadcasts.
But what I didn't know was, the bloody Banyan Vines protocol was doing something crazy: use broadcasts and even assemble multiple broadcasts into bigger packets.
Just after the set up, the global directory went VERY SLOW, indeed, which prompted a queue of users at my office. Didn't take me long to fix it, though.
Bloody Vines !
What's the password?
Walt sent me!