News: 1733124671

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

NetAdmin learns that wooden chocks, unlike swipe cards, open doors when networks can't

(2024/12/02)


Who, Me? Welcome once more, valued readers, to another Monday and another instalment of Who, Me? in which Reg readers like your good selves share tales of tech misadventure.

Last week's reminiscence by "Erik" about locking himself in a client's premises inspired several readers to share similar tales. "Bilbo," for instance (not his real name) wanted to reassure us that "When working into the early hours of the morning it's easy to make mistakes" and that he had also locked himself both within and without the places he was meant to be getting work done. He had on various occasions resorted to popping out ceiling tiles and scrambling through crawl spaces as well as popping doors off hinges in the name of getting the job done.

He assured us he has learned from his mistakes. We can only hope – for his sake.

[1]

Another heroic Regizen was "Hudson" (also a Regonym) who was tasked with replacing the entire network stack in a branch office of his employer – "new router, about six switches in the main office/server room, and two other buildings on the property that each had a switch." Quite a task.

[2]

[3]

What's more, the existing cabling, he told us, was "a crime against all that is good and decent," so he brought all new kit to bring the place up to his exacting standards.

As with Erik last week, the place Hudson was working on used a combination of some physical keys and some electronic ones. This will be important later.

[4]

He set to work at about 5PM when the office closed for the day, and started deactivating and disconnecting the various cabling and equipment in the main building. By midnight he had rebuilt the stack with all the new kit in place and cabled up on racks and looking absolutely tickety-boo.

At this point he decamped to one of the satellite buildings – an equipment shop full of various tools – to replace the switch there, thinking it would only take a minute and it would be handy if that new switch were already in place when he powered on the main network stack.

With that task done he returned to the main building, only to discover that his keycard, required for entry, would not work. He tried it again, with no response.

[5]

You may have already realized his mistake.

[6]Network engineer chose humiliation over a night on the datacenter floor

[7]Undergrad thought he had mastered Unix in weeks. Then he discovered rm -rf

[8]The sad tale of the Alpha massacre

[9]Relocation is a complete success – right up until the last minute

The electronic key system, you see, was IP based, and ran off the very network that Hudson was in the process of rebuilding. He'd gone to the external building with his physical key without turning on the system he would need to get back inside to his gear.

And unlike Erik, he hadn't brought his phone with him to replace the other switch. Everything – phone, laptop, even his hotel room key – was in the deactivated network room, tantalizingly out of reach.

He walked around the building several times trying to find a way in, and settled on an entryway that was undergoing refurbishment. It was destined to become a glass-enclosed foyer, but the glass hadn't yet been fitted so the doors were boarded up with plywood. Most importantly, the hinges were on the outside.

Retrieving a hammer and screwdriver from the equipment shop, he carefully removed the pins from the hinges and shifted one of the heavy doors to gain entry to the building. Lacking the skills or strength to replace the hinge, he rested it more or less in place, attached an apologetic note, and went back to work.

This escapade had cost about an hour of working time, but he was still able to get the network up and running before people started arriving for work at 6AM.

Like Bilbo, Hudson claimed he's learned his lesson. He now keeps his phone with him at all times and chocks a door open if there's any chance he'll be locked out.

We kind of hope his employer knows its state-of-the-art keycard system can be defeated by a hammer and screwdriver – but that's another story.

It is indeed easy to make mistakes. We all do. The best thing to do is pick yourself up, dust yourself off, and [10]click here to send an email to Who, Me? so that we can share the story with your colleagues on some future Monday. ®

Get our [11]Tech Resources



[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Z02TU0x1tDYrMVKhYc4-dwAAARE&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0

[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z02TU0x1tDYrMVKhYc4-dwAAARE&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z02TU0x1tDYrMVKhYc4-dwAAARE&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z02TU0x1tDYrMVKhYc4-dwAAARE&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z02TU0x1tDYrMVKhYc4-dwAAARE&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[6] https://www.theregister.com/2024/11/25/who_me/

[7] https://www.theregister.com/2024/11/18/who_me/

[8] https://www.theregister.com/2024/11/11/who_me/

[9] https://www.theregister.com/2024/11/04/who_me/

[10] mailto:whome@theregister.com

[11] https://whitepapers.theregister.com/



Korev

Was Hudson installing Riverbed network devices?

Same network?

FirstTangoInParis

So the building physical security system is run off the same hardware as the business network. Even allowing for VLANs, is this normal?

Re: Same network?

Lazlo Woodbine

It is everywhere I've worked.

Most lock controllers are now PoE, so when the switch is powered down, the lock will either fail secure, or fail safe. Most sensible installers will add a battery back-up to any external doors, and those controllers have a local database of recently used cards for when they lose contact with the central server.

Re: Same network?

An_Old_Dog

The last big place I worked at had dedicated wiring for the electronic locks. That wiring ended up at a big box in the Facilities/Security building, with a dedicated Windows PC in the Security room connected to the control box via RS232C.

Most servers, the NOC, the mainframes, and the Help Desk all resided in four different buildings.

Re: Same network?

Lazlo Woodbine

That sounds like an old (and probably very reliable) system. The RS232 / 485 systems were great, but more expensive to install, and usually relied solely on the central data store, so power cuts would see the lock controllers fail. The place I currently work had that problem. and we all got locked out one morning after a powercut saw all the external doors firmly locked. Luckily someone had left a window open, so the smallest member of staff had to break in and physically remove the magnet from the front door.

I don't think I've seen a hard wired system that works with anything newer than Windows 7, so probably best to keep the system well away from the internet. Great for a single site, not so great for a multi-site operation.

Power Failures

An_Old_Dog

The security building had its own Diesel generator and battery back-up system, but it was "their" system, IT was kept away from it, and I knew nothing about their maintenance and test procedures, if any.

The locks ran on 24 VDC, and the control PC ran Windows 3.1. (It was in the 1990s/early 2000s.)

Nifty

I'd initially thought this was about building security and the way companies spend thousands on e-security only to leave themselves wide open to a 'people hack'. As in a recent BBC podcast on this topic. There's a book too.

People Hacker: Confessions of a Burglar for Hire https://www.amazon.co.uk/People-Hacker-Jenny-Radcliffe/dp/1398519014

SVD_NL

That actually ended up happening, popping hinges is the oldest trick in the book!

I think one of the issues is that it seems like many "high-tech" security systems are designed without consulting designers of "legacy" security systems, which effectively means every lesson they learned the hard way, has to be learned yet again.

You commonly see logically bulletproof electronic systems, but they can be bypassed by shims, magnets, picking poor backup locks, or simply a large heavy object.

The LockPickingLawyer has a great series of videos comically bypassing locks, i highly recommend watching them if you want a good laugh.

Lazlo Woodbine

A few years ago I was working at a large boarding school.

Its location in a small, respectable town meant that security wasn't really a high priority, with most buildings unlocked for much of the day, even though the local towns people would regularly walk their dogs in the grounds.

During lockdown though, we had a few issues with visitors to the area wandering around the buildings, somehow assuming we were a local attraction (the buildings were open so the site team and IT department could take advantage of the empty site to do essentially upgrades)

One of the leadership decided we needed to join the 21st century and ordered a fancy Paxton 10 system. At no point did he let us know in IT until the engineer arrived on site to do the install.

This system was so new, the engineer had never seen one before, and as he'd only just returned from furlough that morning, he'd not had any training - hurdle number 1

Hurdle number 2 - these locks all used PoE+, and we had no spare capacity on the PoE switches, so we needed new switches for every building, 15 new switches were ordered for next day delivery.

Hurdle number 3 - there was no structured cabling anywhere near most of the doors, so we had to run new wiring.

The system was installed, tested and commissioned just in time for the kids to return from lockdown 2

Hurdle number 4 - the locks were bluetooth, something the guy who ordered the locks hadn't thought about, naturally. His answer, the kids can download the app to their phones. Ahh, they're not allowed their phones in lessons. We could order 600 bluetooth fobs, I suggest. Ahh, nothing left in the budget.

We estimate the 30 locks, with 15 switches, cabling, 4 days installation time, new server and software, cost around £3,000 per door, and they were never used.

The server was quite nice though, an i7 Intel NUC which found a use controlling a display wall in the new 6th Form building

How typical

Pascal Monett

Manglement gets a "bright" idea and goes about spending money without any analysis whatsoever.

The new kit cannot be installed without yet more kit, so another round of spending because hey, manglement is never wrong, right ?

Finally, new kit is installed at x times the cost, hits another hurdle and manglement gives up.

At least you got a nice server out of it.

SVD_NL

That's hilarious, i see this sort of thing a lot but they rarely score that high on the incompetence scale!

You're also damn lucky you could get your hands on that many switches during that time period. I have pretty good rapport with a couple of large vendors but i was really struggling to get my hands on switches during that time period.

Lazlo Woodbine

We were going to order those switches for a different project at another school site, so they'd been reserved for us.

The huge overspend on the access control SNAFU meant that project was sidelined.

What we did have a huge problem with was webcams, they were rare as hens teeth, and kept going missing, so had to be replaced. I ended up ordering a box full direct from Logitech at about 3x the pre-pandemic price, it was handy having the IT director's credit card details...

Trygve Henriksen

I believe Deviant Ollam is the better choice for intrusion videos.

Flightmode

Darknet Diaries and, to a lesser extent, Malicious Life have both done multiple episodes on using social engineering to bypass both physical and logical barriers put up by companies with huge security budgets. Jenny Radcliffe (who wrote the book Nifty linked to above) was on Darknet Diaries ep #90 (simply called "Jenny"), and there have been many others. Anything with Rachel Tobac is worth listening to, as is Alethe Denis; but pretty much all of them are fascinating. (There are occasional exceptions where you can tell that the person being interviewed is CLEARLY exaggerating what happened to them and don't get challenged by the interviewer.)

Some ploys used by social engineers are extremely elaborate, but in many cases the old trope of people always stopping to let a person wearing a high-viz vest and carrying a clipboard through any door still seems to hold scarily true.

"So how did you manage to get in?"

SVD_NL

...

"Manual override, sir"

A different kind of oops ...

Michael H.F. Wilkinson

occurred at my university when the builders of a shiny new building forgot to install electronic locks on certain doors. These were doors to labs with kit like powerful lasers or other expensive and potentially dangerous kit was housed. Apart from not installing an electronic lock, no mechanical lock was installed either, so anyone could just stroll into these labs. Quite apart from kit being stolen, sauntering into a lab where powerful lasers are being operated without wearing suitable safety goggles is not high on my list of recommendations. The building itself is open to the public during daytime, so a lot of people were less than pleased. Quite the security and safety nightmare. Moving labs from the old building to the new had to be delayed.

Remember the Watergate Scandal?

An_Old_Dog

The Nixon Administration's "plumbers" were undone when they taped-over the openings of striker plates of doorlocks while they were breaking into the Democratic Party's headquarters in the Watergate Hotel.

A semi-ept security guard found the tape and removed it (yet did not call in a general alert).

I expect a techie's hold-the-door-open chocks would be removed, if discovered by Security.

Re: Remember the Watergate Scandal?

Lazlo Woodbine

much better to stuff the striker plate full black fabric, harder to see than tape over the opening.

Re: Remember the Watergate Scandal?

Sam not the Viking

I have some little blocks of wood which fit into the latch-hole for this very purpose. A bit of silver or black duct-tape and they are almost invisible.....

How did you get into this room?

ColinPa

We were at bank's HQ, and were meant to be escorted every where. We were only allowed in the operations room in special circumstances.

On our last day we had a meeting scheduled for the conference room in the operations area.

We parked our car, walked to the building and saw the door locks were green, so in we went, and found the operations area. The cleaner was there doing the room, and had blocked the door open to use the power supply outside the area. We smiled at the cleaner, said Hello, and sat down and did our preparation.

A little while later someone(we were working with) came past and asked "How did you guys get in here". We explained.

He came back later and said that there had been a little glitch with the outside doors (power ?) and they all unlocked for about 30 seconds. He also said he would fix the cleaner. I think this meant education and the provision of a "cleaner's socket" inside the secure area.

Re: How did you get into this room?

lglethal

Did you see the cleaner again? Or was anyone perchance laying new concrete as you were leaving?

Enquiring minds and all that...

Anonymous Coward

Working overnight when the power went out. Generator didn't kick in (broken switch in the start panel), and because of a fault on the building management system, switching to UPS failed too - and was behind 3 locked doors (locks failed shut with loss of power).

We had no choice but to kick the doors open (which was fun with only a small torch, especially as one of the rooms was a lights-out machine room - eerie in its own right as we were so used to the noise we usually heard in there).

Still, one screwdriver in the switch started up the generator. The rest of the night was spent recovering systems.

We were so poor we couldn't afford a watchdog. If we heard a noise at night,
we'd bark ourselves.
-- Crazy Jimmy