News: 1687764851

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

Security? Working servers? Who needs those when you can have a shiny floor?

(2023/06/26)


Who, Me? Ah, gentle reader, once again it is Monday and all that entails. But fear not, for The Reg is here with Who, Me? and another tale of things going not quite so well as might have been hoped. Perhaps this will lift your day.

This week's raconteur, who we'll Regomize as "Murph," brings us a tale tying together two of our recent themes: security tech that doesn't behave as it should, and power points being used in ways they were not meant to be.

Murph works in an older building that's been renovated, with "marble entry and tasteful carpeting in the work areas." The server room inhabits a repurposed gymnasium (no servers under the break room bench here) and there are tiled floors everywhere. In fact, "with lots of tile across seven floors, we have a decent size maintenance staff to keep it looking good."

[1]

Now as you might imagine in a facility like that, different staff have access to different areas. Security fobs ensure that people go only where they are meant to. All well and good.

[2]

[3]

Until one day, when a new member joined the maintenance staff. Decent bloke. Conscientious. Hard worker. Well liked.

For some reason – we will likely never know the reason – when this new affable cleaner signed on, he was not given the standard security fob that maintenance staff were issued. He was given an old fob that had formerly belonged to a retired member of the IT team. Obviously this is a problem for a bunch of reasons. But that's not where our story ends.

[4]

The new guy was asked to clean the floors in the IT department. All well and good – and with his super-powered fob he could clean even more of it than anyone expected.

Take, for example, that former gymnasium, with its expanse of dingy tile that hadn't had a good polish in who knows how long.

[5]Data cleanser did its job, but – oopsie! – also doubled customers' bills

[6]A toast to being in the right place at the right time

[7]Fed up with slammed servers, IT replaced iTunes backups with a cow of a file

[8]Seriously, boss? You want that stupid password? OK, you get that stupid password

That grime was just fine, of course, because it's well known that mops and datacenters don't mix.

Conscientiously – diligently, even – he plugged his industrial floor polisher in to a power board labelled "A". As it turned out, this connected to the "A" rack of uninterruptible power supplies.

"Nine minutes and twenty-three seconds later (confirmed by the logs and surveillance system)," Murph tells us, "alarm emails and cell phone vibrating sensor alerts were sent out" as the UPS batteries were depleted by the unexpected load.

[9]

Murph got his alert and dashed to the server room, arriving just in time to stop the highly motivated cleaner from diligently – conscientiously, even – plugging in the polisher (which had lost power when the UPS died) to the rack labelled "B".

It's a story in which not one thing but several had to go wrong – but it has a happy ending. You see, that "B" rack of UPSes kept everything going. In spite of the alarms and alerts and whooping sirens (OK, there weren't whooping sirens) the system stayed up and worked as it should.

Between a well-liked cleaner and an anonymous screw-up in HR, the system was given one heck of a stress test, and passed. Murph just hopes it doesn't happen again.

Sadly we're unsure if the server room floor ever got that much-needed polish.

Have you ever given a system a necessary if unintentional stress test? Tell us all about your exploits in [10]an email to Who, Me? and we'll tell the world.

Get our [11]Tech Resources



[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZJlhwGR7elhNMdxAJhPaAQAAAMM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0

[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZJlhwGR7elhNMdxAJhPaAQAAAMM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZJlhwGR7elhNMdxAJhPaAQAAAMM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZJlhwGR7elhNMdxAJhPaAQAAAMM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[5] https://www.theregister.com/2023/06/19/who_me/

[6] https://www.theregister.com/2023/06/12/who_me/

[7] https://www.theregister.com/2023/06/05/who_me/

[8] https://www.theregister.com/2023/05/29/who_me/

[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZJlhwGR7elhNMdxAJhPaAQAAAMM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[10] mailto:whome@theregister.com

[11] https://whitepapers.theregister.com/



Clean keyboards

trevorde

Worked at a company in the early 90s when PCs were being introduced in the workplace. The Managing Director's Personal Assistant got a bee in her bonnet about hygiene of the PC keyboards and got our regular cleaning company to 'sort it out'. The cleaning company got their best people onto it which were little old ladies who had never seen a computer in their lives. They dunked the keyboards into a bucket of warm, soapy water and proceeded to scrub them with a big brush, previously used for the floors. Amazingly, some of the keyboards even worked after being cleaned!

Re: Clean keyboards

phuzz

It's amazing how waterproof electronics can be. Don't put keyboards in a modern dishwasher though, the high temperature drying cycle can melt them.

Re: Clean keyboards

Little Mouse

As can the hot tap in the kitchenette that dispenses near-boiling water for cups of tea etc.

I saw an engineer attempt to rinse a laptop keyboard (detached) under one of these once. Unsurprisingly, the keyboard was barely recognisable afterwards. It melted like the Nazi's face in Raiders of the Lost Ark.

Re: Clean keyboards

imanidiot

Can't entirely fault the MD though. Keyboards ARE filthy, filthy things and under ideal circumstances shouldn't be shared.

Re: Clean keyboards

big_D

Doing support over TeamViewer is much nicer these days, than having to visit a user's PC, where the keycaps are so black that you can't read the letters underneath, and sticky as well...

One positive from Corona, at least we always have a bottle of hand disinfectant, for when we do actually have to make a personal visit...

Re: Clean keyboards

Evil Auditor

Get a small wireless keyboard for personal visits.

Re: Clean keyboards

Evil Auditor

Eww, still remember the filthy keyboard of a manager in one of my former lives. Each key had a dark, brownish boarder with a brighter circular area in the middle. And the darkness/thickness of the boarder indicated which keys were used most. The same filth patina was present on everything on his desk, being it pens, stapler, calculator, desk pad, mouse obviously. I didn't want to touch any of that, including the manager, with a pole. It if it was unavoidable, treatment of hands with alcohol was the minimum.

No gloves icon? At least a filthy key.

Re: Clean keyboards

big_D

A friend loved his Siemens keyboard from the late 80s. He treasured it for years, but he was a chain smoker... So it went into the dishwasher every 6 months for a clean. He had to finally replace it in the early 2000s.

Re: Clean keyboards

Anonymous Coward

I recall a story, doing the rounds >20 years ago, about a garden centre which had a similar way to clean out the manky keyboards. They had a spare and the owners wife would periodically swap one out for the (clean) spare and take it home and into the shower with her. After a good rinse it went into the airing cupboard for a few days before becoming the new spare.

That story may have been apocryphal, but I can vouch for the fact that a good rinse and a few days in an airing cupboard used to work (mainly after someone had spilled coffee or other drink over their keyboard - if they took a lot of sugar, or it was a soft drink like Coke, the rinsing needed to be particularly thorough); crisp crumbs were the main keyboard bugbear, though, as you usually had to take the case apart to clean those out. I had a few old keyboards with worn or otherwise unreliable keys that were loaned to the culprit whilst theirs was being cleaned - went a little way to teaching them to be a bit more careful...

Re: Clean keyboards

big_D

If they aren't plugged in at the time, and are given time to dry, they are pretty resilient. It is the drying time that is critical (as is not using abrasive or corrosive detergents).

Re: Clean keyboards

Little Mouse

Back in the '90s we employed a cleaning company who sent a similarly enthusiastic team of pensioners to do a one-off clean of all the desktop IT kit at our head office, during office hours .

They used those mini vacuum cleaners to clean the keyboards, but they might as well have just rubbed the keyboards on their polyester jumpers for all the good it did.

The resulting pile of fried keyboards was bad enough, but it was the relentless jaunty whistling that ensured they never got invited back.

Korev

It's a story in which not one thing but several had to go wrong – but it has a happy ending. You see, that "B" rack of UPSes kept everything going. In spite of the alarms and alerts and whooping sirens (OK, there weren't whooping sirens) the system stayed up and worked as it should.

Looks like the cleaner showed that the IT setup was floorless

Korev

This week's raconteur, who we'll Regomize as "Murph," brings us a tale tying together two of our recent themes: security tech that doesn't behave as it should, and power points being used in ways they were not meant to be.

I think today's raconteur should have been "called" Jim for obvious reasons...

Multiple things lead to the conclusion

Caver_Dave

In my very first full time job, the offices were repurposed chicken sheds (don't ask me how they ever got the smell out - it was done before they were sold to my boss.)

We were having a problem with the water drain in the tarmac between the U shape of single storey wooden buildings.

Being very fit, and having access to a pick axe and shovel, the owner (my direct boss) asked me to have a quick look at the pipes.

I did the usual due diligence with him and got the site plans out. The mains feed was along the North side of the building on the North side of the tarmac. I could see that the drain was at least 30cm to water level and so the drain was presumably at this depth (otherwise the trap would not work).

And so I swung the pick axe just once and saw sparks!

Very carefully I investigated, with the owner standing a few feet out of the way.

It turned out that:

1) the building on the South side of the U shape had a different mains feed (that joined the other feed before the meter).

2) that mains feed had been placed in the water drainage pipe's trench above the pipe.

3) the building on the South side had been built later and the original North side drawings only partially updated.

4) the delivery lorries we had in perhaps once a year, followed the same route as the agricultural vehicles that originally serviced the chicken sheds - no other vehicle was allowed down there. It turns out this was the only part of the tarmac with any form of hardcore underneath. The pick axe had gone into tarmac that was 2cm deep over hard packed soil over the trench.

So multiple things that shouldn't have happened did and resulted in a lot of cost for:

rewiring (to standard)

replacing the old clay drainage pipe that was partially collapsed (under the old tractor route), full of soil and chicken shed litter

a new layer of tarmac over the whole area to a reasonable depth

newly corrected plans

Re: Multiple things lead to the conclusion

big_D

I worked in a business center that was a converted slaughter house... They only partially managed to get rid of the smell, in some offices, the sealing on the floor must have cracked underneath the linoleum and the smell wafted into the offices...

All that glitters ...

jake

When I was at SAIL, we had problems with the VAX disk drives going walkabout during overnight, disk intensive runs. Sometimes they'd walk far enough to pull their own power and/or data cables. I had one block the only door in once, in the throws of committing suicide ... I had to climb over the hanging ceiling to get in and restore order.

I had lunch with a colleague at SLAC and the subject came up. He allowed as to how they had had a similar problem, but the fix was simple: First, ban floor wax from the glass room (that wasn't glass). Pull the floor tiles a few at a time and take them outside. Scuff the Formica with 120 grit on an orbital sander. Dust off the tiles with a tack-cloth. Reinstall. No more walkies.

The precedent of "telling the cleaning staff how to do their their job" (as it was claimed) made it easier when we finally kicked them (almost) completely out of the machine room about three months later ... but that's a story for another day.

Re: All that glitters ...

Herring`

This can happen when the heavy data ends up on one side of the disk.

Re: All that glitters ...

WhoMeGuvNah

Here's me thinking that the heavy data got balanced out over time because of the centrifugal force. OK, it's all on the outermost track but it should be evenly spread.

Re: All that glitters ...

Norman Nescio

Pull the floor tiles a few at a time

Ah yes. Sage advice.

At one of the places that employed me, they had a (quite serious) workplace accident.

A new cable needed to be laid in a machine room with a raised floor. So the person doing the work removed a line of floortiles two or three metres long along the front of a line of racks.

The racks tilted over and collapsed on the person, causing life-changing injuries. I saw the pictures of the aftermath, without the blood.

People were most assiduous in removing only two adjacent tiles at a time after that.

You are a bundle of energy, always on the go.