A server room with two air-con systems, with the first failing over to the second on failure. But we almost always had one of the air-con out of action awaiting repair.

Took me a while to figure it out, but the thermostat was set for 16 degrees and the units installed weren't capable of actually pulling the server room down to that temperature. So the main unit would run continuously ...... until it failed. Then while the main unit was waiting for repair the second would run continuously.

I finally just started sneaking the thermostat up by half a degree every few days. After a couple of weeks the server room was at a nice cool 18 degrees, and you could hear that the units were only running about 20% of the time.

From that point on we didn't have another air-con failure, and the only people disappointed with this were the air-con repair company who lost a nice little earner!

Client of mine rents office space from another company, so our servers are in a mini rack in their computer room. there's 3 part trunking along the wall with the 2 air cons on it.

When I started, said trunking had plastic sheeting taped to the wall above it, under the right aircon. It was like that for 3 years.

Some managers really are complete arseholes, completely ignore how someone actually resolved an issue and restored service and instead focus on how some 'process' wasn't followed or what that person did wrong. This is an easy 'lessons learned' example of documenting and making the location of the keys or the name of the primary keyholder available to the person who is looking after the system. Ok, he broke the locks but the managers didn't even know how the servers were configured and tried to say Manny was in the wrong when he did everything he could to get them up and running again.

That said, if I had been in Manny's place and didn't have the details of where the keys were, I'd have escalated to someone who did know and then when everything is back running as normal, have the relevant documentation updated with the correct details.

Unless that was already part of Manny's job.

And he hadn't done it, yet.

Icon because I had an arsehole manager once and revenge is sweet.

If the use case is to secure a server rack in a secure room that's in a secure room, and the protection was a lock that could be foiled with just a screwdriver, then I'd say the manager is at fault for signing off on such shoddy locks that aren't fit for purpose.

They make locks which are direct replacements that are difficult for neophyte pickers to bypass (proper fitting disc tumblers, for example (I think the LPL calls them disc detainers)). They usually cost a couple dollars more than the stock ones, though, and what manager will pay for that?

Doesn't know the specifics of the hardware, doesn't care why the server was down or the aircon had failed to restart properly, but knows just enough about your department to blame you for everything.

Typical.

When I was a field engineer I visited a large Telco DC to be greeted with various people lifting floor tiles looking for some specific cabling - I immediately advised the customer contact who was escorting me that there seemed to be too many tiles up and the floor might be in danger of collapse. I made quite a fuss as I thought their people might be in some real danger of injury and they did yank everyone out of the room. Whilst the senior DC manager was sought for his opinion part of the floor collapsed tipping a VAX Mini onto its front (Though impressively it kept running).

In the aftermath I was blamed as the customer said if their people had been in the room they would have been able to prevent the floor collapse - They asked my company not to sent me to site in the future as I was banned ....

PS: The ban only last about 2 months until the Senior DC owner was moved onwards to a new position, this was not the first major incident I found out that had happened under his stewardship.

Depends on the style of floor. I've driven an F250 onto a raised floor with over half the tiles lifted, loaded the truck with equipment, and driven it out again. Several times (we were moving the DC a couple blocks to the East).

I would not recommend doing this unless you know for a fact that it is safe, though ...

It depends entirely on what tiles are removed. In some patterns it's entirely fine as the floor can retain it's rigidity, in other patterns (especially long uninterrupted lines) the stands lose support and buckle, causing the floor to start collapsing (and cascade to any still connected parts of the floor)

Again, it depends on the style of floor.

Consider, for example, that I'm in earthquake country.

1) Don't pry the silly little things open. Rather, learn to pick them. Equipment cabinet locks, desk drawer locks, file cabinet locks and the like are among the easiest locks to learn on. Making picks is easy, instructions can be found online. All you need is a few tines from a street sweeper and a Dremel. Or, if you live in an enlightened country you can buy kits of assorted picks and turning tools for very little money. Try Covert Instruments for a good selection of quality tools. Lock picking is a useful skill, and not all that difficult to learn. Recommended.

2) Those equipment cabinet locks are usually fairly generic, and OEM parts were probably available if you knew where to look. In fact, I'll bet a wooden nickle that they are still available. Call your nearest locksmith, or just look em up online. Both Grainger and McMaster-Carr sell a wide variety at reasonable prices. Also recommended.

Try Covert Instruments for a good selection of quality tools.

'This is the Lock Picking Lawyer, and today I'm going to show you how to open a data cabinet with some simple tools...'

Is Jake the LPL?

V.

Is it bad that I first worked out how to rake a lock in Scouts with a paperclip when we couldn't open the door in the scout hut?

I always found server racks / cabs fairly easy to open as the lock was always just clipped into a circular hole in the metal or glass and could easily be rotated with a screwdriver in the keyhole.

Perhaps they wernt the highest quality.

The slightly higher end ones nowadays have D shaped hole with a flat on the lock body. Attempting to rotate those out of their retaining nut works just fine, but by shattering the glass instead of freeing the lock. Takes a bit more explaining...

Did you get a badge for managing to do it? Scouts seem to have badges for just about everything.

No badge for that here in the US. Several years ago, one of my nephews tried to convince the BSA to let him write a new locksmithing merit badge. They flat turned him down ,,, and threatened to throw him out if he let on to his buddies that he knew how. When I pointed out that their Automotive Maintenance merit badge would give a scout the ability to hotwire a car, I thought they'd have a collective coronary ... and I was asked to leave. The automotive badge requirements have since been dumbed down.

The world is filled with namby-pamby hand-wringers. Sad, that.

I hope you got your badge for that.

No, I'm not the LPL.

Agree, essential skill. At the apprentice training centre one of the firtst things you made was your own toolbox. You then got to padlock it to keep your tools safe. Next skill was picking locks so you could nick tools from other apprentices, or, in the case of one imaginative oik, open the box, fill it with hydraulic oil, refit lock leaving no trace.

I don't think my apprentice toolbox would have held oil for long. I still have it though some 50 years later.

There is an old adage about locks only keeping honest people out, and it's so true.

None of these trivial locks which secure server racks, little steel cash boxes and similar take more than about five seconds to open with a rake pick and that doesn't even require more than a modicum of skill to use.

Many moons ago, the shared laundry room in my apartment block was locked when my booked time came around. The person who had locked it with their booking tag wasn't even home so that I could get them to remove their clothes from the machine so that I could start my wash. What to do? Then I noticed that the door was hardly a great fit in its frame - the simple application of a large screwdriver between the door and its frame was sufficient to open it. After dumping the missing resident's wet clothes in a trolley and starting my wash, I was on my way out when another resident saw me leaving the room and commented "Oh, have they finally come and collected their washing?". I replied "No, it's all in that trolley over there". "But how did you get in?" she said.

The look on her face when I simply smiled and showed her the big screwdriver I had in my back pocket was classic!

"Who are you? And how did you get in?"

"I'm a locksmith... and I'm a locksmith"

Actually, locks don't keep honest people out, either.

However, honest people don't do anything illegal with their knowledge and ability.

Locks keep honest people honest and delay dishonest people to varying degrees is the most accurate descriptor.

My ex-boss and myself keep a few choice tools around that look somewhat like some stiff wire and some thin bits of metal.

We're also both subscribed to the lock picking lawyers channel...

We also really keep seriously eyeing ordering a pair of covert companions.

Anon because I'm not putting my name to knowledge of physical security bypasses...

You just know you are in trouble. Hiring people just because they have management experience does not work in the Tech world because you have to dumb down your explanation of why you had to do something in a particular way.

95% I agree with you (and 100% if the manglement are also self-aggrandizing and/or a moron).

But you do get the occasional gem of a manager who knows what they don't know, are comfortable that you know more than them on the subject and will support and facilitate you doing your work.

I had a boss a while back who fitted that category, and was happy for me to use him as a sounding board to think things through by explaining it to him, which often helped me sort things in my mind too (there's no better way to be sure you know and understand something fully than to have to explain it to someone else).

Plus he may not have had deep technical skills, but he did have decent common sense and a logical way of thinking, meaning he'd often ask not-so-technical questions that again made you stop and think or reconsider how you'd approached the problem. Often got over the old "not seeing the wood for the trees" scenarios, or when you'd got too focussed on a given possible solution and had missed something else or a test that should be done.

If you luck onto one of them, hang onto them for dear life...

Where's the ON CALL: tag in the article, so we can find the story should we be otherwise occupied on the day it is released?

Use the "Send corrections" link (https://www.theregister.com/Author/Email/corrections?message=re:%20https%3A%2F%2Fwww.theregister.com/2024/04/05/on_call/) at the top of the Comments page to report this. That usually gets reacted to quite fast. Don't bother now anymore, I' ve already done it. :-)

Flipping aircon- 7 hour round trip as the 'has anything changed' question was met with a 'no' from a customer. Open doors and fans were fairly conclusive even without stepping foot into the sauna.

"On Call" missing from article headline / title ! nearly missed it!

Use the "Send corrections" link (https://www.theregister.com/Author/Email/corrections?message=re:%20https%3A%2F%2Fwww.theregister.com/2024/04/05/on_call/) at the top of the Comments page to report this. That usually gets reacted to quite fast. Don't bother now anymore, I' ve already done it. :-)

I'm in a similarly locked down facility, and I would have slid to a halt with the locked servers.

A) You have to be by-the-book around here security-wise. I don't fancy explaining things to a tribunal.

B) It would inconvenience the smarmy idiot with the keys, but he would not be able to bitch.

C) It would inconvenience upper management, and if they bitched, you'd be able to kick it to upper upper management.

D) All of the above has resulted in me getting the keys.

Of course. Always follow policy and procedure.

But if you know how, let the Boss know you can pick the lock(s) and minimize damage (costs). Make certain you get a wet signature authorizing it before making the attempt, though..