Keep your cables tidy. You never know when someone might need some wine
- Reference: 1660552209
- News link: https://www.theregister.co.uk/2022/08/15/who_me/
- Source link:
Our story comes from a reader Regomized as "Tom" and concerns the time his employer, a major parcel delivery company, was planning to open a new international hub in a far-flung continent at the turn of the century.
The buildout had gone well. The hub was ready to go. The IT systems were up, running and tested. Parcels could be tracked, and their details sent back to the mothership in Western Europe where eager customers could log in and check the progress of their precious packages. Everything was going swimmingly. Possibly a bit too swimmingly, as things turned out.
[2]
"Everything was tested and ready for the official opening in partnership with the country's local Post Office, and high-ranking government officials were present," recalled Tom.
[3]
[4]
"Food and wine were also present," he added.
And then it all went wrong. To be fair, not a lot in IT goes particularly well when alcohol is stirred into the mix, but the "incident" that disrupted the best laid plans was not related to consumption. It was related to retrieval.
[5]
The country in question was a warm one, but the server closet was cool. It had to be air-conditioned to prevent all that hardware from overheating. What better place, therefore, for the caterers to keep the party consumables?
All went well, the celebrations kicked off, party food was scoffed and a caterer headed to the server closet for more supplies.
At this point we'd have to wonder about cable hygiene, since the caterer, while stumbling around the server closet in search of food and wine, managed to get his leg tangled up in wiring. He could have carefully extracted himself from the resulting knot of cables. However, he didn't.
[6]
"Rather than disentangle," said Tom, "[he] just decided to give his leg an almighty jerk... and ripped out the backplane of the server it was connected to."
[7]Yeah, we'll just take that first network handshake. What could possibly go wrong?
[8]IT advice fuelled by beer is the best IT advice of all, right?
[9]A time when cabling was not so much 'structured' than 'survival of the fittest'
[10]How to stop a content filter becoming a career-shortening network component
Servers don't respond well to such treatment and the grand opening celebration swiftly became more of a wake for shattered hopes and dreams as the destruction became clear.
Still, all it actually needed was some new parts. A new motherboard. That sort of thing. This was, after all, an international courier company. How hard could it be?
"Despite running an express parcel delivery service," said Tom, "the replacement motherboard got snarled up in local customs for more than 3 months before we could finally get the local hardware support company to re-install the parts, recommission the system, and finally declare the site fully operational."
A multiple-month delay thanks to a bit of careless footwork on the part of a caterer. Something to ponder the next time you watch a package sail over the garden fence after being kicked by an overworked operative. It could always be worse.
Ever accidentally pulled this only to see that happen? Or partied a little prematurely? Confess all with an email to [11]Who, Me? ®
Get our [12]Tech Resources
[1] https://www.theregister.com/Tag/Who,%20Me?/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YvoZRRwovgk2@iAY8@ASRQAAAJA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YvoZRRwovgk2@iAY8@ASRQAAAJA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YvoZRRwovgk2@iAY8@ASRQAAAJA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YvoZRRwovgk2@iAY8@ASRQAAAJA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YvoZRRwovgk2@iAY8@ASRQAAAJA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2022/08/08/who_me/
[8] https://www.theregister.com/2022/02/28/who_me/
[9] https://www.theregister.com/2022/01/03/who_me/
[10] https://www.theregister.com/2021/08/31/who_me/
[11] mailto:whome@theregister.com
[12] https://whitepapers.theregister.com/
Leg pull, surely?
I’ll see myself out …
I hope whoever agreed to allowing catering to use the server room for cooling received their pink slip before the day was over. Dear lord that's a horrendours decision.
Who could ever think letting a load of unknown, low paid, catering staff to run loose in your brand new server room was ever a good idea. They maybe saved $100 by not having to hire a refrigerated trailer to store there stuff, and look what it cost them!!!
My Flabber is well and truly gasted...
And please, management should be publicly flogged and fired on the spot because they allowed critical infrastructure accessible without proper locks and other restrictions.
Russia
"Who could ever think letting a load of unknown, low paid, catering staff to run loose in your brand new server room was ever a good idea."
Do you really think the oligarch was fired?
Re: Russia
Russia?
It was in Florida...
Blame shifting
A multiple-month delay thanks to a bit of careless footwork on the part of a caterer
Not a fault of caterer for being a clumsy caterer. The blame is on anyone responsible for the server room security.
Also when you run a server room yourself, you need to have some redundancy. I know servers are expensive, but that's just how it is. Even one spare server would get them out of trouble.
far flung kit issues...
VOIP Telco in early 2000's (startup so many ethical corners cut - oh the memories...)
wanted a local gateway in a particularly high volume destination of our phone cards (remember them?)
anyway... official telco licenses were impossible, we had support of a local general, who was willing to house equipment at his house and arrange the local E1 drops - for a rental fee and revenue share deal.
so we shipped out half a dozen preconfigured AS5300's they got escorted through customs to the generals house, and appeared on-net rapidly. Traffic flowed and the backhanders seemed to have paid off.
until the rainy season.
boxes went down constantly. came back up, but were particularly flaky.
we sent out an engineer to "get it back to the reliability we had at the start" on landing at this tropical hellhole and getting transported to the generals pad was pointed to the stack of boxes in a corner of the veranda. he took a picture (with a real camera) of the cables coming down the building acting as a perfect rain conduit directly to the boxes.
asked if they could get moved in to the house until the rain stopped was met with grumbling... made comments about revenue share needed the boxes up to work...and it happened. they called in a local labourer to literally knock a hole in the wall so the cables could get fed through (hole stuffed with newspaper afterwards to "seal") . once in the lounge the kit worked without issue
eye opening all round.
"snarled up in local customs for more than 3 months"
Local customs!
Maybe the caterer should have been told off to pay all the local expedited clearance fees.
fail - over?
> more than 3 months before we could finally get the local hardware support company to re-install the parts
So an annualised uptime of less than 75%