Ad agency boss owned two Ferraris but wouldn't buy a real server
- Reference: 1710487747
- News link: https://www.theregister.co.uk/2024/03/15/on_call/
- Source link:
This week, meet a reader we'll Regomize as "Aaron" who very recently worked for a managed services provider in Australia. One of his clients was an ad agency that he described as "insanely flashy and demanding."
"The owner has two Ferraris. They have that kind of money," Aaron told On Call.
[1]
The agency also had an all-Apple environment – other than a SharePoint server that was used to store, among other things, plenty of uncompressed video and audio.
[2]
[3]
SharePoint choked on those files, and users wanted faster access. So the agency built a file server to host that in-demand data.
Except it wasn't really a server. It was a Mac Mini with some external USB drives. Data from SharePoint was stored on those drives in directories set up as SMB shares.
[4]
This arrangement was obviously unstable, and the server would become unavailable if client devices updated macOS before the server. So every time Apple issued a new OS, Aaron rushed to update the sever before the mismatch meant the ad agency lost access to its files.
"This was so common that we ended up creating an entire ticket stream just to monitor for macOS update emails as soon as Apple sent them," Aaron told On Call. But even with that early warning system in place, downtime was regular.
The Ferrari-owning boss got wind of this mess and would berate Aaron and his crew whenever the server became unavailable, citing the enormous cost of outages.
[5]
Aaron countered by pointing out that a real file server would make the problem go away.
"The bosses at my MSP explicitly told everyone to remind the client of this because they were personally sick of the problem," Aaron told On Call.
But the agency boss wouldn't buy a server, so Aaron and his colleagues lived in a horrible cycle of trying to keep the server updated, falling behind, fixing it, being berated by their rich client, recommending he pay for a new server, and being told "No."
Until the day Aaron updated the Mac Mini, and the SharePoint directories on the external drives disappeared.
[6]Job interview descended into sweary shouting match, candidate got the gig anyway
[7]They call me 'Growler'. I don't like you. Let's discuss your pay cut
[8]Work for you? Again? After you lied about the job and stole my stuff? No thanks
[9]Dave's not here, man. But this mind-blowingly huge server just, like, arrived
"I had to go on site, and try as I might the damn thing wouldn't read," Aaron recalled. "We could see the hardware ID for the external drives. But they would not mount."
After an hour without finding a fix, Aaron called a colleague who also struggled.
The two sweated it out – literally, because this incident took place on a hot Australian summer day in an airless office without air conditioning – until they figured out the fresh macOS update wasn't loading USB drivers and therefore wouldn't talk to the external disks.
Reinstalling the drivers did not work … until Aaron and his sidekick reinstalled macOS and the external drives came alive.
After three hours of downtime.
A couple of months later, the ad agency agreed to buy a new file server. A real one. Aaron built the box with a glad heart.
But the agency never paid for it.
Sick of the cycle of rushed repairs and recriminations, Aaron quit a few months later.
"And that file server was still sitting next to my desk, unpaid for, on the day I left" he told On Call. "Lord have mercy on whoever has to deal with their bullshit now."
What's the cheapest thing you've seen a boss do that made needless work for IT? [10]Click here to send On Call your story and we may feature it on a future Friday. ®
Get our [11]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZfQqSBFYancHB1hCMqpFGQAAAJU&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZfQqSBFYancHB1hCMqpFGQAAAJU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZfQqSBFYancHB1hCMqpFGQAAAJU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZfQqSBFYancHB1hCMqpFGQAAAJU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZfQqSBFYancHB1hCMqpFGQAAAJU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2024/03/08/on_call/
[7] https://www.theregister.com/2024/03/01/on_call/
[8] https://www.theregister.com/2024/02/23/on_call/
[9] https://www.theregister.com/2024/02/16/on_call/
[10] mailto:oncall@theregister.com
[11] https://whitepapers.theregister.com/
Sadly all too common - bean counters baulk at the cost of enterprise grade disks for the server when they can by a consumer grade external drive for a fraction of the price.
To be fair, the markup on "Enterprise discs" is crazy
The issue here is putting the primary and backup copies on an external drive which is more likely to get damaged or stolen, and the same drive meaning no backup.
When it comes to consumer Vs enterprise drives if you are talking less than a dozen drives you probably won't notice any difference. Statistically over thousands of drives you may have fewer failures with enterprise drives and therefore less maintenance etc. but for a small deployment drive replacement is an irregular task.
You should always assume a drive will fail, and if using RAID that another drive will fail during the restore process. If buying consumer drives means you can buy an extra parity drive, I would say that is a good idea... Like off-site backups.
My small fileserver (Celeron + 4 disks in 2x RAID 1) went through a phase of appearing to eat disks ... even though I bought fileserver grade disks (eg Seagate Ironwolf). The problem turned out to be the Ubuntu kernel's use of SATA Native Command Queueing by default. There is a kernel option to turn that off, and it's all been sweetness and light ever since.
Aaron?
This wouldn't happen to be the famous Aaron A Aardvark would it? Who changed his name so that he would be the first entry in the phone book?
Bonus points to anyone who can remember the story where this took place! ..... Without simply Goggling it that is!
Re: Aaron?
He became the first cit to be killed when Cal started to purge the entire population. The day the law died? Proper early 2000AD, double digit prog no, I still have it in storage!
How many bonus points?? ;-P
Re: Aaron?
Classic Judge Dredd - The Judge Cal saga
Re: Aaron?
Not far off my real name. I can tell you it is not always fun to be first in line for stuff.
Been there, done that.
Client insisted they needed the latest greatest IT environment, gave me what looked like a pretty well thought out list of goals, requirements etc.
Budget?
Nah, can't you just do all that with what we already have?
What, you mean we need to *buy* software, we've already got Office and WIndows (one copy and licence for each)?
Walk away
Sometimes it's better to walk away from bad business. They can't have been paying for all the extra time and effort it was costing Aaron's employer to support them either. His boss should have served them notice to terminate the contract and ended the relationship.
"They have that kind of money"
And very little sense to go with it, apparently.
Normal, it's an ad agency. There's no intelligence there.
Maybe a typical Oz thing?
In 1998/9 I was asked by a customer what was needed to update the 6 PC LAN that ran our software. I suggested Pentiums with Windows NT4 Workstation running peer-to peer with one of the PCs running the "new" SQL Server 7. A few weeks later I was asked to install the new equipment that the owner had bought. It was one Pentium for himself, with the instruction to use his 2 year old desktop PC as the "server". I noticed that he had leased a brand new Holden Commodore 5.3 litre V8, and suspect that he had gone to the bank for a loan based on my specs and used the money for the car lease...
Invoice
Hopefully Aaron sent an invoice and if they didn't pay lodged the wind up petition.
The owner has two Ferraris. They have that kind of money
I've worked for one Ferrari owning boss, and judging by this tale they're all equally stupid. I once agreed to accompany him in it to his house, during work time, to get something working on his PC there. This hideous yellow car made an equally hideous racket, and could accelerate like a rocket. It turned out that his daily commute in it involved an 8 mile journey, mostly along an extremely busy and urban dual carriageway, with traffic lights about every third of a mile. So the drive involved being rapidly accelerated and decelerated over very short distances, with no time advantage gained at all because the world's fastest car would have taken exactly the same amount of time to do the journey as a £300 old banger.
Re: The owner has two Ferraris. They have that kind of money
Most people who buy a Ferrari don't to it to be faster. They buy it because they want to be seen as a Ferrari owner.
Re: The owner has two Ferraris. They have that kind of money
My very small and not at all powerful car can generally do those sorts of journeys faster, because it can fit into spaces that bigger cars can't. Use the right tool for the job, and for that job, the right tool is a city car.
Re: The owner has two Ferraris. They have that kind of money
Better by far to use the right tool for the job, than having to do jobs for a right tool.
We once took over a customer who had a server for their Exchange environment. Proper brand name server with hardware RAID and multiple disks. The problem was the storage provided by the disks on the RAID controller was far too small. The previous vendors "solution" was to install the OS and Exchange binaries on the RAID volume, but put the Exchange databases on an external USB drive bought from Harvey Norman.
To add insult to injury, the backups were saved to the same external drive.