News: 1620977169

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

Protip: If Joe Public reports that your kit is broken, maybe check that it is actually broken

(2021/05/14)


On Call Welcome back to [1]On Call where this week we peek behind the scenes and see what happens when public-facing kit is reported as borked.

Our reader, Regomised as "Sean", had a varied brief at a company based in Northern England. "My responsibilities," he told us, "included software development, CAD design, electronics, and 'whatever else needs doing'."

The "else" included being called out to smooth the ruffled feathers of uppity customers.

[2]

[3]

[4]

"We had a London-based customer to whom we had a few years previously sold some hardware, software, and support," he recalled. The problem was that all those involved had since left the company and so, as is often the case, there was precious little documentation on what had been actually been installed.

Not that such an omission stood in the way of a support contract. And before long, Sean received The Call (general fix-it person was another of his duties).

The devices themselves formed part of an interactive exhibit at a tourist attraction. One we can't name. 12 of the gizmos were supported by Sean's company and nine of those had been flagged as borked.

[5]

Sadly, Sean's tale predates our inexplicably long-running Bork series so it's unlikely that the machines have featured in our pages.

From his perspective, all he had in terms of diagnostic data was "they're not working". All attempts to clarify in what way the devices were not working were rebuffed. Oh, and the customer wanted them fixed by tomorrow.

[6]The swift in-person response is part of the service (and nothing to do with the thing I broke while trying to help you)

[7]Can't get that printer to work? It's not you. It's that sodding cablin.... oh beautiful job with that cabling, boss

[8]George Clooney of IT: Dribbling disaster and damp disk warnings scare the life out of innocent user

With nothing to go on, Sean ordered in nine complete sets of hardware and paid the eye-watering extra cost to get the kit delivered same day. The next morning, armed with a bulging site pack full of gear, he hopped on a train to London and set off on a voyage into the unknown.

He was at least a little familiar with the client's site and, upon arrival, headed directly to the location where the stricken devices had been installed.

They were all working perfectly. Apart from one, which just needed to be switched on.

[9]

The client turned up shortly after, wanting to know how long the job was going to take.

Sean had a choice. Should he play the "I have fixed everything, behold my genius" card, or tell the truth?

He told the truth, of course. Nothing had been broken.

Now it was time for the client to also make a confession: "It transpired that a member of the public had reported a catastrophic failure, and no one had thought to spend five minutes and double-check."

The cost was impressive. Because there was nothing wrong, the callout wasn't covered by the maintenance contract. Nor was all the hardware Sean had brought. "The bill ran to many thousands of pounds," he said.

"In the end, the client asked me to replace all the (working) equipment with the spares I'd brought with me so they could report that work had been undertaken (and presumably hide the fact they'd never checked things were broken by claiming that vandalism was suspected)."

"Most pointless job I've ever done," he sighed. If only Bork!Bork!Bork! had been around to record it for posterity.

It's always easier for a user to pick up the phone than actually check on a problem themselves. Hiding a five-figure invoice can, however, be slightly trickier. Tell us about your experiences on the front line with an email to [10]On Call . ®

Get our [11]Tech Resources



[1] https://www.theregister.com/Tag/on-call

[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YJ5KNzg2URIFyC2r--A4pwAAAIA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0

[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YJ5KNzg2URIFyC2r--A4pwAAAIA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YJ5KNzg2URIFyC2r--A4pwAAAIA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YJ5KNzg2URIFyC2r--A4pwAAAIA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[6] https://www.theregister.com/2021/05/07/on_call/

[7] https://www.theregister.com/2021/04/30/on_call/

[8] https://www.theregister.com/2021/04/23/on_call/

[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YJ5KNzg2URIFyC2r--A4pwAAAIA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[10] mailto:oncall@theregister.com

[11] https://whitepapers.theregister.com/

sanmigueelbeer

First question: Have you turned it off and on again?

Dave K

Most likely response:

"Never mind that, they're broken and I demand you come and fix them NOW".

Either that, or they fib and say "of course I have, I'm not stupid".

"Either that, or they fib and say "of course I have, I'm not stupid"."

CustomCruiser

Been there too many times to count...and yet if I was to plant an axe in their gormless faces after rocking up and power-cycling the kit in question I'd go to jail. How is that justice?

Re: "Either that, or they fib and say "of course I have, I'm not stupid"."

IHateWearingATie

Jail for an axe in the face, definitely.

A 'tickle' with an electric cattle prod..... well that's a bit more understandable

Martin-R

To which I can add "have you turned it off and left it for four weeks?" Power shower stopped working after a couple of power cuts in quick succession. "The Internet" says yeah, can happen, turn it off for an hour, it'll reset. Nope. Left if off 24h. Nope. Decided to survive on feeble shower for a few weeks as bathroom needs completely refitting anyway. Chap comes out to do survey and estimates, turns it on, works... aargh!

Default response

DwarfPants

In the politest possible way assume the caller is

- Misguided

- Does not know what they are doing

- Cannot be bothered or too overloaded to do their job / check

- Lying

Until you have proof to the contrary

Re: Default response

Anonymous Coward

Do you happen to be a dev at a certain Scotland-based video game company? Back when I was in QA that was how a lot of our bug reports seemed to be routinely treated, even WITH proof to the contrary.

Not that I blamed them towards the end of my time there, mind. The introduction of minimum weekly bug report quotas resulted in some absolutely dire stuff getting sent in just to fulfil the management's desire for 'good metrics'.

GlenP

Abbreviated and paraphrased version of a support call over 30 years ago (I've probably recounted this before on here):

User: "It's completely dead!"

Me: "Have you checked the power lead?"

User: "Yes!"

Me: "Can you unplug the power lead and plug it back in, making sure it's properly in?"

Pause

User: "Yes, I've done that - it's still dead!"

Me: "OK, I'll call an engineer but if they don't find a fault it will be chargeable."

User: "Whatever!"

Next Day

Engineer: "I've plugged the power lead in properly."

IBM Service (to User's boss): "Here's the call out invoice!"

User's Boss: "*&())*&%$("

Chloe Cresswell

Had the son of MD at a client's site call us out due to not being able to email colleague sat opposite him.

Walked up, looked at screen.. "You've spelt the company name wrong", walked out again. Invoiced for time. Son sends email to all on site about how we are ripping them off.

Civil service paying for excuses

Anonymous Coward

I’d seen several times in the civil service where someone had screwed up and instead of owning up and sorting the mess out they just carried on and paid for stuff that wasn’t needed.

We had a consultant in who advised on getting 3G pcmcia cards for laptops, at the start of the trial they ordered ~1000 of these cards from orange that arrived in our store rooms.

At the time remote access was dial up, 12 months later an ssl vpn was trialled with connectivity across the internet which was great, months later it was rolled out to all. The 3G cards where then looked into and it was decided that because the corporation used Vodafone they needed new sims and then they realised the frequencies of the cards where incompatible so they all got junked, only a handful ever being used.

3G modems, plus monthly sims sat in a room for ~3 years all paid for but never used.

Re: Civil service paying for excuses

DailyLlama

I had a similar thing at a previous job. My tight-fisted manager ordered us all brand new shiny laptops, but decided to save £50 per laptop by not purchasing the built in WiFi cards, as none of our sites had wireless capability.

Less than 6 months later, she's signed an order for £100 per laptop to buy Cisco PCMCIA WiFi cards for us all, as all our sites were getting WiFi installed.

Re: Civil service paying for excuses

Anonymous Custard

Not civil service, but we experienced something similar.

Our head IT drone was spec'ing our new corporate laptops, and decided (on his own, without consulting anyone) that Bluetooth wasn't something anyone ever used, and so *paid extra* to have the standard BT hardware in the laptops removed.

Of course, said laptops arrived to we minions, who regularly used BT for phone sync'ing, connection to peripherals and various other misdemeanours. Cue many support calls and requests to IT, and a large batch of BT dongles being ordered and distributed to fill up one of the few USB ports that the laptops had.

At least the lesson was learned, and subsequent replacements have all had full specs and we even get a say in what "options" we get or may not want/need (e.g. my laptop has no camera, as various customer sites ban laptops with them from cleanrooms and other sensitive areas).

Re: Civil service paying for excuses

Emir Al Weeq

I've been on the other side of that:-

To the planning team: will you be using X in the next 5 years? If not I can save ££££ on my project.

Planning team: no we won't.

To the design team: will you be using X in the next 5 years? If not I can save ££££ on my project.

Design team: no we won't.

To the architect team: will you be using X in the next 5 years? If not I can save ££££ on my project.

Architct team: no we won't.

Six months later... We've just started a project to install X.

too many times

chivo243

I've spotted the issue the minute I walk into the room. Latest one was just a power cable not seated properly. The user had received a second monitor and loved to pivot it to show examples... Can I get a D'oh?!

Re: too many times

Anonymous Custard

D'oh!

PC Upgrade

trevorde

Told to me as a true story:

User: What do I have to do to get a new PC?

IT: We don't have the budget. It has to be broken before we can replace it.

User: [pours cup of sugary coffee over PC]

Re: PC Upgrade

Admiral Grace Hopper

I used to work with a very effective project manager who simply parked her BMW on top of her laptop when she wanted an upgrade.

Local council wasting money by not spending it

Boothy

Many years ago (early 90s), I worked for a local engineering firm, long since gone the way of the dodo.

We did a lot of work for local councils all across the UK, door entry systems, security cameras, TV distribution etc etc.

Typically any control gear would be put in a plant room somewhere, and wherever possible would have power hard wired into the nearest fuse box, on it's own fuse of course.

We had some door entry systems going in for this council, small blocks of flats, four per block, two down, two up.

This particular council wouldn't allow us to wire into the fuse box (we had our own qualified electricians), and wouldn't use their own sparkies to do this either (some places insisted on using their own people). Instead insisting that we simply mount the gear with a standard 240v plug, and use the single available socket, next to the fuse box, which was basically there for trades people to use when working in the blocks. We warned them what was going to happen, and suggested they at least buy the battery backup option. But nope. No battery, and use the socket!

I'm sure everyone knows where this is going now!

Also bear in mind, this is the early 90s, so no plethora of battery powered tools at this point (and those early ones didn't last long on a single charge), so just about every trades person had mains powered gear, drills etc, and so needed power at some point (even if it's just for their kettle!).

Sure enough the council had some work done some time later, who unplugged our gear to use the socket (not their fault, it was the only socket in the blocks), and the council started to get calls about the failing door entry systems, who in turn called us out.

We'd turn up, typically the next day, and everything was working fine. The trades person had plugged the gear in at the end of the day before, and wasn't around when we turned up. So we weren't even aware that anything else was going on at this point.

Several false calls later, and we managed to turn up on the same day, trades person still there, our equipment unplugged, theirs's plugged in.

Reported back to the council what was going on. They also got a bill, as per the article, false callouts were not covered. No more false calls after this point.

I turned up to the same site some months later for a real issue, and noticed all the blocks had had a new fuse fitted in the fuse box, and our gear was now all hard wired in!

Icon --> It's Friday, cheers all.

His life was formal; his actions seemed ruled with a ruler.