Compression? What's that? And why is the network congested and the PCs frozen?
- Reference: 1728890893
- News link: https://www.theregister.co.uk/2024/10/14/who_me/
- Source link:
This week's legit hero is a guy we'll Regomize as "Vivian" who worked as a personal assistant to the general manager of a regional office of a hardware manufacturer.
Viv shared his office with around 300 employees. Most of the PCs in the place were '486s running either Windows for Workgroups or – for the lucky few – Windows NT. They were all networked at the breakneck speed of 10 megabits per second, which was actually kind of fast for the time.
[1]
Crucially for the story, internet access for the office relied on a Unix box connected to the mail server in the main office, down the coast. All of the company mail for all offices went through the server in the main office – a classic bottleneck if ever there was one.
[2]
[3]
One of the tasks that landed on Viv’s desk was helping to arrange an office move, and an employee poll to let workers have their say about the company’s next home
Viv decided staff deserved to see the options management had picked, so in the spirit of "we're all one big family here" he took photos of three prospective new locations with a mid-'90s digital camera, attached them to an email to all staff, and hit "Send."
[4]After we fix that, how about we also accidentally break something important?
[5]Personalized pop-up was funny for about a second, until it felt like stalking
[6]Did you hear the one about the help desk chap who abused privileges to prank his mate?
[7]I don't know what pressing Delete will do, but it seems safe enough!
When recipients received the email they tried to open the pictures – and that's when all hell broke loose.
Or, more accurately, when all hell came to a grinding standstill because all those emails, and the load they imposed on the network, saw every computer freeze.
[8]
When the IT guy was called, he discovered that the problem was that those photos were .BMP format – a file format that to this day eschews compression. JPEGs were a thing at the time, but clearly a fairly new thing, and Viv wanted each and every employee to savour the images in all their magnificent mid-'90s uncompressed pixely glory.
Each image was over three megabytes, so the email weighed in at a hefty 10MB. Even these days you'd get a stern talking-to for sending that to all staff, so you can imagine how Viv's superiors felt.
The IT guy did sort the hung servers – both of them, as the main office mail server didn't like the traffic either – and re-sent the images as far more petite JPEGs.
[9]
Tragically, our correspondent could not recall whether one of the offices whose picture crashed the company ended up chosen as the new location. We suspect they'd have been at a disadvantage in the vote.
Ever found your enthusiasm for a project exceeding the capabilities of the tech – or your knowledge of how it worked? [10]Click here to send an email to Who, Me? and tell us the tale. We might share it on some future Monday to remind fellow readers that we all mess up sometimes – so it's OK. ®
Get our [11]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZwzrxYV9VxBt4bCF0GptYAAAAIc&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZwzrxYV9VxBt4bCF0GptYAAAAIc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZwzrxYV9VxBt4bCF0GptYAAAAIc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://www.theregister.com/2024/10/07/who_me/
[5] https://www.theregister.com/2024/09/30/who_me/
[6] https://www.theregister.com/2024/09/23/who_we/
[7] https://www.theregister.com/2024/09/16/who_me/
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZwzrxYV9VxBt4bCF0GptYAAAAIc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZwzrxYV9VxBt4bCF0GptYAAAAIc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[10] mailto:whome@theregister.com
[11] https://whitepapers.theregister.com/
Did the colours in the photos POP3 out?
Mental image
Did anyone else have a mental image of Vivian being similar to the Vivian from "The Young Ones", complete with metal studs in his forehead? Granted, not the type to be personal assistant to anyone, especially as nobody would probably dare give him a stern talking to about anything.
I'd better be going. The black leather jacket today
Re: Mental image
With 'The Young Ones' in mind, I was thinking more along the lines of the IT guy called upon to fix the issue referring to him as Rick with a silent P...
Re: Mental image
To me, Vivian is forever associated with The Bonzo Dog Doo-Dah Band: Vivian Stanshall...... After a quick Intro/Outro, the Canyons of your Mind onto the Urban Spaceman..... Sorry, I'm off to Jollity Farm.
You've made my day ---->
Re: Mental image
By a loose coincidence Ade Edmonson who played Vivyan in TYO has also been in The Doodahs. He tells the story on Desert Island Discs (repeated recently, BBC Sounds…) and probably in his book Berserker which I have but am yet to read.
Re: Mental image
Who could forget Rhinocratic Oaths - "Excuse me sir, I have reason to believe you can.. turn me on".
I had a client that decided to email some data over just as they headed home (to populate a database), all 3 Gb of it.
my own mail server sat at the end of an adsl connection, a solution what had never been an issue before with it having about 4mb of bandwidth.
Their server started delivering the mail and mine was accepting it, slow but it should get there after about an hour of using all available bandwidth.
After 20 minutes their server started resending the email (retry as the email hadn't been sent)
1st copy will now complete in 80 minutes
2nd copy will take 100 minutes
20 mins later their server tries again etc etc
By the next morning my mail server was on it's knees with no disk space remaining in the partition used for mailbox storage (every attempt at sending had allocated 3gb of disk space) and at this stage there were 42 incoming mail connections from their server with none showing any sign of completing this year.
Their mail server wasn't fairing much better, each retry had generated an email back to them warning them that the email had not been delivered and was being resent (with the original email as an attachment), this had quite quickly taken them over their mailbox size limit which then meant that their server was sending itself failure notices about that (still with the attachment) which was then generating even more failure notices.
The aftermath took the rest of the day to sort out at both ends (mine was fairly simple as I just took smtp offline and wiped the queue), theirs was more complicated as they were struggling to get a connection in to manage the server.
Final outcome was that they imposed a 200mb attachment size limit and changed their retry limit from infinite to 4 times and increased the retry delay to 1 hour - 3 hours - 6 hours - 24 hours
Best of it was that the data they were sending over was the data I had already pulled a copy of from their production database.
Those are the stories that demonstrate how we got to where we are now. Better management of email is, just like laws about security, due to the abysmal failures of existing procedures.
The eggheads who thought of email management (back in the day where a 10MB hard disk cost a huge chunk of money) didn't think about the real world, they just thought that a message should never be lost - and that's how the Real World TM lost plenty of messages.
Nowadays, we have servers with terabytes of disk space, and we still limit message size to a very reasonable 10MB. If you have more than that to send, you can arrange for a shared cloudy thing and not bother the email server with it.
But that takes experience, and experience always means experiencing failure and finding out why.
Way Back...
A friend's daughter sent him a photo from her work, all 30MB of it, and he only had a dial up modem! Inevitably this blocked all his incoming email* so he called his unofficial support, i.e. me (payment used to be made in bottles of decent Scotch :) ).
Fortunately I immediately spotted what was happening and was able to delete the message from the mailbox as I dread to think how long it would have taken to download.
*He was a local councillor and sat on several committees so this was a significant issue.
BTDTNT
Oh, look: sendmail is not running. "startsrc -s sendmail" (yes, that OS. VIOS, actually).
3
2
1
Guess what, there is a limit how much mail a sophos mail gateway can process. I really should have checked how much there was in the queue beforehand. Took the team a few hours to clear the queue and cost me a few beer.
First, I wanted to ask Vivian whether thereafter he moved on to our coloured pencil department. But since he's aware of the mishap he caused, that can't be him.
Near identical
Around about the same time as Vivian's escapade (judging from the use of 486es) I had a near identical experience which I think I've recounted here before. In this case it was an ad-hoc dial-up MS Mail system with low-spec "mail servers" at each site (I think there were six or seven sites at the time) and local networking on 10Mbps thin Ethernet with a four-port isolating hub in the middle. If memory serves correctly, our mail server was a '286 with 1MB RAM and a 40MB HDD running DOS6.
One of the sites had a new logo which a manager scanned as a glorious 24bit black and white TIFF (or might have been a BMP) and emailed to everyone, everywhere. Fortunately the system only sent one copy over the dial-up link, but distributing 10MB each to a mixed bag of 4MB 486SX, 486DX and even one 1MB 386SX took two or three hours out of everyone's working morning. Manager escaped with nothing more than a terse email from higher-ups.
Later that year a sales wag sent an email containing a topical joke. Nothing particularly offensive, only a couple of kB, delivered in a couple of minutes, but because it was again to "everyone everywhere" he got a final written warning.
M.
Re: Near identical
He was in sales, so expendable by definition.
We had a marketing guy try to email the contents of a CD to a client.
It took most of the friday and sat to crawl out over their ISDN line.
It took most of sunday and monday for the rejection bounce to crawl back in...
similar
I told a customer I could read vcd/evidence files. The customer sends me a 750Mb vcd file
It killed the office email system. The office manager got the email queue deleted. I only had the email addresses in the queue.
I had to email each customer on the email list apologise and ask them to resend the email.
Most embarrassing.
Re: similar
I told a customer I could read evcd/vcd files. Sorry about that.
A simple yes/no would have done
In the early days of PDF documents, someone created a questionnaire, with a few boxes to tick, and had lots of glossy photographs, so the PDF was large. This was emailed out. It filled my inbox space (but I ran at 90% full anyway) We duly ticked the boxes and sent the document back. The originator had not planned on getting 100 large documents back in email, and their mail box filled up. They panicked and purged lots of these emails.
As a result they had to resend the original document by sending us a link, and in the email asked us to answer the following questions.... The originator had to eat humble pie, especially when people asked why we needed all the photos, wasting their time creating the PDF, and wasting our time having to read it.
PPT Is the new BMP
Funnily enough, 30 years later emails over 10Mb still cause slowdowns.
Moore's law and all that
Did they send out an IMAP of the locations as well as just photos?