How do you solve the problem that is Twitter?
- Reference: 1670319015
- News link: https://www.theregister.co.uk/2022/12/06/twitter_opinon_column/
- Source link:
Twitter is in trouble. I mean, who [1]blunders his way into a fight with Apple only to later claim it was all a misunderstanding ? But, as idiotic as that is combined with [2]alienating advertisers, Elon Musk's wreaking havoc with Twitter's technical staff may end up causing more damage in the long run.
True, Twitter is still up and running, but some people have already seen smaller failures. For example, [3]Twitter's SMS-based Two-Factor Authentication (2FA) has already failed for some users.
[4]
As a reminder, the Titanic didn't sink immediately after hitting the iceberg. It took its own sweet time. And, many passengers still thought all would be well until their feet were awash in the frigid North Atlantic waters.
[5]
[6]
Don't believe me? Perhaps you'll believe Twitter site reliability engineer (SRE) Ben Krueger, who told [7]MIT Technology Review , "The larger catastrophic failures are a little more titillating, but the [8]biggest risk is the smaller things starting to degrade. " Exactly.
Another SRE, [9]MosquitoCapital, says that something as mundane as a [10]random hard drive filling up can "cause cascading failures across systems, even well-engineered fault-tolerant ones with active maintenance. Where's the box? What's filling it up? Who will figure that out?"
[11]
Good questions, and with Musk firing half the staff and another [12]thousand plus Twitter employees quitting , it's possible not many at the Big Tweet knows the answer.
And, although you may not know it, that example's more apt than you might think. It's estimated that half of Twitter's infrastructure is devoted to data storage and management. If a leak pops up in Twitter's data lake, there could be a flood of bad things downstream for users quickly.
Then, there's networking. For now, Twitter still [13]chiefly relies on its own datacenters rather than a hyper-cloud. So what happens if ye olde backhoe rips out a key length of fiber? I don't know, and we can only hope someone left at Twitter does.
[14]
Or, say, a Distributed Denial of Service (DDoS) attack comes along and kicks Twitter off the net? After all, [15]it has happened to Twitter before . It can happen again. Where are the net admins? Many of them have new, better jobs without a boss who requires them to be [16]hardcore .
Let's look at another example from MosquitoCapital. What if there's a really, really bad code push, instead of the [17]lame failure of paid Blue verification marks ? This time around, instead of just a flood of fake users, the new code locks up Twitter's Identity and Access Management (IAM) service for some users. Suddenly, some people, say old blue verified users, can't log in to Twitter. Now what?
You roll back the change, of course. That even a short-handed tech staff can probably pull off. But, what, oh my friends, what happens if they can't pull the change out of the code without freezing IAM up? Or, worse still, the site just goes down. Period. End of statement.
It would require staffers with deep system knowledge, and we'd better hope at least one or two of those are still around.
As Peter Clowes, a former Twitter senior project manager, explained, he quit because he saw no upside to being " [18]the last one standing in multiple large slack rooms and JIRA boards." Clowes continued, "If I stayed, [19]I would have been on-call constantly with little support for an indeterminate amount of time on several additional complex systems I had no experience in."
Musk still has the programmers who stayed until 1:30 in the morning but many of Twitter's top developers have left.
[20]Twitter set for more layoffs as Musk mulls next move
[21]Elon Musk issues ultimatum to Twitter staff: Go hardcore or go home
[22]Investor tells Google: Cut costs now and stop paying staff so much
[23]Email domain for NPM lib with 6m downloads a week grabbed by expert to make a point
As Will Norris, former Twitter open-source lead, told The Register , "Pretty much all of the key people that were working on open source at Twitter have left. All of the engineers that I worked with on open source are gone." Oh, and in case, you didn't know, Twitter relies on open-source software for pretty much everything.
Many people still believe Twitter will be fine. I can't see it. Between Musk's management missteps, [24]Twitter's ad cash flow receding quickly , and simply not having enough in-house technical expertise to keep things running smoothly, sooner or later, Twitter will stumble badly. My only question is, "How bad will its fall be?" Stick around. We'll soon see. ®
Get our [25]Tech Resources
[1] https://www.theregister.com/2022/12/01/musk_app_store_dispute_over/
[2] https://www.theregister.com/2022/11/02/advertisers_twitter_concerns/
[3] https://www.zdnet.com/article/stop-using-twitter-to-log-in-to-other-websites/
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Y48g1OMsP90J@qHz4xMKwAAAAJY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y48g1OMsP90J@qHz4xMKwAAAAJY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y48g1OMsP90J@qHz4xMKwAAAAJY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://www.technologyreview.com/
[8] https://www.technologyreview.com/2022/11/08/1062886/heres-how-a-twitter-engineer-says-it-will-break-in-the-coming-weeks/
[9] https://twitter.com/MosquitoCapital
[10] https://twitter.com/MosquitoCapital/status/1593541187675516928
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y48g1OMsP90J@qHz4xMKwAAAAJY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[12] https://www.reuters.com/technology/after-elon-musks-ultimatum-twitter-employees-start-exiting-2022-11-18/
[13] https://blog.twitter.com/engineering/en_us/topics/infrastructure/2017/the-infrastructure-behind-twitter-scale
[14] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y48g1OMsP90J@qHz4xMKwAAAAJY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[15] https://spectrum.ieee.org/what-is-a-distributed-denialofservice-attack-and-how-did-it-break-twitter
[16] https://www.theregister.com/2022/11/16/musk_twitter_ultimatum/
[17] https://arstechnica.com/tech-policy/2022/11/twitter-quietly-drops-8-paid-verification-tricking-people-not-ok-musk-says/
[18] https://twitter.com/peterclowes/status/1593458247972839426
[19] https://twitter.com/peterclowes/status/1593458253014265856
[20] https://www.theregister.com/2022/11/21/more_twitter_layoffs/
[21] https://www.theregister.com/2022/11/16/musk_twitter_ultimatum/
[22] https://www.theregister.com/2022/11/16/tci_fund_google_cut_costs_waymo_compensation/
[23] https://www.theregister.com/2022/05/10/security_npm_email/
[24] https://www.reuters.com/markets/deals/twitters-massive-revenue-drop-adds-heavy-debt-burden-2022-11-07/
[25] https://whitepapers.theregister.com/
Given the recent drops in Tesla's share price, is 6 a number of months or hours?
"How bad will its fall be?"
I'm hoping for a Looney Tunes kind of bad fall
Re: "How bad will its fall be?"
Don’t forget the anvil drop immediately after…
Re: "How bad will its fall be?"
And a group of birds flying around in a circle over the head twitchering?
Musk's management missteps
Understatement of the year.
Modern slavery
If a journalist was interested...
It would be interesting to analyse how many engineers who've stayed at Twitter through the recent chaos are on H1B Visas. Though it's not explicitly stated in the various reports, it's implied it's a high proportion of the engineering work force.
That raises the question of whether Musk's "hardcore" ultimatum - Work all hours, twice as hard or be fired - constitutes a form of modern slavery for workers that essentially rely on Twitter's sponsorship to remain in the country.
(Posted anonymously because... can of worms)
Re: Modern slavery
The Journalists would like to investigate this and maybe talk to someone from HR (probably on the quiet), but since Musk sacked everyone they cant seem to find any HR staff to talk to...
Another cunning plan from old Musky...
Problem isn't technical
Good article but I suspect what will do Twitter in won't be technical, it will be financial. He's found an unprofitable company and decided the solution is to slash costs, but by doing so he's slashed revenue too. Advertisers don't want to be associated with hate speech, but it's categorically going to happen (see Private Eye's "Malgorithms" for inspiration) and when it does it will be publicised, widely. You'll be left with people punting cryptocurrencies and other unsavouries, revenue will collapse and that will be the end. A whimper, not a bang.
There is a clear correlation between staff costs and revenue, but it's not a direct link. Somehow he seems to have failed to notice it.
As an aside, I would love to be a fly on the wall the next time Elon approaches a bank for a loan...
Re: Problem isn't technical
Twitter's product isn't technology, it's a huge, volatile, worldwide community that the company has been desperately trying to keep from flipping out and falling apart for years, like someone trying to balance a broom handle in the palm of their hand. Elon doesn't seem to have grasped that - he may not understand technology very well but he really doesn't understand people and communities - and everything he does is basically poking at that precariously balanced stick. Whatever comes of it, Twitter will look very, very, different afterwards, if it exists at all.
Unpopular opinion incoming...
...how about we just wait and see?
Give it six months, or even better twelve, and see what the landscape looks like then? What's the point of all the catastrophising and doom-mongering? Does it make you feel better? Do you, perhaps, have some sort of emotional investment in Twitter that has been somehow trodden on by Musk? Do you, in short, really care?
I don't. I'm watching it with interest, and perhaps ironically, I'm using Twitter quite a lot more than I did previously (but from a very, very low starting point). Will it work, or will it collapse? I have no opinion. I can wait, because I am an adult.
GJC
Re: Unpopular opinion incoming...
That's a very reasonable take - a lot of people are heavily invested in the future of Twitter, either emotionally or politically, and it's colouring their crystal ball gazing quite heavily. Musk is divisive, and for many who have decided on which side of the fence they sit, there's a strong desire to be proven right.
It goes further than that. Musk's behaviour is stereotypical of quite a large swathe of US tech leaders, and that culture has spread over the years. For those that don't believe it's a very good way of running a company (for various values of good - social, financial, technical, political), this very public test of that sort of management style could have a huge impact on many other companies.
Finally of course, there's the inherent belief that "this company couldn't survive without me", and we now have some very experienced and talented engineers discovering that (if they've done their jobs well!), actually a company can survive a surprising length of time without their input.
Re: Unpopular opinion incoming...
I assume that everybody opposed to Musk's ownership of Twitter is an antisemite who thinks that legitimate news stories that are harmful to their favoured candidate should be suppressed, thatTrump is worse than the Taliban and is a fan of child pornography.
Otherwise why else would they want to maintain the status quo?
Maybe, just maybe
This is the one time when putting our fingers in our ears and ignoring the problem might be the correct solution? It's already going away, like the Hindenburg in slow motion...
Just let it die…
and, for a few minutes at least, the world will be a better place. No doubt, something equally useless will pop up to take its place so journalists don't have to worry they'll be forced to do real work!
Given how upbeat Elon Musk is:
it's not so surprising he's an anagram of Me No Sulk
Re: Given how upbeat Elon Musk is:
He's also an anagram of "Lone Kums," make of that what you will.
Musk has just released evidence that Twitter interfered with elections, suppressing legitimate news about their preferred candidate - going so far as to block private messages sharing the truth.
And he's the problem? Jesus.
But given how he meddles, he'll have a view on which bit of elections he thinks *should* be interfered with. On a whim.
Yes, he is still a problem.
The report actually said that they responded to requests from both sides of the political divide (the outrage bus conveniently ignores this detail) - but that the higher proportion of Democrat-leaning employees (no surprise in a tech company) tended to be more supportive of requests from their preferred party.
Why this should be a surprise to anyone, or why anyone believes a company has any obligation to be neutral in the highly partisan American media environment is beyond me.
It's like Oprah buying Fox News and complaining it has a right-wing bias.
Additionally, Musk distracts everyone with this *explosive* revelation just at the point where the latest (dire) financial numbers come out - and then goes on to personally select 'favourites' to ban or unban without any oversight at all. How are all the people who are being loudly outraged about Twitters' past political interference strangely silent whilst Musk picks and chooses who he wants on the site? And no - he is not being a 'free speech absolutist', whatever he may claim (and however many accounts pointing this out he might choose to remove from the site).
The level is low
Christ - a common garden slug would be able to see that the apple twitter spat was Elon preemptively acting before Apple even suggested pulling twitter from the App Store.
Elon effectively fired the first shot and effectively brought the fight to Apple - its the equivalent of “you can think about trying to ban us but I will make your name mud; and I’ll bring to bear many millions of users against you”.
Question - why has EL reg seemly lost a percentage of IQ whenever it starts ranting about Elon and Twitter ??
Given that Twitter's "technical difficulties" result from Musk's "management style", it's all on Musk no matter how you slice it.
IT is not hardware manufacturing, which is all Musk really has experience with. He's 44 billion dollars in over his head, and he won't be "the richest man in the world" by a long shot within 12 months, maybe even as little as 6.