News: 1607950901

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

World+dog share in collective panic attack as Google slides off the face of the internet

(2020/12/14)


Google services such as YouTube and Gmail started the week with an almighty bang as the Chocolate Factory's cloud came crashing to the ground.

Despite an insistence from the company's various health dashboards that all was fine and dandy, it most definitely was not.

Those seeking distraction in video form were treated to YouTube's "Something went wrong..." monkey, while others wishing to express their disquiet via Gmail were shown a 502 code or a suggestion to try again in five minutes.

[1]

The issue appears to have afflicted vast swathes of the globe, with users in the Philippines and India joining Europeans and US early birds in being unexpectedly ejected from the Chocolate Factory's services.

Problems seemed to start at around 11:30 GMT. At time of writing YouTube was inaccessible, Gmail was borked, Drive was down, image search failed (unless an error code was what you were looking for), and Docs didn't seem happy.

Some things still worked – we found links to existing Google Docs were working and the search for which the company is famed appeared to be running. So there was no need to resort to something like Bing.

Google is no stranger to outages. Pretty much everything from GCP to G Suite fell over into a heap [2]back in August .

As for today's outage, Google's Workspace dashboard was aglow with green lights, even if the reality was quite different.

[3]

Though it did eventually right itself...

[4]

The day came. When you can’t even google what happened with google.

It’s still 2020. [5]#googledown [6]#YouTubeDOWN [7]#gmaildown [8]pic.twitter.com/76CMpJrA0k — Hum Tum Hub (@humtumhub) [9]December 14, 2020

The Register contacted Google to find out what has befallen its services and will update should the ad slinger respond. ®

* Total Inability To Support Usual Procrastination

Get our [10]Tech Resources



[1] https://regmedia.co.uk/2020/12/14/youtubemonkey.png

[2] https://www.theregister.com/2020/08/20/gmail_outage/

[3] https://regmedia.co.uk/2020/12/14/dashboard2.png

[4] https://regmedia.co.uk/2020/12/14/googledashboard.png

[5] https://twitter.com/hashtag/googledown?src=hash&ref_src=twsrc%5Etfw

[6] https://twitter.com/hashtag/YouTubeDOWN?src=hash&ref_src=twsrc%5Etfw

[7] https://twitter.com/hashtag/gmaildown?src=hash&ref_src=twsrc%5Etfw

[8] https://t.co/76CMpJrA0k

[9] https://twitter.com/humtumhub/status/1338457806383431682?ref_src=twsrc%5Etfw

[10] https://whitepapers.theregister.com/

“So there was no need to resort to something like Bing”

Winkypop

Steady on now Ted!

Re: “So there was no need to resort to something like Bing”

Lomax

I've been using [1]Qwant as my default search engine for months now, and have found it to be really quite good. Previously tried to use DuckDuckGo as the default but found myself constantly repeating the search in Google - not so with Qwant!

[1] https://www.qwant.com

Re: “So there was no need to resort to something like Bing”

MatthewF

Careful Now

Re: “So there was no need to resort to something like Bing”

Imhotep

Does anyone have a suggestion for a good alternative to Google search? None that I've used measure up.

That explains it.

Victor Ludorum

Thunderbird asked me to re-authorise access to GMail - I thought it had just expired or something, but I was then getting a 500 error. All seems normal now (well, as normal as 2020 can be).

Re: That explains it.

Anonymous South African Coward

Same here - but I needed to re-authenticate myself after the charlie foxtrot event have passed.

Everything's fine and dandy now.

The entire g string fell off...

sorry, what?

Exposing friends and foe alike to naked robots with their innards on show. Not just gmail and YouTube etc. (my son found he could still access YouTube through a desktop browser, just not on his device, so could continue to chortle at those kittens and their antics); gsuite was borked too. Looked like a serious authentication infrastructure boo-boo. I was told that my gsuite email address wasn't recognized at all when attempting to login. Ooof.

Brewster's Angle Grinder

THIS WAS SO SERIOUS I HAD TO GO ROUND TO MY EIGHT-FOUR YEAR OLD NEIGHBOUR AND SEE IF SHE COULD COPE WITHOUT GMAIL.

YES, I DO NEED A COAT - IT'S RAINING AGAIN.

Raining again?

TRT

That’ll be the one cloud service that you can guarantee will be working.

Re: Raining again?

richardcox13

Except when it comes to summer... and all the gardeners start moaning after 30min of hot weather.

Impressive

JonW

Last Month

AWS: Here's a major outage for you

This Month:

Google: Hold my beer...

"Google: Hold my beer..."

LDS

"Oooops, I poured it on the Master Console and Main Power Controller"....

Re: Impressive

Captain Scarlet

Office 362: Been there done that, oh no not again

Enterprise stuff

chivo243

G-Suite\ GAffE went offline... my personal gmail was up the whole time. I received an error message trying to authenticate into the google chat app stating my domain didn't have access to the service???

Things are back to normal for the time being?

Fábio Rabelo de Deus

You can add Brazil in the list of affected Countries .

TRT

I hear things were left alone there, but only in a narrow strip down the middle of the country.

Anonymous Coward

It's been too humid here, that's probably related.

I think you'll find...

Sgt_Oddball

The word is 'moist'...

The Register contacted Google

Warm Braw

Given we're not talking about Apple, you might be able to infer from their silence that they eat their own dog, er, food.

Try again *later*

andy 103

It always makes me chuckle when you get error messages that say

"Try again in five minutes"

Why five? Where has that figure come from? They don't even know what the error is! So how can you reliably determine when someone can "try again"?

But it's somehow better than "try again later ". When is later? 10 mins, tomorrow, next year?

Either way we'll all keep madly hitting Ctrl+F5 until we get the response we want.

Re: Try again *later*

TRT

How long does it take YOU to make a cup of tea?

Re: Try again *later*

andy 103

How long does it take YOU to make a cup of tea?

Well I made at least 2 in the time it took Google to fix this.

So, considerably longer than 5 mins.

Re: Try again *later*

TRT

I’ve always been fond of HandBrake’s policy on timing user activity.

Re: Try again *later*

Anonymous Custard

Or the Windows file transfer dialogue box, where the last 10s often seem to take at least 10 minutes...

Never a finer example of quantum time dilation!

Re: Try again *later*

bigphil9009

Obligatory XKCD:

https://xkcd.com/612/

Fail-over/Fall-over

thondwe

What's the betting that's another "transparent fail-over" fail in the ultra-clever routing domain then?

Re: Fail-over/Fall-over

TRT

It’s good that the search engine is on a different system though. I mean how else do they find a fix for a problem?

Re: Fail-over/Fall-over

Anonymous Coward

Well, they'll have a job logging in to StackExchange sites to find solutions, since those rely on JavaScripts hosted by googleapis (much to the annoyance of many).

(Makes you wonder how many sites will also be borked if Google's captcha service is also down: even if you don't use sign in with Google, they still try to ensnare you...)

The Register contacted Google to find out

Anonymous Coward

not until they get their gmail sorted, I suppose... We're all (?) DOOMED!!! :)

Ain't those Cloud services supposed to be up 100% of the time?

Anonymous Coward

Asking for a friend...

Re: Ain't those Cloud services supposed to be up 100% of the time?

TRT

To be fair the services were probably all up and working just fine. It’s just that no one could get to them to use them.

Re: Ain't those Cloud services supposed to be up 100% of the time?

Anonymous Coward

None of our GCP hosted instances went down even if we couldn't load anything in the GCP console

Re: Ain't those Cloud services supposed to be up 100% of the time?

Antonius_Prime

Global Warming. It's getting hotter, so the Cloud is evapourating a little...

Re: Ain't those Cloud services supposed to be up 100% of the time?

storner

Supposed to? Yes.

Did Google/AWS/Microsoft/Oracle/RedHat promise that they would be? No.

Are they? Apparently not.

DJV

As reported by the BBC, a guy on Twitter said: "I’m sitting here in the dark in my toddler’s room because the light is controlled by @Google Home. Rethinking... a lot right now."

And THAT is why I don't have anything essential in my house controlled via the internet in any form or fashion. But you just can't tell people that - they have to experience the downsides for themselves before they get the clue!

TRT

Mine all have a local controller on the inside of my network as well as being RF controlled without IP so I don’t need internet or network. The light bulbs I have (those that are smart) rather sensibly also turn on if the supply is interrupted and restored so I reckon Twitterman is spouting bullshit.

Cederic

IoT was crushed by corporation demands to control all the data, resulting in cloud services the users don't control. It's lovely to see the people that didn't realise this get a helpful education.

My favourite quote on the BBC though was "Google has been contacted for comment, but one spokesperson said they were unable to access their email."

Anonymous Custard

If you really must use the things, then Home Assistant is your man (or Pi)...

Use the competition?

Bitsminer

Most mobile phone companies around here have repair crews equipped with....wait for it....phones linked to their competitor's networks. Because, obviously. Repair crews, obviously.

I wonder how long it takes Microsoft to switch to posting their status on GCP and Google to set their email backup plan to Office 362.

Until they're both down. Then, what?

Anonymous Coward

When I'm unable to control my Philips Hue bulbs via apps or Alexa I use this cunning workaround device called a "light switch", or did he remove them?

The hard part…

KarMann

…is figuring out whether 'failsafe' for a door lock means it's locked or unlocked if the Internet or cloud or power fails. Either way can be pretty bad. Which is why, while I'm not totally opposed to such 'smart' IoT things*, door locks will be about the last thing I'd 'upgrade' that way.

* Brought to you by the Redundant Department of Redundancy Department. Ask us about our PIN number for the ATM machine!

Re: The hard part…

bigphil9009

"Redundant Department of Redundancy Department". Congratulations! You have totally melted my brain with that one. It feels like I'm one re-read away from entering an infinite loop...

Re: The hard part…

Lazlo Woodbine

Within the realm of Access Control, 'Fail Safe' means the lock opens when power is lost, 'Fail Secure' means the lock closes when power is lost.

Re: The hard part…

DJV

A school local to me found out the hard way that their electronic door locks (the only type they had on them at first) were "fail safe". When they did have a power outage, they discovered (to their surprise), that all and sundry could just walk in and out of the school with no problem. They rapidly "downgraded" to proper old fashioned locks after that.

Anonymous South African Coward

All your IoT devices are belong to us

Martin Summers

Normally smart lights have an override. When you switch the power on to them first time they come on and they function just like any other. This is what happens with my TP-Link bulbs and I've never had any issues if there's been Internet connectivity problems. In fact the app looks for them locally. Just because I can control my lights via Google home or Alexa, it doesn't mean I can't independently control them. Either he's got crap smart bulbs or he doesn't know how to use them (or possibly the switch for that matter).

Natalie Gritpants Jr

Given the outage was at lunchtime, why not open the curtains? Also, why remote control the light in a toddler's room? Does the toddler have a smartphone? If it's for the adults, why not use the conventional light switch when you walk in the room? Also^2, why haven't you fitted a smart switch with a local button instead of a stupid 'smart' bulb that requires the normal light switch to be left on?

If you really really want to gaslight your toddler reliably try one of these: https://www.home-assistant.io/blue no internet required as long as you have the correct bulbs/switches.

Anonymous Coward

It's not lunchtime all over the world when it's lunchtime in the UK.

richardcox13

[1]https://twitter.com/internetofshit has had a busy time...

[1] https://twitter.com/internetofshit

Anonymous Coward

Did he not think of walking over to the light switch to turn on the light.........

Its called manual override

I was appalled by this story of the destruction of a member of a valued
endangered species. It's all very well to celebrate the practicality of
pigs by ennobling the porcine sibling who constructed his home out of
bricks and mortar. But to wantonly destroy a wolf, even one with an
excessive taste for porkers, is unconscionable in these ecologically
critical times when both man and his domestic beasts continue to maraud
the earth.
Sylvia Kamerman, "Book Reviewing"