Update 'designed to improve user experience' takes down the Microsoft 365 Admin Portal
- Reference: 1643888661
- News link: https://www.theregister.co.uk/2022/02/03/ms365_portal_down/
- Source link:
Without a hint of irony, the company [2]posted : "We've identified a recent service update designed to improve user experience is causing impact."
The impact in question was an inability to access the admin portal, something unlikely to affect an end user pootling around in Excel, but a huge headache for an administrator trying to manage their tenant.
[3]
Microsoft 365 Portal problems
The problem appears to be specific to users in Europe, the Middle East and Africa, according to Microsoft (although some users in [4]North America and [5]Asia reported issues).
Other services, such as the Teams Admin Center also dropped over in sympathy, although Teams itself continued functioning. And there was the additional irony that to view Microsoft 365 Service Health from the somewhat sluggish status.office.com, one was directed to portal.office.com, which was... a little poorly when we checked it.
[6]ISO.org outage hits day 3: Still in the dark as the important matter of bunk bed standards enters discussion
[7]Data centre outfit Interxion hit with outage at central London facility
[8]What begins with a 'B' and is having problems at tsoHost? Hopefully not your website
[9]Nationwide Building Society's Faster Payments turn into Slower Payments for 2022
Microsoft confirmed the fault by 0936 UTC this morning. An excruciating 45 minutes later, the company was hurriedly backing out of whatever it had done to its systems.
In the latest update to its admin portal, Microsoft said: “The revert of the update is taking longer than expected to complete. We’re continuing our efforts to resolve the issue by also manually restarting the affected infrastructure to expedite the recovery.”
[10]
Administrators, doubtless weary from dealing with the creative approach to quality control, reacted predictably to the outage.
Thanks [11]@MSFT365Status ....I'm in the middle of a migration as well....good timing. Dogs crying..... — Lee Sainsbury (@Lee_Sainsbury) [12]February 3, 2022
The Register contacted Microsoft to learn more about this change to improve user experience, which has had the opposite effect for administrators, and will update this piece should the company respond. ®
Updated to add at 1502 UTC:
Microsoft's 365 admin portal is still down.
The good news is that backing out of the update and, er, turning it off and on again seems to be working.
[13]
The bad news is that "the recovery process has been slower than anticipated" according to Microsoft, which noted: "The revert of the update is taking longer than expected to complete. We're continuing our efforts to resolve the issue by also manually restarting the affected infrastructure to expedite the recovery."
The company has also confirmed that other regions might also be affected, although service health was improving as of its latest update an hour ago.
Get our [14]Tech Resources
[1] https://twitter.com/MSFT365Status/status/1489170822375911424
[2] https://status.office365.com/
[3] https://regmedia.co.uk/2022/02/03/ms365_down.png
[4] https://mobile.twitter.com/DespicableDD/status/1489171299855486981
[5] https://mobile.twitter.com/ohwhatmenope/status/1489175950969348097
[6] https://www.theregister.com/2022/01/28/iso_down/
[7] https://www.theregister.com/2022/01/11/interxion_suffers_outage_at_central/
[8] https://www.theregister.com/2022/01/11/tsohost_b/
[9] https://www.theregister.com/2022/01/04/nationwide_processing_slowdown/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YfwKNrmYSCvTuBftLrpPTgAAANQ&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[11] https://twitter.com/MSFT365Status?ref_src=twsrc%5Etfw
[12] https://twitter.com/Lee_Sainsbury/status/1489179151869394951?ref_src=twsrc%5Etfw
[13] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YfwKNrmYSCvTuBftLrpPTgAAANQ&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[14] https://whitepapers.theregister.com/
I wondered what it was...
I must have caught the last few minutes of it.
I'm afraid there is one thing I can say about this....
WHAT A FUCK UP!!!!!
Still down with:
Service Unavailable
HTTP Error 503. The service is unavailable.
3 9s....
I'm guessing even 4 9s is aspirational at Microsoft.
Still think Office 365 is false advertising...
Re: 3 9s....
Still think Office 365 is false advertising...
Made me think - maybe Microsoft are being honest: there are actually 365.256 days in a true year, so maybe they think they can afford up to 6 hours of down time per annum without being accused of false advertising...
Re: 3 9s....
I still favour the report that the number part of Microsoft's xx365 products is actually in Octal which equates to a decimal 245 days allowing them decimal 120 days outage per year.
Re: 3 9s....
For a certain definition of hours.
Like 6 hours of your work day.
Re: 3 9s....
Maybe that would be 3 Neins:
You want it to work? Nein!
You want it to behave consistently? Nein!
You want to be recompensed for your testing, and fiddling with it to make it work? Nein!
Re: 3 9s....
Still think Office 365 is false advertising...
No; the service level guarantee refers to their billing, which is always for the full 365 days.
At last - the truth
'designed to improve user experience'
It certainly did that.
Re: At last - the truth
While waiting for MS to clean up the mess, have one of those -->
A much better user experience than using Office 3xx (going down almost daily, not even the 6 is safe)
Of all the days, today was the wrong day to cut down on the smokes
But was it Johnny Henshaw-Jacobs who pulled the big plug in Microsoft's datacenter?
"Just kidding! Haha!"
From DevOps to DevOooooooops!
And from Azure to out in the blue...
Interesting way to "improve" the user experience, by removing it..
I dunno...not having to deal with a Microsoft beta release product for a day would certainly improve my "user experience"....
I would call not dealing with a Microsoft interface for the day an improvement on the normal workday experience! :)
evangelist
If O365 wasn't such great value I would be angry!
Re: evangelist
The sarcasm is strong in this one!
Oh why are we waiting (sung in a very loud voice)
It's still f'in down. I have tasks queuing up!!!!
Still fucked. Just getting a timeout message at the moment.
Main Admin portal is affected but most of the individual portals are not. Here's a handy list of the others if you are tearing your hair out:
https://msportals.io/?search=
Hope it comes back soon but have found out can do most tasks in the others..
Yup. Saved a project.
If it ain't broke...
They managed to leave out the "don't".
Seems to finally back up and running so I was able to do the very minor job I needed to do.
i've enjoyed the opportunity
to explain to the management that this is what can happen when you rely on cloud systems.
Was a bit of a pain too.
We made an update. We fucked up. Again.
Please accept this 2% increase in contract fees due to supply chain issues. We will correct the problem as soon as we've hired enough of our old employees back as consultants.
Thank you for testing our applications in our place.