IBM's 18-month company-wide email system migration has been a disaster, sources say
- Reference: 1625082330
- News link: https://www.theregister.co.uk/2021/06/30/ibm_email_outage/
- Source link:
Current and former IBMers have confirmed to The Register that the migration, 18 months in the making, has been a disaster.
"I feel bad for bringing this to the press but I'm afraid that I'm only one of the many thousands at Big Blue who are utterly disgruntled at the moment," one employee told us. "If we can't even handle our own cloud migration program then why would any customer trust us?"
[1]
The incident has spilled out onto social media as IBMers vent their frustration.
[2]
[3]
"Every IBMer has descended into a dark, chaotic pit of not being able to access email or calendars for the past 3 days.... wondering where we are??" lamented IBM designer Jessica Tremblay, [4]via Twitter . "Well, probably pulling our hair out [5]#pleasesendwine "
Every IBMer has descended into a dark, chaotic pit of not being able to access email or calendars for the past 3 days.... wondering where we are?? Well, probably pulling our hair out 😂😂 [6]#pleasesendwine 🍷 — Jessica Tremblay (@poofichu) [7]June 30, 2021
We've been told that email service has been intermittent for the past four or five days, and not everyone has been affected in the same way. Lack of access has been shorter for some – one source told us email was back after two days of downtime.
Slack is said to be working though Outlook, Verse (IBM's webmail), and Notes have been unreliable.
"Outlook won't work with the new system, IBM Notes won't work and the online email called Verse has now gone down," a tipster told us. "Everyone has been affected and no fix is in sight."
[8]
One source we spoke with laid the blame on IBM CFO James Kavanaugh for cutting costs and not hiring the right people.
Everything's fine! For the moment
An IBM spokesperson contacted by phone declined to confirm that anything is amiss or to comment on the situation. The IT giant subsequently got back in touch with The Register , and we understand Big Blue may have something to say at a later time, or not.
Over the weekend, a source told us, a blog post to IBM's internal network [9]w3 said the migration had been planned for 18 months and that everything should go fine provided everyone follows the instructions emailed to them. Evidently, this did not happen.
[10]
A warning banner on IBM's intranet alerting staff to email delays ... Click to enlarge
Since then, a banner has gone up on w3 pointing people to a Slack channel for updates on the situation, and IBM's CIO has posted a note to employees addressing the problems. Presently, the [11]w3 status page returns an error.
Thousands of staff, we're told, have been dealing with poor performance from iNotes on desktop devices and Verse on the web and on mobile devices ( [12]which hasn't been getting great reviews ). Initially, the company tried to resolve complaints through help desk chat but the queue ballooned to the point that employees gave up.
[13]Microsoft loves Linux so much that packages.microsoft.com has fallen and can't get up
[14]IBM pulls up the ladder behind some supercomputer customers
[15]Monitoring is simple enough – green means everything's fine. But getting to that point can be a whole other ball game
[16]No change control? Without suitable planning, a change can be as good as an arrest
We're told that the migration plan followed from IBM's decision in 2018 to sell various software products, including Notes, [17]to India-based HCL Technologies . Following the sale, Big Blue didn't want its data on HCL's servers.
"They were transitioning to IBM-owned servers," one source told us. "That's where it broke down."
Among those employees unable to access their email, there's concern that disappeared messages may not be restored. We've even heard that IBM employees have been approached by recruiters posing questions like, "Why are you still at IBM? They can't even get email straight."
“We sold Notes to HCL a few years back,” a source told us. “The problem is we are late getting off the version that becomes unsupported tomorrow.”
[18]
If IBM decides it wishes to comment, we will let you know. ®
Updated to add
Minutes after this article was published, an IBM spokesperson dictated the company’s statement over the phone, presumably because email is a bit spotty: “Some IBM employees are experiencing email service delays. We are working to restore full service as quickly as possible, and we are leveraging a variety of alternative communications tools to ensure minimal disruption to our clients and to our business while we work to restore full email capabilities.”
Get our [19]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YNzpfPrmrhSxafmXzoHGrQAAAIc&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YNzpfPrmrhSxafmXzoHGrQAAAIc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YNzpfPrmrhSxafmXzoHGrQAAAIc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://twitter.com/poofichu/status/1410251987229233153?s=20
[5] https://twitter.com/hashtag/pleasesendwine?src=hashtag_click
[6] https://twitter.com/hashtag/pleasesendwine?src=hash&ref_src=twsrc%5Etfw
[7] https://twitter.com/poofichu/status/1410251987229233153?ref_src=twsrc%5Etfw
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YNzpfPrmrhSxafmXzoHGrQAAAIc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://login.w3.ibm.com/
[10] https://regmedia.co.uk/2021/06/30/ibm_w3_screenshot.jpg
[11] https://ibm.biz/w3id-status
[12] https://play.google.com/store/apps/details?id=com.lotus.sync.traveler&hl=en_US&gl=US&showAllReviews=true%20target=
[13] https://www.theregister.com/2021/06/17/microsoft_packages_404/
[14] https://www.theregister.com/2021/06/14/ibm_discontinues_ladder_sales/
[15] https://www.theregister.com/2021/06/22/monitoring/
[16] https://www.theregister.com/2021/06/10/change_control_planning/
[17] https://newsroom.ibm.com/2018-12-06-HCL-Technologies-to-Acquire-Select-IBM-Software-Products-for-1-8B
[18] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YNzpfPrmrhSxafmXzoHGrQAAAIc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[19] https://whitepapers.theregister.com/
As an IBMer, I'd like to say that the article implies that every user at IBM is having issues. I'm not, and I suspect that I'm not alone. Yeah, they didn't handle this as well as they could have, but it isn't a total disaster.
No, it's just a major disaster.
Dark, chaotic pit of not being able to access email or calendars
Sounds idyllic. I would sign up, but I hear IBM's record with the elderly experienced is similarly flawed.
Bean counters?
The implication of the article is that IBM is yet another company which has allowed operational decisions to be made by bean counters. From flogging off a product they needed to use themselves* to trying to replace it on the cheap.
Nail, meet coffin.
*I kind of wonder if those migration costs were included in the calculations that lead to the sell-off. Or indeed the saved costs/subsidy of having an in-house system that was also a product.
Re: Bean counters?
More to the point if IBM can't use Notes because it would be insecure having their data on another companies servers - why would anyone else run Notes ?
Migration costs
Of course they weren't - that would have affected the bonus paid on savings.
Maybe they should outsource?
Have they thought about outsourcing their e-mail system upgrade to someone? Capita for example? :)
Re: Maybe they should outsource?
Notes is horrible, I've been scarred.
Having said that, the ability to mail out structured forms and get data back direct into a database seems like such a core feature.
Both Notes and Outlook forms are horrible, are there any alternatives?
Youd want anything that follows standards or, like IBM, you will be locked in forever.
Perhaps the world needs a standard for html form validation that is safe/sane in a email client.
Incredibly Borked Mail system
>>>
we are leveraging a variety of alternative communications tools to ensure minimal disruption to our clients and to our business while we work to restore full email capabilities
<<<
Carrier pigeon?
Yum
It's hilarious to see a consulting company get sick from eating its own dog food.
"laid the blame on IBM CFO James Kavanaugh"
Okay, I am well aware that large companies do have the despicable habit of being run by the beancounters, but in this case shouldn't it be the CTO getting the flack ?
IBM does have a CTO, right ? If he can't stand up for what is needed, isn't it his fault in the first place ?
Re: "laid the blame on IBM CFO James Kavanaugh"
>IBM does have a CTO, right
No, it turns out he was over 50 so they made him redundant and outsourced the job
Re: "laid the blame on IBM CFO James Kavanaugh"
I think many CTOs have learned the adage of "the nail sticking out". If they try to use logic and reason to stop a "cost-savings" they tend to get hammered flush with the floor - much like a nail sticking up.
That makes it hard to stay on the merry-go-round that is corporate executive hiring. It is much better to say nothing, be a pleasant personality and parachute out of the ball of fail only to land in another cushy post.
Re: "laid the blame on IBM CFO James Kavanaugh"
Of course IBM has a CTO. But guess who the CTO reports to? Yup, Kavanaugh
"are leveraging a variety"
FFS what's wrong with "using" or "trying"?
"Why are you still at IBM? They can't even get email straight."
These guys were important once, right? I don't think I'm old enough to remember such a time. I thought they were a law firm for a while because they only seemed to be mentioned in court cases.