Kaseya delays SaaS restore to Sunday, CEO says ‘this sucks’ but decision was his alone
(2021/07/09)
- Reference: 1625804698
- News link: https://www.theregister.co.uk/2021/07/09/kaseya_saas_restoration_july_11/
- Source link:
Beleaguered IT management software vendor Kaseya has delayed the restoration of its SaaS services until Sunday, July 11th.
An update to the company’s [1]incident guidance report includes a video message from CEO Fred Voccola, who took personal responsibility for the delay.
“It is my decision to do this to pull the release from yesterday,” he said. “We had all the vulnerabilities managed and felt comfortable with the release, but third-party engineers made suggestions to add extra layers of protection to guard against things we could not foresee.”
[2]
Adding those extra protections led to the delays.
[3]
[4]
“This was the hardest decision of my career,” Voccola said, but assured customers the result will be a product that is “hardened as much as we feel we can do”. Later in the video he said Kaseya’s VSA product will be “exponentially more secure” due to the changes.
The scheduled time for restoration of SaaS services is 4PM Eastern Daylight Time, July 11th.
[5]
“We feel extremely confident … we will have customers coming back online,” Voccola said.
[6]Bogus Kaseya VSA patches circulate, booby-trapped with remote-access tool
[7]Bogus Kaseya VSA patches circulate, booby-trapped with remote-access tool
[8]Report shines light on REvil's depressingly simple tactics: Phishing, credential-stuffing RDP servers... the usual
[9]Kaseya says it's seen no sign of supply chain attack, sets SaaS restoration target of Tuesday afternoon, on-prem fix to follow
The CEO also sketched a program of cash assistance for Kaseya customers that he said will resemble payments made in March and April 2020. Payments for licences will be deferred.
“Throwing money at problems is not a way to solve them,” Voccola said, but “it is better than not throwing money at them. We are doing what we can do.”
Voccola shot this video in his home, and it was a rather more rustic version than his previous effort – complete with sound glitches and dubious focus.
The CEO was also a little more contrite, admitting “I feel like I let this community down, I let my company down, our company let you down.”
[10]
“I am not reading off a script,” he said at one point. “This is not BS – this is the reality.”
But he argued that every software company has flaws, and that the criminals behind the attack bear all responsibility for the incident.
“We love our customers,” he said. “It pisses me off when we do something to hurt them. Especially when it is something like when we have fallen victim to criminal acts, and it has impacted everybody.”
[11]
Kaseya CEO Fred Voccola in his new video.
Click to enlarge
CTO Dan Timpson also appeared in a video, stating that because of the incident Kaseya “is adding a lot more rigour to our processes, to our deployment, to our code base, to keep everyone safe an improve the overall safety of our products.
“We are committed to, and are, working fiercely on our security posture across the board.”
For now, Kaseya has published runbooks for [12]SaaS and [13]on-prem customers. Timpson said both have been peer-reviewed.
Good luck, Kaseya customers. Feel free to [14]let us know how things go on Sunday. ®
Get our [15]Tech Resources
[1] https://www.kaseya.com/potential-attack-on-kaseya-vsa/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YOgeR-1FCEJh14NpRw2GuQAAAFc&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YOgeR-1FCEJh14NpRw2GuQAAAFc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YOgeR-1FCEJh14NpRw2GuQAAAFc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YOgeR-1FCEJh14NpRw2GuQAAAFc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2021/07/07/kaseya_malware_patches_/
[7] https://www.theregister.com/2021/07/07/kaseya_malware_patches_/
[8] https://www.theregister.com/2021/07/07/revil_tactics_and_multimillion_dollar/
[9] https://www.theregister.com/2021/07/06/kaseya_update/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YOgeR-1FCEJh14NpRw2GuQAAAFc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[11] https://regmedia.co.uk/2021/07/09/screenshot_kaseya_ceo_fred_voccola.jpg
[12] https://helpdesk.kaseya.com/hc/en-gb/articles/4403709476369
[13] https://helpdesk.kaseya.com/hc/en-gb/articles/4403709150993)/
[14] mailto:simon.sharwood@sitpub.com
[15] https://whitepapers.theregister.com/
An update to the company’s [1]incident guidance report includes a video message from CEO Fred Voccola, who took personal responsibility for the delay.
“It is my decision to do this to pull the release from yesterday,” he said. “We had all the vulnerabilities managed and felt comfortable with the release, but third-party engineers made suggestions to add extra layers of protection to guard against things we could not foresee.”
[2]
Adding those extra protections led to the delays.
[3]
[4]
“This was the hardest decision of my career,” Voccola said, but assured customers the result will be a product that is “hardened as much as we feel we can do”. Later in the video he said Kaseya’s VSA product will be “exponentially more secure” due to the changes.
The scheduled time for restoration of SaaS services is 4PM Eastern Daylight Time, July 11th.
[5]
“We feel extremely confident … we will have customers coming back online,” Voccola said.
[6]Bogus Kaseya VSA patches circulate, booby-trapped with remote-access tool
[7]Bogus Kaseya VSA patches circulate, booby-trapped with remote-access tool
[8]Report shines light on REvil's depressingly simple tactics: Phishing, credential-stuffing RDP servers... the usual
[9]Kaseya says it's seen no sign of supply chain attack, sets SaaS restoration target of Tuesday afternoon, on-prem fix to follow
The CEO also sketched a program of cash assistance for Kaseya customers that he said will resemble payments made in March and April 2020. Payments for licences will be deferred.
“Throwing money at problems is not a way to solve them,” Voccola said, but “it is better than not throwing money at them. We are doing what we can do.”
Voccola shot this video in his home, and it was a rather more rustic version than his previous effort – complete with sound glitches and dubious focus.
The CEO was also a little more contrite, admitting “I feel like I let this community down, I let my company down, our company let you down.”
[10]
“I am not reading off a script,” he said at one point. “This is not BS – this is the reality.”
But he argued that every software company has flaws, and that the criminals behind the attack bear all responsibility for the incident.
“We love our customers,” he said. “It pisses me off when we do something to hurt them. Especially when it is something like when we have fallen victim to criminal acts, and it has impacted everybody.”
[11]
Kaseya CEO Fred Voccola in his new video.
Click to enlarge
CTO Dan Timpson also appeared in a video, stating that because of the incident Kaseya “is adding a lot more rigour to our processes, to our deployment, to our code base, to keep everyone safe an improve the overall safety of our products.
“We are committed to, and are, working fiercely on our security posture across the board.”
For now, Kaseya has published runbooks for [12]SaaS and [13]on-prem customers. Timpson said both have been peer-reviewed.
Good luck, Kaseya customers. Feel free to [14]let us know how things go on Sunday. ®
Get our [15]Tech Resources
[1] https://www.kaseya.com/potential-attack-on-kaseya-vsa/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YOgeR-1FCEJh14NpRw2GuQAAAFc&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YOgeR-1FCEJh14NpRw2GuQAAAFc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YOgeR-1FCEJh14NpRw2GuQAAAFc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YOgeR-1FCEJh14NpRw2GuQAAAFc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2021/07/07/kaseya_malware_patches_/
[7] https://www.theregister.com/2021/07/07/kaseya_malware_patches_/
[8] https://www.theregister.com/2021/07/07/revil_tactics_and_multimillion_dollar/
[9] https://www.theregister.com/2021/07/06/kaseya_update/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YOgeR-1FCEJh14NpRw2GuQAAAFc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[11] https://regmedia.co.uk/2021/07/09/screenshot_kaseya_ceo_fred_voccola.jpg
[12] https://helpdesk.kaseya.com/hc/en-gb/articles/4403709476369
[13] https://helpdesk.kaseya.com/hc/en-gb/articles/4403709150993)/
[14] mailto:simon.sharwood@sitpub.com
[15] https://whitepapers.theregister.com/
“This was the hardest decision of my career,”
Pascal Monett
I'm pretty sure that there will be harder ones in the future.
That said, I have come to believe that Kaseya should have the benefit of the doubt. White hats have pointed out that Kaseya reacted swiftly and properly to vulnerability alerts, and did everything it could to plug the holes as swiftly as possible.
Unfortunately, the criminals got in first. You cannot guard against bad luck.
I'm now convinced Kaseya is doing what it can to pick up the pieces and put everything back together again. All the noises being made point to a team that is working its ass off and trying its best to recover from the situation.
I no longer think that Kaseya was asleep at the wheel. I feel sorry for everyone involved, and I really wish someone could stop those despicable criminals.
Good luck, Voccola.
I assume Fred Voccola is aware of the history of the three envelopes, and has already progressed to retrieving envelope three from the Company safe.