News: 1649721032

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

At last, Atlassian sees an end to its outage ... in two weeks

(2022/04/12)


The Atlassian outage that began on April 5 is likely to last a bit longer for the several hundred customers affected.

In a statement emailed to The Register , a company spokesperson said the reconstruction effort could last another two weeks.

The company's spokesperson explained its engineers ran a script to delete legacy data as part of a scheduled maintenance for unidentified cloud products. But the script went above and beyond its official remit by trashing everything.

[1]

"This data was from a deprecated service that had been moved into the core datastore of our products," Atlassian's spokesperson said. "Instead of deleting the legacy data, the script erroneously deleted sites, and all associated products for those sites including connected products, users, and third-party applications."

[2]

[3]

Atlassian, which has been trying to repair the damage of its errant script, on Friday [4]said it expected "most site recoveries to occur with minimal or no data loss." And so far, though data has been deleted, Atlassian has been able to revive it.

"We maintain extensive backup and recovery systems, and there has been no data loss for customers that have been restored to date," Atlassian's spokesperson said, and stressed that this was not the consequence of a cyberattack and that no authorized access to customer data has occurred.

[5]

Jira Software, Jira Work Management, Jira Service Management and Confluence continue to show problems on the [6]Atlassian status page , as do Opsgenie and Atlassian Access. Jira provides software issue tracking while Confluence offers a web-based corporate wiki. Opsgenie is an alert service and Atlassian Access is an identity and access management service.

Onsite JIRA installations have not been affected. Self-managed servers are on the way out, however: Back in October, 2020, Atlassian [7]announced the discontinuation of its server products – it stopped selling new licenses on February 2, 2021 and plans to end support for its server products on February 2, 2024. The reason, the company explained last year, is that [8]the cloud is the future .

"We know this outage is unacceptable and we are fully committed to resolving this," Atlassian's spokesperson said. "Our global engineering teams are working around the clock to achieve full and safe restoration for our approximately 400 impacted customers and they are continuing to make progress on this incident."

[9]Day 7 of the great Atlassian outage: IT giant still struggling to restore access

[10]Atlassian outage lingers, sparking data loss fears

[11]Atlassian adds Analytics, Atlas, Compass to line up

[12]Atlassian Jira, Confluence outage persists two days on

As we [13]reported earlier on Monday, the software biz says it has restored functionality for more than 35 per cent of those affected by the service outage.

Atlassian said that the company is doing everything it can to restore service as fast as possible but until today it had been unable to provide a likely recovery date due to the complexity of the rebuilding process.

[14]

"While we are beginning to bring some customers back online, we estimate the rebuilding effort to last for up to two more weeks," the company said.

That's quite a bit longer than the "<6 hours" recovery time [15]promised by the company for Tier 1 services like Jira and Confluence.

"We know this is not the news our customers are hoping for, and we apologize for the length and severity of this incident. We don’t take this issue lightly and are taking steps to prevent future reoccurrence." ®

Get our [16]Tech Resources



[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YlT5bXR1MoGAr5H6uwxZRQAAAAU&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0

[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YlT5bXR1MoGAr5H6uwxZRQAAAAU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YlT5bXR1MoGAr5H6uwxZRQAAAAU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[4] https://www.theregister.com/2022/04/08/atlassian_service_issues_linger_leaving/

[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YlT5bXR1MoGAr5H6uwxZRQAAAAU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[6] https://status.atlassian.com/

[7] https://www.atlassian.com/blog/announcements/journey-to-cloud

[8] https://www.atlassian.com/blog/platform/atlassian-server-is-going-away-next-steps

[9] https://www.theregister.com/2022/04/11/atlassian_still_down/

[10] https://www.theregister.com/2022/04/08/atlassian_service_issues_linger_leaving/

[11] https://www.theregister.com/2022/04/06/atlassian_new_services/

[12] https://www.theregister.com/2022/04/06/atlassian_jira_confluence_outage/

[13] https://www.theregister.com/2022/04/11/atlassian_still_down/

[14] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YlT5bXR1MoGAr5H6uwxZRQAAAAU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[15] https://www.atlassian.com/trust/security/data-management

[16] https://whitepapers.theregister.com/



Cheers to that guy who hit the ENTER key!

oldtaku

I've certainly done some eff-ups, but nothing that comes remotely close to this scale.

chicken & pig

An_Old_Dog

"... we are fully committed to resolving this."

In a ham-and-egg breakfast, the chicken is involved. The pig is committed.

We maintain extensive... DMML

Falmari

"We maintain extensive backup and recovery systems"

Backup maybe but not recovery.

There does not seem to be a system in place to recover and restore this particular set of data. Because if there had been a well maintained and tested recovery system in place the data would have been restored days ago.

It would not have taken a week to restore just 9% of their customers data.

Never explain. Your friends do not need it and your enemies will never
believe you anyway.
-- Elbert Hubbard