Atlassian outage (April 2022)
The accidental deletion of 883 Atlassian customer sites by a maintenance script, followed by a manual restoration that took up to fourteen days for some customers.
An online software provider hosts thousands of customers on a shared platform, and its teams run maintenance scripts there with broad privileges. In April 2022, Atlassian set out to deactivate a legacy app built into its products; the list handed to the team running the job contained the IDs of entire sites instead of the app's, and the script, which did not check what kind of ID it had received, deleted those sites. Backups existed and no data was lost beyond a few minutes, but the restoration tooling had been built to recover one site at a time, not hundreds at once: recovery was largely manual. The case separates two notions that policies often blur, data loss and restoration time, and shows that the second can drive most of the business interruption loss even when the first is zero. It also illustrates human operational error at the provider, a peril the customer bears without any way to prevent it.
On April 5, 2022, between 7:38 and 8:01 UTC, a script deleted 883 sites belonging to 775 Atlassian customers, who lost access to Jira, Confluence and the company's other products. According to the post-incident review Atlassian published, the first customers were restored on April 8 and the last on April 18, up to fourteen days of downtime, with no customer losing more than five minutes of data.
Follow the research
One letter a week. First, the week's underwriting note, when one is published: a dated, argued decision, signed by Paul Bernardski. Then what the observatory published on cyber risk, political risk and insurance. No account, no tracking.
No spam, one-click unsubscribe, no data ever sold.