Google Cloud outage of June 12, 2025
A global Google Cloud outage caused by an incomplete policy record that crashed Service Control, the system that authorizes every API call, in every region at once.
A cloud provider splits its services into independent regions so that a local failure stays local, but some control functions, such as authorizing calls and enforcing quotas, read data replicated everywhere. The June 12, 2025 outage started in one of those functions: code added a few weeks earlier to check quota policies had neither error handling nor a feature flag, and it ran only when a policy of a certain type was written. When a policy with blank fields was inserted, it replicated to every region within seconds, and every Service Control instance hit the same null pointer. The gradual, region-by-region rollout could not have caught it, since the faulty path had never been exercised before. For the underwriter, the case shows that geographic separation protects against a fire or a power cut, not against shared configuration data, which turns a single provider into a global accumulation point.
On June 12, 2025, at around 10:45 a.m. Pacific time, a policy change inserted into Service Control's Spanner tables crashed the service in every Google Cloud region. According to Google's report, the root cause was identified within ten minutes and the emergency kill switch fully rolled out within forty, but the us-central1 region took up to 2 hours 40 minutes longer to recover, because its tasks all restarted at once without randomized backoff and overloaded the database they depended on.
Follow the research
One letter a week. First, the week's underwriting note, when one is published: a dated, argued decision, signed by Paul Bernardski. Then what the observatory published on cyber risk, political risk and insurance. No account, no tracking.
No spam, one-click unsubscribe, no data ever sold.