Google Cloud has revealed that the cause of the October 20 outage in its europe-west-2-b availability zone was a ‘misconfiguration’ of its systems that meant its emergency generators didn’t start.
The outage at the London facility caused service interruptions at outfits including Lloyds, HSBC, Royal Mail, and Revolut.
In a November 4 update to its service status page, Google explained what went wrong.
“The site relies on a 2N utility power architecture — where the facility has complete duplicate power paths for every critical load — backed by a central high-voltage generator bank managed by a dual-redundant Programmable Logic Controller (PLC),” Google revealed.
PLC misconfiguration prevented the facility’s generator plant from receiving an automatic start signal
The facility gets its juice from two independent utility feeders, which Google calls “A” and “B”. There’s also a “standby utility connection configured for manual campus-wide transfer during maintenance operations or emergency events.”
Google says the root cause of the incident was failure in an external underground cable through which its power supplier brings ‘leccy to its bit barns.
That cable connects to primary feeder A and it broke at a time when feeder B wasn’t working.
“Circuit breaker feeder B was physically opened to allow for corrective repairs following issues identified during a quarterly generator load test on 14 Oct 2026,” Google admitted. “Consequently, primary utility feed A was supporting the total critical load while the feed B breaker remained in an open position to allow repairs to be conducted on the B feeder pathway.”
That shouldn’t be a problem for a top-tier operator like Google, which knows it needs to install backup sources of electricity like uninterruptible power supplies (UPS) and on-site generators.
Indeed, europe-west-2-b employs UPSes that Google says can keep things up and running for five minutes if the availability zone loses its connection to the grid.
The UPSes worked, but a “voltage-sensing design flaw and PLC misconfiguration prevented the facility’s generator plant from receiving an automatic start signal to assume the critical load.”
Google’s UPSes therefore ran out of juice, but its generators didn’t start. The rest is history.
The search giant has figured out that its PLCs weren’t set up to handle the conditions it faced on this fateful day.
“PLC logic indicated that a healthy utility source was available, even though the primary and secondary distribution feeds were de-energized, and did not issue a start command to the generator plant,” the status update reveals.
Once Google was able to reconnect to its power utility, it reprogrammed the PLC so it wouldn’t make the same mistake again.
The Big G has since made many changes to the electrical setup at europe-west-2-b and says it’s tested them and that they all work.
It’s also given itself two action items, one of which it describes as a “Communications Enhancement” that will see it “bolster notification speed by developing an automated alert-to-communication mechanism for location-impacting incidents.”
The other is to “expand platform-wide testing outcomes for infrastructure-level events.”
Once Google gets that done, perhaps it will be able to handle outages better – and if not, at least advise customers about its troubles more quickly. ®