Polygon Network Experiences Outage After Upgrade

Layer-2 Ethereum scaling solution, Polygon, suffered an outage for over 11 hours resulting from an issue with one of its nodes.

On March 10, Polygon, a leading sidechain-based scaling solution for Ethereum, announced that its network had been experiencing downtime since 5:50pm UTC as a result of its Heimdall node halting. The network developers notified users on the project’s forum that there would likely be downtime due to maintenance required on one of the network’s three layers. However, a recent upgrade is thought to have caused an error in the network’s ability to achieve consensus.

?Network update
Due to issues with the Tendermint implementation used by the Polygon PoS chain, users can expect downtime starting at 5:50 PM UTC.

We appreciate your patience. More details are in the forum and we’ll share updates when possible. ?https://t.co/O1MFUrKuAr

— Polygon Developers ? (@0xPolygonDevs) March 10, 2022

What Caused The Outage?

According to the official blog post, the outage seemed to have originated from an earlier upgrade consisting of a minor parameterization fix to the Ethereum to Polygon PoS state sync/bridging module. The developers emphasized that there may have been a bug in the upgrade which affected consensus, and caused different Heimdall validators to be on different versions of the chain, thereby not reaching 2/3 consensus. When using Tendermint consensus, this situation will cause the Heimdall chain to halt.

The blog post also stated that the Bor chain relies on Heimdall for block proposer committee selection, specifically span creation. So once the last span that Heimdall specifies passes, the Bor chain also halts. Bor is the block producer layer of the PoS Chain architecture that is responsible for aggregating transactions into blocks.

The team went on to add that Heimdall does not handle user transactions. It is used for validator related transactions and bridgingall user funds and data remain fully secure. The team assured that all user funds and data remain fully secure. Devs at Polygon Network remarked,

“At that time, the Bor chain (or the user-facing Polygon PoS chain) will then only be halted, and while there is a liveness issue, there are no security or safety issues. Meaning that while there will be downtime, state and funds are not affected.”

Earlier Issue Faced By Polygon Network

Polygon has recently come under severe criticisms for suffering multiple outages since the start of the year. Last month, transactions were halted as a result of a gas-station outage. Prior to that, an issue with gas estimation resulted in users being unable to transact using NFTs on the network for several days in January. Polygon developers had said that the network was facing difficulties because there was a high level of activity from bots farming on some decentralised finance (DeFi) applications.