A large number of inbound transactions and heavy traffic caused the Solana network to be suspended on the night of April 30-May 1. This was reported by the developers following the results of the investigation.

On April 30, Solana’s Mainnet Beta cluster suffered a 7-hour outage due to stalled consensus. Below is an investigation into the cause of the outage and the steps core developers have taken to improve network resilience and stability. https://t.co/pfkMTEwPda

— Solana Status (@SolanaStatus) May 3, 2022

Network downtime was linked to the activity of bots competing to print NFTs released using the Candy Machine tool.

The unwanted transactions resulted from the Dutch auction being abandoned in favor of selling at a fixed price. This created an incentive for users and bots to submit large numbers of transactions in hopes of getting ahead of the competition.

Today #fading mainnet-beta partially crashed due to bots in Metaplex Candy Machine. We have united to combat this and will soon impose a botched penalty on the program as part of a wider effort to stabilize the network. https://t.co/QaAZT3VxXz

— Metaplex (@metaplex) May 1, 2022

As a result, the validators ran out of memory and crashed. The total number of forks validators need to evaluate has exceeded their ability to do this automatically. This led to the need for manual intervention to fix the problem.

“TPS exceeded 100 Gb/s traffic for individual nodes, reaching 6 million. No signs of DoS attacks detected”, says report.

On average, the Solana network handles approximately 2,700 TPS with an upper limit of over 710,000 TPS.

The incident that occurred on the night of May 1 was of the same nature as the malfunction on September 14, 2021.

In order to prevent the recurrence of such situations in the future, the developers are preparing the v1.10 update. It includes improvements to the memory usage algorithm that will extend the time that nodes can maintain slow or stalled consensus.

In the next version v 1.11, it is planned to introduce a commission prioritization mechanism.

Recall that at the beginning of the year, the Solana network was unavailable for 48 hours. At that time, the developers did not rule out that the problem may recur in the future. The event that occurred at that time was the sixth of the previous three months.