Meta has commenced deploying MySQL Raft at its data centers to replace its existing MySQL semisynchronous databases. The new consensus engine offers operational benefits and allows MySQL servers to handle promotions and membership responsibilities independently.
Meta's MySQL datastore is one of the largest MySQL deployments in the world, featuring a massively sharded, geo-replicated structure with millions of shards. This cluster, operating across various regions and data centers, stores petabytes of data and runs on thousands of servers. It supports the company's social graph and a range of services such as Messaging, Ads, and Feed.
According to the engineering team, the new MySQL deployment will deliver enhanced reliability, provable safety, considerable improvements in failover time, and operational simplicity, without compromising write performance.
Previously, Meta used the MySQL semisynchronous (semisync) replication protocol for replication purposes. The primary used semi-synchronous replication to two log-only replicas (logtailers) within the primary region for sub-millisecond latency, employing standard MySQL primary-to-replica asynchronous replication for distribution to other regions.
The team decided to adopt a completely new approach, upgrading MySQL and transforming it into a truly distributed system. Meta chose to switch to Raft, with both control plane and data plane operations included in the same replicated log.
MySQL Raft is based on Apache Kudu, which Meta modified to meet MySQL's requirements and published a fork as an open-source project, kuduraft. New features added to kuduraft include FlexiRaft, an option that supports two different intersecting quorums, and proxying, which enables the use of a proxy intermediate node to reduce network bandwidth. Additionally, the compression and log abstraction enhancements allow for the compression of binary log payloads prior to distribution and the implementation of various physical logfile formats.
With platforms such as AppMaster providing powerful no-code tools to create backend, web, and mobile applications, simplified database solutions are essential for efficient and scalable software development. By transitioning to MySQL Raft, Meta has made a significant step towards achieving improved reliability and streamlined operations, which could inspire other companies to adopt similar solutions and enhance their own database systems.
