Paxos Made Parallel Zhenyu Guo, Chuntao Hong, Mao Yang, Dong Zhou, Lidong Zhou and Li Zhuang Microsoft Research Asia Abstract Standard state-machine replication involves consensus on a sequence of totally ordered requests through, for ex- ample, the Paxos protocol. Serialized request processing seriously limits our ability to leverage prevalent multi- core servers. This tension between concurrency and con- sistency is not inherent because the total-ordering of re- quests is merely a simplifying convenience that is un- necessary for consistency. Replicated state machines can be made parallel by consensus on partial-order traces, rather than on totally ordered requests, that capture causal orders in one replica execution and to be replayed in an order-preserving manner on others. The result is a new multi-core friendly replicated state-machine frame- work that achieves strong consistency while preserving parallelism in multi-threaded applications. On 12-core machines with hyper-threading, evaluations on typical applications show that we can scale with the number of cores, achieving up to a 16 times throughput increase over standard replicated state machines. 1 I NTRODUCTION Server applications that power on-line services typically run on a cluster of multi-core commodity servers. These applications must leverage an underlying multi-core ar- chitecture for highly concurrent request-processing, but must also often resort to replication to guard against the failure of commodity components. To guarantee consis- tency, replicas in the standard replicated state-machine approach [26, 36] start from the same initial state and process the same set of requests in the same total or- der deterministically. They use a consensus protocol to agree on each request in a total order that implies se- quential request processing, leading to performance loss on a multi-core machine as many requests are in practice non-conflicting and can be processed concurrently. We present Tribble, a new replication framework that maintains replica consistency while preserving concur- rency in request processing, thereby refuting the miscon- ception that serialized execution is necessary for state- machine replication. Tribble processes multiple requests concurrently on replicas; application writers provide pro- cessing logic that uses a set of synchronization primitives (e.g., locks and semaphores) for coordinating concurrent access to shared data. Tribble achieves replica consis- tency with the following key techniques. First, Tribble has a primary replica process requests concurrently, with others as secondary replicas that concurrently replay the primary execution. Second, Tribble uses a consensus protocol, such as Paxos, to maintain consistency even when replicas fail and the primary changes. Rather than agreeing on a sequence of total-ordering requests, Trib- ble replicas agree on a partially ordered trace that cap- tures causal dependencies among synchronization events that occur in request processing. Replay of the trace on a secondary preserves these causal dependencies and the replica reaches the same consistent state as the primary, while still allowing for concurrency during replay. Tribble is the first practical replication framework that approaches the ideal of achieving both high re- liability through Paxos and high performance through concurrent multi-threaded execution. We design Trib- ble to be the new “replicated state-machine” approach that is multi-core friendly. To evaluate whether Tribble preserves concurrency in request processing under var- ious circumstances and to understand its overhead, we have further developed a set of micro-benchmarks and identified several representative applications, including a global lock service, a thumbnail service, a key/value store, a simple file system, and Google’s LevelDB. Our experiments on 12-core servers with hyper-threading have shown that applications achieve as high as 16 times the throughput on Tribble when compared to standard replicated state-machine. The rest of the paper is organized as follows. Sec- tion 2 presents an architectural overview of Tribble. Sec- tion 3 details how Tribble uses replica consensus to agree on a trace. Section 4 presents how Tribble captures causal orders in traces during execution and replays execution while respecting the causal orders. We discuss state di- vergence and Tribble application scope in Section 5. Sec- tion 6 presents experimental results on both our micro- benchmarks and representative applications. We survey related work in Section 7 and conclude in Section 8. 2 OVERVIEW We consider on-line service applications that serve in- coming client requests with request handlers and a thread pool with a fixed number of threads to process requests. In contrast to standard state-machine replica- tion, where totally ordered requests execute sequentially, request handlers in Tribble are executed concurrently us-