Rethinking Distributed Systems for the Datacenter

Play Rethinking Distributed Systems for the Datacenter
Sign in to queue

Description

Today's most popular applications are deployed as massive-scale distributed systems in the datacenter. Keeping data consistent and available despite server failures and concurrent updates is a formidable challenge. Two well-known abstractions, strongly consistent replication and serializable transactions, can free developers from these challenges by transparently masking failures and treating complex updates as atomic units. Yet the conventional wisdom is that these techniques are too expensive to deploy in high-performance systems.

I will demonstrate a new approach to designing distributed systems that allows strongly consistent distributed systems to be built with little to no performance cost. Taking advantage of the properties and capabilities of the datacenter environment, we can co-design distributed protocols and the network layer. Specifically, I will
describe two systems for state machine replication, Speculative Paxos and Network-Ordered Paxos, and one for distributed transaction processing, Eris, built using this approach. They are able to achieve 5-17x performance improvements over conventional designs. Moreover, they achieve performance within 2% of their weakly consistent alternatives, demonstrating that strong consistency and high performance are not incompatible.

Embed

Download

Download this episode

The Discussion

Add Your 2 Cents