Presentation: The Anatomy of a Distributed System
Abstract
Much has been written and said about distributed systems of many different sizes, scales, and complexities. Most of the time though, we either talk about algorithms and theoretical ideas or entirely concrete specific systems: Abstract or Concrete. Gossip, failure detection, and leader election are discussed as if they’re a magical salve that can be rubbed onto a project to make it distributed. But the practical realities of distributed systems are far from straightforward.
In this talk, I will take a new approach, talking through the components and design of a real system, as well as the theory behind them. The system we’ll discuss is built to perform very high volumes of health checks, done across a cluster of machines for reliability and scalability. It’s complex enough that we need to worry about things like gossip, failure detection, leader election, logical clocks, and consistency tradeoffs. We’ll walk through each of the major components, see how they are practically built, and learn about the pain and compromises that each bring.
The audience will leave the talk with a better understanding of how the basic components fit together in a real-world system. With that knowledge, they will be better equipped to approach such projects themselves with confidence.
Similar Talks
Evolution of Edge @Netflix
Engineering Leader @Netflix
Vasily Vlasov
Future of Data Engineering
Distinguished Engineer @WePay
Chris Riccomini
Observability in the Development Process: Not Just for Ops Anymore
Cofounder @honeycombio
Christine Yen
Data Mesh Paradigm Shift in Data Platform Architecture
Principal Technology Consultant @ThoughtWorks
Zhamak Dehghani
Scaling Patterns for Netflix's Edge
Playback Edge Engineering @Netflix
Justin Ryan
Architectures Panel
Playback Edge Engineering @Netflix
Justin Ryan
Secrets at Planet-Scale: Engineering the Internal Google KMS
Software Developer @Google
Anvita Pandit
Evolutionary Architecture as Product @ CircleCI
CTO @CircleCI
Robert Zuber
Architectures That Scale Deep - Regaining Control in Deep Systems
CEO and co-founder @LightStepHQ, Co-creator @OpenTracing API standard