The system design library
Search and filter every curated system design video, article, repository and paper in one place.
31 resources
Concurrent and Distributed Systems (Cambridge lecture notes)
Eight lectures on time, broadcast, replication, consensus and transactions, with notes, slides and video. Free from the author under CC BY-SA.
BooksIntermediateFree from the authorBy Martin Kleppmann · University of Cambridge
Distributed Systems for Fun and Profit
A short, readable tour of time, consistency and replication in distributed systems, free to read online from the author.
BooksBeginnerFree from the authorBy Mikito Takada · mixu.net
Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases (paper)
Why Aurora moves the redo log to a distributed storage layer, so replication and crash recovery become cheap.
PapersAdvancedRead hereBy Verbitski et al. · Amazon Web Services
Apache Cassandra
The source of the wide-column, leaderless database.
GitHubAdvancedBy Apache Software Foundation · GitHub
AWS Database Blog
Guidance on choosing, migrating and tuning relational and NoSQL databases.
BlogsIntermediateBy AWS database teams · Amazon Web Services
Black-Box Auditing: Verifying End-to-End Replication Integrity between MySQL and Redshift at Yelp
Black-Box Auditing: Verifying End-to-End Replication Integrity between MySQL and Redshift Jacob Park, Software Engineering Intern Apr 12, 2018 Since Yelp introduced its real-time streaming data infrastructure, “Data Pipeline”, it has...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Cassandra architecture overview
The ring, replication, tunable consistency and the storage engine of Cassandra.
DocumentationIntermediateBy Apache Cassandra · Apache Software Foundation
Cockroach Labs Blog
Distributed SQL internals: consensus, transactions and multi-region design.
BlogsAdvancedBy Cockroach Labs engineers · Cockroach Labs
CockroachDB
A distributed SQL database built on ranges, Raft and MVCC.
GitHubAdvancedBy Cockroach Labs · GitHub
Designing Data-Intensive Applications
The standard book on data systems: storage engines, replication, partitioning, transactions, batch and stream processing.
BooksIntermediateSold by publisherBy Martin Kleppmann · O'Reilly
High-performance Replicated Log Service at Twitter
Building DistributedLog: High-performance replicated log service
Case StudiesIntermediateBy Twitter engineers · Twitter
In Search of an Understandable Consensus Algorithm (Raft paper)
The Raft paper: leader election, log replication and safety explained so that engineers can implement them.
PapersAdvancedBy Ongaro and Ousterhout · Stanford University
Inside the Magic Pocket
How Dropbox built its own exabyte-scale block storage, including durability and cost choices.
Case StudiesAdvancedRead hereBy Dropbox
Megastore: Providing Scalable, Highly Available Storage (paper)
Synchronous replication across data centres with Paxos while keeping a transactional data model.
PapersAdvancedBy Baker et al. · Google
Mitigating MySQL Replication Lag and Reducing Read Load at Github
At GitHub, we use MySQL as the main database technology backing our services.
Case StudiesIntermediateRead hereBy Github engineers · Github
MySQL High Availability at GitHub
How GitHub detects primary failure and promotes a replica safely using orchestration and service discovery.
Case StudiesAdvancedRead hereBy GitHub Engineering · GitHub
Patterns of Distributed Systems
A catalogue of named building blocks (replicated log, lease, quorum, gossip, high-water mark) found in systems like Kafka and Cassandra.
BlogsAdvancedBy Unmesh Joshi · Martin Fowler
Read Consistency with Database Replicas at Shopify
At Shopify, we’ve long used database replication for redundancy and failure recovery, but only fairly recently started to explore the potential of replicas as an alternative read-only data source for applications.
Case StudiesIntermediateRead hereBy Shopify engineers · Shopify
Spanner: Google's Globally Distributed Database (paper)
A globally distributed database that gives strongly consistent transactions by combining consensus groups with clocks whose uncertainty is bounded.
PapersAdvancedBy Corbett et al. · Google
The Google File System (paper)
How Google built a file system for huge files and constant machine failures: one master for metadata, many chunk servers, and append-heavy workloads. The ancestor of HDFS.
PapersAdvancedBy Ghemawat, Gobioff and Leung · Google
Types of Replication - SQL Server
Learn about the different types of replication that SQL Server provides for use in distributed applications.
Case StudiesIntermediateBy Microsoft engineers · Microsoft
ZippyDB: Facebook's key-value store
Facebook's strongly consistent key-value store built on RocksDB and Paxos, with data sharded and replicated.
Case StudiesAdvancedRead hereBy Meta Engineering · Meta
Active-Active for Multi-Regional Resiliency
How Netflix routes traffic across regions and survives regional outages.
Case StudiesAdvancedBy Netflix Engineering · Netflix
Uber Schemaless: A Scalable MySQL Datastore
How Uber built an append-only, sharded datastore on top of MySQL.
Case StudiesAdvancedRead hereBy Uber Engineering · Uber