The system design library
Search and filter every curated system design video, article, repository and paper in one place.
133 resources
Scalable Web Architecture and Distributed Systems
A full chapter on the building blocks of large websites: availability, performance, reliability, scalability, caches, proxies, indexes, load balancers and queues. Free to read here under CC BY 3.0.
BlogsBeginner40 minRead hereFree from the authorBy Kate Matsudaira · The Architecture of Open Source Applications
Security Engineering
How to build systems that stay secure when things go wrong: threat models, protocols, access control and failures. The author publishes the book free online.
BooksAdvancedFree from the authorBy Ross Anderson · University of Cambridge
Site Reliability Engineering
How Google runs production systems: SLOs, error budgets, monitoring and incident response. Free to read online from Google.
BooksIntermediateFree from the authorBy Google SRE team · Google
10 Tips for Building Resilient Payment Systems
Practical rules from Shopify's payments team: idempotency, retries, reconciliation and more.
Case StudiesIntermediateRead hereBy Shopify Engineering · Shopify
Airflow at Adyen
Explore Adyen's journey and challenges in achieving reliable Apache Airflow operations at scale for data engineering.
Case StudiesIntermediateRead hereBy Adyen engineers · Adyen
Anthropic Blog
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
BlogsIntermediateBy Anthropic engineers · Anthropic
Apache Flink: Concepts overview
Stateful stream processing concepts: time, windows, state and fault tolerance.
DocumentationAdvancedBy Apache Flink · Apache Software Foundation
API Platform at Riot Games
Hello all, Leigh Estes, aka RiotSchmick, here. I’m a software engineer at Riot Games working in the Service Availability initiative. Our first article on the API covered our goals, the responsibilities of the Developer Relations and Develo…
Case StudiesIntermediateRead hereBy Riot Games engineers · Riot Games
Automating Datacenter Operations at Dropbox
How Dropbox's Pirlo system automates switch provisioning and server validation in its data centres, raising first-pass success and engineer output.
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox
Autoscaling AWS Step Functions Activities at Yelp
Autoscaling AWS Step Functions Activities Devesh Chourasiya, Jingjing Gu Jun 26, 2019 In an ongoing effort to break down our monolithic applications into microservices here at Yelp, we’ve migrated several...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Autoscaling Bigtable Clusters based on CPU Load at Spotify
At Spotify we are moving to managed infrastructure solutions to focus more on our core competencies. In the databases space that means moving to .
Case StudiesIntermediateRead hereBy Spotify engineers · Spotify
Autoscaling Jenkins at Trivago
I love to take complex and tedious processes and automate the pain out of them until they are reduced to three or four steps!As part of the release team ...
Case StudiesIntermediateBy Trivago engineers · Trivago
Autoscaling Pub-Sub Consumers at Spotify
is responsible for delivering hundreds of billions of events every day. Most of the events are generated as a response to a user action, such as playing a song, following an artist or clicking on an ad. All in all, more than 300 differen…
Case StudiesIntermediateRead hereBy Spotify engineers · Spotify
Availability in Globally Distributed Storage Systems at Google
A real-world engineering write-up from Google on availability in Globally Distributed Storage Systems at Google.
PapersIntermediateBy Google
Avoiding fallback in distributed systems
Why rarely used fallback paths often fail when needed, and what Amazon does instead.
BlogsAdvancedBy Amazon Builders' Library · Amazon Web Services
Avoiding insurmountable queue backlogs
Why queues can accumulate work faster than it can be done, and design choices that keep recovery fast.
BlogsAdvancedBy Amazon Builders' Library · Amazon Web Services
AWS Architecture Blog
Reference architectures and design lessons for building on AWS.
BlogsIntermediateBy AWS architects · Amazon Web Services
AWS Compute Blog
AWS Lambda now exposes Availability Zone metadata through an endpoint in the execution environment, so your function can discover its AZ ID with an HTTP request. Use it to prefer same-AZ endpoints for services like ElastiCache, Amazon RD…
BlogsIntermediateBy Amazon Web Services engineers · Amazon Web Services
Back-end at Flickr
By Peter Welch, Senior Principal Engineer Today, Yahoo Mail introduced a feature that allows you to automatically sync your mobile photos to Yahoo Mail so that they’re readily available when you’re composing an email from your computer. …
Case StudiesIntermediateBy Flickr engineers · Flickr
Bandcamp Engineering Blog
Technical posts by Bandcamp's engineers, such as write-ups of outages, tools and tricky bugs.
BlogsIntermediateBy Bandcamp engineers · Bandcamp
Bank Backend at Monzo
At Monzo, we’re building a banking system from scratch. Our systems must be available 24x7 no matter what happens, scalable to hundreds of millions of customers around the world, and very extensible. This first post in a series about our…
Case StudiesIntermediateRead hereBy Monzo engineers · Monzo
BookKeeper: Distributed Log Storage at Yahoo
Apache Software Foundation recently announced that BookKeeper has been promoted as an Apache Top Level Project. BookKeeper is near and dear to Yahoo: it was actually created here in 2009 to provide horizontally scalable, reliable, replic…
Case StudiesIntermediateBy Yahoo engineers · Yahoo
Brooklin: Distributed Service for Near Real-Time Data Streaming at LinkedIn
Brooklin—a distributed service for streaming data in near real-time and at scale—has been running in production at LinkedIn since 2016, powering thousands of data streams and over 2 trillion messages per day. Today, we are pleased to ann…
Case StudiesIntermediateRead hereBy LinkedIn engineers · LinkedIn
Building Reliable Social Infrastructure for Google - Marc Alvidrez, Senior Manager at Google
Book your room for $225 single or double plus tax or call (800) 255-9925 or (408) 988-8411 and reference USENIX Association or Billing ID #32992. Room rate includes WiFi and complimentary shuttle to the Hyatt Regency Santa Clara.
Case StudiesIntermediateBy Google engineers · Google