System design case studies
How real companies build and scale their systems. Read an in-depth write-up, filter hundreds of engineering case studies, or open a similar design in the Builder.
In-depth write-ups
Each has an architecture diagram, the key components, scaling challenges and trade-offs, videos to watch and reading.
Architecture at a glance
- Service mesh of APIs
- Dynamo / DynamoDB
- SQS / SNS
- Cart & checkout
Architecture at a glance
- Application servers
- Sharded relational database
- Cache tier
- Media storage and CDN
Instagram
A photo and video feed that stays fast as the user base grows by orders of magnitude.
4 components3 challenges
Architecture at a glance
- Edge gateway (Zuul)
- Open Connect CDN
- Microservices
- Cassandra / EVCache
- +1 more
Netflix
Streaming to hundreds of millions of devices, built to survive failure.
5 components3 challenges
Architecture at a glance
- Streaming and CDN layer
- Metadata and catalogue services
- Event pipeline
- Recommendation platform
Spotify
Instant playback and personalised discovery for a global catalogue of audio.
4 components3 challenges
Architecture at a glance
- Dispatch service
- Location pipeline
- Schemaless / Docstore
- Kafka
- +1 more
Architecture at a glance
- Chat servers
- Message queue / store
- Presence service
- Media storage + CDN
Architecture at a glance
- Upload service
- Transcoding workers
- Metadata DB (Vitess)
- CDN / edge caches
- +1 more
439 engineering case studies
Write-ups from the companies themselves, with credit. Pick a company, or use the filters for topic, design area, difficulty and creator.
13 resources
Athenz: Fine-Grained, Role-Based Access Control at Yahoo
yahoocs: By Lee Boynton, Henry Avetisyan, Ken Fox, Itsik Figenblat, Mujib Wahab, Gurpreet Kaur, Usha Parsa, and Preeti Somal Today, we are pleased to offer Athenz, an open-source platform for fine-grained access control, to the community…
Case StudiesIntermediateBy Yahoo engineers · Yahoo
Benchmarking Streaming Computation Engines at Yahoo
(Yahoo Storm Team in alphabetical order) Sanket Chintapalli, Derek Dagit, Bobby Evans, Reza Farivar, Tom Graves, Mark Holderbaugh, Zhuo Liu, Kyle Nusbaum, Kishorkumar Patil, Boyang Jerry Peng and Paul Poulosky. DISCLAIMER: Dec 17th 2015 …
Case StudiesIntermediateBy Yahoo engineers · Yahoo
BookKeeper: Distributed Log Storage at Yahoo
Apache Software Foundation recently announced that BookKeeper has been promoted as an Apache Top Level Project. BookKeeper is near and dear to Yahoo: it was actually created here in 2009 to provide horizontally scalable, reliable, replic…
Case StudiesIntermediateBy Yahoo engineers · Yahoo
Bullet: Forward-Looking Query Engine for Streaming Data at Yahoo
A real-world engineering write-up from Yahoo on distributed Messaging, Queuing, and Event Streaming.
Case StudiesIntermediateBy Yahoo engineers · Yahoo
CaffeOnSpark: Distributed Deep Learning on Big Data Clusters at Yahoo
yahoohadoop: By Andy Feng(@afeng76), Jun Shi and Mridul Jain (@mridul_jain), Yahoo Big ML Team Introduction Deep learning (DL) is a critical capability required by Yahoo product teams (ex. Flickr, Image Search) to gain intelligence from …
Case StudiesIntermediateBy Yahoo engineers · Yahoo
Cloud Bouncer: Distributed Rate Limiting at Yahoo
Most platforms at Yahoo serve dozens of Yahoo Properties and Apps at any given time (think Mail, Finance, Flickr etc.) and are operated as multi-tenant services on shared infrastructure for cost efficiency and on demand elasticity. As su…
Case StudiesIntermediateBy Yahoo engineers · Yahoo
Cloud Object Store at Yahoo
Yahoo stores more than 250 Billion objects and half an exabyte of perpetually durable user content such as photos, videos, email, and blog posts. Object storage at Yahoo is growing at 20-25% annually. The growth is primarily driven by mo…
Case StudiesIntermediateBy Yahoo engineers · Yahoo
HaloDB: Embedded Key-Value Storage Engine at Yahoo
yahoodevelopers: By Arjun Mannaly, Senior Software Engineer At Oath, multiple ad platforms use a high throughput, low latency distributed key-value database that runs in data centers all over the world. The database stores billions of re…
Case StudiesAdvancedBy Yahoo engineers · Yahoo
NodeJS High Availability at Yahoo
Do you have a NodeJS application serving production traffic? Ever wonder how Yahoo might handle this? If yes, at some point or other you must have thought it would be nice to have a clean and simple solution for high availability. Read b…
Case StudiesIntermediateBy Yahoo engineers · Yahoo
Omid: Transaction Processing Platform at Yahoo
yahoodevelopers: By Ohad Shacham, Yonatan Gottesman, Edward Bortnikov Scalable Systems Research, Verizon/Oath Omid, an open source transaction processing platform for Big Data, was born as a research project at Yahoo (now part of Verizon…
Case StudiesAdvancedBy Yahoo engineers · Yahoo
Screwdriver: Continuous Delivery Build System for Dynamic Infrastructure at Yahoo
By James Collins, Sr. Director, Developer Platforms and Services, and St. John Johnson, Principal Engineer Continuous Delivery enables software development teams to move faster and adapt to users’ needs quicker by reducing the inherent f…
Case StudiesIntermediateBy Yahoo engineers · Yahoo
Sherpa: Distributed NoSQL Key-Value Store at Yahoo
Overview Sherpa is Yahoo’s distributed NoSQL key-value store offered as a hosted service. It is the cornerstone of Yahoo properties and hosts over 2,000 tables, 1 trillion records, and serves over 1 million queries a second across a wide…
Case StudiesIntermediateBy Yahoo engineers · Yahoo
TensorFlowOnSpark: Distributed Deep Learning on Big Data Clusters at Yahoo
yahoohadoop: By Lee Yang, Jun Shi, Bobbie Chern, and Andy Feng (@afeng76), Yahoo Big ML team Introduction Today, we are pleased to offer TensorFlowOnSpark to the community, our latest open source framework for distributed deep learning o…
Case StudiesIntermediateBy Yahoo engineers · Yahoo