System design case studies
How real companies build and scale their systems. Read an in-depth write-up, filter hundreds of engineering case studies, or open a similar design in the Builder.
In-depth write-ups
Each has an architecture diagram, the key components, scaling challenges and trade-offs, videos to watch and reading.
Architecture at a glance
- Service mesh of APIs
- Dynamo / DynamoDB
- SQS / SNS
- Cart & checkout
Architecture at a glance
- Application servers
- Sharded relational database
- Cache tier
- Media storage and CDN
Instagram
A photo and video feed that stays fast as the user base grows by orders of magnitude.
4 components3 challenges
Architecture at a glance
- Edge gateway (Zuul)
- Open Connect CDN
- Microservices
- Cassandra / EVCache
- +1 more
Netflix
Streaming to hundreds of millions of devices, built to survive failure.
5 components3 challenges
Architecture at a glance
- Streaming and CDN layer
- Metadata and catalogue services
- Event pipeline
- Recommendation platform
Spotify
Instant playback and personalised discovery for a global catalogue of audio.
4 components3 challenges
Architecture at a glance
- Dispatch service
- Location pipeline
- Schemaless / Docstore
- Kafka
- +1 more
Architecture at a glance
- Chat servers
- Message queue / store
- Presence service
- Media storage + CDN
Architecture at a glance
- Upload service
- Transcoding workers
- Metadata DB (Vitess)
- CDN / edge caches
- +1 more
439 engineering case studies
Write-ups from the companies themselves, with credit. Pick a company, or use the filters for topic, design area, difficulty and creator.
24 resources
30x Performance Improvements on MySQLStreamer at Yelp
Making 30x performance improvements on Yelp’s MySQLStreamer Abrar Sheikh, Software Engineer Feb 5, 2018 Introduction MySQLStreamer is an important application in Yelp’s Data Pipeline infrastructure. It’s responsible for streaming high-vo…
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Active Directory Password Blacklisting at Yelp
Active Directory Password Blacklisting Leeren Chang, Software Engineer Apr 16, 2018 Many enterprise professionals use passwords that are weak and easily compromised. Equipped with this knowledge, as well as the...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Architecting Restaurant Wait Time Predictions at Yelp
Architecting Restaurant Wait Time Predictions Shweta Joshi, Machine Learning Engineer Dec 12, 2019 Is there a restaurant you’ve always wanted to check out, but haven’t been able to because they...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Autoscaling AWS Step Functions Activities at Yelp
Autoscaling AWS Step Functions Activities Devesh Chourasiya, Jingjing Gu Jun 26, 2019 In an ongoing effort to break down our monolithic applications into microservices here at Yelp, we’ve migrated several...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Black-Box Auditing: Verifying End-to-End Replication Integrity between MySQL and Redshift at Yelp
Black-Box Auditing: Verifying End-to-End Replication Integrity between MySQL and Redshift Jacob Park, Software Engineering Intern Apr 12, 2018 Since Yelp introduced its real-time streaming data infrastructure, “Data Pipeline”, it has...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Clusterman: Autoscaling Mesos Clusters at Yelp
Autoscaling Mesos Clusters with Clusterman David R. Morrison, Compute Infra Tech Lead Feb 19, 2019 Here at Yelp, we host a lot of servers in the cloud. In order to...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Discovering Popular Dishes with Deep Learning at Yelp
Discovering Popular Dishes with Deep Learning Anna F. and Parthasarathy Gopavarapu, Machine Learning Engineers Oct 8, 2019 Introduction Yelp is home to nearly 200 million user-submitted reviews and even more...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Fast Order Search using Data Pipeline and Elasticsearch at Yelp
Fast Order Search Using Yelp’s Data Pipeline and Elasticsearch Dmitriy Kunitskiy, Software Engineer Jun 1, 2018 Since its inception in 2013, Yelp has grown its transactions platform to tens of...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Kafka at Yelp
Billions of Messages a Day - Yelp's Real-time Data Pipeline Justin C., Software Engineer Jul 14, 2016 This post is part of a series covering Yelp's real-time streaming data infrastructure....
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Kafka on PaaSTA: Running Kafka on Kubernetes at Yelp (2 parts)
Kafka on PaaSTA: Running Kafka on Kubernetes at Yelp (Part 2 - Migration) Lennart Rudolph, Software Engineer Mar 3, 2022 In a previous post we detailed the architecture and motivation...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
LogFeeder: Log Collection System at Yelp
Introducing LogFeeder - A log collection system Yonny Tonui, Information Security Engineer Mar 2, 2018 Introduction The collection and processing of logs is essential to good security. One of the...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Machine Learning Platform at Yelp
Introducing Yelp's Machine Learning Platform Jason Sleight, ML Platform Group Tech Lead Jul 1, 2020 Understanding data is a vital part of Yelp’s success. To connect our consumers with great...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Migrating Kafka's Zookeeper with No Downtime at Yelp
Migrating Kafka's Zookeeper With No Downtime Toby Cole, Engineering Manager Jan 17, 2019 Here at Yelp we use Kafka extensively. In fact, we send billions of messages a day through our various...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Minimizing Read-Write MySQL Downtime at Yelp
Minimizing read-write MySQL downtime Nick Del Nano, Database Reliability Engineer Nov 9, 2020 The relational database of choice at Yelp is MySQL and it powers much of the Yelp app...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Moderating Inappropriate Video Content at Yelp
Moderating Inappropriate Video Content at Yelp Prateek Yadav, Machine Learning Engineer Mar 27, 2024 One of Yelp’s top priorities is the trust and safety of our users. Yelp’s platform is...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Moving Core Business Search to Elasticsearch at Yelp
Moving Yelp's Core Business Search to Elasticsearch Umesh Dangat, Software Engineer Jun 29, 2017 While newer search engines at Yelp typically use Elasticsearch as a backend, Yelp’s core business search...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Pagelets Parallelize Server-side Processing at Yelp
Generating Web Pages in Parallel with Pagelets, the Building Blocks of Yelp.com Arnaud Brousseau, Software Engineer Jul 6, 2017 At Yelp, pagelets are a server-side optimization to parallelize the rendering...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Progressive Enhancement with Brotli at Yelp
Progressive Enhancement with Brotli Stephen Arthur, Software Engineer Jul 26, 2017 Brotli is a compression algorithm from Google that has been making waves recently. You may have heard about its...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Scaling Ad Analytics with Cassandra at Yelp
How We Scaled Our Ad Analytics with Apache Cassandra Qui N. and Shafi B., Software Engineers Aug 8, 2016 On the Ad Backend team, we recently moved our ad analytics...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Scaling Gradient Boosted Trees for Click-Through-Rate Prediction at Yelp
Scaling Gradient Boosted Trees for CTR Prediction - Part I Niloy Gupta, Software Engineer - Machine Learning Jan 9, 2018 Building a Distributed Machine Learning Pipeline As a part of...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Scaling Jira Server at Yelp
Jira & Ansible: Scaling Jira Server Administration for the Enterprise Jose Martin de Vidales Biurrun, Application Engineer Apr 3, 2019 In 2017, Yelp had over 40 Jira administrators to allow...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Seagull: Distributed System that Helps Running > 20 Million Tests Per Day at Yelp
How Yelp Runs Millions of Tests Every Day Chunky G., Software Engineer Apr 26, 2017 Fast feature development is critical to a company’s success. We all strive to increase developer...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
Secret Detector: Preventing Secrets in Source Code at Yelp
Yelp's Secret Detector: Preventing Secrets in Source Code Aaron Loo, Engineering Manager Jun 11, 2018 We are always looking for new ways to bolster our security posture to keep our...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp
TTL as a Service: Automatic Revocation of Stale Privileges at Yelp
TTL as a Service: Automatic Revocation of Stale Privileges Aaron Loo, Engineering Manager Nov 19, 2018 Security and usability are often at odds with one another, a fact that is...
Case StudiesIntermediateRead hereBy Yelp engineers · Yelp