The system design library
Search and filter every curated system design video, article, repository and paper in one place.
24 resources
Athena: Automated Build Health Management System at Dropbox
At Dropbox, we run more than 35,000 builds and millions of automated tests every day. With so many tests, a few are bound to fail non-deterministically or “flake.” Some new code submissions are bound to break the build, which prevents de…
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox
Automating Datacenter Operations at Dropbox
How Dropbox's Pirlo system automates switch provisioning and server validation in its data centres, raising first-pass success and engineer output.
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox
Cannes: ML saves $1.7M a year on document previews at Dropbox
Recently, we translated the predictive power of machine learning (ML) into $1.7 million a year in infrastructure cost savings by optimizing how Dropbox generates and caches document previews. Machine learning at Dropbox already powers co…
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox
Cape: Event Stream Processing Framework at Dropbox
We launched Cape towards the end of 2016 to replace Livefill, which we last talked about in 2014. In this post, we’ll give an overview of Cape and its key features, high-level architecture, and varied use cases at Dropbox.
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox
Continuous Integration and Deployment with Bazel at Dropbox
Dropbox server-side software lives in a large monorepo. One lesson we’ve learned scaling the monorepo is to minimize the number of global operations that operate on the repository as a whole. Years ago, it was reasonable to run our entir…
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox
Cross Shard Transactions at 10 Million RPS at Dropbox
How Dropbox added cross-shard transactions to Edgestore, its metadata store, with a modified two-phase commit proven on a live system at 10 million requests a second.
Case StudiesAdvancedRead hereBy Dropbox engineers · Dropbox
Data Checking at Dropbox
A SREcon talk on designing consistency checkers that verify data at rest: goals, components, alert triage and auto-remediation.
Case StudiesIntermediateBy Dropbox engineers · Dropbox
Deploying Brotli for Static Content at Dropbox
Most representations of data contain a lot of redundancy, which provides an opportunity for greater communication efficiency by compressing the content. Compression is either built-in into the data format — like in the case of images, fo…
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox
Dropbox Engineering Blog
Dropbox's engineering blog on infrastructure, machine learning, mobile, security and front end.
BlogsIntermediateBy Dropbox engineers · Dropbox
Dropbox Tech Blog
Infrastructure, storage and sync write-ups from Dropbox.
BlogsIntermediateBy Dropbox engineers · Dropbox
Finding Kafka throughput limit in infrastructure at Dropbox
How Dropbox's Jetstream team built an automated platform to measure the maximum throughput of its Kafka clusters, finding about 60 MB/s per broker.
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox
Inside the Magic Pocket
How Dropbox built its own exabyte-scale block storage, including durability and cost choices.
Case StudiesAdvancedRead hereBy Dropbox
Intelligent DNS based load balancing at Dropbox
The Dropbox Traffic team is charged with innovating our application networking stack to improve the experience for every one of our users—over half a billion of them. This article describes our work with NS1 to optimize our intelligent D…
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox
Machine Learning for Indexing Text from Billions of Images at Dropbox
In our previous blog posts, we talked about how we updated the Dropbox search engine to add intelligence into our users’ workflow, and how we built our optical character recognition (OCR) pipeline. One of the most impactful benefits that…
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox
Messaging System Model at Dropbox
The MSM helped us build a unified event-driven system capable of orchestrating a wide range of asynchronous tasks and meeting future needs, especially as we focus on AI.
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox
Nautilus: Search Engine at Dropbox
Over the last few months, the Search Infrastructure engineering team at Dropbox has been busy releasing a new full-text search engine called Nautilus, as a replacement for our previous search engine.
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox
Optimizing Payments with Machine Learning at Dropbox
It’s probably happened to you at some point: You go to use a service for which you believe you’ve got a paid subscription, only to find that it’s been canceled for non-payment. That’s not only bad for you the customer: It causes negative…
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox
Optimizing Web Servers for High Throughput and Low Latency at Dropbox
Dash knows your context, your team, and your work, so your team can stay organized, easily find and share knowledge, and keep projects secure, all from one place. And soon, Dash is coming to Dropbox.
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox
Rewriting the heart of our sync engine
Why and how Dropbox rewrote its file-sync engine for correctness, with testing at the centre.
Case StudiesAdvancedRead hereBy Dropbox
Scaling Dropbox - Kevin Modzelewski, Back-end Engineer at Dropbox
(Feburary 22, 2012) Kevin Modzelewski talks about Dropbox and its History. He describes the technological issues faced by Dropbox and the actions they have t...
YouTubeIntermediateBy Dropbox
Site Reliability Engineering at Dropbox - Tammy Butow, Site Reliability Engineering Manager at Dropbox
Tammy Butowhttps://linux.conf.au/schedule/30330/view_talkTammy is a Site Reliability Engineering Manager at Dropbox.Dropbox is the home for your most importa...
YouTubeIntermediateBy Dropbox
Sync Engine (2 parts) at Dropbox
Executing a full rewrite of the Dropbox sync engine was pretty daunting. (Read more about our goals and how we made the decision in our previous post here.) Doing so meant taking the engine that powers Dropbox on hundreds of millions of …
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox
Traffic Infrastructure (Edge Network) at Dropbox
As you can see, by simply putting the PoP close to the user one can improve latency by more than factor of two.
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox
Vortex: Monitoring Server Applications at Dropbox
Dropbox's redesigned monitoring system for server applications, Vortex: its goals, architecture, query language and scale.
Case StudiesIntermediateRead hereBy Dropbox engineers · Dropbox