Library / Papers
Facebook’s photo storage
This paper describes Haystack, an object storage system optimized for Facebook’s Photos application. Facebook currently stores over 260 billion images, which translates to over 20 petabytes of data.
By USENIX · USENIX · Added 2026-10-12
All content belongs to USENIX. We only link to it and give credit. Report a problem
Study path: How big storage systems are builtFrom spreading keys over servers to a database built for the cloud.
0 of 7 finished- 1Consistent Hashing and Random Trees (paper)
- 2CAP Twelve Years Later: How the Rules Have Changed
- 3Amazon Dynamo: Highly Available Key-Value Store
- 4Bigtable: A Distributed Storage System for Structured Data (paper)
- 5Cassandra: A Decentralized Structured Storage System (paper)
- 6Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases (paper)
- 7Facebook’s photo storage
Learn this paper
About 37 min left. Your work is saved as you go.
What you are about to study
USENIX wrote this paper. Read the abstract and headings first. You should know the basics: caching, queues and replication.
Ideas you will meet
- Distributed Systems
- Consensus, time, failure and the fundamentals of scale-out.
Go deeper on this
A path through videos, articles, other case studies and code on the same ideas as Facebook’s photo storage.
Warm upGet the vocabulary first.
Books · Free from the author
Concurrent and Distributed Systems (Cambridge lecture notes)
Eight lectures on time, broadcast, replication, consensus and transactions, with notes, slides and video. Free from the author under CC BY-SA.
University of CambridgeIntermediate
Books · Free from the author
Computer Networks: A Systems Approach
How networks are built and why: protocols, routing, congestion control and the internet's architecture. The authors publish it free online under CC BY 4.0.
Systems ApproachIntermediate
WatchHow others solve the same problem, on video.
Videos belong to their creators and play from YouTube. More on YouTube (opens in a new tab)
ReadEngineers explain it in their own words.
Blogs · Free from the author · Read here
Scalable Web Architecture and Distributed Systems
A full chapter on the building blocks of large websites: availability, performance, reliability, scalability, caches, proxies, indexes, load balancers and queues. Free to read here under CC BY 3.0.
The Architecture of Open Source ApplicationsBeginner40 min
Papers · Read here
MapReduce: Simplified Data Processing on Large Clusters (paper)
The programming model that made it routine to process huge datasets on thousands of ordinary machines by splitting work into map and reduce steps.
GoogleIntermediate
See it in another companyThe same ideas, a different system.
Case Studies · Read here
Aroma: Using ML for Code Recommendation at Facebook
Thousands of engineers write the code to create our apps, which serve billions of people worldwide. This is no trivial task—our services have grown so diverse and complex that the codebase contains…
FacebookIntermediate
Case Studies · Read here
Brooklin: Distributed Service for Near Real-Time Data Streaming at LinkedIn
Brooklin—a distributed service for streaming data in near real-time and at scale—has been running in production at LinkedIn since 2016, powering thousands of data streams and over 2 trillion messages per day. Today, w...
LinkedInIntermediate
Read the codeSee how it looks when it is real.
GitHub
Engineering Blogs
A huge directory of company engineering blogs to learn from real production systems.
kilimchoiIntermediate
GitHub
Awesome Scalability
Curated reading list of articles on the architecture of scalable, reliable systems.
binhnguyennusIntermediate
Keep learning
Loading community…