The system design library
Search and filter every curated system design video, article, repository and paper in one place.
24 resources
Brooklin: Distributed Service for Near Real-Time Data Streaming at LinkedIn
Brooklin—a distributed service for streaming data in near real-time and at scale—has been running in production at LinkedIn since 2016, powering thousands of data streams and over 2 trillion messages per day. Today, we are pleased to ann…
Case StudiesIntermediateRead hereBy LinkedIn engineers · LinkedIn
Concourse: Generating Personalized Content Notifications in Near-Real-Time at LinkedIn
Social media are computer-mediated platforms that facilitate creation and sharing of information, ideas, career interests, and other forms of expression via virtual communities and networks. In the last few years, notifications have been…
Case StudiesIntermediateRead hereBy LinkedIn engineers · LinkedIn
Druid - Real-time Analytics Database
Apache Druid, a real-time analytics database: the project site and its documentation.
DocumentationIntermediateBy Apache engineers · Apache
Erlang at Facebook (slides)
How Facebook's chat backend used Erlang processes to hold huge numbers of connections and route messages.
Case StudiesIntermediateBy Eugene Letuchy · Facebook
High Performance Browser Networking
How TCP, TLS, HTTP/2, WebSocket and mobile networks affect web performance; free to read.
BooksIntermediateFree from the authorBy Ilya Grigorik · O'Reilly
How Discord Scaled Elixir to 5,000,000 Concurrent Users
How a chat service handles millions of concurrent connections with Erlang/Elixir processes and consistent hashing.
Case StudiesIntermediateRead hereBy Discord
How Uber Scales Their Real-time Market Platform - High Scalability -
Reportedly Uber has grown an astonishing 38 times bigger in just four years . Now, for what I ...
Case StudiesIntermediateRead hereBy High Scalability engineers · High Scalability
MDN: WebRTC connectivity
ICE, STUN, TURN and how peers find a path through NATs.
DocumentationIntermediateBy MDN contributors · MDN Web Docs
MemSQL and Kinesis for Real-time Insights at Disney
We’ve made the very difficult decision to cancel all future O’Reilly in-person conferences. Instead, we’ll continue to invest in and grow O’Reilly online learning, supporting the 5,000 companies and 2.5 million people who count on our ex…
Case StudiesIntermediateBy Disney engineers · Disney
ML Data Pipelines for Real-Time Fraud Prevention at PayPal
Mikhail Kourjanski focuses on the architectural approach towards PayPal’s internally built real-time service platform that leverages Machine Learning models and delivers performance and quality of decisions. This platform has established…
Case StudiesIntermediateBy PayPal engineers · PayPal
Real-Time Data Warehouse with MemSQL on Cisco UCS
We are at an inflection point where we are moving to a world where everything is connected to everything else. That’s what the Internet of Things (IoT)
Case StudiesIntermediateRead hereBy Cisco engineers · Cisco
Real-time Distributed Tracing at LinkedIn
Co-authors: Chris Coleman , Toon Sripatanaskul With LinkedIn’s service-oriented architecture, a single page view request can fan out calls to downstream services spanning multiple backend tiers, many levels deep. Though applications depe…
Case StudiesIntermediateRead hereBy LinkedIn engineers · LinkedIn
Real-time Messaging
How Slack delivers real-time messages to many clients through gateway servers and a messaging pipeline.
Case StudiesIntermediateRead hereBy Slack Engineering · Slack
Real-Time Predictions at DoorDash
Raghav Ramesh presents DoorDash’s thoughts on how to structure machine learning systems in production to enable robust and wide-scale deployment of machine learning, and shares best practices in designing engineering tooling around machi…
Case StudiesIntermediateBy DoorDash engineers · DoorDash
Real-time Presence Platform at LinkedIn
Have you ever wondered how those green online indicators (or “presence indicators”) that are so pervasive in every instant messaging application and social network actually work? How does one display them in real time for roughly 500 mil…
Case StudiesIntermediateRead hereBy LinkedIn engineers · LinkedIn
RFC 6455: The WebSocket Protocol
The standard behind WebSocket, including the upgrade handshake and framing.
DocumentationAdvancedBy IETF
RFC 8445: Interactive Connectivity Establishment (ICE)
The standard that defines how peers gather and test candidate network paths.
PapersAdvancedBy IETF
Scaling Real-time Infrastructure at Alibaba for Global Shopping Holiday - Xiaowei Jiang, Senior Director at Alibaba
Alibaba’s 11.11 Global Shopping Festival, hosted annually on November 11, is the world’s largest 24-hour online shopping event. On this day, we process trillions of events in realtime to give the best experience to users of our e-commerc…
Case StudiesIntermediateBy Alibaba engineers · Alibaba
Slack Engineering
How Slack builds messaging, search, job queues and infrastructure.
BlogsIntermediateBy Slack engineers · Slack
Uber's Real-Time Push Platform
How Uber pushes live updates to millions of phones instead of having apps poll.
Case StudiesAdvancedRead hereBy Uber Engineering · Uber
WebRTC: getting started overview
The official overview of peer connections, signalling and media in WebRTC.
DocumentationBeginnerBy WebRTC project · webrtc.org
The WhatsApp Architecture Facebook Bought for $19B
How a tiny team served hundreds of millions of users with Erlang and FreeBSD.
Case StudiesIntermediateRead hereBy High Scalability
MDN: WebSockets API
Browser API reference and guides for opening persistent two-way connections.
DocumentationBeginnerBy MDN Contributors · Mozilla
Hussein Nasser: Backend Engineering Show
Protocols, proxies, databases and load balancers explained from first principles.
YouTubeIntermediateBy Hussein Nasser · Hussein Nasser (YouTube)