Sources and further reading
The lessons are our own writing. These are the books, papers and articles they point to for more depth, credited to their authors. We link to them; we do not copy them. Image credits are on the image credits page.
System Design Foundations
Books for the whole course
- Designing Data-Intensive Applications · Martin Kleppmann · the best single book on storage, replication and consistency
- Site Reliability Engineering · Google SRE team · free to read online; reliability and operations
- System Design Interview, An Insider's Guide · Alex Xu · worked designs for the classic interview problems
- The Amazon Builders' Library · Amazon engineers · free articles on timeouts, retries, load shedding and more
What system design is, and how to approach it
- Designing Data-Intensive Applications, chapter 1 · Martin Kleppmann · reliability, scalability and maintainability explained
From one server to many: scaling and load balancing
- Timeouts, retries, and backoff with jitter · Amazon Builders' Library
- Elastic Load Balancing: sticky sessions · Amazon Web Services · what sticky sessions are and why stateless servers avoid needing them
- Single point of failure · Wikipedia · the idea behind redundancy at every tier
Back-of-the-envelope estimation
- Latency numbers every programmer should know · Jeff Dean (popularised) · search for the table; exact figures change, the orders of magnitude do not
- Latency numbers every programmer should know (interactive) · Colin Scott · how the figures change by year
- Compute service level agreements · Amazon Web Services · real availability promises to compare with your own targets
- Compute Engine service level agreement · Google Cloud · another provider's published availability figures
Choosing and using a database
- Designing Data-Intensive Applications, chapters 2 and 3 · Martin Kleppmann · data models, storage engines and indexes
Replication and sharding
- Designing Data-Intensive Applications, chapters 5 and 6 · Martin Kleppmann · replication and partitioning in depth
Caching
- Caching challenges and strategies · Amazon Builders' Library
Consistency, availability and the CAP theorem
- Designing Data-Intensive Applications, chapters 7 to 9 · Martin Kleppmann · transactions, the trouble with distributed systems, consistency and consensus
Queues, streams and asynchronous work
- Designing Data-Intensive Applications, chapters 11 and 12 · Martin Kleppmann · stream processing and the future of data systems
Networking, APIs and CDNs
- High Performance Browser Networking · Ilya Grigorik · free online; how TCP, TLS and HTTP actually behave
Designing for failure: reliability
- Timeouts, retries and backoff with jitter · Amazon Builders' Library
- Site Reliability Engineering, Addressing Cascading Failures · Google SRE team
Observability and running a system
- Site Reliability Engineering, Monitoring Distributed Systems · Google SRE team
Putting it together: design a URL shortener
- System Design Interview, An Insider's Guide (the URL shortener chapter) · Alex Xu · a longer treatment of the same problem
- A RESTful tutorial · restapitutorial.com · background for designing the shorten and redirect endpoints
- Bloom filter · Wikipedia · used to skip database lookups for keys that cannot exist
Scaling from one user to millions
- The Architecture of Open Source Applications · various authors · free book: real systems and how they evolved
Classic System Designs
Books for the whole course
- System Design Interview, An Insider's Guide (volumes 1 and 2) · Alex Xu · book-length treatments of many of these problems
- Designing Data-Intensive Applications · Martin Kleppmann · the theory behind storage, replication and streams
- The Amazon Builders' Library · Amazon engineers · real-world write-ups on reliability and scaling
Design a rate limiter
- Using load shedding to avoid overload · Amazon Builders' Library
- Rate-limiting strategies and techniques · Google Cloud · an overview of approaches
- Throttle API requests for better throughput · Amazon Web Services · how a managed gateway applies limits
- Scaling your API with rate limiters · Stripe · several kinds of limiter used together in production
- Counting things, a lot of different things · Cloudflare · how the sliding-window counter behaves at huge scale
- Lyft ratelimit · Lyft (GitHub) · an open-source limiter whose rules are plain configuration
- Better rate limiting with Redis sorted sets · ClassDojo · implementing a sliding-window log
- Request rate limiter (Lua) · Paul Tarjan (gist) · an atomic script version
- Rate limit requests with iptables · Programster · limiting at the network layer instead of in the application
Design consistent hashing
- Dynamo: Amazon's Highly Available Key-value Store · DeCandia et al. · the paper that popularised consistent hashing for storage; easy to find online
- Consistent hashing · Wikipedia · definition and variants
- Consistent hashing · Tom White · an experiment showing how virtual nodes smooth the distribution
- Dynamo: Amazon's highly available key-value store · Amazon · the paper that popularised the ring for storage
- Maglev: a fast and reliable software network load balancer · Google · consistent hashing inside a load balancer
- How Discord scaled Elixir to 5,000,000 concurrent users · Discord · a production use of the ring
Design a key-value store
- Dynamo: Amazon's Highly Available Key-value Store · DeCandia et al. · the original paper; widely available online
- Designing Data-Intensive Applications, chapters 3, 5 and 6 · Martin Kleppmann
- Dynamo: Amazon's highly available key-value store · Amazon · quorums, vector clocks, hinted handoff and Merkle trees in one paper
- Bigtable: a distributed storage system for structured data · Google · the log-structured storage model behind many stores
- Cassandra architecture · Apache Cassandra · gossip, replication and the write and read paths
- SSTable and log-structured storage · Ilya Grigorik · how sorted tables and merging work
- Merkle tree · Wikipedia · the structure used for replica comparison
Design a unique ID generator
- Announcing Snowflake · Twitter Engineering · the original description of the 64-bit time-based ID design
- Ticket servers: distributed unique primary keys on the cheap · Flickr · the central counter approach and how it was made redundant
- Universally unique identifier · Wikipedia · versions of UUID and collision odds
- Network Time Protocol · Wikipedia · how machines keep their clocks close
Design a web crawler
- Mercator: A Scalable, Extensible Web Crawler · Heydon and Najork · a classic design paper; easy to find online
- Dynamic rendering · Google Search Central · why scripts change what a crawler sees
- Web crawling lecture notes · Donald Patterson (UC Irvine) · frontier, politeness and priority in a course setting
Design a notification system
- Designing Data-Intensive Applications, chapter 11 · Martin Kleppmann
- You cannot have exactly-once delivery · Brave New Geek · why duplicates must be handled by the receiver
- Twilio SMS · Twilio · an example SMS provider
- SendGrid · Twilio SendGrid · an example email delivery provider
- Mailchimp · Mailchimp · an example email marketing provider
Design a news feed
- Feeding Frenzy: Selectively Materializing Users' Event Feeds · Silberstein et al. · a research paper on push, pull and hybrid feeds
Design a chat system
- How WhatsApp scaled with a small team (engineering talks) · various · search for conference talks on Erlang and WhatsApp
- Erlang at Facebook · Eugene Letuchy (Facebook) · early messaging architecture
- How Discord stores billions of messages · Discord · partitioning and storage choices for chat history
- Apache ZooKeeper · Apache · coordination and service discovery
- Flannel: an application-level edge cache · Slack Engineering · reducing load time with an edge cache
Design search autocomplete
- Introduction to Information Retrieval, chapter 3 (tolerant retrieval) · Manning, Raghavan and Schutze · free online textbook
- Prefix hash tree: an indexing data structure over distributed hash tables · Berkeley · a way to spread a prefix index across nodes
- Unicode frequently asked questions · Unicode Consortium · what to know before supporting other languages
Design a video streaming service
- Netflix Tech Blog, video encoding and Open Connect · Netflix engineers · free articles on encoding pipelines and content delivery
- SVE: distributed video processing at Facebook scale · Princeton and Facebook · the task-graph model for video processing
- Streaming protocols explained · Dacast · the main adaptive streaming protocols compared
- Understanding the characteristics of internet short video sharing · Cheng et al. (arXiv) · measurements behind the long-tail viewing pattern
- Delegate access with a shared access signature · Microsoft Azure · Azure's version of pre-signed upload links
- Netflix on AWS · Amazon Web Services · a large streaming service built on rented cloud capacity
- Amazon CloudFront pricing · Amazon Web Services · how delivery cost is billed
Design cloud file storage and sync
- Designing Data-Intensive Applications, chapters 5 and 7 · Martin Kleppmann · replication, conflicts and transactions
- Upload file data (resumable uploads) · Google Drive API · simple versus resumable upload
- Differential synchronization · Neil Fraser · a way to keep documents in step
- librsync · GitHub · the library behind block-level delta sync
- How we've scaled Dropbox · Dropbox (talk) · block servers and long-polling notification at scale
- Amazon S3 Glacier FAQ · Amazon Web Services · cold storage for rarely used data
- ACID · Wikipedia · the guarantees relational metadata stores provide
Core Concepts: How Things Actually Work
Books for the whole course
- Designing Data-Intensive Applications · Martin Kleppmann · transactions, time, consensus and streams in depth
- High Performance Browser Networking · Ilya Grigorik · free; TCP, TLS and HTTP
- OWASP Cheat Sheet Series · OWASP · free; practical guidance on passwords, sessions and more
How AWS Lambda and serverless platforms work
- Firecracker: Lightweight Virtualization for Serverless Applications · Agache et al. (Amazon Web Services) · an NSDI paper describing the micro-VM technology behind AWS Lambda; easy to find online
SOAP, REST, GraphQL and gRPC compared
- Microsoft REST API Guidelines and the GraphQL and gRPC documentation · Microsoft, GraphQL Foundation, gRPC authors · search for each; practical guidance on design, versioning and errors
Where online payments are heading
- Bank for International Settlements, committee on payments and market infrastructures · BIS · search the site for reports on fast payments and the future of money
Choosing a programming language for a system
- Stack Overflow Developer Survey, TIOBE index, GitHub Octoverse · Stack Overflow, TIOBE, GitHub · search for the latest editions: three public views of language popularity, each measured differently
DevOps in practice, and where to learn more
- Accelerate, The DevOps Handbook, The Phoenix Project, Site Reliability Engineering, Continuous Delivery, Release It! · Forsgren, Humble and Kim; Kim, Humboldt and Debois; Kim, Behr and Spafford; Google SRE team; Humble and Farley; Michael Nygard · the standard reading list on DevOps, delivery performance, reliability and production-ready design
Real-World Systems
Books for the whole course
- Designing Data-Intensive Applications · Martin Kleppmann · the foundations behind almost every lesson here
- Site Reliability Engineering · Google SRE team · free; monitoring, incidents and postmortems
- OWASP API Security Project · OWASP · free; the most common API security mistakes
- System Design Interview, An Insider's Guide (volume 2) · Alex Xu · worked designs for several of these systems
Design collaborative editing (Google Docs)
- Yjs and Automerge documentation · open-source CRDT libraries · search for their docs: practical introductions to CRDT-based collaboration
Design an email system (Gmail)
- How email works: RFC 5321 (SMTP) overview · IETF · search for the standard; the first sections are readable
Design a maps and navigation service
- OpenStreetMap wiki: Slippy map tilenames · OpenStreetMap community · a clear description of the z/x/y tile scheme
Design a nearby-places service (Yelp)
- Geohash on Wikipedia, and the S2 Geometry and H3 libraries · various · search for their documentation; H3 is by Uber, S2 by Google
Design a stock exchange
- The LMAX Architecture · Martin Fowler · a well-known description of a single-threaded, event-sourced, very fast trading system
Design a flash sale system
- Using load shedding to avoid overload · Amazon Builders' Library
Design a payment system
- Stripe API reference: idempotent requests · Stripe · a clear description of idempotency keys for payment APIs
Cross-border payments: SWIFT and foreign exchange
- Bank for International Settlements: cross-border payments · BIS · search the site for cross-border payments; reports on why they are slow and costly
Design a metrics and alerting system
- Gorilla: A Fast, Scalable, In-Memory Time Series Database · Facebook engineers · a paper on compressing time series; easy to find online
Why SSDs are fast, and block, file and object storage
- Designing Data-Intensive Applications, chapter 3 · Martin Kleppmann · how storage engines use disks
How a browser turns a URL into a page
- How browsers work · Google Chrome developers (web.dev) · search the site for 'critical rendering path' and 'Inside look at modern web browser'
Diagnosing a slow machine: CPU, memory and I/O
- The USE Method · Brendan Gregg · free; a checklist for finding performance bottlenecks
How microservices work together
- Microservices patterns: Saga and Transactional Outbox · microservices.io (Chris Richardson) · free catalogue of patterns with diagrams and trade-offs
Securing a web API
- OWASP API Security Top 10 · OWASP · the most common API security risks with examples
Shipping safely and handling a large outage
- Site Reliability Engineering: Managing Incidents and Postmortem Culture · Google SRE team · free; practical guidance on incident response and learning from failure
How a Twitter-style service works
- Timelines at Scale (talk) and the Twitter engineering blog · Twitter engineers · search for the talk and blog posts; details of the real system have changed over the years
Uploading large files reliably
- Amazon S3 multipart upload documentation, and the tus resumable upload protocol · Amazon Web Services; the tus community · tus is an open protocol for resumable uploads
Design an AI coding assistant
- Retrieval-augmented generation and LLM serving (vLLM, PagedAttention) · Lewis et al.; Kwon et al. · search for the papers and documentation; background for the retrieval and serving parts
Big Data and Cloud Data Platforms
Books for the whole course
- Designing Data-Intensive Applications · Martin Kleppmann · chapters 10 and 11 cover batch and stream processing
- The Data Warehouse Toolkit · Ralph Kimball and Margy Ross · the classic reference on dimensional modelling
How big data evolved, and the papers behind it
- The papers listed in this lesson · authors named in the lesson · search each title; most are free to read online from the publishers or authors
Batch and stream processing
- Streaming Systems · Tyler Akidau, Slava Chernyak, Reuven Lax · a book on the dataflow model; the earlier articles 'Streaming 101' and 'Streaming 102' are free online
Data lakes, warehouses and lakehouses
- The Data Warehouse Toolkit, and the Apache Iceberg and Delta Lake documentation · Kimball and Ross; the open-source projects · dimensional modelling and the table formats behind the lakehouse
Choosing a cloud provider for a big data solution
- Each provider's data architecture guides · Amazon Web Services, Microsoft Azure, Google Cloud · search for their reference architectures for data analytics; compare with your own requirements
Software Design Essentials
Books for the whole course
- Design Patterns: Elements of Reusable Object-Oriented Software · Gamma, Helm, Johnson and Vlissides · the original catalogue of 23 patterns
- Clean Architecture · Robert C. Martin · the dependency rule and architecture boundaries
- Domain-Driven Design · Eric Evans · the book that introduced the approach; Vaughn Vernon's 'Domain-Driven Design Distilled' is a shorter start
- Refactoring Guru: design patterns catalogue · Refactoring Guru · free explanations and examples of each pattern
How to learn design patterns
- Refactoring Guru: design patterns · Refactoring Guru · free catalogue with examples in several languages
- Head First Design Patterns · Freeman and Robson · a beginner-friendly, example-led introduction to the same catalogue
The SOLID principles
- Agile Software Development, Principles, Patterns and Practices; Clean Architecture · Robert C. Martin · where the SOLID principles were collected and explained
Creational patterns: Factory, Builder, Singleton
- Refactoring Guru: creational patterns · Refactoring Guru
Structural patterns: Adapter, Decorator, Facade, Proxy
- Refactoring Guru: structural patterns · Refactoring Guru
Behavioural patterns: Strategy, Observer, Command, State
- Refactoring Guru: behavioural patterns · Refactoring Guru
Clean and hexagonal architecture
- The Clean Architecture (blog), Hexagonal Architecture · Robert C. Martin; Alistair Cockburn · search for both articles; short, free introductions to the same family of ideas
Domain-driven design essentials
- Domain-Driven Design (the 'blue book') and Domain-Driven Design Distilled · Eric Evans; Vaughn Vernon · the original and a short introduction; both worth buying
Low-Level Design Practice
Books for the whole course
- Head First Design Patterns · Freeman and Robson · a gentle, example-led introduction to the patterns used here
- Design Patterns: Elements of Reusable Object-Oriented Software · Gamma, Helm, Johnson and Vlissides · the original catalogue
- Refactoring Guru: design patterns · Refactoring Guru · free explanations with examples
How to approach a low-level design problem
- Refactoring Guru: design patterns · Refactoring Guru · use as a catalogue once you know the problem
Parking lot
- Refactoring Guru: strategy pattern · Refactoring Guru · swappable pricing rules
Cab booking
- Refactoring Guru: state pattern · Refactoring Guru · for the trip lifecycle
Chat application
- Chat system design (classic system designs course) · This site · for the infrastructure side: WebSockets, presence and storage
Social feed (Instagram-style)
- Design the news feed (classic system designs course) · This site · for fan-out on write versus read at scale
Advanced Distributed Systems
Books for the whole course
- Designing Data-Intensive Applications · Martin Kleppmann · the standard book on these topics
- In Search of an Understandable Consensus Algorithm (Raft) · Ongaro and Ousterhout · the Raft paper; short and readable
- Site Reliability Engineering · Google · free online; reliability practice
Consensus: Raft and Paxos
- In Search of an Understandable Consensus Algorithm · Ongaro and Ousterhout · the Raft paper
- The Raft visualisation · Raft authors · an interactive animation of leader election and replication
Write-ahead logs, B-trees and LSM-trees
- Designing Data-Intensive Applications, chapter 3 · Martin Kleppmann · storage engines in depth
Kafka internals: segments, replication and consumer groups
- Apache Kafka documentation · Apache Kafka · design and operations sections
Redis: persistence, replication, Sentinel and Cluster
- Redis documentation: persistence, Sentinel, cluster · Redis · operations guides
Resilience: timeouts, retries, bulkheads, backpressure and circuit breakers
- Release It! · Michael Nygard · the book that popularised circuit breakers and bulkheads
- Exponential Backoff and Jitter · AWS Architecture Blog · why jitter matters
Kubernetes essentials
- Kubernetes documentation: concepts · Kubernetes · clear and complete
- Kubernetes the Hard Way · Kelsey Hightower · to see every moving part
Cloud architecture principles
- AWS Well-Architected Framework · Amazon Web Services · one vendor's pillars; the ideas apply across clouds
- Google Cloud Architecture Framework · Google Cloud · a similar set of guidelines
A map of distributed-system patterns
- Patterns of Distributed Systems · Unmesh Joshi (on martinfowler.com) · the original catalogue with code and diagrams
Databases Deep Dive
Books for the whole course
- Use The Index, Luke · Markus Winand · free online guide to SQL indexing
- Designing Data-Intensive Applications · Martin Kleppmann · storage, replication and data models
- PostgreSQL documentation: performance tips · PostgreSQL · query planning and EXPLAIN
Normalisation and schema design
- Database normalization (overview) · Wikipedia · a reference for the normal forms
Indexes in depth
- Use The Index, Luke · Markus Winand · free and excellent
Query optimisation, EXPLAIN and connection pooling
- PostgreSQL: Using EXPLAIN · PostgreSQL · how to read plans
Replication topologies and multi-region databases
- Designing Data-Intensive Applications, chapter 5 · Martin Kleppmann · replication in depth
Choosing a database type
- Which database should I use? (cloud decision guides) · Cloud provider documentation · each major cloud publishes a decision tree for its own services
Networking and Web Security Deep Dive
Books for the whole course
- High Performance Browser Networking · Ilya Grigorik · free online; networking for web developers
- OWASP Top 10 · OWASP · https://owasp.org/www-project-top-ten/
- The Web Application Hacker's Handbook · Stuttard and Pinto · classic reference on web attacks
Network layers, TCP, UDP, ports and sockets
- High Performance Browser Networking · Ilya Grigorik · free online book
Web attacks: XSS, CSRF, injection, SSRF and clickjacking
- OWASP Cheat Sheet Series · OWASP · concise defences for each attack
JWT, cookies, CORS and security headers
- JSON Web Token introduction · jwt.io · structure and use
- MDN: Cross-Origin Resource Sharing · MDN Web Docs · the definitive explanation
The OWASP Top 10 and secure design
- OWASP Top 10 · OWASP · the current list and explanations
- OWASP Application Security Verification Standard · OWASP · a detailed checklist
Messaging and Microservice Patterns
Books for the whole course
- Designing Data-Intensive Applications · Martin Kleppmann · messaging, streams and consistency
- Microservices Patterns · Chris Richardson · sagas, outbox, CQRS
- Site Reliability Engineering · Google · free online; SLOs and error budgets
Message brokers compared
- Apache Kafka documentation · Apache Kafka · design chapter
- RabbitMQ tutorials · RabbitMQ · hands-on patterns
Reliable messaging: retries, dead letters, ordering and the outbox
- Pattern: Transactional outbox · microservices.io · the pattern in detail
Event-driven architecture: sagas, CQRS and event sourcing
- Microservices Patterns: Saga · microservices.io · orchestration versus choreography
- CQRS · Martin Fowler · a short, clear explanation
Service discovery, service mesh and sidecars
- Envoy proxy documentation · Envoy · the proxy under many meshes
- Istio documentation · Istio · concepts of a service mesh
Operating microservices: config, secrets, feature flags, tracing and logging
- OpenTelemetry documentation · OpenTelemetry · the common standard for traces, metrics and logs
- The Twelve-Factor App · Adam Wiggins · configuration, logs and other practices
Observability: metrics, SLIs, SLOs and alerting
- Site Reliability Engineering: Service Level Objectives · Google · the classic chapter
- Prometheus documentation · Prometheus · metrics and alerting
Interview Preparation
Books for the whole course
- System Design Interview, volumes 1 and 2 · Alex Xu · classic interview-style walkthroughs
- Designing Data-Intensive Applications · Martin Kleppmann · depth behind the answers
- Grokking the System Design Interview · Educative · a structured practice course
A method for system design interviews
- Back-of-the-envelope estimation · This site · see the estimation lesson in the foundations course
Keep learning: papers, blogs and a practice habit
- System design primer · Donne Martin (GitHub) · a free, community-maintained study guide with many links
- The Google File System · Google · distributed storage that shaped the big data era
- MapReduce: simplified data processing on large clusters · Google · a model for batch processing at scale
- Bigtable: a distributed storage system for structured data · Google · wide-column storage built on a log-structured design
- Dynamo: Amazon's highly available key-value store · Amazon · availability-first replication, quorums and conflict handling
- In search of an understandable consensus algorithm (Raft) · Ongaro and Ousterhout · consensus explained for implementers
- Patterns of distributed systems · Unmesh Joshi (martinfowler.com) · named building blocks you will see in real systems
- Site Reliability Engineering · Google · reliability practice from running very large systems
- Amazon Builders' Library · Amazon Web Services · practical articles on timeouts, retries, deployment and more
- Engineering at Meta · Meta · write-ups of large-scale systems
Search, Analytics and Geo
Books for the whole course
- Introduction to Information Retrieval · Manning, Raghavan and Schutze · free online textbook behind most search lessons
- Designing Data-Intensive Applications · Martin Kleppmann · partitioning and stream processing background
- Elasticsearch: the definitive guide and reference · Elastic · how a production engine exposes these ideas
TF-IDF: scoring words by how useful they are
- Introduction to Information Retrieval: inverse document frequency · Manning, Raghavan and Schutze · the textbook definition with worked figures
- The probabilistic relevance framework: BM25 and beyond · Robertson and Zaragoza · where TF-IDF came from and what replaced it
BM25: the ranking function behind most search engines
- The probabilistic relevance framework: BM25 and beyond · Robertson and Zaragoza · the definitive survey
- Practical BM25, part 2: the algorithm and its variables · Elastic · k1, b and their effect with examples
- BM25Similarity · Apache Lucene · the implementation used by Elasticsearch and Solr
Query understanding: from typed text to a search request
- Introduction to Information Retrieval: tolerant retrieval · Manning, Raghavan and Schutze · spelling correction and wildcard queries
- Query Understanding · Daniel Tunkelang · a book and blog series on the whole topic
Measuring search: precision, recall, MRR and NDCG
- Introduction to Information Retrieval: evaluation · Manning, Raghavan and Schutze · precision, recall, MAP and NDCG explained
- Assessing relevance · Manning, Raghavan and Schutze · test collections and judge agreement
Sharding a search index: scatter, gather and merge
- Scalability and resilience: clusters, nodes and shards · Elastic · how a production engine thinks about shards and replicas
- Shards and indexing data in SolrCloud · Apache Solr · routing documents to shards
- Designing Data-Intensive Applications, chapter on partitioning · Martin Kleppmann · document versus term partitioning
Reservoir sampling: a fair sample of a stream you cannot store
- Random sampling with a reservoir · Jeffrey Vitter · the 1985 paper that defined the algorithms
- Reservoir sampling · Wikipedia contributors · variants, weighted sampling and proofs
Percentiles at scale: t-digest and mergeable summaries
- Computing extremely accurate quantiles using t-digests · Dunning and Ertl · the paper that defines the structure
- t-digest reference implementation · Ted Dunning · code and documentation
- Site Reliability Engineering: monitoring distributed systems · Google · why percentiles beat averages for latency
Geofencing: is this point inside that area?
- Point in polygon · Wikipedia contributors · ray casting and winding number
- S2 Geometry · Google · cell covering of regions on the sphere
- PostGIS documentation · PostGIS project · ST_Contains and spatial indexes in a database
Nearby search with Redis GEO
- Geospatial data type · Redis · the data type and its limits
- GEOSEARCH command · Redis · searching by radius or box
- GEOADD command · Redis · adding members and accuracy notes
Seen filtering: Bloom filter versus exact sets
- Bloom filter · Wikipedia contributors · definition, error rate and sizing formulas
- Redis data types: sets and bitmaps · Redis · the exact-set alternatives
- Network applications of Bloom filters: a survey · Broder and Mitzenmacher · variants and uses
Storage, Queues and Real-time
Books for the whole course
- Designing Data-Intensive Applications · Martin Kleppmann · storage engines, replication and stream processing
- Database Internals · Alex Petrov · how storage engines and distributed databases are built
- Patterns of Enterprise Application Architecture · Martin Fowler · repository, unit of work and related patterns
Bitcask: a tiny log-structured key-value store
- Bitcask: a log-structured hash table for fast key/value data · Sheehy and Smith · the original design note
- Designing Data-Intensive Applications, chapter 3 · Martin Kleppmann · hash indexes, SSTables and LSM-trees compared
Changing a live database: online indexes and schema changes
- CREATE INDEX (CONCURRENTLY) · PostgreSQL · exact behaviour and caveats
- gh-ost: GitHub's online migration tool for MySQL · GitHub Engineering · the shadow-table approach explained
- gh-ost repository · GitHub · documentation on throttling and cut-over
A job queue on PostgreSQL with SKIP LOCKED
- SELECT: the locking clause · PostgreSQL · reference for FOR UPDATE and SKIP LOCKED
- What is SELECT SKIP LOCKED for in PostgreSQL? · 2ndQuadrant · motivation and examples
- Scaling Slack's job queue · Slack Engineering · a production story of queue design
A rule engine and trigger framework
- Rules engine · Martin Fowler · a balanced view on when engines help
- Drools documentation · Drools community · a mature open-source rules engine
- Enterprise Integration Patterns · Hohpe and Woolf · idempotent receiver and message router patterns
The repository pattern: hiding storage behind a domain interface
- Repository · Martin Fowler · the original catalogue entry
- Domain-Driven Design · Eric Evans · repositories in the context of aggregates
- Patterns of Enterprise Application Architecture · Martin Fowler · unit of work and data mapper
WebRTC: peer-to-peer media, signalling, NATs and media servers
- WebRTC: getting started overview · webrtc.org · the official introduction
- WebRTC connectivity · MDN Web Docs · ICE, STUN and TURN in plain language
- RFC 8445: Interactive Connectivity Establishment · IETF · the standard behind candidate gathering