Backend System Design

Design backends that scale and stay up: estimates, caching, partitioning, consistency, queues and observability, with the core algorithms built in code.

32 lessons across 8 units: estimation, load balancing, caching and eviction, consistent hashing and sharding, replication, quorums and CAP, rate limiting, gateways and circuit breakers, queues, idempotency, outbox and sagas, search, feeds and SLOs, then three design walkthroughs, with 32 runnable examples, quizzes and 33 coding problems.

Units
8
Lessons
32
Coding problems
33
Examples
32

Free: units 1 to 2 (9 lessons). Units 3 to 8 with DevArcade Pro.

See pricing

What you'll learn

  • Design FundamentalsA design process and Little's law, back-of-the-envelope estimates, latency numbers, and availability math.
  • Scaling OutStateless fleets, load balancing algorithms and the power of two choices, caching patterns, and LRU and LFU eviction.
  • Caching and Partitioning at ScaleCache stampedes and single-flight, consistent hashing with virtual nodes, and sharding.
  • Replication and ConsistencyReplication lag and read-your-writes, quorums and read repair, CAP, PACELC and vector clocks.
  • Traffic and BoundariesRate limiting algorithms, GCRA for shared limits, API gateways and service boundaries, circuit breakers.
  • Queues and EventsDelivery semantics and visibility timeouts, idempotency keys, the outbox, event sourcing, and sagas.
  • Search, Feeds and OperationsInverted indexes, feed fan-out, SLOs and error budgets, percentiles, and retries with jitter.
  • Design WalkthroughsA URL shortener, a chat system, and the boss: a rate-limited public API.

Course outline

8 units and 32 lessons. Each lesson has a short read with examples you run, a quiz, and coding problems tested in your browser; most have a step-through visualizer.

  1. Unit 1:Design Fundamentals

    A design process and Little's law, back-of-the-envelope estimates, latency numbers, and availability math.

    Free
    1. How to Design a Backend System A repeatable process: requirements, estimates, API, data, high-level design, deep dives and trade-offs, plus Little's law, the first tool for sizing anything.
      1 problem
    2. Back-of-the-Envelope Estimation Turn users and behavior into requests per second, storage and bandwidth with round numbers: 86,400 seconds a day, peak factors, read/write ratios and powers of ten.
      1 problem
    3. Latency Numbers and Request Paths Approximate orders of magnitude for memory, SSD, network and disk, why they matter more than exact values, and computing the latency of sequential and parallel calls.
      1 problem
    4. Availability and the Nines What 99.9% and 99.99% mean in minutes of downtime, why components in series multiply their availability, and how redundancy in parallel buys it back.
      1 problem

  2. Unit 2:Scaling Out

    Stateless fleets, load balancing algorithms and the power of two choices, caching patterns, and LRU and LFU eviction.

    Free
    1. Scaling Up and Scaling Out Bigger machines versus more machines, why horizontal scaling needs stateless services, and sizing a fleet for peak load with headroom and spare capacity.
      1 problem
    2. Load Balancing Algorithms Layer 4 versus layer 7 balancers, health checks, and the classic algorithms: round robin, weighted round robin (and nginx's smooth variant), least connections and hashing.
      1 problem
    3. Least Load and the Power of Two Choices Why "send it to the least loaded server" goes wrong with many balancers and stale information, and how picking the better of two random servers fixes it.
      1 problem
    4. Caching Patterns Where caches sit, cache-aside versus read-through, write-through, write-back and write-around, TTLs, and invalidation on writes.
      1 problem
    5. Eviction Policies: LRU and LFU When the cache is full, what goes? Least recently used, least frequently used, FIFO and random, their failure modes, and O(1) implementations.
      2 problems

  3. Unit 3:Caching and Partitioning at Scale

    Cache stampedes and single-flight, consistent hashing with virtual nodes, and sharding.

    Pro
    1. Cache Stampedes When a hot key expires, thousands of requests rebuild it at once. Request coalescing (single-flight), early probabilistic refresh, jittered TTLs and serving stale data prevent it.
      1 problem
    2. Consistent Hashing Why hash(key) mod N reshuffles almost everything when N changes, the hash ring that moves only about 1/N of keys, and virtual nodes for even spread.
      1 problem
    3. Sharding Splitting one dataset over many databases: choosing a shard key, range versus hash partitioning, hot spots, cross-shard queries, and splitting a shard as it grows.
      1 problem

  4. Unit 4:Replication and Consistency

    Replication lag and read-your-writes, quorums and read repair, CAP, PACELC and vector clocks.

    Pro
    1. Replication and Replication Lag Leader-follower, multi-leader and leaderless replication, synchronous versus asynchronous, the anomalies of lag, and guaranteeing that users read their own writes.
      1 problem
    2. Quorums Leaderless replication with N replicas, W write acknowledgments and R read responses: why R + W > N makes reads see the latest write, versions, read repair, and sloppy quorums.
      1 problem
    3. CAP, PACELC and Vector Clocks What the CAP theorem actually says (a choice during partitions), PACELC's latency versus consistency trade-off the rest of the time, and vector clocks for detecting concurrent writes.
      1 problem

  5. Unit 5:Traffic and Boundaries

    Rate limiting algorithms, GCRA for shared limits, API gateways and service boundaries, circuit breakers.

    Pro
    1. Rate Limiting Algorithms Fixed window, sliding log, sliding window counter, token bucket and leaky bucket: what each allows, what it costs in memory, and where each is used.
      1 problem
    2. Distributed Rate Limits and GCRA Enforcing one limit across many servers: shared counters with atomic updates, the race in read-then-write, and the generic cell rate algorithm, which needs one timestamp per key.
      1 problem
    3. API Gateways and Service Boundaries What an API gateway does (routing, authentication, limits, aggregation), backends for frontends, and drawing service boundaries around business capabilities with their own data.
      1 problem
    4. Timeouts, Retries and Circuit Breakers Every remote call can hang or fail: deadlines and timeouts, bounded retries for idempotent calls, and circuit breakers that stop hammering a dependency that is down.
      1 problem

  6. Unit 6:Queues and Events

    Delivery semantics and visibility timeouts, idempotency keys, the outbox, event sourcing, and sagas.

    Pro
    1. Queues and Delivery Semantics Why queues decouple producers from consumers, at-most-once versus at-least-once delivery, visibility timeouts and acknowledgments, and dead-letter queues.
      1 problem
    2. Idempotency Keys Making retries safe: idempotent operations, idempotency keys on APIs (as payment APIs use them), storing results to replay, rejecting reused keys with different requests, and deduplicating consumers.
      1 problem
    3. The Transactional Outbox The dual-write problem (saving to the database and publishing an event can half-fail), writing events to an outbox table in the same transaction, and relaying them at least once.
      1 problem
    4. Event-Driven Design and Event Sourcing Events versus commands, ordering per key with partitions, consumer groups, and event sourcing: state as a fold over an append-only log of events.
      1 problem
    5. Sagas Business transactions across services without distributed locks: a sequence of local steps with compensating actions, orchestration versus choreography, and designing compensations.
      1 problem

  7. Unit 7:Search, Feeds and Operations

    Inverted indexes, feed fan-out, SLOs and error budgets, percentiles, and retries with jitter.

    Pro
    1. Search and Inverted Indexes Why LIKE '%term%' does not scale, how an inverted index maps terms to documents, tokenizing and normalizing text, AND queries by intersecting postings, and keeping a search index in sync.
      1 problem
    2. Feeds: Fan-Out on Write or Read Building home timelines: precomputing per-user feeds on write, merging followed timelines on read, the celebrity problem, and the hybrid most large services use.
      1 problem
    3. SLIs, SLOs and Error Budgets Measuring reliability the way users feel it: service level indicators, objectives and agreements, error budgets that turn reliability into a decision, and burn-rate alerts.
      1 problem
    4. Observability and Percentiles Metrics, logs and traces and what each answers; why averages hide the latency users feel; computing percentiles; and the golden signals.
      1 problem
    5. Retries, Backoff and Jitter Retrying without making outages worse: exponential backoff, full jitter to spread retries out, caps and budgets, and load shedding when demand exceeds capacity.
      1 problem

  8. Unit 8:Design Walkthroughs

    A URL shortener, a chat system, and the boss: a rate-limited public API.

    Pro
    1. Design: A URL Shortener The classic warm-up, done properly: requirements and estimates, short codes from base62 counters versus hashes, a read-heavy redirect path with caching, and 301 versus 302.
      1 problem
    2. Design: A Chat System One-to-one and group chat: persistent connections and a presence service, ordering messages per conversation with sequence numbers, deduplicating client retries, delivery and read receipts, and offline sync.
      1 problem
    3. Boss: A Rate-Limited Public API Design the front door of a public API: API keys and plans, a per-second burst limit plus a daily quota, informative 429 responses with headers, and where each piece runs.
      1 problem

Start Backend System Design for free

Enroll for free and units 1 and 2 are yours. DevArcade Pro opens every unit of every course, including new ones as they launch, monthly or yearly.

All courses