System Design Index

This is a compact map of system design. Each entry says what a tool or technique does, why it exists, and what it costs.

Core Measurements

• Latency: How long one operation takes. Track percentiles because averages hide slow requests.
• Throughput: How much work a system completes per unit of time.
• Availability: The share of time a system can serve valid requests.
• Durability: The chance that acknowledged data survives failures.
• Reliability: The ability to behave correctly and consistently over time.
• Scalability: The ability to handle more load by adding resources without redesigning everything.
• Vertical scaling: Give one machine more CPU, memory, or storage. It is simple but has a ceiling.
• Horizontal scaling: Add more machines. It raises capacity but introduces coordination and distributed failure.
• Capacity planning: Estimate traffic, storage, bandwidth, and growth before choosing an architecture.

Traffic and Delivery

• DNS: Maps names to network addresses and can steer traffic across regions.
• Load balancing: Distributes requests across healthy servers to improve capacity and availability.
• Reverse proxy: Receives client traffic and forwards it to internal services while hiding their topology.
• CDN: Serves content near users to reduce latency and origin traffic.
• Rate limiting: Restricts request volume to protect capacity, fairness, and cost.
• Backpressure: Slows producers when consumers cannot safely keep up.
• Load shedding: Rejects less important work so critical paths remain healthy during overload.
• Batching: Groups work to reduce overhead. Items should share a destination and failure boundary.
• Compression: Trades CPU for fewer transferred bytes and lower bandwidth cost.
• Pagination: Returns large result sets in bounded pieces. Cursor pagination handles changing data best.

Caching

Caching stores expensive or frequently requested results closer to the people and systems that need them. It can reduce latency, lower pressure on databases and services, and avoid repeating work that has already been completed.

The difficult part is deciding how long cached data can remain useful and how it should be refreshed. Faster reads come with the possibility of stale data, while aggressive invalidation can remove much of the benefit.

• Cache aside: The application reads the cache first. When the value is missing, it loads the source data and stores the result for later requests.
• Write through: Every write updates the cache and backing store together. Reads stay fresh, but writes carry more work and latency.
• Write back: Writes reach the cache first and durable storage later. This improves write speed but increases the risk of data loss when the cache fails.

Data and Storage

Relational Databases

Store structured data with schemas, joins, constraints, and strong transaction support.

SQL

A language for querying and changing relational data.

Key Value Stores

Map unique keys to values for simple, predictable, and fast access.

Document Stores

Store flexible records as documents when objects vary in shape.

Wide Column Stores

Organize sparse data by partition and column family for massive distributed workloads.

Graph Databases

Store relationships as first class data for traversal heavy queries.

Time Series Databases

Optimize timestamped measurements, retention, and time window queries.

Object Storage

Stores large immutable objects cheaply with simple key based access.

Data Warehouse

Organizes historical data for analytics across many business sources.

Data Lake

Keeps large amounts of raw data for later processing and analysis.

Indexes

Add lookup structures that speed reads while consuming space and slowing writes.

Normalization

Separates repeated data to improve consistency and reduce duplication.

Denormalization

Duplicates data to make important reads faster and more predictable.

Materialized Views

Store computed query results and refresh them when source data changes.

Bloom Filters

Quickly prove an item is absent using little memory, with possible false positives.

Distributed Data

Partitioning

Splits data across machines by a chosen key to spread storage and work.

Sharding

Partitions a data set into independently stored pieces called shards.

Replication

Keeps copies of data on multiple machines for availability and read capacity.

Leader And Followers

One replica accepts writes while followers copy changes and often serve reads.

Leader Election

Chooses one coordinator when the current leader fails or becomes unreachable.

Consistent Hashing

Maps keys to changing nodes while moving only part of the data.

Consensus

Lets distributed nodes agree on ordered state despite failures.

Quorums

Require enough replicas to participate so reads and writes overlap safely.

Strong Consistency

Every successful read sees the latest completed write.

Eventual Consistency

Replicas may temporarily differ but converge when updates stop.

Causal Consistency

Preserves the order of operations that could have influenced each other.

ACID

Atomicity, consistency, isolation, and durability describe reliable database transactions.

CAP

During a network partition, a distributed system chooses consistency or availability.

PACELC

Adds the normal choice between latency and consistency when no partition exists.

Idempotency

Repeating the same operation produces the same intended result.

Read Repair

A read detects stale replicas and updates them with newer data.

Anti Entropy

Replicas compare and reconcile data in the background.

Messaging and Events

Message Queues

Buffer work between producers and consumers so each can fail or scale independently.

Publish And Subscribe

Publishers send events to a topic while many interested consumers receive them.

Event Streams

Store ordered events so consumers can process and replay history.

At Most Once Delivery

A message is attempted once, so duplicates are avoided but loss is possible.

At Least Once Delivery

A message is retried until acknowledged, so consumers must handle duplicates.

Exactly Once Effects

Idempotency and transactions make repeated delivery produce one business result.

Ordering

Preserves event sequence globally or within a chosen partition.

Dead Letter Queues

Isolate repeatedly failing messages for inspection and recovery.

Event Sourcing

Stores state changes as an event history instead of only the latest value.

CQRS

Uses separate models for writes and reads when their needs differ greatly.

Change Data Capture

Turns database changes into an event stream for other systems.

Search and Discovery

Inverted Index

Maps terms to documents so text search avoids scanning every record.

Elastic

Elasticsearch is a distributed search engine built on Lucene with filtering, aggregation, sharding, and replication.

Search Routing

Sends a query to the smallest set of shards that can answer it.

Relevance Ranking

Scores matches so the most useful search results appear first.

Full Text Search

Finds natural language terms across documents using tokenization and indexes.

Vector Search

Finds nearby embeddings to retrieve semantically similar items.

Hybrid Search

Combines keyword and vector results to balance precision and meaning.

Backfills

Processes historical data so a new index or system starts complete.

Dual Writes

Writes to old and new destinations during a migration, then compares their results.

Service Architecture

Monolith

Deploys the application as one unit, which simplifies coordination and transactions.

Modular Monolith

Keeps one deployment while enforcing clear internal module boundaries.

Microservices

Deploy capabilities independently at the cost of networking and operational complexity.

Cell Architecture

Repeats small, isolated service groups to limit failures and scale in units.

Serverless

Runs managed functions on demand while the platform owns most server operations.

Containers

Package an application and its runtime into a portable isolated unit.

Kubernetes

Schedules containers and manages rollout, recovery, scaling, and service discovery.

Service Discovery

Lets services find healthy instances without fixed addresses.

Service Mesh

Moves service networking, security, and telemetry into shared infrastructure.

Saga

Coordinates a long business operation through local transactions and compensating actions.

APIs and Connections

REST

Models resources through standard HTTP methods and representations.

GraphQL

Lets clients request precise fields through a typed query schema.

gRPC

Uses typed contracts and compact binary messages for efficient service calls.

WebSockets

Keep a two way connection open for low latency updates.

Server Sent Events

Stream updates from a server to a browser over one HTTP connection.

Webhooks

Send an HTTP request when an event happens instead of waiting for polling.

Polling

Checks repeatedly for changes. It is simple but can waste requests.

Long Polling

Keeps a request open until data changes or a timeout occurs.

API Gateway

Centralizes routing, authentication, limits, and policy for external API traffic.

Reliability Patterns

Timeouts

Stop waiting after a bounded period so stuck work cannot consume resources forever.

Retries

Repeat transient failures carefully and only when the operation is safe.

Exponential Backoff

Wait progressively longer between retries to reduce pressure during failure.

Jitter

Adds randomness to retry timing so clients do not retry together.

Circuit Breaker

Stops calls to an unhealthy dependency and periodically tests recovery.

Bulkhead

Separates resource pools so one overloaded workload cannot sink everything.

Health Checks

Test whether an instance can safely receive traffic.

Failover

Moves work to a healthy replica, region, or dependency after failure.

Redundancy

Provides extra components so one failure does not remove the capability.

Graceful Degradation

Keeps essential behavior available while optional features are reduced.

Checkpoints

Record progress so interrupted work can resume instead of restarting.

Backups

Keep recoverable copies of data outside the primary failure path.

Disaster Recovery

Defines how systems and data return after a major failure.

RPO

The maximum amount of recent data a recovery plan may lose.

RTO

The maximum acceptable time required to restore service.

Security

Authentication

Proves who a user or service is.

Authorization

Decides what an authenticated identity may do.

Least Privilege

Grants only the permissions required for the current responsibility.

Encryption In Transit

Protects data moving across networks, usually with TLS.

Encryption At Rest

Protects stored data and backups if storage is exposed.

Secrets Management

Stores, rotates, and audits credentials outside application code.

Zero Trust

Verifies every request instead of trusting its network location.

Threat Modeling

Identifies assets, attackers, entry points, and controls before incidents happen.

Observability

Logs

Record discrete events with enough context to explain behavior.

Metrics

Measure system behavior over time for dashboards and alerts.

Traces

Follow one request across services to reveal latency and failure paths.

Correlation IDs

Attach one identifier to related work across service boundaries.

SLI

A measured signal such as availability, latency, or correctness.

SLO

The reliability target a service intends to meet.

SLA

A formal reliability promise with business consequences.

Error Budgets

Translate an SLO into the amount of unreliability a team can spend.

Sampling

Records a useful subset when observing every event is too expensive.

Design Process

Functional Requirements

Describe what the system must let users accomplish.

Quality Requirements

Define expectations for scale, latency, reliability, security, and cost.

Data Model

Shapes entities and relationships around required reads and writes.

Access Patterns

Describe which data is read or written, how often, and in what combinations.

Bottlenecks

The resources or dependencies that limit total system capacity.

Hot Keys

Popular partition keys concentrate traffic and defeat otherwise balanced distribution.

Failure Domains

Components likely to fail together because they share infrastructure or dependencies.

Tradeoffs

Every design improves some qualities by spending complexity, cost, latency, or consistency.

Cost Modeling

Estimates compute, storage, transfer, and operational cost as usage grows.

Migration Planning

Moves systems through reversible stages while old and new paths coexist safely.

Validation

Uses tests, metrics, load, and failure exercises to prove design assumptions.