System Design Visual Encyclopedia

Translates domain names to IP addresses via hierarchical tree lookups, utilizing BGP Anycast to direct users to the topologically closest network PoP.

Foundational distributed computing paradigm where client applications initiate request packets and distributed servers listen and compute responses.

Decomposes a single unified deployment unit into autonomous, independently deployable services organized around business bounded contexts.

Vertical scaling upgrades a single node (CPU/RAM). Horizontal scaling adds stateless worker nodes across an autoscaling pool behind a load balancer.

Distributes incoming client requests across a pool of backend servers using algorithms like Round Robin, Least Connections, and IP Hash.

An intermediary proxy server that sits in front of web servers to terminate TLS, compress payloads (Brotli/Gzip), cache responses, and hide internal network topologies.

Single unified gateway entry point for microservices that handles JWT verification, rate limiting, protocol translation (REST to gRPC), and request routing.

SQL databases enforce structured schemas and ACID transactions with relations. NoSQL systems optimize for flexible schemas, horizontal partition scale, and eventual consistency.

Self-balancing search tree data structure that keeps data sorted and allows searches, sequential access, insertions, and deletions in logarithmic O(log N) time.

Guarantees Atomicity (all-or-nothing), Consistency (rules preserved), Isolation (concurrent safety), and Durability (WAL commits survive power loss).

In any distributed data store, network partitions (P) are inevitable. When a partition occurs, the system must choose between Consistency (CP) or Availability (AP).

A primary leader node accepts all write transactions and streams change logs to secondary read replicas to scale read QPS and enable high availability.

Partitions massive database tables across independent physical server nodes based on a partition key (e.g. hash(user_id) % N).

High-speed in-memory key-value data stores positioned in front of slower relational databases to reduce read latency from 20ms to <1ms.

A geographically distributed network of proxy edge servers that caches static assets (images, videos, JS/CSS) and terminates TLS close to end users.

The application code explicitly queries the cache first. On a cache miss, it reads from the database and populates the cache for subsequent requests.

Data is written simultaneously to the cache and the primary database in a single synchronized transaction before acknowledging the client.

Writes are acknowledged immediately upon being written to RAM cache. A background worker daemon asynchronously batches and writes changes to the database.

The cache automatically predicts hot keys nearing TTL expiration and asynchronously refreshes their values from the database before clients experience a cache miss.

Occurs when a heavily requested hot key expires and thousands of concurrent requests simultaneously miss the cache and overwhelm the underlying database.

A space-efficient probabilistic data structure that tests whether an element is a member of a set. False positives are possible, but false negatives are impossible.

Point-to-point asynchronous buffers where producer services enqueue jobs, and competing consumer workers process tasks at their own decoupled pace.

Publishers broadcast events to topics without knowing who the subscribers are. Multiple independent consumer groups read from the same topic simultaneously.

Systems coordinate state changes through state-change notifications and events rather than synchronous request/reply HTTP/RPC calls.

Divides large software domains into discrete logical boundaries where ubiquitous language and domain models are strictly defined and isolated.

Separates read models from write models. Commands mutate state using normalized relational databases; queries read from denormalized read-optimized views.

Instead of storing only the current state of an entity, systems persist an immutable append-only sequence of domain events that represent every state change.

Isolates core business domain logic from external dependencies (HTTP frameworks, databases, messaging brokers) via explicit interface ports and pluggable adapters.

Organizes software into concentric rings (Entities -> Use Cases -> Interface Adapters -> Frameworks) governed by the Dependency Inversion Principle.

A dedicated infrastructure layer that transparently handles service-to-service communication, mutual TLS encryption, traffic routing, retries, and observability via sidecar proxies.

Serverless compute scales to zero to eliminate idle server costs, but new container provisioning incurs cold start latency penalties.

Decomposes monolithic web frontend applications into independent, semi-autonomous frontend apps composed together at runtime or build time.

Decentralized peer-to-peer communication protocol where nodes periodically exchange membership and state updates with randomly selected peers.

Minimizes database bottlenecks by distributing application processing units and shared in-memory data grids across replicated RAM spaces.

Data processing architecture that balances latency, throughput, and fault tolerance by running two parallel paths: a real-time Speed Layer and a comprehensive Batch Layer.

Simplifies big data pipelines by eliminating the dual batch layer. Everything is treated as a continuous event stream processed through a single streaming engine.

A fundamental durability mechanism where database modifications are sequentially appended to non-volatile disk logs before changes are written to table pages in memory.

Log-Structured Merge-Trees write updates to an in-memory sorted MemTable. When full, MemTables flush immutably to disk as SSTables, which are compacted in the background.

Manages distributed multi-service transactions via a centralized coordinator service that explicitly tells each participant which local transactions to execute and rollback.

Decentralized distributed transaction pattern where microservices communicate via published domain events without any central orchestrator.

A blocking atomic commitment protocol where a coordinator asks all participating distributed database nodes to prepare to commit, and only commits if all nodes vote yes.

An extension of 2PC that avoids blocking during coordinator crashes by introducing a Pre-Commit phase and timeout-based state transitions.

Guarantees reliable message publishing by writing business entity updates and message payloads into the same relational database in a single local ACID transaction.

Captures row-level database changes by reading the database replication log (PostgreSQL WAL or MySQL Binlog) and streaming events to Kafka in real time.

Synchronizes secondary read replicas from primary database write nodes using physical or logical streaming replication logs.

Precomputes and physically stores the result of expensive multi-table aggregation queries on disk to serve heavy analytics lookups in under 1ms.

Maintains a cached pool of reusable established database connections to eliminate the heavy latency overhead of repeatedly opening and tearing down TCP & TLS connections.

The deterministic algorithmic strategy used by routers to direct database queries to the correct physical shard node based on entity attributes.

A logical clock mechanism used to determine causal partial ordering of events and detect concurrent conflicting updates in leaderless distributed databases.

Multiple data center leader nodes accept write transactions concurrently. Writes are replicated asynchronously between leaders, requiring conflict resolution.

Monitors downstream network calls. When failures exceed a threshold, it trips the circuit to OPEN, failing immediately without waiting for timeouts to prevent cascading system collapse.

Partitions system resources (CPU threads, memory pools, connection slots) into isolated compartments so a failure in one area cannot sink the entire ship.

Retries transient network failures by progressively doubling wait times (2s, 4s, 8s) combined with random full jitter to prevent synchronized retry waves.

Clients attach a unique UUID idempotency key to mutating requests. The server records the key and cached response in Redis/DB to prevent duplicate executions.

When downstream services or databases fail, the application falls back to cached data, static defaults, or simplified views rather than crashing with an HTTP 500.

Because distributed messaging brokers guarantee at-least-once delivery, consumer workers must track processed message IDs to discard duplicate messages safely.

Poison pill messages that continuously fail processing after max retry attempts are moved to a Dead Letter Queue (DLQ) to prevent blocking the main pipeline.

Periodic lightweight ping signals exchanged between nodes and orchestrators to rapidly identify crashed or unresponsive instances.

A consensus algorithm designed to be understandable, dividing consensus into Leader Election, Log Replication, and Safety invariants.

Under severe overload, the system proactively drops lower-priority requests (e.g. background telemetry) to guarantee resources for critical user transactions.

Multiple geographical regions run concurrently, each serving live customer traffic and cross-syncing data. If one region dies, DNS/Anycast redirects traffic to the other.

A primary active region processes 100% of live traffic while a secondary standby region remains idle, taking over only when the primary suffers catastrophic failure.

Compares human-readable text serialization (JSON) against strictly-typed compact binary serialization (Protocol Buffers) for internal service communication.

A feedback mechanism where a slow consumer signals an upstream fast producer to throttle its emission rate, preventing consumer memory buffer exhaustion.

A workflow pattern where a coordinator splits a large job into multiple parallel sub-tasks (Fan-Out), and collects and merges the results once completed (Fan-In).

Routes packets across private, optimized fiber backbones rather than the public congested internet, bypassing BGP routing inefficiencies.

Multiple worker instances listen to a single shared message queue. Each message is delivered to and processed by exactly one competing consumer worker.

Overcomes message broker payload size limits (e.g. SQS 256KB limit) by storing the large payload in blob storage and sending a tiny reference pointer ticket in the queue message.

Inspects the internal payload or metadata of incoming messages and dynamically routes them to specific destination queues based on message values.

Combines multiple related individual messages belonging to the same correlation ID into a single unified composite message before downstream processing.

A standard HTTP mechanism allowing servers to continuously push real-time text data to browsers over a single persistent HTTP connection.

Establishes a persistent, bidirectional, full-duplex TCP socket connection between client and server via an initial HTTP Upgrade handshake.

The client sends an HTTP request, and the server hangs open the connection until new data is available or a timeout occurs, after which the client immediately repolls.

Combines multiple independently maintained subgraph services into a single unified GraphQL schema served through a central gateway router.

Creates dedicated backend gateway services tailored specifically to the needs of individual frontend client types (e.g. Mobile iOS BFF vs Desktop Web BFF).

Controls the rate of incoming network traffic to protect services from abusive traffic, scraping, DDoS attacks, and resource starvation.

A bucket holds up to B tokens and refills at a constant rate R tokens/sec. Each incoming request consumes 1 token. If no tokens remain, requests are dropped.

Requests enter a FIFO queue bucket. The bucket leaks requests out to the backend at a strictly constant smooth rate, smoothing out spiky traffic.

Tracks request timestamps in a sorted set (Redis ZSET) to compute precise request counts within any rolling 60-second window, eliminating boundary burst bugs.

Maps both server nodes and cache keys onto a circular hash ring (0 to 2^32-1). When a server node is added or removed, only K/N keys need to be remapped.

Coordinates mutually exclusive access to shared resources across multiple independent servers and processes in a distributed cluster.

High-throughput system that generates unique 7-character base62 short keys for long URLs, serving 100,000+ redirect queries per second with sub-5ms latency.

Massively scalable real-time messaging architecture serving millions of concurrent WebSocket connections, handling 1-on-1 and group chats with message persistence.

Calculates the optimal lowest-latency or lowest-cost routing path across weighted graphs representing road networks or computer networks.

Systematic approaches to visiting all nodes in hierarchical tree and graph data structures (DOM trees, AST parsers, B-Tree index pages).

A compact, URL-safe means of representing signed claims (Header.Payload.Signature) transferred between client and microservices statelessly.

Industry-standard authorization framework that allows third-party applications to obtain limited access to user accounts without sharing passwords.

The cryptographic protocol that establishes authenticated, end-to-end encrypted HTTPS communication over TCP in a single 1-RTT round trip.

Restricts system access by grouping permissions into predefined roles (Admin, Editor, Viewer) and assigning users to roles.

Evaluates access dynamically using boolean policy rules combining Subject attributes, Resource attributes, Action attributes, and Environmental context.

Allows resource servers to query the authorization server to determine the active state and meta-information of an opaque access token.

Both the client and the server authenticate each other using cryptographic X.509 certificates before establishing an encrypted tunnel.

Monitors, inspects, and filters HTTP/HTTPS packets traveling to web applications, shielding servers from SQL injection, cross-site scripting (XSS), and bot scrapers.

Automates testing, linting, container building, and deployment of code updates, accelerating release velocity from months to minutes.

Maintains two identical production environments (Blue and Green). One serves live traffic while the other is updated. Traffic is cut over instantly at the router.

Rolls out new software to a tiny subset of users (e.g. 2%), monitors error rates and latency, and incrementally shifts remaining traffic if health checks pass.

Incrementally updates instances of an application by replacing old pods/servers with new ones one by one or in small batches.

A declarative infrastructure practice where Git is the single source of truth. Automated controller agents continuously reconcile cluster state to match Git.

Clones and mirrors real incoming production traffic asynchronously to a newly deployed shadow version without affecting live user responses.

Enables modifying system behavior and releasing features to specific user cohorts at runtime without deploying new code or restarting services.

Technique for modifying relational database schemas (e.g. renaming columns, adding indexes) without locking tables or interrupting active user traffic.

Detects discrepancies between declared IaC configuration files (Terraform/OpenTofu) and actual real-world cloud resource states caused by manual console edits.

Architectural audit challenge: Identifying any single component whose failure will cause an entire distributed platform to become unavailable.

Augments LLM prompt context by retrieving semantically relevant text chunks from private vector databases before generating answers.

A unified reverse proxy sitting between applications and LLM providers that manages semantic caching, cost rate-limiting, and automated failover.

An autonomous agent paradigm that interleaves reasoning (Thought), tool execution (Action), and observation (Observation) to solve multi-step problems.

Combines sparse keyword search (BM25) with dense semantic vector search via Reciprocal Rank Fusion (RRF), followed by a Cohere cross-encoder reranker.

Programmable safety filters placed around LLM inputs and outputs to prevent prompt injection, PII leakage, toxic outputs, and unauthorized system access.

A model architecture where a learned gating router dynamically routes each token to a specialized subset of feed-forward expert networks (e.g. 2 of 8 experts active).

Accelerates LLM inference by using a tiny draft model (e.g. 1B) to generate K candidate tokens, which are verified in parallel by the target model (e.g. 70B) in a single forward pass.

Freezes pre-trained base model weights and injects trainable low-rank rank-decomposition matrices into attention layers, cutting trainable parameters by 99%.

Combines vector retrieval with structured knowledge graphs (Entities, Relationships, Communities) to answer complex multi-hop global questions.

Prompts LLMs to break down complex problems into explicit intermediate reasoning steps, sampling multiple diverse reasoning paths to take a majority vote.

Aligns LLMs with human preferences directly on pairs of chosen vs rejected responses without training an explicit reinforcement learning (PPO) reward model.

Hierarchical Navigable Small World graphs organize high-dimensional vectors into multi-layer skip-list graphs for sub-10ms Approximate Nearest Neighbor (ANN) search.

A multi-agent coordination architecture where lightweight autonomous agents with specific capabilities hand off conversations to specialized peer agents dynamically.

Enforces 100% adherence to strict JSON Schemas during LLM token generation by masking out invalid token logits that violate Context-Free Grammars (CFG).

Uses a high-capability frontier model (e.g. GPT-4o) to evaluate and score the output quality of smaller models on rubrics like correctness, tone, and safety.

Replaces fragile manual prompt engineering with algorithmic compilation. Optimizers (e.g. MIPROv2, BootstrapFewShot) automatically synthesize optimal prompts and few-shot examples.

An agent evaluates its own intermediate outputs against linters, unit tests, or self-critique prompts, iteratively refining code until tests pass.

Evaluates incoming prompt complexity to dynamically route simple queries to cheap/fast models (e.g. GPT-4o-mini) and hard queries to expensive reasoning models (e.g. o1/Claude 3.5).

One LLM generates candidate solutions while a second evaluator LLM provides targeted critique, feeding feedback back in an iterative improvement loop.

A central orchestrator LLM breaks a large complex task into subtasks, delegates them to parallel worker LLMs, and synthesizes the outputs into a coherent result.

Multiple independent LLM agents propose differing perspectives or solutions and critique each other’s arguments across rounds to reach high-confidence consensus.

Caches LLM responses by query embedding similarity rather than exact string equality. If a new question is semantically identical (e.g. Cosine > 0.96), serves from cache in 2ms.

Partitions high-dimensional vector space into Voronoi cells using k-means clustering. Queries only search vectors inside the nearest neighboring centroids.

Extracts and stores knowledge as RDF triplets (Subject -> Predicate -> Object) to perform multi-hop graph graph traversals during retrieval.

Techniques to prevent untrusted user inputs from overriding system instructions (indirect prompt injection, jailbreaks, data exfiltration).

Automated post-processing heuristics and repair passes that fix common formatting anomalies (missing brackets, trailing commas, markdown fences) in LLM outputs.

Extends Chain-of-Thought by exploring multiple reasoning branches as a tree, using search heuristics (BFS/DFS) and backtracking to find the optimal solution.

Rewrites, expands, or breaks complex user questions into multiple sub-queries before querying vector stores (e.g. Sub-Question Querying, HyDE).

Submits the same question to multiple distinct prompt variations (or diverse expert personas) and aggregates the predictions via majority voting or meta-synthesis.

Tracks user requests as they traverse across 20+ microservices using propagated `trace_id` and `span_id` context headers.

Collects, parses, and centralizes structured JSON logs across thousands of server containers into an indexed search engine for rapid troubleshooting.

Time-series numeric telemetry monitoring system performance (CPU, Memory, Request Rate, Error Rate, Duration - RED metrics).

Aggregates health statuses across internal microservices, third-party payment providers, and databases into a public or internal status dashboard.