System Design: A Beginner's Guide

Software Design Software Design 8 min read 1558 words Beginner ExcellentWiki Editorial Team

System design is the process of defining the architecture, components, modules, interfaces, and data flow of a system to satisfy specified requirements. It bridges the gap between product requirements and implementation.

This guide covers the fundamental concepts of designing large-scale systems — scalability, load balancing, caching, databases, and common architectural patterns.

Scalability

Scalability is a system’s ability to handle increased load by adding resources.

Vertical Scaling (Scale Up)

Add more power to a single machine — more CPU, more RAM, faster disk. Simple but limited by hardware constraints and increasingly expensive at high ends.

Horizontal Scaling (Scale Out)

Add more machines to the pool. Virtually unlimited, but introduces complexity — you need load balancing, distributed coordination, and data consistency strategies.

The CAP Theorem

A distributed system can guarantee at most two of three properties:

Consistency — Every read receives the most recent write or an error
Availability — Every request receives a non-error response
Partition Tolerance — The system continues operating despite network failures

In practice, you must choose between CP (consistency + partition tolerance) and AP (availability + partition tolerance). Network partitions are inevitable, so you cannot sacrifice partition tolerance.

Load Balancing

Load balancers distribute incoming traffic across multiple servers. They prevent any single server from becoming a bottleneck.

Algorithms

Round Robin — Requests distributed sequentially
Least Connections — Sends to the server with fewest active connections
IP Hash — Client IP determines which server receives the request (useful for sticky sessions)
Weighted Distribution — Servers with higher capacity receive more requests

Layer 4 vs Layer 7

Layer 4 load balancers operate at the transport layer (TCP/UDP) — fast but limited to IP and port routing. Layer 7 load balancers operate at the application layer (HTTP/HTTPS) — they can inspect request content, route based on URL or headers, and handle SSL termination.

Caching

Caching stores frequently accessed data in a faster storage layer, reducing latency and load on backend systems.

Cache Strategies

Cache-Aside: Application checks cache first. On a miss, it loads from the database and populates the cache.

def get_user(user_id):
    user = cache.get(f"user:{user_id}")
    if user is None:
        user = db.query("SELECT * FROM users WHERE id = ?", user_id)
        cache.set(f"user:{user_id}", user, ttl=300)
    return user

Write-Through: Every write goes through the cache to the database. Read performance is excellent, but write latency increases.

Write-Behind: Writes go to cache, then asynchronously propagate to the database. Fast writes but risk data loss if the cache fails.

Cache Eviction Policies

LRU (Least Recently Used) — Removes items not accessed for the longest time
TTL (Time To Live) — Items expire after a fixed duration
LFU (Least Frequently Used) — Removes items accessed least often

CDN (Content Delivery Network)

CDNs cache static assets (images, CSS, JavaScript, video) at edge locations worldwide. Users download from the nearest server, dramatically reducing latency.

Databases

SQL vs NoSQL

System Design Interview Framework

System design follows a structured process regardless of the specific system being designed:

Step 1: Requirements Clarification

Before proposing solutions, understand the scope. Ask about:

Functional requirements — What should the system do? (e.g., “Users should be able to upload and share photos”)
Non-functional requirements — What qualities matter? (e.g., “99.99% availability, <200ms latency, eventually consistent”)
Scale — How many users? How much data? Read/write ratio?

Step 2: High-Level Design

Propose a high-level architecture showing the main components and their interactions. A typical design includes:

Clients — Web, mobile, API consumers
Load balancer — Distributes traffic across servers
Application servers — Handle business logic
Database — Primary data store (SQL or NoSQL)
Cache — Redis/Memcached for hot data
CDN — Static asset delivery
Message queue — Async task processing
Search — Elasticsearch for full-text search

Step 3: Deep Dive

Drill into specific components that address the most interesting challenges. For a URL shortener, this might be the ID generation scheme. For a chat system, it is the webSocket connection management and message ordering. For a news feed, it is the fan-out strategy (push vs pull vs hybrid).

Step 4: Scale and Trade-offs

Discuss how the design scales and what trade-offs were made. Common scaling strategies:

Vertical scaling — Bigger machines (simple but limited)
Horizontal scaling — More machines (complex but elastic)
Database replication — Read replicas for read-heavy workloads
Database sharding — Distribute data across databases
Caching — Reduce database load for hot data
CDN — Serve static content from edge locations
Async processing — Decouple slow operations from request path

Consistent Hashing

Consistent hashing distributes data across a cluster of nodes while minimizing reorganization when nodes are added or removed. Each node is assigned one or more points on a hash ring. Data keys hash to a point on the ring and are stored on the nearest node clockwise. When a node is added, only the data between the new node and its predecessor must be redistributed. When a node is removed, only data from that node moves to the next node on the ring. Virtual nodes (multiple hash ring positions per physical node) improve load distribution.

Leader Election

Distributed systems require leader election when exactly one node must coordinate an operation. Common algorithms include:

Bully Algorithm — The node with the highest ID becomes leader. Simple but O(n^2) messages in the worst case.
Raft — Uses randomized election timeouts to avoid split votes. Nodes are in one of three states: follower, candidate, or leader. Raft is designed for understandability and is used in etcd and Consul.
Zab — Used by ZooKeeper. Similar to Paxos but optimized for primary-backup replication.

CAP Theorem Trade-offs

The CAP theorem states that a distributed system can provide at most two of three guarantees: Consistency (all nodes see the same data at the same time), Availability (every request receives a response), and Partition Tolerance (the system continues operating despite network partitions). In practice, partitions are inevitable, so the choice is between CP (sacrifice availability during partitions — typical for financial systems) and AP (sacrifice consistency — typical for social media feeds). Most systems choose AP with eventual consistency.

Database Indexing

Indexes speed up read queries at the cost of slower writes and increased storage. Use B-tree indexes for range queries and equality checks. Use hash indexes for point lookups.

Sharding

Sharding distributes data across multiple database instances. Choose a shard key that evenly distributes data and query load.

Replication

Replication copies data to multiple nodes. Leader-based replication: one leader accepts writes, followers replicate. Multi-leader and leaderless replication provide higher availability but increase conflict resolution complexity.

Message Queues

Message queues decouple services and handle asynchronous communication.

queue.publish("order_created", {
    "order_id": 123,
    "user_id": 456,
---)

def handle_order_created(message):
    order = message.body
    send_confirmation_email(order)

Popular systems: RabbitMQ, Apache Kafka, Amazon SQS, Redis Streams.

API Gateway

An API gateway is a single entry point for all client requests. It handles request routing, authentication, rate limiting, caching, and response transformation. It simplifies clients (they talk to one endpoint) and provides a consistent security layer.

Capacity Estimation

System design interviews and real-world planning require capacity estimation. Calculate expected traffic: daily active users × requests per user per day. Estimate storage: average data size per entity × number of entities. Estimate bandwidth: average response size × requests per second. These rough calculations determine whether a monolithic or distributed architecture is appropriate and guide technology choices.

Database Selection

Choose databases based on data access patterns. Relational databases (PostgreSQL, MySQL) suit structured data with complex queries and transactions. Document stores (MongoDB) suit flexible schemas. Key-value stores (Redis) suit caching and session management. Column stores (Cassandra) suit time-series and analytics. Graph databases (Neo4j) suit relationship-heavy data like social networks. Many systems use multiple databases — a polyglot persistence approach.

Monitoring and Observability

Metrics — CPU, memory, request latency, error rates (Prometheus, Grafana)
Logging — Structured logs for debugging and audit trails
Tracing — Distributed tracing across services (Jaeger, Zipkin)

Conclusion

Design large systems by starting simple and adding complexity only when needed. Understand the trade-offs: consistency vs availability, read performance vs write performance, vertical vs horizontal scaling. Practice with common design problems — design a URL shortener, a chat system, or a news feed — to build intuition.

FAQ

How do I start learning system design? Study the fundamentals (CAP theorem, caching, load balancing, database scaling), then practice designing common systems: URL shortener, chat system, rate limiter, news feed, and web crawler.

What is the most important system design concept? Understanding trade-offs. Every design decision involves a trade-off — consistency vs availability, read performance vs write performance, simplicity vs scalability. Good system design is about choosing the right trade-off for your requirements.

When should I use a monolith vs microservices? Start with a monolith. Extract services only when you need independent scaling, independent deployment, or team autonomy. Premature microservices add complexity without benefit.

How do I estimate capacity for a system design interview? Start with daily active users, estimate requests per user per day, then calculate requests per second. Estimate storage per entity and multiply by the number of entities. Estimate bandwidth from average response size and requests per second.

Share this article

X LinkedIn Facebook Email