System design basics for freshers

In short

System design is the practice of choosing the components of a software system — web servers, load balancers, databases, caches, queues, storage — and the way they talk to each other, so that the system meets its functional needs while staying fast, available and affordable under real traffic. For a fresher it is examined less as "build Twitter" and more as "can you reason about load, bottlenecks, trade-offs and failure". It appears in Software Engineering and Distributed Systems papers of most AICTE/CBCS curricula, in GATE CS indirectly (through operating systems, DBMS transactions and computer networks), and almost universally in the design round at Indian product companies and service firms — Flipkart, Zomato, Swiggy, Razorpay, PhonePe, Zoho, Freshworks, Infosys, TCS Digital and Wipro Turbo. The core skill is a structured conversation: clarify requirements, estimate numbers, draw a high-level diagram, pick data stores, then attack the bottlenecks you created.

Explained

What a design question is actually asking

• Functional requirements (FR): what the system does — "shorten a URL", "post a photo", "book a ticket". List 3-5; do not expand scope.
• Non-functional requirements (NFR): how well it does it — latency, availability, consistency, durability, cost, security. These drive every architectural choice.
• Constraints: read-heavy or write-heavy, expected DAU, data retention, region (single-region India vs global), budget.
• HLD (High-Level Design): boxes and arrows — clients, API gateway, services, data stores, caches, queues. LLD (Low-Level Design): classes, interfaces, schemas, API signatures, design patterns. Freshers are usually asked HLD plus an LLD class diagram or schema.
• A standard answer order: requirements → scale estimation → API design → data model → high-level diagram → deep dive on one or two components → bottlenecks, failure modes, trade-offs.

Scalability: vertical, horizontal and statelessness

• Vertical scaling (scale up): bigger machine — more CPU, RAM, faster disk. Simple, no code change, but has a hard ceiling, is costly at the top end and leaves a single point of failure.
• Horizontal scaling (scale out): more machines behind a load balancer. Near-unlimited, cheap commodity nodes, fault-tolerant — but forces you to handle distributed state, consistency and coordination.
• Stateless services are the enabling trick: a server keeps no per-user data in local memory, so any request can go to any server. Push session state into a shared store (Redis) or a signed token.
• Sticky sessions (the load balancer pins a user to one server) are a crutch — they break even load distribution and lose state when that server dies.
• Load balancer algorithms: round-robin, weighted round-robin, least-connections, IP hash, least response time. Layer 4 balances on IP/port (fast, protocol-agnostic); Layer 7 reads HTTP and can route by path, host or header.
• Health checks remove unhealthy nodes from rotation; the LB itself must be redundant (active-passive pair with a floating IP) or it becomes the SPOF.
• Auto-scaling adds or removes instances on a metric (CPU %, request rate, queue depth) — needs a warm-up buffer because instances take 1-3 minutes to become useful.

Databases: SQL, NoSQL, replication, sharding

• Relational (SQL): MySQL, PostgreSQL. Fixed schema, joins, ACID transactions. Choose when data is structured and relationships and correctness matter — payments, orders, ledgers, inventory.
• ACID: Atomicity (all or nothing), Consistency (constraints hold), Isolation (concurrent transactions do not interfere), Durability (committed data survives crash).
• NoSQL families: key-value (Redis, DynamoDB), document (MongoDB), wide-column (Cassandra, HBase), graph (Neo4j). Flexible schema, horizontal scale, usually BASE — Basically Available, Soft state, Eventual consistency.
• Indexing: a B-tree index turns an O(n) table scan into roughly O(log n) lookup; the cost is slower writes and extra storage. Index the columns in WHERE, JOIN and ORDER BY.
• Replication: one primary takes writes, replicas serve reads. Gives read scaling, failover and backups. Asynchronous replication is fast but can lose the last writes on failover and shows replication lag (stale reads); synchronous is safe but slower.
• Sharding (horizontal partitioning): split rows across databases. Range-based (user_id 1-1M on shard 1) is simple but creates hotspots; hash-based spreads evenly but makes range queries hard; directory-based uses a lookup service — flexible, but the directory is a new SPOF.
• Consequences of sharding: no cross-shard joins, no cross-shard transactions without two-phase commit, resharding is painful, and a celebrity/hot key can overload one shard.
• Consistent hashing places servers and keys on a hash ring so adding or removing one node moves only about K/N keys instead of remapping everything; virtual nodes even out the distribution.
• Vertical partitioning splits columns (rarely-read blobs into a separate table/store); federation splits by function (users DB, orders DB, analytics DB).

CAP, consistency and availability

• CAP theorem (Brewer): during a network partition, a distributed system can keep either consistency (every read sees the latest write) or availability (every request gets a non-error response) — not both. When there is no partition you can have both C and A.
• CP systems: HBase, MongoDB (default), ZooKeeper, traditional RDBMS clusters — refuse or block rather than serve stale data. Use for payments, inventory, ticket seats.
• AP systems: Cassandra, DynamoDB (tunable), CouchDB — always answer, reconcile later. Use for feeds, likes, analytics, logs.
• PACELC extends it: else (no partition), choose between latency and consistency.
• Consistency models: strong (read-after-write always), eventual (converges if writes stop), causal, read-your-own-writes (route a user's reads to the primary after their write — a common practical fix).
• Availability maths: components in series multiply (99.9% × 99.9% = 99.8%); redundant components in parallel give 1 − (1−A)ⁿ. More dependencies in a chain means lower availability.
• Durability ≠ availability: data can be safe on disk (durable) while the service is down (unavailable).

Making it fast: caching, async work, CDNs at the design level

• Cache placement: client/browser, CDN edge, reverse proxy, application-level (Redis/Memcached), database query cache. Cache the expensive, frequently-read, rarely-changed data.
• Patterns: cache-aside (read-through on miss — the default), write-through (write both, consistent but slower), write-back (write cache, flush later — fast, risk of loss).
• Invalidation and TTL: stale data is the main cache bug. Use short TTLs, versioned keys, or explicit invalidation on write. Thundering herd / cache stampede when a hot key expires — fix with a lock, staggered TTL or background refresh.
• Asynchronous processing: anything the user need not wait for — sending an email/SMS, generating a PDF invoice, image resizing, analytics — goes on a message queue (Kafka, RabbitMQ, SQS) and is handled by workers. This decouples producers from consumers and absorbs traffic spikes.
• Idempotency: a retried request must not create a second order or a second payment. Use an idempotency key (client-generated UUID) stored with the result; return the stored result on a repeat.
• Rate limiting: protects from abuse and overload. Algorithms — fixed window (simple, allows a burst at the boundary), sliding window log/counter (accurate), token bucket (allows controlled bursts, most common), leaky bucket (smooths output).
• Back-pressure, timeouts, retries with exponential backoff and jitter, circuit breakers (stop calling a failing dependency for a while) and bulkheads (isolate thread pools) keep one slow service from taking the whole system down — the cascading failure pattern.
• Graceful degradation: show a cached/older feed, hide recommendations, queue the write — serve something rather than a 500.

Architecture styles and service communication

• Monolith: one deployable. Simple to build, test and debug; fast local calls. Scales only as a whole; one bad module can take down everything; large team coordination is hard.
• Microservices: independently deployable services owning their own data. Independent scaling and releases, technology freedom, fault isolation. Costs: network latency, distributed debugging, data consistency across services, heavier DevOps. Recommended path for a fresher to state: start monolithic, extract services when a clear bottleneck or team boundary appears.
• Communication: synchronous REST/gRPC when the caller needs an answer now; asynchronous events/queue when it does not. gRPC (HTTP/2, protobuf) is faster and typed; REST is simpler and universal.
• API gateway: single entry point doing routing, authentication, rate limiting, TLS termination, request aggregation. Service discovery (Consul, Eureka, Kubernetes DNS) lets services find each other's changing IPs.
• Data consistency across services: avoid distributed transactions; prefer the Saga pattern (a sequence of local transactions with compensating actions, e.g. refund if shipment fails) and the outbox pattern for reliable event publishing.
• Blob/object storage for files (S3-style) with the metadata in the database — never store images as BLOBs in the main transactional DB.
• Observability: metrics (Prometheus/Grafana), centralised logs (ELK), distributed tracing (correlation/trace IDs). State SLI/SLO — an indicator you measure (p99 latency) and the target you promise (p99 < 300 ms, 99.9% monthly).

Estimation and capacity planning

• Why: numbers decide the design. 100 requests per second needs one server; 100,000 needs a fleet, a cache and sharding.
• Method: DAU → requests per user per day → requests per second (average) → peak (typically 2-5× average, higher for an IPL match or a flash sale) → storage per record × records per day × retention → bandwidth = QPS × payload size.
• Read:write ratio decides the shape: a read-heavy 100:1 system is solved with caching and read replicas; a write-heavy system needs sharding, queues and LSM-tree stores like Cassandra.
• Rules of thumb (order-of-magnitude, not exact): memory read ~100 ns, SSD read ~100 µs, disk seek ~1-10 ms, same-datacentre round trip ~0.5 ms, Mumbai→Singapore round trip ~60-80 ms, Mumbai→US East ~220-250 ms. One modern server handles roughly 1,000-10,000 simple QPS depending on work per request.
• Indian regulatory context that affects design: the Digital Personal Data Protection Act, 2023 governs personal-data handling and requires breach notification to the Data Protection Board (from 13 May 2027 under the DPDP Rules 2025; until then IT Act s.43A and the SPDI Rules 2011 govern); the RBI mandates that payment-system data be stored only in India (payment data localisation, 2018 directive), which is why payment systems are designed on India-region infrastructure; CERT-In directions (2022) require logs to be maintained for 180 days and certain incidents reported within 6 hours. Details of these rules change — confirm the current text before quoting in a workplace.

Key points

• FR vs NFR — what the system does versus how fast, available and consistent it must be; NFRs drive the architecture.
• Vertical scaling — bigger machine, simple, has a ceiling and a single point of failure.
• Horizontal scaling — more machines behind a load balancer; needs stateless services.
• Stateless service — stores no per-user state locally, so any node can serve any request; state lives in Redis or a token.
• Load balancer — distributes traffic (round-robin, least-connections, IP hash); L4 on IP/port, L7 on HTTP content.
• Replication — primary for writes, replicas for reads; async replication causes replication lag and stale reads.
• Sharding — splitting rows across databases by range, hash or directory; kills cross-shard joins and transactions.
• Consistent hashing — hash ring that moves only ~K/N keys when a node joins or leaves.
• CAP theorem — under a partition, pick consistency or availability; no partition means you can have both.
• ACID vs BASE — strict transactional guarantees versus eventual consistency for scale.
• Cache-aside — check cache, miss → read DB → populate cache; the default caching pattern.
• Cache invalidation — stale data is the hardest cache problem; use TTL plus explicit invalidation on write.
• Message queue — decouples producers and consumers, absorbs spikes, enables async work and retries.
• Idempotency key — makes a retried write safe; essential in payments and order creation.
• Rate limiting — token bucket is the usual choice; protects against abuse and overload.
• Circuit breaker — stops calls to a failing dependency to prevent cascading failure.
• Single point of failure (SPOF) — any component whose death kills the system; remove by redundancy.
• Back-of-the-envelope estimation — DAU → QPS → peak QPS → storage and bandwidth; always state assumptions.

Formulas and examples

Example 1 — Capacity estimate for a URL shortener. Assume 100 million new URLs created per month, read:write ratio 100:1, each record 500 bytes, retention 5 years.

• Writes/s = 100,000,000 / (30 × 86,400) = 100,000,000 / 2,592,000 ≈ 38.6 writes/s
• Reads/s = 100 × 38.6 ≈ 3,860 reads/s
• Peak (×3) ≈ 11,580 reads/s
• Storage/month = 100,000,000 × 500 bytes = 50,000,000,000 B = 50 GB/month
• 5 years = 50 GB × 60 months = 3 TB; with replication factor 3 → 9 TB raw.
• Read bandwidth = 3,860 × 500 B ≈ 1,930,000 B/s ≈ 1.93 MB/s (about 15.4 Mbps).

Conclusion: writes are trivial, reads are 100× heavier — so cache the hot short-codes in Redis and use read replicas; a single sharded key-value store suffices.

Example 2 — Servers and availability for a food-delivery API. 2 million DAU, 20 API calls per user per day, each server handles 500 QPS, peak factor 4.

• Average QPS = (2,000,000 × 20) / 86,400 = 40,000,000 / 86,400 ≈ 463 QPS
• Peak QPS = 463 × 4 ≈ 1,852 QPS
• Servers = 1,852 / 500 = 3.7 → 4 servers, plus 1 for redundancy = 5 servers
• If each server is 99.9% available, 5 in parallel (need any one) = 1 − (0.001)⁵ ≈ 99.999999…% — but the *chain* matters: LB (99.99%) × app tier (99.99%) × DB (99.95%) = 0.9999 × 0.9999 × 0.9995 = 0.99930 ≈ 99.93%
• Downtime = (1 − 0.99930) × 525,600 ≈ 0.0007 × 525,600 ≈ 368 minutes/year (~6.1 hours). The database is the weakest link — add a replica with automatic failover.

Example 3 — Cache effectiveness. DB read = 20 ms, Redis read = 1 ms, cache hit rate 90%.

• Effective latency = (0.90 × 1) + (0.10 × 20) = 0.9 + 2.0 = 2.9 ms (about 7× faster than 20 ms).
• At 10,000 QPS, DB load falls from 10,000 QPS to 10,000 × 0.10 = 1,000 QPS.
• Raise the hit rate to 99%: (0.99 × 1) + (0.01 × 20) = 0.99 + 0.2 = 1.19 ms, DB load 100 QPS. The last few percent of hit rate matter most.

Example 4 — Little's Law for thread-pool sizing. A service receives 2,000 requests/s and each takes 150 ms (0.15 s) on average.

• L = λ × W = 2,000 × 0.15 = 300 concurrent requests in flight.
• With 4 servers, each must hold 75 concurrent requests — so a 100-thread pool per server, or an async/non-blocking model.
• If the downstream DB slows to 600 ms, L = 2,000 × 0.6 = 1,200 concurrent — the 400 available threads are exhausted, queues build and the service times out. This is why timeouts and circuit breakers exist.

Example 5 — Consistent hashing benefit. 1,000,000 cache keys across 10 nodes; add an 11th node.

• Plain modulo hashing (hash % N): N changes 10 → 11, so almost every key maps elsewhere — roughly 900,000+ keys (≈90%) move and the cache is effectively cold.
• Consistent hashing: keys moved ≈ K/N = 1,000,000 / 11 ≈ 90,909 keys (≈9%); the other 91% keep their hits.

Client → DNS → CDN (static) → Load Balancer → API Gateway (auth, rate limit)
                                                   │
                        ┌──────────────────────────┼──────────────────────┐
                        ▼                          ▼                      ▼
                  Service A (stateless)      Service B              Service C
                        │                          │                      │
              Redis cache ─── Primary DB ── Replicas          Queue (Kafka) → Workers
                                   │                                        │
                              Sharded by user_id                     Object store (files)

Where it is used

• E-commerce and quick commerce (Flipkart, Myntra, Zepto, Blinkit): flash-sale design — queueing, rate limiting, inventory consistency, cache warming before a sale.
• Fintech and payments (Razorpay, PhonePe, Paytm, NPCI/UPI stack): idempotency keys, exactly-once semantics via ledgers, CP data stores, RBI data-localisation constraints, four-nines availability targets.
• Food delivery and mobility (Swiggy, Zomato, Ola, Uber India): geospatial indexing, real-time location streams on Kafka, surge handling at peak dinner hours.
• Streaming and content (Hotstar/JioHotstar during IPL, Netflix India): CDN-first design, massive read fan-out, graceful degradation when concurrency spikes to tens of millions.
• Job roles: backend engineer, SDE-1, site reliability engineer (SRE), solutions architect, platform engineer — the design round is a standard SDE-1 interview stage at Indian product companies.
• Enterprise IT (TCS, Infosys, Wipro, Accenture): modernising monoliths into microservices on AWS/Azure for banking and insurance clients; HLD/LLD documents are formal deliverables.

Interview questions

• Q: What is the difference between vertical and horizontal scaling? A: Vertical scaling adds resources to one machine — simple but capped and a single point of failure; horizontal scaling adds more machines behind a load balancer, which scales further and tolerates failures but requires stateless services and distributed-state handling.
• Q: Explain the CAP theorem in one line and give a practical choice. A: During a network partition you can guarantee consistency or availability, not both; a payment ledger chooses consistency (CP) and refuses the write, while a social feed chooses availability (AP) and serves slightly stale data.
• Q: What makes a service stateless, and why does it matter? A: It keeps no client-specific data in local memory between requests — session state lives in Redis or a signed token — so any instance can serve any request, which allows free horizontal scaling and safe instance replacement.
• Q: When would you shard a database, and how would you pick the shard key? A: When a single node can no longer hold the data or serve the write load; pick a high-cardinality key that spreads load evenly and matches your dominant query pattern — usually user_id — so most queries hit a single shard.
• Q: What is consistent hashing and what problem does it solve? A: It maps servers and keys onto a hash ring so that adding or removing a node remaps only about K/N keys instead of nearly all of them, which prevents a mass cache miss when the cluster size changes.
• Q: Your read latency is high. Walk through how you would fix it. A: Measure first — find whether it is the application, the DB or the network; then add indexes and optimise queries, add a cache for hot reads, add read replicas, and only then consider sharding or denormalisation.
• Q: Why use a message queue instead of calling the service directly? A: It decouples producer and consumer, absorbs traffic spikes, lets slow work happen asynchronously, and gives retries and durability if the consumer is temporarily down.
• Q: What is idempotency and where is it essential? A: An operation that gives the same result however many times it is applied; it is essential for payments and order creation, where a client retry after a timeout must not charge the customer twice — implemented with a client-supplied idempotency key.
• Q: Monolith or microservices for a new startup? A: Start with a well-modularised monolith — it is faster to build, deploy and debug — and extract microservices only when a specific component needs independent scaling or a team boundary demands it.
• Q: How do you prevent a cascading failure? A: Set aggressive timeouts, retry with exponential backoff and jitter, use circuit breakers to stop calling a failing dependency, isolate resources with bulkheads, and degrade gracefully instead of returning errors.
• Q: How do you estimate the number of servers for 1 million daily users? A: Convert DAU and requests per user into average QPS, multiply by a peak factor of 2-5, divide by the QPS one server handles, then add redundancy — and state every assumption aloud.
• Q: What is a single point of failure, and how do you find one? A: Any component whose failure brings the system down; find it by walking the request path and asking "if this box dies, does the system still serve?" — the usual culprits are a lone load balancer, a single primary DB or a shared config service.

Common mistakes

• Jumping straight to a diagram without clarifying requirements or asking about scale, read:write ratio and consistency needs.
• Giving no numbers — an estimate with stated assumptions beats a vague "it will be scalable".
• Designing for a billion users when the question implies a thousand; over-engineering is penalised as much as under-engineering.
• Reciting "use microservices and Kafka" as a default without naming the cost (operational complexity, latency, distributed debugging).
• Saying CAP means "pick two of three" — C and A are both available when there is no partition; the choice only arises during one.
• Adding a cache or a load balancer and forgetting it is now a SPOF or a stale-data source — ignoring invalidation and redundancy.
• Confusing latency (time for one request) with throughput (requests per second), and quoting an average latency when p95/p99 is what users feel.
• Forgetting to multiply storage by the replication factor and retention period in capacity estimates.

Related topics

Caching, queues and CDNs · REST APIs · Cloud computing basics · Containers: Docker and Kubernetes · System design case studies · Authentication and authorisation: sessions, OAuth and JWT