Scalability

CAP Theorem

In a distributed system, you can only guarantee two out of three:

In practice, network partitions happen — so you must choose between CP (consistent but may be unavailable during a partition) or AP (available but may serve stale data).

• CP — ZooKeeper, HBase, traditional SQL
• AP — Cassandra, DynamoDB, CouchDB

Load Balancers

A load balancer distributes incoming traffic across multiple servers to prevent any one server from being overwhelmed.

Algorithms

• Round Robin — requests distributed in rotation
• Least Connections — route to the server with fewest active connections
• IP Hash — same client always goes to same server (sticky sessions)
• Weighted Round Robin — more powerful servers get more traffic

Layer 4 vs Layer 7

• L4 (Transport) — routes by IP/TCP without looking at content; faster
• L7 (Application) — can route by URL, headers, cookies; enables path-based routing

Caching

Caching stores frequently accessed data in fast memory (RAM) to reduce DB load and response latency.

Cache Strategies

• Cache-Aside (Lazy Loading) — app checks cache first; on miss, loads from DB and writes to cache. Most common.
• Write-Through — write to cache and DB simultaneously. Consistent but slower writes.
• Write-Back — write to cache only; flush to DB asynchronously. Fast writes, risk of data loss.

Eviction Policies

• LRU — Least Recently Used (most common)
• LFU — Least Frequently Used
• TTL — expire after a time period

Tools

• Redis — in-memory data store; supports strings, hashes, lists, sets. Use for session, leaderboards, pub/sub.
• Memcached — simple key-value cache; faster for pure caching use cases.

Databases — SQL vs NoSQL

Database Replication

• Master-Slave — writes go to master, reads from replicas. Most common.
• Multi-Master — writes to any node; complex conflict resolution.

Message Queues

Message queues decouple producers from consumers, enabling async processing and absorbing traffic spikes.

Use Cases

• Sending emails/notifications asynchronously after a user action
• Processing uploads (video encoding, image resizing) in the background
• Distributing work across multiple worker instances
• Event-driven microservices communication

Tools

• Kafka — high-throughput event streaming; replay messages; used at Netflix, LinkedIn
• RabbitMQ — traditional message broker; flexible routing; easier to set up than Kafka
• AWS SQS — fully managed queue; great for AWS-native apps

CDN (Content Delivery Network)

A CDN caches static assets (images, CSS, JS, videos) on servers geographically close to users, reducing latency and origin server load.

How It Works

1. User requests cdn.example.com/logo.png
2. CDN routes to the nearest edge server (PoP)
3. If cached → serve immediately (cache hit)
4. If not → fetch from origin, cache, then serve (cache miss)

Tools

• CloudFront — AWS CDN, integrates with S3 and EC2
• Cloudflare — CDN + DDoS protection + DNS
• Fastly — programmable CDN, used by GitHub, Stripe

Microservices Architecture

Microservices is an architectural style that structures an application as a collection of small, independently deployable services. Each service owns a specific business capability, its own database, and communicates with others over well-defined APIs (REST, gRPC, or message queues).

Monolith vs Microservices

When Microservices Make Sense

• Teams are large and need to deploy independently without stepping on each other
• Different parts of the system have radically different scaling needs (e.g. order processing vs. recommendation engine)
• Parts of the system need different reliability SLAs (payments vs. notifications)
• You need to mix technologies — ML in Python, APIs in Java, real-time in Go

22 Patterns — 7 Groups

Decomposition Patterns

// order-service/src/main/java/com/example/order/OrderServiceApplication.java
@SpringBootApplication
public class OrderServiceApplication {
    public static void main(String[] args) {
        SpringApplication.run(OrderServiceApplication.class, args);
    }
}

@RestController
@RequestMapping("/api/orders")
public class OrderController {
    @Autowired private OrderRepository orderRepository;

    @GetMapping("/{id}")
    public Order getOrder(@PathVariable Long id) {
        return orderRepository.findById(id).orElseThrow();
    }
}
// order-service/src/main/resources/application.yml
// Each service has its OWN database — no sharing
spring:
  datasource:
    url: jdbc:postgresql://orders-db:5432/orders_db
    username: ${DB_USERNAME}
    password: ${DB_PASSWORD}

Domain
  │
  ├── Core Domain (competitive advantage — invest most here)
  │     └── Identity Service  ←  Auth Service + Profile Service
  │
  ├── Supporting Subdomain (needed but not differentiating)
  │     └── Catalog Service  ←  Product Service + Search Service
  │
  └── Generic Subdomain (commodity — buy or outsource)
        └── Shipping  ←  delegated to 3rd party (FedEx API)

// Package structure reflects bounded context — not technical layers
com.example
  ├── identity/          // Core domain — auth + profile (same bounded context)
  │     ├── auth/
  │     │     ├── AuthService.java
  │     │     └── TokenRepository.java
  │     └── profile/
  │           ├── ProfileService.java
  │           └── ProfileRepository.java
  │
  ├── catalog/           // Supporting subdomain
  │     ├── product/
  │     └── search/
  │
  └── shipping/          // Generic — thin adapter over 3rd party API
        └── ShippingGateway.java

// Spring Cloud Gateway — route specific paths to new services,
// everything else falls through to the monolith
spring:
  cloud:
    gateway:
      routes:
        - id: order-service
          uri: http://order-service:8081
          predicates:
            - Path=/api/orders/**
        - id: payment-service
          uri: http://payment-service:8082
          predicates:
            - Path=/api/payments/**
        - id: monolith-fallback
          uri: http://legacy-monolith:8080
          predicates:
            - Path=/**          # catch-all — everything else still goes to monolith

API & Communication Patterns

// build.gradle — Spring Cloud Gateway dependency
// implementation 'org.springframework.cloud:spring-cloud-starter-gateway'

// application.yml
spring:
  cloud:
    gateway:
      routes:
        - id: order-service
          uri: lb://order-service          # lb:// uses service discovery
          predicates:
            - Path=/api/orders/**
          filters:
            - AddRequestHeader=X-Internal-Auth, ${INTERNAL_SECRET}
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 100
                redis-rate-limiter.burstCapacity: 200
        - id: user-service
          uri: lb://user-service
          predicates:
            - Path=/api/users/**

Client
  │
  ▼
Aggregator Service
  │   (calls all 3 in parallel)
  ├──▶  Order Service    → { orderId, status, items }
  ├──▶  User Service     → { name, email, address }
  └──▶  Inventory Service → { stockLevel, warehouse }
  │
  ▼  (merges responses)
{ order: {...}, user: {...}, inventory: {...} }
  │
  ▼
Client  ←  single response, one round trip

@Service
public class OrderDetailAggregator {
    private final WebClient webClient;

    public Mono<OrderDetailResponse> aggregate(Long orderId, Long userId) {
        Mono<OrderDto> order = webClient.get()
            .uri("http://order-service/api/orders/{id}", orderId)
            .retrieve().bodyToMono(OrderDto.class);

        Mono<UserDto> user = webClient.get()
            .uri("http://user-service/api/users/{id}", userId)
            .retrieve().bodyToMono(UserDto.class);

        Mono<InventoryDto> inventory = webClient.get()
            .uri("http://inventory-service/api/stock/{id}", orderId)
            .retrieve().bodyToMono(InventoryDto.class);

        // Parallel calls — waits for the slowest, not the sum of all
        return Mono.zip(order, user, inventory)
            .map(tuple -> new OrderDetailResponse(
                tuple.getT1(), tuple.getT2(), tuple.getT3()));
    }
}

Browser assembles page from independent fragments:

┌──────────────────────────────────────────────────┐
│  [ Header Fragment ]       ← nav team            │
├────────────────────────┬─────────────────────────┤
│  [ Product Detail ]    │  [ Cart Widget ]        │
│  ← catalog team        │  ← cart team            │
├────────────────────────┴─────────────────────────┤
│  [ Review Section Fragment ]  ← review team      │
└──────────────────────────────────────────────────┘

Each fragment fetches its own data from its own service.

// Each team exposes a REST endpoint for their fragment's data

// catalog-service — Product Detail fragment data
@GetMapping("/api/fragments/product/{id}")
public ProductFragmentDto getProductFragment(@PathVariable Long id) {
    return productService.getFragmentData(id);
}

// cart-service — Cart Widget fragment data
@GetMapping("/api/fragments/cart/{userId}")
public CartFragmentDto getCartFragment(@PathVariable Long userId) {
    return cartService.getFragmentData(userId);
}

// Thymeleaf server-side composition alternative:
// th:replace="~{fragments/cart :: cart-widget}"

Data Management Patterns

// order-service/application.yml — its OWN datasource
spring:
  datasource:
    url: jdbc:postgresql://orders-db:5432/orders_db
    username: ${DB_USERNAME}
    password: ${DB_PASSWORD}
  jpa:
    hibernate:
      ddl-auto: validate

// OrderRepository — only Order Service can touch this
public interface OrderRepository extends JpaRepository<Order, Long> {
    List<Order> findByUserIdOrderByCreatedAtDesc(Long userId);
}

// If User Service needs order data, it calls the API:
// GET http://order-service/api/orders?userId=123
// NEVER: SELECT * FROM orders_db.orders WHERE user_id = 123

Order Service ──┐
                 │
User Service  ───┼──▶  ⚠️  SHARED DATABASE  ⚠️
                 │         (single schema, single connection pool)
Inventory Svc ──┘

Problems:
- Order Service changes orders table → breaks User Service JOIN
- Inventory Service slow query → exhausts connection pool → all services hang
- Cannot deploy services independently (schema migrations are global)

// ANTI-PATTERN: Two services pointing to same datasource URL
// order-service/application.yml
spring.datasource.url: jdbc:postgresql://shared-db:5432/app_db

// user-service/application.yml  ← same URL — BAD
spring.datasource.url: jdbc:postgresql://shared-db:5432/app_db

// FIX: Each service gets its own datasource
// order-service/application.yml
spring.datasource.url: jdbc:postgresql://orders-db:5432/orders_db

// user-service/application.yml
spring.datasource.url: jdbc:postgresql://users-db:5432/users_db

@Entity
@Table(name = "event_store")
public class DomainEvent {
    @Id @GeneratedValue
    private Long id;
    private String aggregateId;
    private String eventType;
    @Column(columnDefinition = "jsonb")
    private String payload;
    private Instant occurredAt;
}

public class OrderAggregate {
    private String status;
    private List<OrderItem> items;
    private BigDecimal total;

    // Rebuild current state by replaying stored events
    public static OrderAggregate reconstituteFrom(List<DomainEvent> events) {
        OrderAggregate order = new OrderAggregate();
        for (DomainEvent event : events) {
            order.apply(event);
        }
        return order;
    }

    private void apply(DomainEvent event) {
        switch (event.getEventType()) {
            case "OrderCreated"    -> this.status = "PENDING";
            case "ItemAdded"       -> this.items.add(parseItem(event.getPayload()));
            case "PaymentReceived" -> this.status = "PAID";
            case "OrderShipped"    -> this.status = "SHIPPED";
        }
    }
}

Client
  │
  ├──▶  Command Side (writes)
  │       OrderCommandHandler
  │         └──▶  Write DB (normalized, ACID)
  │                 │
  │                 └──▶ publishes OrderCreatedEvent
  │                           │
  │                           ▼
  └──▶  Query Side (reads)   EventListener updates Read Model
          OrderQueryHandler
            └──▶  Read DB (denormalized, pre-joined, fast)
                    e.g. order_summary view with user name already embedded

// Command side — write to normalized DB
@Service
public class OrderCommandHandler {
    @Autowired private OrderWriteRepository writeRepo;
    @Autowired private ApplicationEventPublisher events;

    @Transactional
    public void handle(CreateOrderCommand cmd) {
        Order order = new Order(cmd.getUserId(), cmd.getItems());
        writeRepo.save(order);
        events.publishEvent(new OrderCreatedEvent(order.getId(), order));
    }
}

// Read side — query from denormalized read model
@Service
public class OrderQueryHandler {
    @Autowired private OrderReadRepository readRepo;

    public OrderSummaryDto getOrderSummary(Long orderId) {
        return readRepo.findOrderSummaryById(orderId);
        // returns pre-joined view: order + user name + item count
    }
}

// Bridge — EventListener keeps read model in sync
@Component
public class OrderReadModelUpdater {
    @EventListener
    public void on(OrderCreatedEvent event) {
        // update the denormalized read model table
        readRepo.upsertOrderSummary(buildSummary(event));
    }
}

@Service
public class CheckoutSagaOrchestrator {
    @Autowired private OrderServiceClient orderClient;
    @Autowired private PaymentServiceClient paymentClient;
    @Autowired private InventoryServiceClient inventoryClient;

    public void execute(CheckoutRequest req) {
        String orderId = null;
        String paymentId = null;
        try {
            orderId = orderClient.createOrder(req);
            paymentId = paymentClient.reservePayment(req.getUserId(), req.getAmount());
            inventoryClient.reserveItems(req.getItems());
            // All steps succeeded
        } catch (InventoryException e) {
            // Compensate in reverse order
            if (paymentId != null) paymentClient.releasePayment(paymentId);
            if (orderId != null)   orderClient.cancelOrder(orderId);
            throw new CheckoutFailedException("Out of stock", e);
        }
    }
}

Resilience Patterns

// build.gradle
// implementation 'io.github.resilience4j:resilience4j-spring-boot3'

// application.yml
resilience4j:
  circuitbreaker:
    instances:
      paymentService:
        slidingWindowSize: 10
        failureRateThreshold: 50        # open if 50% of 10 calls fail
        waitDurationInOpenState: 10s
        permittedNumberOfCallsInHalfOpenState: 2

// Service
@Service
public class OrderService {
    @CircuitBreaker(name = "paymentService", fallbackMethod = "paymentFallback")
    public PaymentResult chargePayment(PaymentRequest request) {
        return paymentClient.charge(request);
    }

    public PaymentResult paymentFallback(PaymentRequest request, Exception ex) {
        // Return a graceful fallback — queue for retry or notify user
        return PaymentResult.pending("Payment service temporarily unavailable");
    }
}

ANTI-PATTERN — Shared thread pool:
┌─────────────────────────────────────────────┐
│  Thread Pool: 200 threads (shared by all)   │
│  Order calls + Payment calls + User calls   │
│  → Payment service slows → eats all 200     │
│  → Order calls and User calls also starve   │
└─────────────────────────────────────────────┘

BULKHEAD — Isolated pools:
┌─────────────────┐  ┌──────────────────┐  ┌────────────────┐
│ Order Service   │  │ Payment Service  │  │ User Service   │
│ Thread Pool: 50 │  │ Thread Pool: 50  │  │ Thread Pool:100│
└─────────────────┘  └──────────────────┘  └────────────────┘
Payment pool exhausted → Order and User pools unaffected

// application.yml
resilience4j:
  bulkhead:
    instances:
      paymentService:
        maxConcurrentCalls: 50
        maxWaitDuration: 100ms
  thread-pool-bulkhead:
    instances:
      inventoryService:
        maxThreadPoolSize: 20
        coreThreadPoolSize: 10
        queueCapacity: 50

// Service
@Service
public class CheckoutService {
    @Bulkhead(name = "paymentService", type = Bulkhead.Type.THREADPOOL)
    public CompletableFuture<PaymentResult> processPayment(PaymentRequest req) {
        return CompletableFuture.supplyAsync(() -> paymentClient.charge(req));
    }
}

Infrastructure Patterns

  Order Service ─┐
                  ├──▶ register on startup
  User Service  ─┤                          ┌─────────────────────┐
                  ├──▶ register on startup   │   Service Registry  │
  Payment Svc   ─┘                          │  (Eureka / Consul)  │
                                             │                     │
                                             │ order-service:      │
Client-Side Discovery:                       │   10.0.0.1:8081     │
  Client ──▶ Registry: "where is order-svc?"│   10.0.0.2:8081     │
  Registry ──▶ Client: [10.0.0.1, 10.0.0.2] └─────────────────────┘
  Client picks one, calls directly

Server-Side Discovery:
  Client ──▶ Load Balancer ──▶ Registry: "which order-svc instance?"
  LB picks an instance, routes request (client doesn't know IPs)

// Eureka Server — discovery-server/application.yml
spring.application.name: discovery-server
eureka.client.register-with-eureka: false
eureka.client.fetch-registry: false

// Eureka Server main class
@SpringBootApplication
@EnableEurekaServer
public class DiscoveryServerApplication { ... }

// Each microservice — application.yml
spring.application.name: order-service
eureka.client.service-url.defaultZone: http://discovery-server:8761/eureka/

// Each microservice — main class
@SpringBootApplication
@EnableDiscoveryClient
public class OrderServiceApplication { ... }

// Calling another service by name — no hardcoded IP
@Configuration
public class WebClientConfig {
    @Bean
    @LoadBalanced                   // resolves service names via Eureka
    public WebClient.Builder webClientBuilder() {
        return WebClient.builder();
    }
}

// Usage: http://order-service resolves to a healthy instance
webClient.get().uri("http://order-service/api/orders/{id}", id)

Kubernetes Pod
┌────────────────────────────────────────────────┐
│  ┌──────────────────────┐  ┌─────────────────┐ │
│  │  Main App Container  │  │ Sidecar (Envoy) │ │
│  │  (Spring Boot)       │  │                 │ │
│  │  Business logic only │  │ mTLS            │ │
│  │  No TLS, no metrics  │  │ Rate limiting   │ │
│  │  No service mesh     │  │ Metrics export  │ │
│  └──────────────────────┘  │ Access logging  │ │
│          ▲                 └───────┬─────────┘ │
│          │ localhost traffic       │            │
└──────────┼─────────────────────────┼────────────┘
           │                         │
  Inbound requests              Outbound to
  via sidecar proxy             Prometheus, Jaeger

# kubernetes/order-service-deployment.yaml
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: order-service          # Main app — pure business logic
          image: order-service:1.2.0
          ports:
            - containerPort: 8080

        - name: envoy-proxy            # Sidecar — all infrastructure concerns
          image: envoyproxy/envoy:v1.28
          ports:
            - containerPort: 9901      # admin port
          volumeMounts:
            - name: envoy-config
              mountPath: /etc/envoy

# Spring Boot app does NOT need to know the sidecar exists.
# Istio can inject Envoy automatically via admission webhook.

                 ┌──────────────────────────────────┐
                 │  Config Store                    │
                 │  (Spring Cloud Config / Vault /  │
                 │   Kubernetes ConfigMap + Secrets) │
                 └───────┬──────────┬───────────────┘
                         │          │
            ┌────────────┘          └───────────────┐
            ▼                                       ▼
  Order Service                           Payment Service
  pulls: db.url, timeout.ms               pulls: stripe.key, db.url

  Dev env  → db.url=jdbc:postgresql://localhost
  Staging  → db.url=jdbc:postgresql://staging-db
  Prod     → db.url=jdbc:postgresql://prod-db-cluster
  (same config key, different values per environment)

// Config Server — config-server/application.yml
spring:
  application.name: config-server
  cloud.config.server.git:
    uri: https://github.com/myorg/config-repo
    default-label: main

// Each microservice — bootstrap.yml
spring:
  application.name: order-service
  config:
    import: "configserver:http://config-server:8888"

// Config repo: order-service.yml (loaded per service name)
order:
  timeout-ms: 5000
  max-retry: 3
spring.datasource.url: jdbc:postgresql://orders-db:5432/orders_db

// @RefreshScope enables live reload without restart
@RestController
@RefreshScope
public class OrderController {
    @Value("${order.timeout-ms}")
    private int timeoutMs;
}

Observability Patterns

Load Balancer pings /actuator/health on all instances:

  Instance 1 → UP   { db: UP, redis: UP, downstream: UP }   ✓ receives traffic
  Instance 2 → DOWN { db: DOWN, redis: UP }                  ✗ removed from rotation
  Instance 3 → UP   { db: UP, redis: UP, downstream: UP }   ✓ receives traffic

Kubernetes:
  livenessProbe  → if fails, container is restarted
  readinessProbe → if fails, pod removed from Service endpoints (no traffic)

// build.gradle
// implementation 'org.springframework.boot:spring-boot-starter-actuator'

// application.yml
management:
  endpoints:
    web:
      exposure:
        include: health,info,metrics,prometheus
  endpoint:
    health:
      show-details: always

// Custom health indicator — checks downstream payment service
@Component
public class PaymentServiceHealthIndicator implements HealthIndicator {
    @Autowired private PaymentServiceClient paymentClient;

    @Override
    public Health health() {
        try {
            paymentClient.ping();
            return Health.up().withDetail("payment-service", "reachable").build();
        } catch (Exception ex) {
            return Health.down().withDetail("payment-service", ex.getMessage()).build();
        }
    }
}

# kubernetes/deployment.yaml — liveness and readiness probes
livenessProbe:
  httpGet:
    path: /actuator/health/liveness
    port: 8080
  initialDelaySeconds: 30
  periodSeconds: 10
readinessProbe:
  httpGet:
    path: /actuator/health/readiness
    port: 8080
  initialDelaySeconds: 10
  periodSeconds: 5

Trace ID: abc-123-xyz — GET /checkout

  Order Service     |████████████████████████████| 0–50ms
    Payment Service     |████████████████| 10–40ms
    Inventory Service     |████████████| 15–35ms
      DB Query                |████| 20–30ms

Each block = one span. Indentation = call hierarchy.
Trace assembled from spans reported by each service independently.

Span metadata: traceId, spanId, parentSpanId, service, operation, startMs, durationMs, tags

// build.gradle — Spring Boot 3 + Micrometer Tracing + Zipkin
// implementation 'io.micrometer:micrometer-tracing-bridge-brave'
// implementation 'io.zipkin.reporter2:zipkin-reporter-brave'

// application.yml
management:
  tracing:
    sampling:
      probability: 1.0         # 100% in dev; use 0.1 (10%) in prod
spring:
  zipkin:
    base-url: http://zipkin:9411

// Trace ID is automatically propagated via HTTP headers:
//   traceparent: 00-abc123xyz-def456-01  (W3C standard)
//   X-B3-TraceId: abc123xyz  (Zipkin B3 format)

// WebClient and RestTemplate propagate headers automatically.
// Custom span for a specific operation:
@Service
public class InventoryService {
    @Autowired private Tracer tracer;

    public StockResult checkStock(Long itemId) {
        Span span = tracer.nextSpan().name("inventory.checkStock").start();
        try (Tracer.SpanInScope ws = tracer.withSpan(span)) {
            return inventoryRepository.findStock(itemId);
        } finally {
            span.end();
        }
    }
}

Spring Boot Actuator
  /actuator/prometheus  ←  Prometheus scrapes every 15s
                                    │
                                    ▼
                            Prometheus (time-series DB)
                                    │
                                    ▼
                             Grafana Dashboard

  Request Rate:    120 req/s
  Error Rate:        0.2%
  P99 Latency:     245 ms
  DB Pool Active:   15 / 20 connections

Alerting: if error_rate > 1% for 5 minutes → PagerDuty alert

// build.gradle
// implementation 'io.micrometer:micrometer-registry-prometheus'

// application.yml
management:
  endpoints.web.exposure.include: prometheus,health,info
  metrics.export.prometheus.enabled: true

// Custom metrics in service code
@Service
public class OrderService {
    private final Counter orderCounter;
    private final Timer orderTimer;

    public OrderService(MeterRegistry registry) {
        this.orderCounter = Counter.builder("orders.created")
            .tag("region", "india")
            .description("Total orders created")
            .register(registry);
        this.orderTimer = Timer.builder("orders.processing.time")
            .description("Order processing latency")
            .register(registry);
    }

    public Order createOrder(OrderRequest req) {
        return orderTimer.record(() -> {
            Order order = processOrder(req);
            orderCounter.increment();
            return order;
        });
    }
}

Order Service  ─┐  JSON logs to stdout
User Service   ─┤  JSON logs to stdout    Fluentd agent
Payment Svc    ─┘  JSON logs to stdout ──▶ (on each node) ──▶ Elasticsearch
                                                                      │
                                                               Kibana Dashboard
                                                            (search, filter, alert)

Each log line includes:
{
  "timestamp": "2025-01-15T10:23:44Z",
  "level": "INFO",
  "service": "order-service",
  "traceId": "abc-123",      ← correlates with distributed trace
  "userId": "u-456",
  "message": "Order created",
  "orderId": "ord-789"
}

// build.gradle — Logstash encoder for structured JSON logs
// implementation 'net.logstash.logback:logstash-logback-encoder:7.4'

// src/main/resources/logback-spring.xml
<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder class="net.logstash.logback.encoder.LogstashEncoder">
      <customFields>{"service":"order-service"}</customFields>
    </encoder>
  </appender>
  <root level="INFO"><appender-ref ref="STDOUT"/></root>
</configuration>

// MDC injects traceId and userId into every log line automatically
@Component
public class MdcFilter implements Filter {
    @Override
    public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain)
            throws IOException, ServletException {
        HttpServletRequest httpReq = (HttpServletRequest) req;
        MDC.put("traceId", httpReq.getHeader("X-Trace-Id"));
        MDC.put("userId",  httpReq.getHeader("X-User-Id"));
        try {
            chain.doFilter(req, res);
        } finally {
            MDC.clear();
        }
    }
}

Deployment Patterns

# docker-compose.yml — two service instances
services:
  order-service-blue:
    image: order-service:1.0.0
    ports: ["8080:8080"]
    environment:
      - SPRING_PROFILES_ACTIVE=prod

  order-service-green:
    image: order-service:1.1.0
    ports: ["8081:8080"]
    environment:
      - SPRING_PROFILES_ACTIVE=prod

# nginx.conf — toggle upstream to switch blue↔green
upstream order_backend {
    server order-service-blue:8080;   # comment out to switch to green
    # server order-service-green:8081;
}

# Kubernetes — blue-green via Service selector switch
# Blue deployment has label: version=blue
# Green deployment has label: version=green
# Service selector: { version: blue }  ← change to green to switch
kubectl patch service order-service \
  -p '{"spec":{"selector":{"version":"green"}}}'

Rate Limiting

Rate limiting controls how many requests a client can make in a time window, protecting APIs from abuse and DDoS attacks.

Algorithms

• Token Bucket — tokens refill at a fixed rate; burst allowed up to bucket capacity. Most common.
• Leaky Bucket — requests processed at a constant rate regardless of burst; smooths traffic.
• Fixed Window Counter — count requests per time window; simple but vulnerable to edge spikes.
• Sliding Window Log — accurate but memory-intensive.

Implementation

• Store counters in Redis (fast, shared across server instances)
• Return 429 Too Many Requests with a Retry-After header
• Use Nginx limit_req module for simple cases

Authentication & Authorization

Two foundational, often-confused concerns in any distributed system:

• Authentication (AuthN) — who are you? Verifying identity (login, tokens, biometrics).
• Authorization (AuthZ) — what are you allowed to do? Granting/denying access to resources.

At scale these are usually delegated to a dedicated Identity Provider using open standards — OAuth 2.0 and OpenID Connect — so individual services never handle passwords directly.

OAuth 2.0 & OpenID Connect (OIDC)

OAuth 2.0 is a delegated authorization framework — it lets an app access a user's data in another service without sharing passwords. OIDC is a thin authentication (identity) layer built on top of OAuth 2.0.

The 4 Key Roles

• Resource Owner — the user who owns the data
• Client — the app requesting access (your Spring Boot app)
• Authorization Server — validates identity and issues tokens (Google, Okta, Keycloak)
• Resource Server — the API being accessed (e.g., Google Calendar API)

Authorization Code Flow (8 Steps)

Token Types

OAuth 2.0 vs OIDC

12-Factor App with Spring Boot

The 12-Factor methodology is a set of best practices for building scalable, maintainable, cloud-native applications. Spring Boot is built to align with these principles.

Factor 3 (Config) — Most Important in Practice

Case Study: Design a URL Shortener

Design a system like bit.ly or TinyURL — convert an arbitrarily long URL into a short 6-7 character code, and redirect anyone who visits that code back to the original URL instantly. Deceptively simple on the surface, this problem covers ID generation, caching strategy, and a read-heavy scale challenge.

Step 1 — Functional Requirements

• User submits a long URL → system returns a unique short code (e.g. 7ds.in/aB3kR)
• User visits a short URL → system redirects to the original URL
• Links can have an optional custom alias (e.g. 7ds.in/summer-sale)
• Links can have an optional expiry date — after which they return 410 Gone
• Basic click analytics: total clicks and last click timestamp per short URL

Out of scope: user accounts, teams, spam/phishing detection, QR code generation, A/B redirect, paid tiers.

Step 2 — Non-Functional Requirements & Scale Estimates

Step 3 — System's API & Sequence of Events

API Design

POST   /v1/urls                           → { shortCode, shortUrl, expiresAt }   # create
GET    /{shortCode}                        → 302 Redirect to longUrl              # redirect
GET    /v1/urls/{shortCode}                → { longUrl, createdAt, expiresAt }    # lookup
GET    /v1/urls/{shortCode}/stats          → { clicks, lastClickAt, createdAt }   # analytics
DELETE /v1/urls/{shortCode}                → 204 No Content                       # owner delete

Create Short URL — Sequence of Events

── Sync: shorten and store ──
Client       →  API Gateway   : POST /v1/urls { longUrl, alias?, expiresAt? }
API Gateway  →  URL Service   : forward (rate-limit · validate URL format)
URL Service  →  ID Generator  : get uniqueId (Twitter Snowflake — 64-bit, distributed)
ID Generator →  URL Service   : uniqueId → Base62 encode → shortCode  (e.g. "aB3kR", 6 chars)
URL Service  →  MySQL         : INSERT (shortCode, longUrl, createdAt, expiresAt)
URL Service  →  Redis         : SET url:aB3kR longUrl EX 86400   # warm the cache on write
URL Service  →  Client        : { shortCode: "aB3kR", shortUrl: "7ds.in/aB3kR" }

Redirect — Sequence of Events

── Cache HIT (fast path, ~3ms) ──
Client       →  API Gateway   : GET /aB3kR
API Gateway  →  URL Service   : forward
URL Service  →  Redis         : GET url:aB3kR
Redis        →  URL Service   : ✓ HIT → longUrl
URL Service  →  Kafka         : publish CLICK_EVENT { shortCode, ts, ip, userAgent }   # fire-and-forget
URL Service  →  Client        : 302 Redirect → longUrl

── Cache MISS (cold path — first hit or cache eviction) ──
URL Service  →  MySQL         : SELECT longUrl, expiresAt WHERE shortCode='aB3kR'
MySQL        →  URL Service   : longUrl (or 404 if not found / 410 if expired)
URL Service  →  Redis         : SET url:aB3kR longUrl EX 86400   # re-warm cache
URL Service  →  Client        : 302 Redirect → longUrl

── Async: analytics aggregation ──
Analytics    ←  Kafka         : consume CLICK_EVENT
Analytics    →  MySQL         : UPSERT url_stats (shortCode, clicks += 1, lastClickAt)

Visual Sequence Diagram

Step 4 — Functional Architecture

Step 5 — Addressing Non-Functional Requirements

High Availability (99.99%)

• URL Service is stateless → N replicas across 3 AZs behind a load balancer
• MySQL: primary (writes) + 2 read replicas across AZs; automatic failover via RDS Multi-AZ
• Redis: ElastiCache cluster mode with Multi-AZ automatic failover (<60s)
• If Redis is completely down, URL Service falls back to MySQL — latency degrades but redirects keep working

Redirect Latency <10ms

• Redis cache hit — sub-millisecond lookup; 302 returned in <3ms end-to-end for cached URLs
• Cache warming on write — every newly created URL is immediately written to Redis, so the very first redirect is always a cache hit
• CDN edge caching — viral links cached at 200+ PoPs globally; zero origin hops for repeated accesses in the same region
• Analytics is async — CLICK_EVENT published to Kafka after the redirect; never blocks the response

Short Code Uniqueness — Why Not Hash?

• Hash (MD5 + truncate) — fast but collision-prone; two URLs can produce the same 6-char prefix. Requires retry loop.
• Auto-increment + Base62 — simple, zero collisions, but requires a central counter (single point of failure).
• Snowflake ID + Base62 — distributed, collision-free, time-sortable, no coordination. Best for production. Base62 with 6 chars = 62⁶ = 56.8B unique codes.
• 301 vs 302: Use 302 (Temporary) so every redirect hits our server and can be counted for analytics. 301 would be cached by browsers, bypassing analytics.

── Final Architecture Summary ──
Client
  │
  ▼
CDN (CloudFront — caches hot 302 redirects at edge)
  │
Load Balancer (Multi-AZ)
  │
  ├──▶ API Gateway (rate-limit · auth · routing)
  │         │
  │    URL Service (stateless, N replicas)
  │    │      │
  │    │    Redis Cache  (url:{shortCode} → longUrl, TTL 24h)
  │    │      │  (cache miss only)
  │    │    MySQL Read Replica   (SELECT longUrl)
  │    │
  │    ├──▶ MySQL Primary  (INSERT on create)
  │    └──▶ Kafka → Analytics Worker → MySQL (url_stats)
  │
  └──── Cleanup Worker  (cron: DELETE WHERE expiresAt < NOW())

Case Study: Design a Scalable News Feed

Design a Facebook/Twitter-style home feed — show each user a personalised, paginated list of posts from people they follow. This is one of the most common and nuanced system design interview questions because it forces you to reason about fan-out trade-offs at massive scale.

Step 1 — Functional Requirements

• Users can create posts (text, image links, URLs)
• Users can follow / unfollow other users
• Users see a home feed — paginated, reverse-chronological list of posts from people they follow
• Users can like posts
• Users can comment on posts

Out of scope: DMs, Stories, Reels, ads, sponsored content, full-text search, trending topics.

Step 2 — Non-Functional Requirements & Scale Estimates

Step 3 — System's API & Sequence of Events

API Design

POST /v1/posts                        → { postId, createdAt, mediaUrl? }
GET  /v1/feed?userId=&after=&limit=20  → { posts:[], nextCursor }
POST /v1/likes                         → { postId, userId, totalLikes }
POST /v1/follows                       → { followerId, followeeId }
DELETE /v1/follows/{followeeId}        → 204 No Content

Create Post — Sequence of Events

── Sync: create post and notify followers ──
Client       →  API Gateway   : POST /v1/posts { text, mediaUrl?, userId }
API Gateway  →  Post Service  : forward (auth validated, rate-limited)
Post Service →  Post DB       : INSERT post (postId, userId, text, mediaUrl, createdAt)
Post Service →  Kafka         : publish POST_CREATED { postId, userId, createdAt }
Post Service →  Client        : { postId, createdAt }

── Async: fan-out to follower feeds ──
Feed Worker  ←  Kafka         : consume POST_CREATED
Feed Worker  →  Follow DB     : GET followers of userId (paginated, up to 10K)
Feed Worker  →  Redis         : ZADD feed:{followerId} createdAt postId  (per follower)
# Celebrity users (>10K followers): skip fan-out; pull-merge on feed load

Load Feed — Sequence of Events

── User opens their home feed ──
Client       →  API Gateway   : GET /v1/feed?userId=U123&after=&limit=20
API Gateway  →  Feed Service  : forward
Feed Service →  Redis         : ZREVRANGE feed:U123 0 19  → [postId1, postId2, ...]
# For celebrity followees: pull-merge their latest posts into result
Feed Service →  Post DB       : MGET post:{postId1}, post:{postId2}, ...  (batch fetch)
Feed Service →  Client        : { posts: [20 hydrated post objects], nextCursor }

── Async: notification delivery ──
Notify Worker ← Kafka         : consume POST_CREATED
Notify Worker →  Push Service : SEND push notification to active followers
Push Service  →  FCM/APNs     : deliver "New post from @username"

Visual Sequence Diagram

Step 4 — Functional Architecture

Step 5 — Addressing Non-Functional Requirements

High Availability

• Post Service and Feed Service are stateless — N replicas across multiple AZs behind a load balancer
• PostgreSQL: primary + read replicas across AZs; automatic failover
• Redis cluster with Multi-AZ replication; if Redis is unavailable, Feed Service falls back to pulling posts directly from PostgreSQL

Feed Load Latency <200ms

• Redis ZREVRANGE is O(log N + K) — returns top 20 posts in under 1ms for any user
• Post metadata batch-fetched with a single MGET (one Redis round-trip for all 20 posts)
• Fan-out happens asynchronously — feed is pre-built before the user requests it

Celebrity Fan-Out Problem

• A celebrity with 10M followers posting = 10M Redis ZADD operations — blocks the Feed Worker for minutes
• Hybrid approach: for users with >10K followers, skip fan-out on write. On feed load, the Feed Service pulls the celebrity's latest posts from Post DB and merges them into the sorted result
• This trades a tiny bit of feed-load latency for a massive reduction in fan-out write amplification

── Final Architecture Summary ──
Client
  │
Load Balancer (Multi-AZ)
  │
  ├──▶ API Gateway (rate-limit · auth)
  │         │
  │    ┌────┴──────────────┐
  │    ▼                   ▼
  │  Post Service      Feed Service
  │    │      │             │
  │    │    Kafka        Redis Feeds
  │    │      │          (ZREVRANGE)
  │    │    ┌─┴───────────────┐
  │    │    ▼                 ▼
  │    │  Feed Worker    Notify Worker
  │    │  (ZADD feeds)   (FCM/APNs)
  │    │       │
  │    │    Follow DB (pgSQL)
  │    │
  │    └──▶ Post DB (pgSQL — read replicas for hydration)

Software Architect's Mindset

Before diving into components and diagrams, the most important thing is how you think. Senior engineers and architects approach system design with three foundational mindsets that separate good designs from great ones.

1. Abstraction

Abstraction means thinking in black boxes — defining what a component does (its inputs, outputs, and contract) without worrying about how it does it. This lets you reason about a system at the right altitude.

2. Every System is Unique

There is no universal template. Two companies building "an image sharing app" may have radically different requirements, constraints, and trade-offs.

• Scale differs — a startup with 1,000 users has different needs than Instagram with 2 billion. Designing for billion-user scale on day one wastes money and complexity.
• Consistency vs. availability trade-offs differ — a banking app needs strong consistency (ACID). A social media feed is fine with eventual consistency (AP system).
• Budget and team size differ — a 5-person team cannot operate a 30-microservice Kubernetes cluster. Operational complexity is a real cost.
• Latency requirements differ — a trading system needs sub-millisecond latency. A batch reporting system can tolerate minutes.
• Read/write ratios differ — a read-heavy news feed optimises for read throughput. A write-heavy IoT time-series system optimises for write throughput.

3. System Design Doesn't Have One Correct Solution

Unlike algorithm problems with optimal solutions, system design has many valid answers, each with different trade-offs. Your job is to make informed decisions and justify them clearly.

System Design — Step-by-step Process

A consistent, structured process prevents you from jumping straight to architecture before you understand what you're building. Use this five-step framework for every system design problem.

Step 1 — Gathering Functional Requirements

Functional requirements define what the system does — the features and use cases it must support. Never start designing until these are clear.

What to ask / clarify

• Who are the users and what actions can they take?
• Which features are in scope for this design? (Explicitly out-of-scope items save time)
• What is the core user journey (happy path)?
• Are there different user roles? (e.g., uploader vs. viewer, rider vs. driver)

Example — Image Sharing Platform:
✅ In scope: upload image, view image, follow users, view home feed
❌ Out of scope: comments, DMs, Stories, payments, ads

Step 2 — Gathering Non-Functional Requirements

Non-functional requirements define how well the system must perform. These drive your architectural choices far more than functional requirements do.

Key dimensions

• Scale — How many total users? How many DAU? What is the peak QPS (queries per second)?
• Availability — 99.9% (8.7 hrs downtime/year)? 99.99% (52 min/year)?
• Latency — p99 read latency target? Write latency? (e.g., feed loads in < 200ms)
• Consistency — Is eventual consistency acceptable, or do you need strong/linearisable reads?
• Durability — Can any data ever be lost? (especially critical for user-generated content)
• Storage — How much data is written per day? Growth rate?

Scale estimation — Image Sharing Platform:
Users: 500M total, 10M DAU
Uploads: 2M images/day  →  ~23 uploads/sec
Reads:   200M/day       →  ~2,300 reads/sec  (100:1 read/write ratio)
Storage: 2M × 3MB avg   =   6 TB/day new data
Feed:    10M users × 5 loads/day = 50M feed requests/day

Step 3 — Defining System's API & Sequence of Events

Before drawing boxes and arrows, define the API contracts and draw a sequence diagram for the core flows. This forces you to think about data ownership and service responsibilities before you commit to an architecture.

API definition

REST API — Image Sharing Platform:
POST   /v1/images                     → Upload image (multipart)
GET    /v1/images/{imageId}           → Image metadata
GET    /v1/images/{imageId}/file      → Image binary (usually via CDN)
POST   /v1/users/{userId}/follow      → Follow a user
DELETE /v1/users/{userId}/follow      → Unfollow a user
GET    /v1/users/{userId}/feed        → Paginated home feed
POST   /v1/images/{imageId}/likes     → Like an image

Sequence of events

Upload flow:
Client  →  API Gateway   : POST /v1/images
API GW  →  Image Service : forward request
Image Service  →  S3            : PUT image bytes → returns s3Key
Image Service  →  Metadata DB   : INSERT (imageId, userId, s3Key, caption, createdAt)
Image Service  →  Kafka         : publish { event: IMAGE_UPLOADED, imageId, userId }
Image Service  →  Client        : 201 { imageId, imageUrl }

[Async] Feed Worker     ← Kafka : consume IMAGE_UPLOADED
        Feed Worker  →  User Service : GET followers(userId)
        Feed Worker  →  Redis        : ZADD feed:{followerId} score=timestamp member=imageId

[Async] Thumb Worker    ← Kafka : consume IMAGE_UPLOADED
        Thumb Worker →  S3      : store resized thumbnails (320px, 640px, 1080px)

View feed flow:
Client  →  API Gateway  : GET /v1/users/{userId}/feed?limit=20
API GW  →  Feed Service : forward
Feed Service  →  Redis         : ZREVRANGE feed:{userId} 0 19
Feed Service  →  Metadata DB   : batch-fetch metadata for returned imageIds
Feed Service  →  Client        : [{ imageId, caption, thumbnailUrl, createdAt, likesCount }]

Step 4 — Designing for Functional Requirements

Now draw the functional architecture — the components required to fulfil the use cases, without worrying about scale yet. Keep it simple; every box must justify its existence.

Step 5 — Addressing Non-Functional Requirements

Take the functional architecture and layer on the components that meet your scale, availability, and latency targets. This is where you earn your NFR SLAs.

• Availability — Deploy every service in multiple AZs. Add health checks and circuit breakers. S3 and managed Redis/Kafka handle AZ failures transparently.
• Read latency — Put CloudFront CDN in front of S3. Cache images at edge nodes globally. Users in Mumbai get images from Mumbai PoP, not US-East.
• Write throughput — Use pre-signed S3 URLs: Image Service generates a URL, client uploads directly to S3 — bypasses your app servers for large payloads entirely.
• DB read scale — Add read replicas to PostgreSQL. Feed and profile reads hit replicas; writes go to primary.
• Feed fan-out for celebrities — Users with >1M followers skip write fan-out (too expensive). Their posts are fetched on read and merged with the pre-built feed (hybrid fan-out).
• Storage growth — 6 TB/day → use S3 Intelligent-Tiering to move infrequently accessed images to cheaper storage classes automatically.

Case Study: Design a Highly Scalable Image Sharing Platform

Design an Instagram-like platform where users can upload photos, follow other users, and see a personalised home feed. We'll follow the five-step process end-to-end.

Step 1 — Functional Requirements

• Users can upload images with a caption
• Users can view images (full-size and thumbnail)
• Users can follow / unfollow other users
• Users see a home feed — a ranked list of recent images from people they follow
• Users can like images

Out of scope: comments, Stories, Reels, DMs, ads, search.

Step 2 — Non-Functional Requirements & Scale Estimates

Step 3 — System's API & Sequence Diagram

API Design

POST   /v1/images/upload-url            → { uploadUrl, imageId }  # request pre-signed S3 URL
PUT    {uploadUrl}                      → 200 OK                  # client uploads directly to S3
POST   /v1/images/{imageId}/confirm     → { imageId, imageUrl }   # notify backend upload complete
GET    /v1/images/{imageId}             → ImageMetadata           # image detail
POST   /v1/users/{userId}/follow        → 200 OK
DELETE /v1/users/{userId}/follow        → 200 OK
GET    /v1/users/{userId}/feed          → [ImageCard]             # paginated, cursor-based
POST   /v1/images/{imageId}/likes       → { likesCount }

Upload — Sequence of Events

── Phase 1: Get upload URL (fast, <50ms) ──
Client         →  API Gateway       : POST /v1/images/upload-url
API Gateway    →  Image Service     : forward (auth validated)
Image Service  →  S3                : generate pre-signed PUT URL (TTL: 5 min)
Image Service  →  Metadata DB       : INSERT image (status=PENDING)
Image Service  →  Client            : { uploadUrl, imageId }

── Phase 2: Direct upload to S3 (bypasses app servers) ──
Client         →  S3                : PUT {image bytes} to pre-signed URL
S3             →  Client            : 200 OK

── Phase 3: Confirm & trigger async processing ──
Client         →  Image Service     : POST /v1/images/{imageId}/confirm
Image Service  →  Metadata DB       : UPDATE image (status=READY, s3Key, caption)
Image Service  →  Kafka             : publish IMAGE_UPLOADED { imageId, userId, timestamp }
Image Service  →  Client            : { imageId, imageUrl }

── Async: Fan-out to followers' feeds ──
Feed Worker    ←  Kafka             : consume IMAGE_UPLOADED
Feed Worker    →  User Service      : GET followers(userId)     # paginated, may be millions
Feed Worker    →  Redis Cluster     : ZADD feed:{followerId} score=timestamp member=imageId
                                                                # done for each follower batch

── Async: Generate thumbnails ──
Thumb Worker   ←  Kafka             : consume IMAGE_UPLOADED
Thumb Worker   →  S3                : GET original image
Thumb Worker   →  S3                : PUT thumbnails (320px · 640px · 1080px)
Thumb Worker   →  Metadata DB       : UPDATE image (thumbnails=READY)

Upload — Visual Sequence Diagram

View Feed — Sequence of Events

Client      →  API Gateway  : GET /v1/users/{userId}/feed?cursor=&limit=20
API Gateway →  Feed Service : forward
Feed Service →  Redis       : ZREVRANGE feed:{userId} 0 19          # O(log n + 20)
Feed Service →  Metadata DB : batch SELECT WHERE imageId IN (...)   # one query, 20 rows
Feed Service →  Client      : [{ imageId, caption, thumbnailUrl, createdAt, likesCount }]

── Image display (independent of feed API) ──
Client       →  CloudFront CDN : GET /images/{imageId}/640px
CDN edge     →  Client         : serve from cache (cache hit, <20ms)
CDN edge     →  S3             : fetch & cache on miss

Step 4 — Functional Architecture

Step 5 — Addressing Non-Functional Requirements

High Availability (99.99%)

• All services deployed across 3 Availability Zones behind a load balancer
• S3, RDS (Multi-AZ), ElastiCache Redis (Multi-AZ) handle AZ failures automatically
• Kafka with replication factor 3 — survives one broker failure with no data loss
• Circuit breakers on all service-to-service calls (fail fast, not cascade)

Low Read Latency (<100ms images, <200ms feed)

• CloudFront CDN — serves images from 450+ edge locations globally. Cache-Control: max-age=31536000 (images are immutable). Mumbai users get images from Mumbai PoP.
• Redis feed cache — pre-built feed means feed reads never hit the DB. ZREVRANGE is O(log n + k) → microseconds.
• PostgreSQL read replicas — Feed Service and Image Service reads go to replicas; only writes hit primary.

High Write Throughput (23 uploads/sec peak)

• Pre-signed S3 URLs — clients upload directly to S3, bypassing application servers. Image Service only handles metadata and URL generation, not bytes.
• Image Service is stateless → horizontal autoscaling (target: <70% CPU triggers scale-out)
• Kafka absorbs write spikes — workers process at their own rate without back-pressure on upload path

Celebrity Problem (Fan-out Bottleneck)

• A celebrity with 50M followers uploading a photo triggers 50M Redis writes in Feed Worker — too slow
• Hybrid fan-out: users with > 1M followers use fan-out on read — their posts are fetched at feed-load time and merged with the pre-built feed. Only normal users get push fan-out.

Storage Cost (3.6 PB/year)

• S3 Intelligent-Tiering — automatically moves images not accessed in 30 days to infrequent-access storage (40-60% cost reduction)
• Images older than 2 years move to S3 Glacier (archive tier, ~$0.004/GB/month)

── Final Architecture Summary ──
Client (Mobile/Web)
  │
  ▼
CloudFront CDN  ──── S3 (images + thumbnails)
  │
  ▼
Load Balancer (Multi-AZ)
  │
  ├──▶  API Gateway  (auth · rate limit · routing)
  │           │
  │    ┌──────┼──────────┬─────────────┐
  │    ▼      ▼          ▼             ▼
  │  Image  Feed      User          Likes
  │  Svc    Svc       Svc           Svc
  │    │      │          │
  │    │    Redis      PostgreSQL
  │    │    Cluster    (primary + 2 replicas)
  │    │
  │    ▼
  │  Kafka (3 brokers, RF=3)
  │    │
  │    ├──▶  Feed Worker (fan-out to Redis)
  │    └──▶  Thumbnail Worker (S3 processing)
  │
  └──── PostgreSQL (Metadata DB)

Case Study: Design Reddit

Reddit is a social news and discussion platform where users submit posts to topic-based communities (subreddits), vote them up/down, and see a ranked feed. Core challenges: vote aggregation at scale, hot-score feed fan-out, nested comment threads.

Step 1 — Functional Requirements

• Users create posts (text/link/image) in subreddits
• Users upvote/downvote posts and comments
• Home feed shows top/hot/new posts from subscribed subreddits
• Users comment on posts; comments are nested (threaded)
• Posts ranked by hot algorithm: score × time-decay
• Moderators can remove posts/comments

Out of scope: Ads, Reddit Premium, Chat DMs, live streaming.

Step 2 — Non-Functional Requirements & Scale Estimates

Step 3 — System's API & Sequence Diagram

API Design

POST /v1/posts                           → { postId, subredditId, title, score:0 }
GET  /v1/feed?sub=&sort=hot&after=       → { posts:[], nextCursor }
POST /v1/votes                           → { postId, direction:+1|-1, newScore }
POST /v1/posts/{postId}/comments         → { commentId, parentCommentId?, body }
GET  /v1/posts/{postId}/comments         → { comments:[] }

Post Submission & Feed Fan-Out — Sequence of Events

Client        →  API Gateway   : POST /v1/posts { subredditId, title, content }
API Gateway   →  Post Service  : forward (auth validated, rate-limited)
Post Service  →  Post DB       : INSERT post (id, subredditId, userId, score=0, createdAt)
Post Service  →  Kafka         : publish POST_CREATED { postId, subredditId }
Post Service  →  Client        : { postId, shortUrl }
Feed Worker   ←  Kafka         : consume POST_CREATED
Feed Worker   →  Sub Service   : GET subscriber list (subredditId, paginated)
Feed Worker   →  Redis         : ZADD feed:{userId} hotScore postId  # per subscriber

Vote & Score Update — Sequence of Events

Client        →  API Gateway   : POST /v1/votes { postId, direction:+1 }
Vote Service  →  Redis         : INCR vote_up:{postId}   # atomic counter
Vote Service  →  Kafka         : publish VOTE_EVENT { postId, direction }
Vote Service  →  Client        : { newScore }
Score Worker  ←  Kafka         : consume VOTE_EVENT
Score Worker  :  hotScore = (ups-downs) / (age_hours+2)^1.8
Score Worker  →  Post DB       : UPDATE posts SET score=hotScore
Score Worker  →  Redis         : ZADD feed:{userId} hotScore postId  # active users

Visual Sequence Diagram

Step 4 — Functional Architecture

Step 5 — Addressing Non-Functional Requirements

Availability

• Multi-AZ PostgreSQL + read replicas; Redis cluster; stateless services.

Vote Latency (<50ms)

• Redis INCR is O(1) sub-ms; DB write deferred to Kafka consumer.

Celebrity Subreddit Fan-out

• r/AskReddit has 40M subscribers — pushing to all feeds on every post = 40M Redis writes. Fix: hybrid fan-out — push to feeds of normal subs, pull-merge from mega-subreddits on feed load.

── Final Architecture Summary ──
Client
  │
  ▼
CDN (static assets, media via S3+CloudFront)
  │
Load Balancer → API Gateway (auth · rate-limit)
                      │
          ┌───────────┼──────────────┐
          ▼           ▼              ▼
      Post Svc    Vote Svc      Comment Svc
          │           │              │
       PostDB      Redis          PostgreSQL
      (pgSQL)    (counters,       (threads)
                  hot feeds)
          │           │
          └───── Kafka ──────┐
                        ┌────┴──────────┐
                        ▼               ▼
                   Feed Worker     Score Worker
                        │               │
                  Redis Feeds       PostDB
                  (ZADD hotScore)   (UPDATE score)

Case Study: Design WhatsApp

WhatsApp delivers 100B messages/day to 2B users with real-time delivery, offline queuing, E2E encryption, and delivery receipts (sent ✓, delivered ✓✓, read ✓✓ blue). The core challenge is maintaining 500M persistent WebSocket connections and routing messages between them in under 100ms.

Step 1 — Functional Requirements

• 1-on-1 and group chat (up to 1024 members)
• Real-time delivery when recipient is online; stored for offline delivery
• Delivery receipts: Sent ✓, Delivered ✓✓, Read ✓✓
• Media sharing (images, video, voice notes, documents)
• Online/offline presence and last-seen timestamps
• End-to-end encrypted (Signal Protocol — only sender/recipient can read)

Out of scope: WhatsApp Pay, Business API, Status stories, voice/video calls.

Step 2 — Non-Functional Requirements & Scale Estimates

Step 3 — System's API & Sequence Diagram

API Design (WebSocket for real-time; REST for setup)

POST /v1/sessions                              → { sessionToken }
WebSocket: wss://msg.7ds.in/v1/connect         # bidirectional channel
POST /v1/chats/{chatId}/messages               → { messageId, status:SENT }
GET  /v1/chats/{chatId}/messages?after=        → { messages:[] }
GET  /v1/users/{id}/presence                   → { online:bool, lastSeen }

Real-Time Message Delivery — Sequence of Events

Sender App    →  WS Server     : SEND { chatId, recipientId, ciphertext, msgId }
WS Server     →  Msg Service   : forward
Msg Service   →  Cassandra     : INSERT (chatId, msgId, ciphertext, sentAt, status=SENT)
Msg Service   →  Redis         : GET presence:{recipientId}  → ONLINE, server_id=ws-07
Msg Service   →  WS Server ws-07: ROUTE { msgId, ciphertext }
WS Server     →  Recipient App : PUSH message
Recipient App →  WS Server     : ACK { msgId, status:DELIVERED }
WS Server     →  Msg Service   : update delivery receipt
Msg Service   →  Cassandra     : UPDATE status=DELIVERED WHERE msgId=?
Msg Service   →  Sender App    : NOTIFY { msgId, status:DELIVERED }   # ✓✓ appears

Offline Delivery & Reconnect — Sequence of Events

── Recipient offline ──
Msg Service   →  Redis         : GET presence:{recipientId}  → OFFLINE
Msg Service   →  Cassandra     : INSERT offline_queue (recipientId, msgId, sentAt)
Msg Service   →  Push Service  : NOTIFY { recipientId, preview:"You have a message" }
Push Service  →  FCM/APNs      : deliver push notification
── Recipient comes online ──
Recipient App →  WS Server     : CONNECT + AUTH
WS Server     →  Msg Service   : user online { recipientId, serverId }
Msg Service   →  Redis         : SET presence:{recipientId} ONLINE serverId
Msg Service   →  Cassandra     : SELECT FROM offline_queue WHERE recipientId=? ORDER BY sentAt
Msg Service   →  WS Server     : BATCH_DELIVER { messages:[] }
Recipient App →  WS Server     : BATCH_ACK
Msg Service   →  Cassandra     : DELETE offline_queue WHERE recipientId=? AND msgIds IN (?)

Visual Sequence Diagram

Step 4 — Functional Architecture

Step 5 — Addressing Non-Functional Requirements

Availability

• WS servers are stateless (user can reconnect to any server); Cassandra quorum writes ensure durability before routing.

Delivery (<100ms)

• Redis presence O(1); WS push avoids HTTP handshake overhead; inter-server routing via internal gRPC.

Group Fan-out (1024 members)

• Publish once to group Kafka topic; Group Fan-out Service reads and pushes to each online member's WS server; offline members get individual push notifications.

── Final Architecture Summary ──
[Sender]  ←─ E2E encrypted (Signal Protocol) ─→  [Recipient]
    │                                                    │
WebSocket Server Cluster (consistent hash routing)
    │
Message Service (stateless)
    │
    ├──▶ Cassandra (messages + offline_queue — durable, write-heavy)
    ├──▶ Redis (presence map: userId → serverId)
    └──▶ Push Notification Service
               ├── FCM  (Android)
               └── APNs (iOS)

Media: client encrypts → S3 pre-signed URL upload → S3 key sent in message

Case Study: Design Uber / Lyft

Uber matches riders with nearby drivers in real-time. The defining challenge is continuously ingesting 333K driver location updates per second, querying geospatial proximity in milliseconds, and completing a match within 5 seconds of a rider's request.

Step 1 — Functional Requirements

• Riders request a trip (pickup + destination); system finds nearby drivers
• Drivers accept/decline trip requests
• Rider sees real-time driver location on a map during the trip
• Fare estimated before trip, calculated after (distance + duration + surge)
• Riders and drivers rate each other after trip completion
• Surge pricing based on local demand/supply ratio

Out of scope: Uber Eats, Uber Freight, scheduled rides, autonomous vehicles.

Step 2 — Non-Functional Requirements & Scale Estimates

Step 3 — System's API & Sequence Diagram

API Design

POST /v1/trips                              → { tripId, estimatedFare, eta }
GET  /v1/drivers/nearby?lat=&lng=&radius=2  → { drivers:[{id,location,eta}] }
PUT  /v1/drivers/{id}/location              → 200 OK
PUT  /v1/trips/{id}/accept                  → { tripId, driverETA }
GET  /v1/trips/{id}/track                   → { driverLocation, status }

Trip Request & Driver Match — Sequence of Events

Rider App     →  API Gateway   : POST /v1/trips { pickupLat, pickupLng, destLat, destLng }
API Gateway   →  Trip Service  : forward (auth validated)
Trip Service  →  Pricing Svc   : GET surge { lat, lng }
Pricing Svc   →  Redis         : GET surge:{geohash}  → 1.3x
Trip Service  →  Location Svc  : FIND_NEARBY { lat, lng, radius:2km }
Location Svc  →  Redis         : GEORADIUS drivers:available lng lat 2km ASC COUNT 10
Redis         →  Location Svc  : [driverId1, driverId2, ...]
Trip Service  →  Notify Svc    : SEND_REQUEST to driverId1 { tripId, pickupLocation }
Trip Service  →  Trip DB       : INSERT trip (tripId, riderId, status=SEARCHING, fare=₹144)
Trip Service  →  Rider App     : { tripId, estimatedFare, status:SEARCHING }

Driver Accept & Location Streaming — Sequence of Events

Driver App    →  API Gateway   : PUT /v1/trips/{tripId}/accept
Trip Service  →  Trip DB       : UPDATE SET driverId=?, status=DRIVER_ASSIGNED
Trip Service  →  Rider App     : NOTIFY { driverName, eta } (WebSocket/Push)
── Real-time location updates ──
Driver App    →  Location Svc  : PUT /location { lat, lng }  (every 4 sec)
Location Svc  →  Redis         : GEOADD drivers:active driverId lng lat
Location Svc  →  Kafka         : publish LOC_UPDATE { driverId, lat, lng, tripId }
Loc Worker    ←  Kafka         : consume LOC_UPDATE
Loc Worker    →  Rider App     : PUSH { driverLocation }  (WebSocket)

Visual Sequence Diagram

Step 4 — Functional Architecture

Step 5 — Addressing Non-Functional Requirements

Availability

• Location Service sharded by city; Redis cluster multi-AZ; Trip Service stateless behind LB.

Match (<5 seconds)

• Redis GEORADIUS sub-ms; driver notification via WebSocket (already connected); sequential with 10s timeout leaves 4+ candidates before expiry.

333K Location Writes/sec

• Redis GEOADD is O(log N); Location Service sharded by geohash (each city shard handles its drivers); Kafka decouples streaming from storage.

── Final Architecture Summary ──
Driver App ──location (4s)──▶ Location Service ──▶ Redis GeoSet
                                                         │
Rider App ──trip request──▶ API Gateway ──▶ Trip Service─┘(GEORADIUS)
                                                │
                               ┌────────────────┤
                               ▼                ▼
                         Pricing Svc        Matching Engine
                         Redis(surge)       → Driver Notification
                                              (FCM / WebSocket)
PostgreSQL: trip lifecycle, payments, ratings
Kafka: location stream → Rider WebSocket (real-time driver pin)

Case Study: Design a Hotel Booking System

A hotel booking system like Booking.com allows users to search available rooms, view pricing, and complete a reservation. The core tension is between read-heavy browsing (1000:1 read/write) which benefits from caching, and booking which requires strong consistency to prevent double-bookings.

Step 1 — Functional Requirements

• Search hotels by city, dates, guests, price range, and amenities
• View hotel details, room types, photos, and reviews
• Book a room for specific dates — guaranteed no double-booking
• Cancel a booking (per hotel's cancellation policy)
• Hotel owners manage room inventory and set pricing
• Email/SMS confirmation after successful booking

Out of scope: Dynamic pricing ML, hotel mobile app, reviews system, payment processing.

Step 2 — Non-Functional Requirements & Scale Estimates

Step 3 — System's API & Sequence Diagram

API Design

GET  /v1/hotels/search?city=&checkin=&checkout=&guests=  → { hotels:[], total }
GET  /v1/hotels/{id}/rooms?checkin=&checkout=            → { rooms:[], available }
POST /v1/bookings                                         → { bookingId, confirmationCode, total }
GET  /v1/bookings/{id}                                   → { booking details }
DELETE /v1/bookings/{id}                                 → { refundAmount }

Hotel Search with Availability — Sequence of Events

Client        →  API Gateway   : GET /hotels/search?city=Mumbai&checkin=2026-07-01&guests=2
API Gateway   →  Search Svc    : forward
Search Svc    →  Redis         : GET search:{hash(params)}  → MISS
Search Svc    →  Elasticsearch : QUERY hotels WHERE city=Mumbai AND available=true
Elasticsearch →  Search Svc    : { hotels:[50 results] }
Search Svc    →  Inventory Svc : CHECK_AVAIL { hotelIds:[50], checkin, checkout }
Inventory Svc →  PostgreSQL    : SELECT rooms WHERE hotel_id IN(...) AND dates NOT OVERLAP
PostgreSQL    →  Inventory Svc : { available rooms per hotel }
Search Svc    →  Redis         : SETEX search:{hash} 60 {results}
Search Svc    →  Client        : { hotels with available rooms and prices }

Room Booking (double-booking prevention) — Sequence of Events

Client        →  API Gateway   : POST /v1/bookings { roomId, checkin, checkout, paymentToken }
API Gateway   →  Booking Svc   : forward (auth required)
Booking Svc   →  Inventory Svc : RESERVE { roomId, checkin, checkout }
Inventory Svc →  PostgreSQL    : BEGIN TRANSACTION
                                  SELECT id FROM room_bookings
                                  WHERE room_id=? AND status='CONFIRMED'
                                  AND NOT (checkout <= ? OR checkin >= ?)
                                  FOR UPDATE  # row-level lock
                                  [conflict found → ROLLBACK → 409]
                                  INSERT INTO room_bookings (roomId, userId, checkin, status=PENDING)
                                  COMMIT
Inventory Svc →  Booking Svc   : { bookingId, reserved:true }
Booking Svc   →  Payment Svc   : CHARGE { paymentToken, amount }
Payment Svc   →  Booking Svc   : { txnId, status:SUCCESS }
Booking Svc   →  PostgreSQL    : UPDATE SET status=CONFIRMED, txnId=?
Booking Svc   →  Kafka         : publish BOOKING_CONFIRMED { bookingId, userId }
Booking Svc   →  Client        : { bookingId, confirmationCode }

Visual Sequence Diagram

Step 4 — Functional Architecture

Step 5 — Addressing Non-Functional Requirements

No Double-Booking

• SELECT FOR UPDATE acquires row-level lock within a transaction; only one concurrent booking for a room+dates succeeds; the other gets 409.

Search (<500ms)

• 90% of searches hit Redis cache (60s TTL); Elasticsearch for cache miss; availability check parallelised for top-10 results.

1000:1 Read/Write

• Elasticsearch + read replicas absorb reads; booking path is the only write-heavy path; PostgreSQL primary handles <116 writes/sec comfortably.

── Final Architecture Summary ──
Client
  │
CDN (hotel photos — S3 + CloudFront)
  │
Load Balancer
  │
  ├──▶ Search Service ──▶ Elasticsearch (geo+text+filter)
  │         └──▶ Redis (search cache 60s TTL)
  │
  ├──▶ Booking Service ──▶ Inventory Service ──▶ PostgreSQL
  │         │                                  (SELECT FOR UPDATE)
  │         └──▶ Payment Service (Stripe/Razorpay)
  │                   │
  │               Kafka → Notification Service (Email + SMS)
  │
  └──▶ Hotel Mgmt API → PostgreSQL (hotel config, pricing)

Case Study: Design a Credit Card Processing System

A credit card processing system handles authorization (does the card have funds?), capture (finalise the charge), and settlement (end-of-day batch to merchants). It operates at 3,000+ TPS with sub-200ms authorization, 99.999% availability, and zero data loss — making it one of the most demanding systems to design.

Step 1 — Functional Requirements

• Merchant submits card details; system approves or declines in real-time (<200ms)
• Approved authorization can be captured (charge finalised) within 7 days
• Merchants can issue full or partial refunds on captured transactions
• Void an authorization before capture
• Fraud detection flags suspicious transactions in real-time
• End-of-day settlement batch moves funds from card network to merchant accounts

Out of scope: Card issuing, rewards/points, chargeback dispute resolution, forex conversion.

Step 2 — Non-Functional Requirements & Scale Estimates

Step 3 — System's API & Sequence Diagram

API Design

POST /v1/transactions/authorize    → { authCode, transactionId, status:APPROVED|DECLINED }
POST /v1/transactions/{id}/capture → { captureId, amount, status:CAPTURED }
POST /v1/transactions/{id}/void    → { status:VOIDED }
POST /v1/transactions/{id}/refund  → { refundId, amount, status:REFUNDED }
GET  /v1/transactions/{id}         → { full transaction history }

Card Authorization — Sequence of Events

Merchant      →  Gateway       : POST /authorize { cardNumber, expiry, CVV, amount }
Gateway       :  tokenize card → HSM encrypts raw PAN → return token
Gateway       →  Fraud Svc     : SCORE { token, amount, merchantId, IP, deviceId }
Fraud Svc     →  ML Model      : score risk (velocity, geo, merchant category)
Fraud Svc     →  Gateway       : { riskScore:23, action:PROCEED }
Gateway       →  Card Network  : AUTH_REQUEST { token, amount, merchantId }  (ISO 8583)
Card Network  →  Issuing Bank  : verify funds, check credit limit
Issuing Bank  →  Card Network  : APPROVED { authCode:XK4921 }
Card Network  →  Gateway       : { authCode, approved:true }
Gateway       →  Txn DB        : INSERT txn (id, token, amount, authCode, status=AUTHORIZED)
Gateway       →  Merchant      : { transactionId, authCode, status:APPROVED }  # within 200ms

End-of-Day Settlement — Sequence of Events

── Settlement batch runs at 23:00 ──
Settlement Svc →  Txn DB       : SELECT WHERE status=CAPTURED AND date=today()
Settlement Svc →  Card Network : BATCH_SETTLEMENT { transactions:[] }
Card Network   →  Settlement   : { batchId, totalAmount, networkFees }
Card Network   :  transfer funds from issuing banks to acquirer
Settlement Svc →  Txn DB       : UPDATE SET status=SETTLED, settledAt=NOW()
Settlement Svc →  Merchant Acct: CREDIT { merchantId, amount - fees }
Settlement Svc →  Audit Log    : WRITE settlement record (immutable append)

Visual Sequence Diagram

Step 4 — Functional Architecture

Step 5 — Addressing Non-Functional Requirements

99.999% (five nines)

• Active-active multi-region; card network connections from both regions; circuit breaker on card network call; no single point of failure on auth path.

Authorization (<200ms)

• Fraud scoring <20ms (Redis velocity counters + lightweight ML); card network ~80ms round-trip; all DB writes async except the final INSERT before response.

Zero Data Loss

• Transaction written to PostgreSQL with WAL-level durability before response sent; Kafka for event stream; nightly reconciliation against card network totals.

── Final Architecture Summary ──
Merchant ──HTTPS──▶ Payment Gateway  (PCI DSS boundary)
                          │
                    ┌─────┴──────────────┐
                    ▼                    ▼
             Fraud Detection        HSM (encryption)
             ML scoring <20ms       PAN tokenization
                    │
                    ▼
             Card Network (Visa/MC — ISO 8583)
                    │
             Issuing Bank (approve/decline)
                    │
             Transaction DB (PostgreSQL, append-only)
             Audit Log (Kafka + S3, immutable)
                    │
             Settlement Service (nightly batch)
                    │
             Merchant Account (ACH/NEFT credit)

Case Study: Design a User Auth System

A user authentication system handles registration, login, MFA, session management, and password reset. It must be secure, fast, and exceptionally available — when auth is down, every other service is inaccessible. The key design insight is that JWT validation must happen locally (without calling the auth service), removing it as a per-request bottleneck.

Step 1 — Functional Requirements

• Register with email + password (bcrypt hashed); verify email address
• Login returns a short-lived access token (JWT, 15 min) and refresh token (30 days)
• MFA via TOTP (Google Authenticator / Authy)
• Access token validated by downstream services locally — no auth service call per request
• Logout invalidates refresh token; logout-all-devices invalidates all sessions
• Password reset via time-limited email link

Out of scope: RBAC permissions, enterprise SSO, biometric authentication, OAuth2 provider.

Step 2 — Non-Functional Requirements & Scale Estimates

Step 3 — System's API & Sequence Diagram

API Design

POST /v1/auth/register           → { userId, verificationSent:true }
POST /v1/auth/login              → { accessToken(JWT), refreshToken, expiresIn:900 }
POST /v1/auth/refresh            → { accessToken, expiresIn:900 }
POST /v1/auth/logout             → 204 No Content
POST /v1/auth/mfa/setup          → { otpSecret, qrCodeUrl }
POST /v1/auth/mfa/verify         → { verified:true }
POST /v1/auth/password/reset     → { resetLinkSent:true }

Login with MFA — Sequence of Events

Client        →  API Gateway   : POST /v1/auth/login { email, password }
API Gateway   →  Auth Service  : forward (rate-limit: 5 attempts/15min per IP, Redis counter)
Auth Service  →  User DB       : SELECT (passwordHash, mfaEnabled, locked) WHERE email=?
User DB       →  Auth Service  : { passwordHash, mfaEnabled:true, locked:false }
Auth Service  :  bcrypt.verify(password, passwordHash)  → matches
Auth Service  →  Redis         : SET mfa:{mfaToken} { userId } EX 300   (5-min pending MFA)
Auth Service  →  Client        : { mfaRequired:true, mfaToken }
── MFA step ──
Client        →  Auth Service  : POST /v1/auth/mfa/verify { mfaToken, totpCode }
Auth Service  →  Redis         : GET mfa:{mfaToken}  → { userId }
Auth Service  :  TOTP.verify(totpCode, userSecret)  → valid
Auth Service  →  JWT Svc       : SIGN { sub:userId, email, roles } RS256 → accessToken
Auth Service  →  Refresh DB    : INSERT (tokenHash, userId, deviceId, expiresAt=30d)
Auth Service  →  Redis         : DEL mfa:{mfaToken}
Auth Service  →  Client        : { accessToken, refreshToken, expiresIn:900 }

Token Refresh and Revocation — Sequence of Events

── Token expired — client refreshes ──
Client        →  Auth Service  : POST /v1/auth/refresh { refreshToken }
Auth Service  →  Redis         : GET revoked:{hash(refreshToken)}  → nil (not revoked)
Auth Service  →  Refresh DB    : SELECT WHERE tokenHash=? AND expiresAt > NOW()
Refresh DB    →  Auth Service  : { userId, valid:true }
Auth Service  →  JWT Svc       : SIGN new accessToken (15 min)
Auth Service  →  Client        : { accessToken, expiresIn:900 }
── User logs out ──
Client        →  Auth Service  : POST /v1/auth/logout { refreshToken }
Auth Service  →  Refresh DB    : DELETE WHERE tokenHash=?
Auth Service  →  Redis         : SETEX revoked:{hash(refreshToken)} 2592000 "1"  (TTL 30d)
Auth Service  →  Client        : 204 No Content

Visual Sequence Diagram

Step 4 — Functional Architecture

Step 5 — Addressing Non-Functional Requirements

99.999% Availability

• Auth Service stateless + N replicas; User DB primary+replicas in 3 AZs; JWT validation fully local — other services keep working even if Auth Service goes down (until tokens expire).

Local JWT Validation (<1ms)

• Services receive Auth Service's RSA public key at startup; verify JWT signature locally (O(1) asymmetric crypto); only calls Auth Service on refresh — removes auth as a per-request network bottleneck.

Brute-force Protection

• Redis rate limiter login_attempts:{ip} with 5/15min threshold; CAPTCHA after 3 failures; IP-level block after 50 in 1 hour; account lock after 10 consecutive failures for the same email.

── Final Architecture Summary ──
Client
  │
API Gateway ──▶ Auth Service (stateless, N replicas)
                      │
         ┌────────────┼──────────────────┐
         ▼            ▼                  ▼
    User DB         Redis            Refresh
  (PostgreSQL)   (MFA state,        Token DB
  (passwords,     revocation,       (PostgreSQL)
   MFA secrets)   rate limits)
                      │
               JWT Service (RS256 sign)
                      │
           accessToken ──▶ All Services verify locally
                           (no Auth Service call per request)
                      │
               Email Service (password reset, verification)

Case Study: Design a Payment System

A payment system like PayPal or UPI manages account balances internally using double-entry accounting, handles money movement between users and merchants, and connects to external banks for fund deposits and withdrawals. Unlike credit card processing, it owns the ledger. Zero money duplication or loss is a hard requirement.

Step 1 — Functional Requirements

• Users fund their wallet via bank transfer, UPI, or card
• Users pay other users or merchants from their wallet
• Merchants receive payments and withdraw to their bank account
• Full or partial refunds on transactions
• Transaction history with search and filtering
• Every money movement recorded in a double-entry ledger (regulatory requirement)

Out of scope: Credit/lending, investments, insurance, crypto, recurring billing.

Step 2 — Non-Functional Requirements & Scale Estimates

Step 3 — System's API & Sequence Diagram

API Design

POST /v1/payments                  → { paymentId, status:PENDING|SUCCESS|FAILED, txnRef }
POST /v1/wallets/fund              → { walletId, newBalance, transactionId }
GET  /v1/wallets/{id}              → { balance, currency, pendingBalance }
POST /v1/refunds                   → { refundId, amount, originalPaymentId }
GET  /v1/payments?userId=&page=    → { payments:[], total }

Wallet-to-Wallet Payment — Sequence of Events

Client        →  API Gateway   : POST /v1/payments { fromWalletId, toWalletId, amount, idempotencyKey }
API Gateway   →  Payment Svc   : forward (auth + idempotency check)
Payment Svc   →  Redis         : SETNX idempotency:{key} "PROCESSING" EX 600  → OK (new)
Payment Svc   →  Ledger Svc    : BEGIN_TXN { from, to, amount }
Ledger Svc    →  PostgreSQL    : BEGIN
                                  SELECT balance FROM wallets WHERE id=? FOR UPDATE → ₹5000
                                  [balance >= amount? YES. NO → ROLLBACK → INSUFFICIENT_FUNDS]
                                  INSERT ledger_entries (debit:from, credit:to, amount, type=PAYMENT)
                                  UPDATE wallets SET balance=balance-amount WHERE id=from
                                  UPDATE wallets SET balance=balance+amount WHERE id=to
                                  COMMIT
Ledger Svc    →  Payment Svc   : { txnId, status:SUCCESS }
Payment Svc   →  Redis         : SET idempotency:{key} txnId EX 600
Payment Svc   →  Kafka         : publish TXN_COMPLETED { paymentId, fromId, toId, amount }
Payment Svc   →  Client        : { paymentId, status:SUCCESS, txnRef }

Refund — Sequence of Events

Client        →  Payment Svc   : POST /v1/refunds { originalPaymentId, amount, reason }
Payment Svc   →  Ledger Svc    : VERIFY original { originalPaymentId }
Ledger Svc    →  PostgreSQL    : SELECT * FROM payments WHERE id=? AND status='SUCCESS'
                                  [not found or already refunded → 400]
Payment Svc   →  Ledger Svc    : BEGIN_TXN { from:merchant, to:user, amount, type:REFUND, refOf:origId }
Ledger Svc    →  PostgreSQL    : [same debit/credit pattern, links to originalPaymentId]
Payment Svc   →  Kafka         : publish REFUND_COMPLETED
Notify Worker ←  Kafka         : SEND "Refund of ₹{amount} received" (SMS/Push)
Payment Svc   →  Client        : { refundId, status:SUCCESS }

Visual Sequence Diagram

Step 4 — Functional Architecture

Step 5 — Addressing Non-Functional Requirements

Zero Money Duplication

• Redis SETNX atomically prevents duplicate processing; ledger transaction is atomic (both debit AND credit in one DB COMMIT); partial update = automatic rollback.

99.999% Availability

• Active-active multi-region PostgreSQL with synchronous replication (cannot lose ledger entries); Payment Service stateless + LB; Redis cluster.

Exactly-once Semantics

• Kafka consumer commits offset only after DB write succeeds; idempotency key prevents re-processing on Kafka retry; bank transfer polled (not fire-and-forget).

── Final Architecture Summary ──
Client
  │
API Gateway (auth · TLS)
  │
Payment Service (orchestrator, stateless)
  │
  ├──▶ Redis (idempotency — SETNX atomic)
  │
  ├──▶ Ledger Service ──▶ PostgreSQL (ACID, double-entry)
  │                        wallets + ledger_entries
  │                        (SELECT FOR UPDATE → debit → credit → COMMIT)
  │
  ├──▶ Kafka ──▶ Notification Worker (SMS/Push)
  │         └──▶ Analytics Worker
  │         └──▶ Reconciliation Worker (daily)
  │
  └──▶ Bank Integration Service (NEFT/IMPS/UPI — async)

Case Study: Design an API Rate Limiter

An API rate limiter controls how many requests a client can make in a time window, protecting APIs from abuse, DDoS, and runaway clients. It must make an allow/deny decision in under 1ms on every request — it sits on the hot path for every API call, so its latency directly becomes your API's latency.

Step 1 — Functional Requirements

• Allow up to N requests per user/API key per time window (e.g. 1,000 req/min)
• Different rate limits per tier (Free: 100/min, Pro: 10K/min, Enterprise: custom)
• Return 429 Too Many Requests with Retry-After header when limit exceeded
• Rate limit by IP, API key, user ID, or combination
• Distributed — same limit enforced across all server replicas (not per-instance counters)
• Per-endpoint overrides (e.g. /search: 10/sec, /upload: 2/min)

Out of scope: DDoS mitigation (Cloudflare layer), cost-based throttling, GraphQL complexity limiting.

Step 2 — Non-Functional Requirements & Scale Estimates

Step 3 — System's API & Sequence Diagram

API Design (internal — called by API Gateway)

POST /v1/ratelimit/check                 → { allowed:bool, remaining:int, resetAt:timestamp }
PUT  /v1/limits                          → { plan, endpoint, limit, window }  # admin config
GET  /v1/limits/{apiKey}                 → { limits:[] }
# Headers added to every API response:
# X-RateLimit-Limit: 1000
# X-RateLimit-Remaining: 847
# X-RateLimit-Reset: 1719900060
# Retry-After: 42  (only on 429)

Request Check (Sliding Window) — Sequence of Events

Client        →  API Gateway   : GET /v1/search?q=hotels (API-Key: abc123)
API Gateway   →  Rate Limiter  : CHECK { apiKey:abc123, endpoint:/search, ts:now() }
Rate Limiter  →  Redis         : PIPELINE
                                  ZREMRANGEBYSCORE rl:abc123:/search 0 (now-60000)
                                  ZADD rl:abc123:/search {now()} {requestId}
                                  ZCARD rl:abc123:/search
                                  EXPIRE rl:abc123:/search 60
Redis         →  Rate Limiter  : count:47
Rate Limiter  :  47 < 1000 → ALLOWED
Rate Limiter  →  API Gateway   : { allowed:true, remaining:953, resetAt:1719900060 }
API Gateway   →  Backend Svc   : forward request
Backend Svc   →  API Gateway   : { response }
API Gateway   →  Client        : 200 OK + X-RateLimit-Remaining:953
── Limit exceeded path ──
Rate Limiter  :  count >= 1000 → DENIED
Rate Limiter  →  API Gateway   : { allowed:false, remaining:0, retryAfter:42 }
API Gateway   →  Client        : 429 Too Many Requests (Retry-After:42)

Admin Config Update — Sequence of Events

Admin         →  Admin API     : PUT /v1/limits { plan:PRO, endpoint:/search, limit:5000, window:60s }
Admin API     →  Config DB     : UPDATE rate_limits SET limit=5000 WHERE plan=PRO AND endpoint='/search'
Admin API     →  Redis         : SET limit:PRO:/search 5000   (fast-path cache)
Admin API     →  Kafka         : publish LIMIT_CHANGED { plan, endpoint, newLimit }
All Instances ←  Kafka         : refresh local config cache (in-process)
Admin API     →  Admin         : { updated:true, effectiveAt:now() }

Visual Sequence Diagram

Step 4 — Functional Architecture

Step 5 — Addressing Non-Functional Requirements

Decision (<1ms)

• Redis pipeline executes 4 commands in one round-trip (~0.3ms local Redis); config read from in-process memory; no DB call on hot path.

Distributed Enforcement

• All 100+ API Gateway instances share the same Redis cluster — a counter per instance would allow 100× more requests than intended.

Fail Open on Redis Unavailability

• Circuit breaker; if Redis unreachable, allow all requests; log and alert; degraded rate-limiting beats blocking all traffic.

── Final Architecture Summary ──
Client
  │
API Gateway (100 replicas)
  │
  ├──▶ Rate Limiter Sidecar ──▶ Redis Cluster
  │     (sliding window,          (ZADD+ZCARD pipeline,
  │      <1ms decision)            shared across all GWs)
  │
  │    allowed? YES ──▶ Backend Service
  │           NO  ──▶ 429 + Retry-After header
  │
  └──▶ Config Service ──▶ PostgreSQL (plan limits)
                              └──▶ Kafka (broadcast to all instances)

Case Study: Design a Video-On-Demand Streaming Service

A VOD platform like Netflix streams high-quality video to 200M subscribers across every device and network condition. The two defining challenges are: (1) transcoding uploaded video into 10+ bitrate/resolution variants per title, and (2) delivering those segments to users with <2-second startup time through a global CDN — serving 100M hours of video per day.

Step 1 — Functional Requirements

• Users browse a catalogue of movies and TV shows with metadata, thumbnails, and ratings
• Stream video with adaptive bitrate — quality adjusts dynamically to network speed
• Studios upload raw video; system transcodes into multiple formats (H.264, H.265) and resolutions (480p, 720p, 1080p, 4K)
• Playback position saved and synced across devices (resume anywhere)
• Content is DRM-protected — only paying subscribers can play
• Users receive personalised recommendations based on watch history

Out of scope: Live streaming, user-generated content, comments, ads.

Step 2 — Non-Functional Requirements & Scale Estimates

Step 3 — System's API & Sequence Diagram

API Design

GET  /v1/catalogue?genre=&page=         → { titles:[], nextCursor }
GET  /v1/titles/{id}                    → { metadata, cast, availableQualities }
GET  /v1/stream/{titleId}               → { manifestUrl (HLS/DASH), licenseToken }
POST /v1/watchprogress                  → 204 OK
GET  /v1/watchprogress/{titleId}        → { positionSeconds }
POST /v1/uploads/{titleId}              → { uploadId, s3PresignedUrl }

Video Upload & Transcoding Pipeline — Sequence of Events

Studio        →  Upload Svc    : POST /v1/uploads { titleId, fileSize }
Upload Svc    →  S3            : generate pre-signed upload URL
Upload Svc    →  Studio        : { presignedUrl }
Studio        →  S3            : PUT raw video file (direct upload — bypasses server)
S3            →  Upload Svc    : ObjectCreated event (SNS → SQS → service)
Upload Svc    →  Kafka         : publish VIDEO_UPLOADED { titleId, s3Key }
Transcode W1  ←  Kafka         : consume VIDEO_UPLOADED → transcode 480p H.264
Transcode W2  ←  Kafka         : consume VIDEO_UPLOADED → transcode 1080p H.265
Transcode W3  ←  Kafka         : consume VIDEO_UPLOADED → transcode 4K H.265
Transcode W4  ←  Kafka         : consume VIDEO_UPLOADED → extract subtitles, thumbnails
All Workers   →  S3            : PUT /video/{titleId}/{quality}.m3u8 + segments
All Workers   →  Metadata DB   : UPDATE title SET status=AVAILABLE, manifests=[...]
All Workers   →  CDN           : invalidate cache for this titleId

User Starts Streaming — Sequence of Events

Client        →  API Gateway   : GET /v1/stream/{titleId}?quality=auto
API Gateway   →  Stream Svc    : forward (auth + subscription check)
Stream Svc    →  Entitlement   : IS_SUBSCRIBED { userId }  → true
Stream Svc    →  DRM Svc       : generate license token { userId, titleId, deviceId, exp }
Stream Svc    →  Client        : { manifestUrl:cdn.7ds.in/video/{titleId}/master.m3u8, licenseToken }
── Client fetches segments from CDN ──
Client        →  CDN Edge      : GET master.m3u8  → [HIT — cached manifest]
Client        :  measure bandwidth → select 1080p
Client        →  CDN Edge      : GET 1080p.m3u8  → segment list
Client        →  CDN Edge      : GET segment_001.ts, segment_002.ts (every 4 sec)
CDN Edge      →  S3            : [MISS on first request — cache and serve]
Client        :  AES-128 decrypt segment → decode H.264/H.265 → render
Client        →  Progress Svc  : POST /watchprogress { titleId, positionSec:240 }  (every 60s)

Visual Sequence Diagram

Step 4 — Functional Architecture

Step 5 — Addressing Non-Functional Requirements

Video Start (<2 seconds)

• Client receives manifest URL immediately (no round-trip to find segments); first segment at nearest CDN PoP is likely cached; DRM license pre-fetched in parallel with first segment.

Zero Buffering (<0.1%)

• ABR algorithm switches bitrate before buffer drops below 10 seconds; CDN ensures segments available <20ms; 300+ PoPs mean user is always within 50ms of an edge node.

Storage Cost at Scale

• Lifecycle rules: raw upload deleted after transcoding (saves 10×); old/rarely-watched titles moved to Glacier; aggressive CDN caching keeps S3 egress minimal.

── Final Architecture Summary ──
Studio ──▶ Upload Svc ──▶ S3 (raw video)
                               │
                         S3 Event → Kafka
                               │
              Transcoding Farm (parallel workers, FFmpeg)
              480p · 720p · 1080p · 4K · subtitles
                               │
                         S3 (transcoded segments)
                               │
                         CDN (300+ PoPs — 98% cache hit)
                               │
Client ──play──▶ Stream Svc (auth + DRM token)
           │
           ▼
         CDN (segment every 4s, ABR quality switching)
           │
       DRM Svc (Widevine/FairPlay license per device)
           │
   Progress Svc (PostgreSQL + Redis — resume on any device)

Metadata: DynamoDB (global tables, browse catalogue)

Case Study: Design a Search Engine like Google

A web search engine crawls billions of pages, builds an inverted index, ranks results by relevance and authority, and returns the top 10 in under 200ms for 98,000 searches per second. It is arguably the most complex distributed system ever built — crawling, indexing, ranking, and serving each run as massive independent pipelines.

Step 1 — Functional Requirements

• Web crawler discovers and fetches pages across the internet (respecting robots.txt)
• Indexer processes fetched pages: extracts text, builds inverted index, extracts links
• Users enter a query → system returns top 10 ranked results in <200ms
• Results include title, URL, and a relevant text snippet from the page
• Results ranked by relevance (keyword match) + authority (PageRank / link graph)
• Search supports operators: "exact phrase", site:, filetype:, -exclude

Out of scope: Image/video search, maps, autocomplete, knowledge graph, ads.

Step 2 — Non-Functional Requirements & Scale Estimates

Step 3 — System's API & Sequence Diagram

API Design

GET /v1/search?q=&page=&lang=&region=   → { results:[{title,url,snippet,rank}], total, timeTaken }
POST /v1/admin/crawl/schedule            → { url, priority }
GET  /v1/admin/index/status?url=         → { indexed:bool, lastCrawled, pageRank }

Web Crawling & Indexing — Sequence of Events

URL Frontier  →  Crawler       : FETCH next URL (priority-ordered: PageRank × freshness)
Crawler       →  example.com   : GET /page (respects robots.txt, rate-limited per domain)
example.com   →  Crawler       : HTML content
Crawler       →  Content Store : STORE raw HTML + metadata (ts, HTTP headers) to S3
Crawler       →  URL Extractor : PARSE → extract all <a href> links from HTML
URL Extractor →  URL Frontier  : ADD_URLS { newUrls:[20 links], source:pageUrl }
URL Frontier  :  dedup (Bloom filter) → prioritise by PageRank estimate → enqueue
Crawler       →  Kafka         : publish PAGE_FETCHED { url, s3Key, fetchedAt }
Indexer       ←  Kafka         : consume PAGE_FETCHED
Indexer       →  Content Store : READ raw HTML from S3
Indexer       :  tokenize → remove stop words → stem → compute tf-idf per term
Indexer       →  Inverted Index: ADD { term → [{url, tf-idf, position, anchorText}] }
Indexer       →  PageRank Eng  : NOTIFY { url, outboundLinks:[20] }

Query Processing & Result Assembly — Sequence of Events

Client        →  Search Gateway: GET /v1/search?q=best+system+design+books
Search Gateway→  Query Proc    : tokenize + parse query
Query Proc    :  tokens: ["best","system","design","books"]
              :  query expansion: "books" → ["textbooks","guides"]
              :  check Redis: GET cache:{"best system design books"}  → MISS
Query Proc    →  Index Shards  : PARALLEL LOOKUP each term (all shards simultaneously)
  Shard 1     :  "system" → [url1:0.92, url3:0.88, url7:0.75...]
  Shard 2     :  "design" → [url1:0.95, url3:0.80, url9:0.70...]
  Shard 3     :  "books"  → [url1:0.90, url5:0.85, url7:0.60...]
Query Proc    :  INTERSECT + SCORE (tf-idf × PageRank × freshness × CTR signals)
Query Proc    →  Snippet Gen   : EXTRACT relevant excerpt for top-10 URLs
Snippet Gen   →  Content Store : FETCH top-10 cached page content
Snippet Gen   →  Query Proc    : { snippets:[10] }
Search Gateway→  Redis         : SETEX cache:{query} 3600 {results}
Search Gateway→  Client        : { results:[10], timeTaken:"142ms" }

Visual Sequence Diagram

Step 4 — Functional Architecture

Step 5 — Addressing Non-Functional Requirements

Query Latency (<200ms)

• Parallel index shard lookups (all terms fetched simultaneously); Redis cache for popular queries (<5ms hit); only top-50 results scored (not full index).

Index Freshness

• News sites and social media crawled within minutes (RSS + API firehose); popular sites every few hours; URL Frontier assigns freshness score based on last-modified and change frequency.

98K Searches/sec

• Search serving is stateless and read-only (trivially horizontally scalable); Redis covers 80% of traffic as cache hits; index servers have read replicas per shard.

── Final Architecture Summary ──
Internet (60T pages)
  │
Crawlers (10K servers) ←── URL Frontier (priority queue + Bloom filter)
  │
Content Store (S3 — raw HTML snapshots)
  │
Indexers (MapReduce pipeline)
  ├──▶ Inverted Index (1000+ shards, sharded by term)
  └──▶ PageRank Engine (link graph → authority scores)

User Query
  │
Search Gateway (logging, A/B, personalisation)
  │
Query Processor → [parallel] → Index Shards × 1000
  │                              (BM25 + PageRank + ML reranker)
Snippet Generator → Content Store (top-10 excerpts)
  │
Redis Cache (popular queries, TTL 1h)
  │
Client (<200ms)

Design: Subscription-Based Healthcare API Marketplace

Design a cloud-native, subscription-based Healthcare API Marketplace where hospitals, clinics, and third-party health apps can securely subscribe to and consume FHIR APIs. It follows the Amazon API Gateway "marketplace" architecture with multi-tenancy, API subscriptions, and usage-based (pay-as-you-go) billing.

Step 1 — Requirements

Functional

• Expose FHIR APIs: Patient, Practitioner (Doctor), Appointment, Observation (Lab Report), Medication, Diagnostic Report.
• Hospitals, clinics & third-party apps register, browse a catalog, and subscribe to APIs.
• Subscription plans: Free, Basic, Premium, Enterprise (different rate limits & prices).
• Pay-as-you-go: charge per API call; track usage, generate invoices, process payments automatically.
• Auth: API keys + OAuth2/JWT; rate limiting & throttling per plan; API analytics.

Non-Functional

Step 2 — High-Level Architecture

Two planes: a control plane (portal, subscriptions, billing, keys, metering) manages who can use what and how they're billed; a data plane (API Gateway → FHIR microservices) serves the actual health data. The gateway is the enforcement point — it authenticates, applies the subscriber's usage plan (rate limits), and emits usage events; it never lets an unbilled or over-quota call through.

Step 3 — Core Components

Step 4 — Sequence: Subscribe → Consume → Bill

── A) Subscribe (control plane) ──
Developer    →  Portal            : sign up, browse catalog
Developer    →  Subscription Svc  : subscribe to "Patient API" · plan = Premium
Subscription →  Key/OAuth Svc     : create API key + OAuth2 client (scoped to tenant + plan)
Key/OAuth    →  API Gateway       : attach key to Usage Plan (rate=100/s, quota=1M/mo)
Subscription →  Developer         : { apiKey, clientId, clientSecret }

── B) Consume an API (data plane, hot path) ──
App          →  API Gateway       : GET /fhir/Patient/123  (x-api-key + Bearer JWT)
API Gateway  →  Key/OAuth Svc     : validate key + JWT · resolve tenant & plan
API Gateway  →  API Gateway       : enforce rate limit / quota (throttle if exceeded → 429)
API Gateway  →  Patient Service   : forward (tenant context injected)
Patient Svc  →  FHIR Data Store   : fetch patient (tenant-isolated, encrypted)
Patient Svc  →  App               : 200 FHIR Patient resource
API Gateway  →  Kafka             : publish USAGE_EVENT { tenant, api, ts, cost }  # async

── C) Meter & bill (async, never blocks B) ──
Metering     ←  Kafka             : consume USAGE_EVENT → aggregate per tenant/API
Billing      ←  (end of cycle)    : compute charges = plan fee + Σ(usage × rate)
Billing      →  Payment Gateway   : charge card (Stripe) · retry on failure
Billing      →  Notification Svc  : "invoice ready / payment processed"

Step 5 — Database Design (per service)

Step 6 — Security Architecture (HIPAA)

── Defense in depth ──
Consumer ──TLS 1.2+──▶ WAF ──▶ API Gateway ──▶ Service ──▶ Encrypted DB (KMS)
                 (blocks OWASP)   │
                                  ├─ AuthN: API key + OAuth2 client-credentials → JWT
                                  ├─ AuthZ: scopes per API + tenant isolation
                                  ├─ Rate limit / quota (per plan)
                                  └─ Audit log EVERY access (who · what PHI · when)

PHI protection:  encryption in transit (TLS) + at rest (AES-256 / KMS) ·
                 field-level encryption for sensitive fields · no PHI in logs/URLs ·
                 signed BAA with cloud provider · least-privilege IAM roles

Step 7 — Billing & Metering Flow (event-driven)

Step 8 — Addressing NFRs & Best Practices

• API Gateway pattern — one enforcement point for authZ, rate limiting, caching, logging.
• OAuth2 + JWT + API keys — client-credentials flow for machine-to-machine; scopes per API.
• Multi-tenant design — tenant_id everywhere (pool model); dedicated DB for Enterprise (silo).
• Event-driven billing — usage via Kafka; metering & billing decoupled from the hot path.
• Caching — gateway caches idempotent GETs; Redis caches key→plan lookups to keep auth fast.
• High availability — multi-AZ, stateless services + auto-scaling, DB read replicas.
• Fault tolerance — circuit breakers/retries to hospital systems; billing failures never block APIs.
• Observability — access logs → analytics; per-tenant usage dashboards; audit trail for HIPAA.

Low-Level Design

Zooming in from the HLD boxes to the concrete API contracts, class structure, database schema, and core algorithms an engineer would implement (shown Spring-Boot-flavoured).

LLD 1 — API Contracts (key endpoints)

LLD 2 — Class / Domain Model

LLD 3 — Database Schema (DDL)

LLD 4 — Core Algorithms

Rate limiting — token bucket (per key)

Usage metering — async aggregation

Billing — flat fee + tiered overage

Payments — idempotent charge

LLD 5 — Design Patterns Used