Scalability & Architecture Patterns

Horizontal vs Vertical Scaling

Vertical scaling buys a bigger machine; horizontal scaling buys more machines. Vertical is simpler and hits a hard ceiling; horizontal is unbounded in theory but demands statelessness and coordination the moment you adopt it.

Vertical scaling (scale up): more CPU/RAM/disk on one box — no architecture changes, but bounded by the biggest instance type that exists and remains a single point of failure
Horizontal scaling (scale out): more machines behind a load balancer — unbounded in principle, but requires the workload to be splittable and the service to be stateless
Vertical scaling has zero coordination cost; horizontal scaling introduces network calls, partitioning, and consistency questions that did not exist before
Cost curves bend against vertical scaling fast — doubling a machine's specs rarely costs 2x near the top of a hardware tier, it costs 3-5x
Horizontal scaling only helps if the bottleneck is actually parallelizable — a single hot shard or a serialized write path will not get faster from adding boxes
Redundancy, not just capacity, is a common reason to go horizontal: three small instances survive one dying; one big instance does not

Most systems scale vertically first because it is free — resize the instance, restart, done. The switch to horizontal scaling is a real architectural commitment: every piece of per-client state has to move out of process memory (Stateless Services And Session Management), and the workload has to actually be parallelizable for adding machines to help at all.

Two ways to add capacity

Vertical vs horizontal scaling
	Vertical (scale up)	Horizontal (scale out)
Mechanism	bigger CPU/RAM/disk on one machine	more machines behind a load balancer
Ceiling	hard limit — the biggest instance that exists	no theoretical limit
Coordination cost	none — still one machine	load balancing, partitioning, distributed state
Failure mode	single point of failure	redundant — one node dying does not take the system down
Requires	nothing — works with any code	stateless services or externalized state

A request handler that assumes nothing about which instance serves it

@RestController
public class OrderController {
    private final OrderRepository repo;     // externalized state — a database
    private final SessionStore sessions;     // externalized session state, e.g. Redis

    @PostMapping("/orders")
    public OrderResponse createOrder(@RequestHeader("Session-Id") String sessionId,
                                      @RequestBody OrderRequest req) {
        Session session = sessions.get(sessionId);   // not an in-memory HashMap
        Order order = repo.save(new Order(session.userId(), req.items()));
        return new OrderResponse(order.id());
    }
}

Any instance can serve any request — nothing about this client is held in the process