Scalability & Architecture Patterns
Service Mesh
A service mesh moves cross-cutting concerns — retries, timeouts, mTLS, load balancing, observability — out of application code and into a sidecar proxy running next to every service instance, controlled by a central control plane.
- Sidecar pattern: a proxy (e.g. Envoy) runs alongside every service instance and intercepts all inbound/outbound traffic — the app talks to localhost, the sidecar handles the network
- Data plane (the sidecars) does the actual proxying; control plane (e.g. Istio's istiod) pushes configuration — routing rules, mTLS certs, retry policy — to every sidecar
- Mesh-provided features: automatic mTLS between services, consistent retry/timeout/circuit-breaker policy, traffic shifting for canary deploys, and uniform metrics without app code changes
- The mesh adds a hop — client to local sidecar, network, remote sidecar, remote app — and a new operational dependency, which is a real cost, not a free lunch
- An API gateway handles north-south traffic (outside to inside); a service mesh handles east-west traffic (service to service inside the cluster) — related but distinct problems
- Service mesh solves problems that only exist once many services call each other over the network — it has little to offer a monolith
| API Gateway | Service Mesh | |
|---|---|---|
| Traffic direction | north-south (client to cluster) | east-west (service to service) |
| Typical concerns | auth, rate limiting, routing | mTLS, retries, load balancing, tracing |
| Where it runs | edge of the cluster | a sidecar per service instance |