Load balancing: layer 4, layer 7 and gateways

What a load balancer sees

A load balancer spreads incoming work across servers, removes failed servers and makes scaling invisible to clients. What it can do depends on which layer it works at.

	Layer 4 (transport)	Layer 7 (application)
Sees	IP addresses, ports, TCP/UDP connections	HTTP method, URL, headers, cookies, body
Decides by	Connection details	Request content
TLS	Passes through, or terminates in some products	Usually terminates TLS
Speed and cost	Very fast, low overhead	More work per request
Typical features	Spread connections, health checks	Path routing, header rules, sticky sessions, rewrites, caching, WAF rules
Examples	Network load balancers, LVS, HAProxy in TCP mode	NGINX, Envoy, HAProxy HTTP mode, cloud application balancers

100%

Figure SDA-ADS-05-1 · Layers of balancing in front of an app · © forcsguy.com

Many designs use both: an L4 tier for raw scale and an L7 tier for smart routing.

Choosing a target: algorithms

• Round robin / weighted round robin: simple rotation; weights favour bigger machines.
• Least connections (and weighted): send to the server with fewest active requests; good when requests vary in length.
• Least response time: combine active load with measured latency.
• IP hash / consistent hashing: the same client or key goes to the same server, useful for caches and sticky state (see consistent hashing).
• Resource-based: servers report load and the balancer follows.

Health checks (active probes and passive error tracking) remove unhealthy servers; connection draining lets in-flight requests finish before a server is removed during a deploy.

Global and DNS balancing

Across regions, DNS-based or global server load balancing directs users to the nearest or healthiest region (see the DNS and CDN lessons). DNS answers are cached, so failover takes as long as the TTL; anycast and global balancers react faster.

API gateway: policy at the edge

An API gateway is an L7 component specialised for APIs. Beyond routing it commonly provides:

• Authentication and authorisation (validate tokens, apply scopes),
• Rate limiting and quotas (see the rate-limiter lesson),
• Request and response transformation, versioning, and protocol translation,
• Aggregation: one client call fans out to several services and merges results, saving round trips for mobile clients,
• Caching, logging, metrics and tracing in one place.

100%

Figure SDA-ADS-05-2 · Gateway routing and aggregation · © forcsguy.com

Risks: the gateway is a single entry point (run several instances, keep it stateless) and can become a bottleneck or a place where business logic piles up. Keep it to cross-cutting concerns. A variation is a backend for frontend: one gateway per client type (web, mobile) tailored to its needs.

Inside the cluster

Between services, balancing is done by client-side libraries or sidecar proxies that know the healthy instances from service discovery, so no central balancer is needed for east-west traffic. See the microservices lessons for service mesh ideas.

Failure habits

• Run at least two of every balancer and gateway instance across zones.
• Keep timeouts and retry budgets (see the resilience lesson) so the balancer does not amplify an outage.
• Use slow start for new instances so they warm up before taking a full share.

Real-world reading