Service discovery, service mesh and sidecars

The problem: moving targets

In a microservice system, instances are created and destroyed constantly by autoscaling, deployments and failures. A caller cannot rely on a fixed address. It needs an answer to "where are the healthy instances of the orders service right now?"

Service discovery

Instances register themselves (or are registered by the platform) in a registry; callers look up the current list.

100%

Figure SDA-MSV-04-1 · Service registry and lookup · © forcsguy.com

• Client-side discovery: the caller queries the registry and balances itself (smart client library).
• Server-side discovery: the caller talks to a load balancer or the platform's service address, which knows the instances (Kubernetes Services and cluster DNS work this way; see the Kubernetes lesson).
• DNS-based discovery is simple but constrained by caching.
• Registries must be highly available (often built on consensus; see the consensus lesson), and the system should keep working with the last known list if the registry is briefly unreachable.

Sidecars

A sidecar is a helper process or container deployed alongside each service instance (in Kubernetes, in the same pod). All the service's network traffic goes through it. It can take over cross-cutting jobs so application code does not have to implement them in every language:

• retries, timeouts and circuit breaking (see the resilience lesson),
• mutual TLS (encrypting and authenticating service-to-service traffic),
• load balancing and service discovery,
• metrics, logs and tracing headers,
• access policy.

Other sidecar uses: log shippers, config reloaders, secret agents.

Service mesh

A service mesh is a dedicated infrastructure layer made of two parts:

• Data plane: the fleet of sidecar proxies (often Envoy) that carry the traffic.
• Control plane: configures and secures the proxies, distributes certificates and policy (for example Istio, Linkerd, Consul).

100%

Figure SDA-MSV-04-2 · Control plane and data plane · © forcsguy.com

What a mesh enables:

• Traffic management: weighted routing for canary releases, mirroring, fault injection, timeouts and retries by route.
• Security: automatic mutual TLS and service-level authorisation, giving each service a cryptographic identity.
• Observability: uniform metrics, traces and access logs for every call, without code changes.

Costs and alternatives

• Resources and latency: a proxy per instance adds CPU, memory and a small latency per hop.
• Operational complexity: another system to upgrade and debug; misconfiguration can break everything.
• Learning curve and sometimes opaque behaviour.
• Newer designs reduce per-pod cost (node-level or "ambient" proxies).

Alternatives: a shared library for resilience and telemetry (language-bound, needs upgrades everywhere), a managed platform's built-in features, or an API gateway for north-south traffic only (see the load balancing lesson).

When a mesh makes sense

Many services, several languages, strong needs for mutual TLS or consistent policy, and a platform team able to run it. For a handful of services, platform defaults and a good library are usually enough. As ever, adopt the pattern for a problem you have.

Real-world reading