Introduction to Envoy

Envoy is a high-performance L4/L7 proxy and communication bus designed for cloud-native applications. Originally built at Lyft and donated to the CNCF (where it graduated in 2018), it is now the networking foundation of major service meshes (Istio, AWS App Mesh, Consul Connect) and the reference implementation behind the Kubernetes Gateway API project Envoy Gateway.

Envoy operates out of process alongside your application. Your code sends traffic to localhost; Envoy handles the rest load balancing, retries, observability, TLS termination, and more without any changes to application code.

Architecture

graph LR
  Downstream["Downstream client"]
  Listener["Listener\n(address:port)"]
  FilterChain["Filter chain\n(L4 + L7 filters)"]
  Router["Router filter"]
  Cluster["Cluster\n(load balancer)"]
  Upstream["Upstream service"]

  Downstream -->|"TCP / HTTP"| Listener
  Listener --> FilterChain
  FilterChain --> Router
  Router -->|"route match"| Cluster
  Cluster -->|"selected endpoint"| Upstream

Every Envoy instance is a self-contained process. Multiple Envoy instances across a fleet share configuration through the xDS API they do not communicate with each other directly.

Core concepts

Listeners

A listener binds to an address and port and waits for inbound connections. Each listener has one or more filter chains selected by SNI, IP range, or other connection metadata.

1
2
3
4
5
6
7
8
9
10
listeners:
- name: ingress_http
  address:
    socket_address:
      address: 0.0.0.0
      port_value: 8080
  filter_chains:
  - filters:
    - name: envoy.filters.network.http_connection_manager
      # ... HTTP filter config

Filter chains and filters

Filters are the processing pipeline. They run in order and can inspect, modify, or terminate connections and requests. There are two layers:

Network filters operate on raw TCP bytes. The most important is http_connection_manager, which parses HTTP and hands off to HTTP filters.

HTTP filters run inside http_connection_manager and process HTTP requests and responses. They are chained each filter can pass, modify, or reject the request before it reaches the next one.

Inbound HTTP request
  └── http_connection_manager (network filter)
        ├── jwt_authn         (HTTP filter verify JWT)
        ├── ext_authz         (HTTP filter call external auth service)
        ├── rate_limit        (HTTP filter check rate limit service)
        ├── fault             (HTTP filter inject delays/errors for testing)
        └── router            (HTTP filter route to upstream cluster, always last)

Clusters

A cluster is a named group of upstream endpoints. It defines how Envoy connects to a backend: load balancing policy, connection pool limits, health checks, circuit breaker thresholds, and TLS settings.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
clusters:
- name: order_service
  connect_timeout: 1s
  type: STRICT_DNS
  lb_policy: ROUND_ROBIN
  load_assignment:
    cluster_name: order_service
    endpoints:
    - lb_endpoints:
      - endpoint:
          address:
            socket_address:
              address: order-svc.default.svc.cluster.local
              port_value: 8080

Load balancing policies include ROUND_ROBIN, LEAST_REQUEST, RING_HASH, and RANDOM.

Routes

Routes live inside http_connection_manager and map incoming requests to clusters using host, path, header, and query-parameter matching.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
route_config:
  virtual_hosts:
  - name: local_services
    domains: ["*"]
    routes:
    - match:
        prefix: /api/orders
      route:
        cluster: order_service
        timeout: 5s
        retry_policy:
          retry_on: 5xx
          num_retries: 3
    - match:
        prefix: /api/customers
      route:
        cluster: customer_service

Endpoints

Endpoints are individual upstream hosts within a cluster. In dynamic mode they are populated by the EDS (Endpoint Discovery Service) and updated without restarting Envoy.

xDS: dynamic configuration

Envoy’s configuration can be fully static (a single YAML file) or fully dynamic via the xDS API a set of gRPC streaming services that push configuration updates to Envoy in real time. Service meshes and control planes (Istiod, xDS-compatible controllers) implement xDS.

With xDS, a control plane can add a new cluster, update a route, rotate a certificate, or drain a listener without touching the Envoy process. This is the mechanism service meshes use to implement traffic shifting, canary deployments, and mTLS certificate rotation.

Static vs dynamic configuration

Key built-in HTTP filters

Where Envoy is used

Service mesh sidecar In Istio, every pod runs an Envoy sidecar injected automatically. Istiod acts as the xDS control plane. Envoy enforces mTLS, applies traffic policies, and emits telemetry without any changes to the application.

Edge/ingress proxy Envoy sits at the cluster edge, terminates TLS, and routes external traffic to internal services. Contour and Emissary-Ingress are Kubernetes Ingress controllers built on Envoy.

Envoy Gateway The CNCF-maintained implementation of the Kubernetes Gateway API. It uses Envoy as the data plane and exposes GatewayClass, Gateway, and HTTPRoute resources. See the Kubernetes Gateway API Setup guide for the Gateway API standard.

AWS App Mesh / Google Traffic Director Managed control planes that use Envoy as the data plane for service-to-service communication in cloud environments.

What to avoid

Do not hardcode static config when your deployment will scale. Static YAML works for a single instance but requires a restart (or config reload) for every change. For production fleets use xDS so configuration updates propagate without downtime.

Do not skip health checks on clusters. Without active health checking, Envoy will continue sending traffic to unhealthy endpoints until the connection times out. Configure both active (health_checks) and passive (outlier detection) health checking.

Do not put auth logic after the router filter. The router filter is always the last HTTP filter in the chain. Any filter placed after it will never execute. Always place jwt_authn, ext_authz, and rate limiting filters before router.

Do not confuse Envoy with Envoy Gateway. Envoy is the raw proxy; Envoy Gateway is a Kubernetes-native control plane that configures Envoy via the Gateway API CRDs. They share the same data plane but have very different configuration surfaces.

Do not ignore the admin interface in production. Envoy exposes an admin endpoint (default port 9901) that can drain connections, change log levels, and dump the current config. Restrict access to it it should never be reachable from outside the cluster.