Architecture Patterns for Building Gateways with Envoy Proxy

This article organizes the architecture patterns for building an API gateway around Envoy Proxy. Envoy is best known as Istio's data plane, but it is an independent, general-purpose proxy born at Lyft that can be used standalone — no Kubernetes, no service mesh required. This "freedom as a raw material" is Envoy's greatest strength, and at the same time it forces a design decision: how much do you build yourself?

The article has three parts. For every pattern, I make explicit which problem it solves.

1. Configuration management and control plane patterns — who delivers configuration to Envoy, and how
2. Authentication and authorization patterns — which layer decides what
3. Comparison with OSS gateways that need no custom authorization service — the structural difference from Kong, APISIX, and friends

---

Background: What Envoy Provides as a Gateway

Envoy is not just a reverse proxy — it ships the building blocks a gateway needs as a filter chain. A request passes through the filters in order, and each filter verifies, transforms, or decides.

Figure: Envoy's filter chain. Requests pass through each filter in order for verification, decision, and transformation

The main building blocks:

• jwt_authn filter: built-in JWKS fetching and caching, signature and exp / aud / iss verification. Verified claims are handed to downstream filters
• RBAC filter: declarative access control such as matching routes against scopes
• ext_authz filter: the official interface for querying an external authorization service via gRPC / HTTP on every request
• Lua / WASM filters: run custom logic inline, inside the proxy process
• mTLS, rate limiting, observability (access logs, metrics, tracing)

In other words, the division of labor — "standard token verification is built in, custom attribute-based decisions live in the extension layer" — is there from the start. Once you see this structure, choosing an authorization pattern reduces to the question: which filter should hold the decision?

---

Part 1: Configuration Management and Control Plane Patterns

The first wall you hit when using Envoy standalone is not the proxy's performance or features — it is configuration management.

Problem: How do you distribute and update Envoy's configuration (routes, clusters, filters)? Redeploy on every change, or push dynamically? And if dynamically — who builds the distribution server?

Envoy accepts configuration in two ways: static config read from YAML at startup, and dynamic delivery from an external server over the standard xDS API. The latter is exactly what Istio's istiod does.

First, the basic shape of xDS-based dynamic delivery. The control plane (istiod in Istio) owns the source of truth and pushes it over xDS to every Envoy — the Gateway Pod and every app Pod's sidecar alike. Envoy itself is an "empty vessel" holding no configuration; this separation is what makes policy changes without proxy restarts possible.

Figure: The basic xDS shape. The control plane dynamically delivers configuration to all Envoys (Gateway Pod and app Pod sidecars)

Four practical patterns sit on top of this basic shape.

Figure: The four control plane patterns. They differ in who delivers configuration to Envoy, and how

Pattern A: Static Configuration (Hand-Written YAML)

Problem: You just want Envoy running as a gateway. No Kubernetes (ECS, EC2).

Solution: Keep Envoy's YAML in a repository and swap containers via CI/CD on every change. Running Envoy as an ECS task behind an NLB as an L7 gateway is a perfectly workable setup.

• Good fit: routing and policy change infrequently (weekly or less), few teams
• Remaining problem: every change triggers a deploy. The moment you want teams to self-serve route additions, this starts to break down. The coupling of "config change = infrastructure deploy" bites harder as the organization grows

Pattern B: Building Your Own xDS Control Plane

Problem: Pattern A needs a task swap for every change. You want dynamic configuration, but Kubernetes is not available.

Solution: Run your own distribution server — the equivalent of istiod — and stream configuration to Envoy over xDS. Routes and policies change without proxy restarts.

• Reality: off-the-shelf options outside Kubernetes are scarce — roll your own with go-control-plane, or Consul, and that's about it. Building your own means committing to "maintaining a small istiod" indefinitely
• Remaining problem: you own the control plane's availability, testing, and versioning. If changes really are frequent, first consider AWS-managed alternatives — ALB, API Gateway, VPC Lattice, ECS Service Connect — before building; a custom xDS server is the last resort

Pattern C: Envoy Gateway (the Official OSS Control Plane on Kubernetes)

Problem: You want dynamic delivery without writing an xDS server — but you don't need Istio's full mesh (sidecar injection, east-west mTLS, istiod operations) either.

Solution: On Kubernetes (e.g. EKS), the official OSS Envoy Gateway is the middle ground. It bundles a Gateway API-compliant control plane: write CRDs like HTTPRoute and they are delivered over xDS. The build-your-own-xDS problem disappears, and the ext_authz / WASM extension points are already wired up.

• Good fit: you only need a north-south (external-to-internal) gateway. Clearly lighter than Istio
• Remaining problem: coverage is north-south only. Service-to-service (east-west) traffic remains unprotected

Pattern D: Istio Ingress Gateway (as Part of the Mesh)

Problem: Protecting only the gateway leaves you with "perimeter defense with a single checkpoint" if internal services talk to each other unauthenticated and in plaintext. You want to satisfy NIST SP 800-207's core tenet: never use network location as a basis for trust.

Solution: Adopt Istio and unify both north-south (Ingress Gateway) and east-west (sidecar / ambient) on Envoy. The Ingress Gateway is just an Envoy Pod (a Deployment) running inside the cluster, receiving configuration from istiod over xDS. Internal services can trust the user-identity headers attached at the edge precisely because east-west traffic is protected by mTLS plus service identity authorization, guaranteeing those headers can only come from the legitimate gateway. Try the same thing without a mesh and header spoofing is wide open.

Figure: The Istio setup at a glance. The NLB is AWS-managed; the Ingress Gateway and app Pods live in the cluster. Every hop past the gateway is protected by mTLS + service identity authorization

• Good fit: organizations needing fine-grained L7 service-to-service authorization or multi-tenant governance
• Remaining problem: operating istiod, sidecar resource overhead, and Pod-restart operations during Istio upgrades. A platform team (roughly six people minimum) is a prerequisite

Part 1 Summary: Problems and Solutions

---

Part 2: Authentication and Authorization Patterns

Authorization is the heart of a gateway. Envoy's design philosophy is PEP / PDP separation — the proxy sticks to being the PEP (Policy Enforcement Point) while the PDP (Policy Decision Point) can be built in or external, depending on the situation. This is the classic pattern from the XACML era, brought into the proxy.

Figure: PEP / PDP separation in its basic form. Envoy enforces; decisions go to the authz service; the truth about consent and tokens lives in the authorization server

As a map of the whole territory, keep in mind that authorization splits into three layers.

Figure: The three layers of authorization. The lower the layer, the more "business-shaped" the data its decisions require

Coarse per-endpoint guardrails belong to the gateway and mesh; per-resource business authorization belongs to the application layer. Break this division — "let the gateway do everything" — and policy maintenance costs explode, because the business data those decisions need is structurally invisible to the gateway. When a resource-level permission graph spans multiple services, that is Zanzibar territory (SpiceDB / OpenFGA) — but that lives outside the gateway.

Below are the edge-layer (gateway) implementation patterns, problem-first. The essence of choosing between them is where the information needed for the decision lives.

Figure: Choosing an authorization implementation pattern. The branching axis is where the decision's inputs live

Pattern 1: Built-in Filters Only (JWT + RBAC)

Problem: Verify the end user's JWT and match scopes against routes — without adding external dependencies or latency.

Solution: The jwt_authn filter completes local verification against JWKS. JWKS holds public keys, so it caches well; the authorization server is involved only at issuance and JWKS fetch. Coarse scope-to-route matching is written declaratively with the built-in RBAC filter. Zero external dependencies, minimal latency, no failure mode to design.

Guiding principle: any decision expressible in configuration (CRDs, in Istio) belongs here by default. East-west traffic in particular completes stateless verification on JWTs, so ninety percent of authorization is covered by this layer. Istio's RequestAuthentication + AuthorizationPolicy is exactly this standard form.

Pattern 2: Inline Extensions (Lua / WASM)

Problem: FAPI-specific checks — matching the cnf claim against the mTLS client certificate, for example — fall outside jwt_authn's remit. Yet the decision needs nothing beyond the request itself, and standing up an external service just for that feels wrong.

Solution: For stateless decisions that need only the request plus pre-distributed configuration, write a small inline filter in Lua or WASM. It runs inside the proxy process, so no external dependency is added.

• WASM modules follow the proxy-wasm spec and can be written in Rust (the primary choice), C++, TinyGo, and more. Compiling OPA's Rego to WASM is a known combo
• Remaining problem (operational weight): hard to debug, a bug takes the proxy down with it, and you must design module distribution and versioning yourself. There is a real distance between "can write it" and "can keep operating it"

Pattern 3: External Delegation via ext_authz

Problem: The decision needs external state at request time — introspection, consent status, a permission DB, revocation lists. None of this can be pre-distributed to the proxy, so inline evaluation is impossible in principle.

Solution: The ext_authz filter queries an external authorization service on every request. The proxy stays a pure PEP; decisions belong to the PDP.

Figure: The ext_authz flow, steps ① through ④. On Denied, it stops at ③ and the client receives a 403 or similar

Design points:

• The interface is a single-method contract: with gRPC, Check(CheckRequest) → CheckResponse. The authorization service can be written in any language; using OPA as the ext_authz server is a standard setup
• Push endpoint knowledge into policy: the table of path patterns × required scopes / consent belongs in the PDP's policy data (Rego, if OPA) — not in the interface. Keep the interface as "facts in → verdict out" and it survives API growth
• Portability: Istio's AuthorizationPolicy CUSTOM action also calls ext_authz under the hood, so the same authorization service is reusable across ECS / EKS and across gateway swaps
• Latency: short-TTL caching of verdicts + placing the authz service as a sidecar / on the same node (sub-millisecond over localhost)
• Remaining problem (the failure-mode dilemma): when the external PDP is down, you must explicitly choose fail-close (deny all; the PDP becomes a SPOF) or fail-open (let everything through; security drops). Inlining (Patterns 1 and 2) eliminates the dilemma itself — the essential advantage of inline is "fail-close-grade safety without sacrificing availability"

In practice a two-stage stack is common: the inline layer cheaply rejects the obviously bad (invalid signature, expired, missing scope), and only what passes goes to external decision-making.

Pattern 4: Phantom Token (Token Exchange at the Edge)

Problem: With opaque (reference) tokens, every verification requires introspection. Every request carries a serial dependency "gateway → authorization server": availability degrades multiplicatively and the authorization server caps the whole system's throughput. But migrating fully to JWTs sacrifices revocation immediacy — a withdrawn consent stays live until expiry.

Solution: The Phantom Token pattern localizes the serial dependency to "once, at the edge." Exchange the opaque token for a JWT at the edge; everything inside is self-contained local JWT verification.

Figure: The Phantom Token flow. Steps ③④ occur only on first sight of a token and on TTL expiry; on a cache hit the flow completes with ②→⑤

Design points:

• One call is enough: have the introspection response itself be a JWT (RFC 9701: JWT Response for OAuth Token Introspection). Verification and acquisition are the same event in the same round trip
• No signature check needed at the edge: the JWT arrives directly from the authorization server over a trusted mTLS / TLS channel. Signature verification does its work inside — each service sidecar's jwt_authn — which dovetails with the "edge + internal re-verification" architecture
• Caching: TTL cache keyed by opaque token compresses actual calls to "first sight + TTL expiry" per token
• The other lineage is Token Exchange (RFC 8693) — issuing a new token, suited to active transformations like narrowing aud to the destination service. For plain opaque→JWT conversion, RFC 9701 is the shortest path
• Remaining problem: RFC 9701 / RFC 8693 support varies by authorization server product. Without it you end up minting JWTs yourself — inheriting key management and getting heavy fast — so put it in the requirements at server selection time

The Deciding Axis for Token Strategy: Revocation SLA

The choice between introspection (opaque) and JWT ultimately comes down to the revocation SLA — how much delay you tolerate before a withdrawn consent takes effect. Introspection's serial dependency was the price of real-time revocation checking.

Part 2 Summary: Problems and Solutions

---

Part 3: Comparison with OSS Gateways That Need No Custom Authorization Service

The Envoy setups so far assumed, in Patterns 3 and 4, that you write an authorization service (adapter). It is only a few hundred lines of glue — introspection calls, certificate-binding checks, header propagation — but it is still one more independent component.

Meanwhile, OSS gateways like Kong, APISIX, Tyk, and KrakenD ship authentication and authorization as bundled plugins: JWT verification, OIDC, API keys, ACLs, and rate limiting work with configuration alone — no custom service.

Problem: Which cost do you take — writing and operating a custom authorization service, or living within the constraints of bundled plugins?

The Structural Difference: PEP / PDP Separated vs. Co-located

This choice is also a difference in philosophy.

Figure: Three PEP / PDP placements — separated, co-located, and the hybrid that takes the best of both: logical separation, physical co-location

• Envoy + ext_authz: PEP and PDP are separated. Authorization logic is an independent service outside the gateway
• Kong / APISIX etc.: PEP and PDP are co-located. Authorization logic runs as plugins (Lua etc.) inside the gateway process

Co-location has historically been the mainstream, and its virtues are fewer hops and simplicity. But it comes with three couplings:

1. Change lifecycle coupling — fixing authorization logic = redeploying the gateway. The more often policy changes, the more it hurts
2. Failure and resource coupling — a buggy or overloaded authorization plugin takes the whole data plane down with it. No fault isolation for authorization
3. Team ownership coupling — code written by the authorization team (security / identity platform) lands in the heart of the platform team's infrastructure. Deploy rights and review responsibility get tangled

In Istio's case these couplings are mitigated by CRDs: policies are declarative configuration dynamically distributed by istiod over xDS, so no proxy redeploy (resolves coupling 1); CRDs are namespaced and ownership separates via Kubernetes RBAC (resolves coupling 3). Part 2's principle — "inline whatever configuration can express; push out only what it can't" — holds because of this mitigation.

Comparison Table

Which to Choose

Bundled OSS gateway fits when:

• Authorization needs are combinations of standard patterns (JWT verification, scope control, API keys, IP restriction, rate limiting)
• No proprietary data (consent, contract tiers, custom permission models) feeds the decisions
• A small team that doesn't want to own an extra authorization component
• You value the surrounding features — admin UI, API catalog

Envoy + custom authorization fits when:

• Decisions must consult external state at request time — FAPI compliance, custom consent verification
• The same authorization logic should be reused in multiple places (edge gateway, mesh, a future different gateway)
• Authorization changes must be independent of gateway deploys (high change frequency, different owning team)
• You already run Istio / Envoy Gateway and want one data plane technology

The middle ground: logical separation, physical co-location. Place the PDP (OPA etc.) as a sidecar in the same Pod / task as the gateway: hop cost is near zero over localhost, while change-lifecycle and failure isolation are preserved. Consider this before embedding authorization logic into the gateway process — you'll regret it less. With OPA, policy distribution runs through the Bundle API under GitOps, enabling the ownership split "platform team owns the runtime; security / service owners own the policy content."

To be fair, Kong and APISIX also have external-delegation plugins (forward-auth style), so external PDPs are not Envoy-exclusive. But Envoy's ext_authz speaks the same contract across the whole ecosystem — Istio and Envoy Gateway included — which makes its portability meaningfully stronger in practice.

---

Overall Summary: Problem→Solution Table and Decision Flow

All the problems and solutions in one place:

Finally, the whole decision flow as one figure.

Figure: The full selection flow. The upper half decides the gateway architecture; the lower half decides the authorization implementation

Building a gateway with Envoy is not buying a finished product — it is assembling enforcement points from parts. In exchange, you cannot escape the authorization design decisions: which layer decides what, where the decision's data lives, which way to fail. Conversely, for systems that want those decisions made explicitly — finance, multi-tenant platforms, regulated industries — that very freedom is the reason to choose it. Start light with a bundled-plugin gateway while standard patterns suffice, and move to ext_authz the moment external-state decisions or portability demands appear. That such a migration path exists at all is the value of PEP / PDP separation as a design.