Securing the Gateway: A Deep Dive Into Envoy Gateway's Advanced Security Pol... Huabing (Robin) Zhao

Securing the Gateway: A Deep Dive Into Envoy Gateway's Advanced Security Policies

Speakers: Huabing (Robin) Zhao, Maintainer of Envoy Gateway, Contributor to Envoy

Conference: KubeCon EU

YouTube: https://www.youtube.com/watch?v=x8wEo6ZDT1g

Overview

This talk, presented by Huabing (Robin) Zhao, a maintainer of the Envoy Gateway project and a contributor to Envoy, delves into the advanced security policies offered by Envoy Gateway. The session focuses on how developers and operators can leverage these policies to secure ingress traffic for applications deployed in Kubernetes, specifically highlighting OpenID Connect (OIDC) authentication and authorization. While security at the gateway level might not be the "flashiest" topic compared to emerging trends like AI, Zhao emphasizes its foundational importance for any application, making it a critical area for discussion and practical implementation.

The core problem addressed is the inherent complexity of configuring Envoy Proxy directly. While Envoy is a powerful and flexible edge and service proxy, manually configuring it for an API gateway can involve thousands of lines of intricate YAML, leading to potential errors and operational overhead. Envoy Gateway abstracts this complexity, providing a Kubernetes-native control plane that simplifies the deployment and management of Envoy Proxy using the standard Kubernetes Gateway API. This talk showcases how Envoy Gateway extends the Gateway API with custom security policies, making robust security configurations accessible and manageable.

The significance of this talk lies in its practical approach to securing modern cloud-native applications. By demonstrating how to declaratively define and apply advanced security measures like OIDC authentication and fine-grained authorization directly at the gateway, Zhao illustrates a powerful paradigm shift. This approach not only centralizes security enforcement but also offloads critical security concerns from individual application services, allowing developers to focus on business logic while ensuring that traffic entering the cluster is properly authenticated and authorized. The ability to integrate with various identity providers and define granular access rules makes Envoy Gateway a compelling solution for robust edge security.

Background

▶ Watch: Introduction and importance of gateway security (0:00)

The foundation of Envoy Gateway lies in Envoy Proxy, a high-performance open-source edge and service proxy designed for cloud-native applications. Envoy Proxy is renowned for its flexibility, extensibility, and rich feature set, including advanced load balancing, traffic management, and robust security capabilities. However, configuring Envoy Proxy directly, especially for complex API gateway use cases, can be notoriously challenging. It often requires deep knowledge of Envoy's intricate configuration language, which can lead to verbose and error-prone YAML files, hindering adoption and increasing operational burden.

To address this complexity, the Envoy Gateway project was introduced as a control plane specifically designed to simplify the deployment and management of Envoy Proxy as an API gateway. Envoy Gateway leverages the Kubernetes Gateway API, a standardized, extensible API for managing ingress traffic in Kubernetes clusters. The Gateway API provides a declarative way to configure common networking resources like Gateway (representing the entry point for traffic) and HTTPRoute (defining routing rules to backend services). Envoy Gateway acts as an implementation of this standard, automatically translating Gateway API resources into the underlying Envoy Proxy configuration.

A key innovation of the Gateway API, and one that Envoy Gateway fully embraces, is the concept of Policy Attachment. This mechanism allows users to define custom policies that enhance the capabilities of core Gateway API resources (like Gateway or HTTPRoute) without directly modifying those resources. Instead, policies are "attached" to these core objects, extending their functionality in a modular and flexible manner. This design principle is crucial for how Envoy Gateway implements its advanced security features. By defining Security Policies as custom resources, users can declaratively specify security requirements—such as authentication, authorization, and Cross-Origin Resource Sharing (CORS)—and attach them to their gateways or routes. This approach isolates security logic, simplifies its management, and promotes reusability, making it much easier to enforce consistent security postures across an entire application landscape. The problem solved is the need for a robust, flexible, and easily manageable security layer for ingress traffic in Kubernetes, moving beyond the limitations of manual Envoy configuration or less feature-rich ingress solutions.

Key Findings

▶ Watch: Policy attachment: Extending Gateway API capabilities (3:00)

Envoy Gateway's approach to security is centered around its Security Policy custom resource, which provides a declarative and highly flexible mechanism for managing all aspects of gateway traffic security. This policy-driven model significantly simplifies the traditionally complex task of securing an Envoy Proxy.

One of the primary findings is the two-level policy application model. Security policies can be applied at the Gateway level, establishing a global security posture that protects all routes under that gateway. Alternatively, more specific policies can be defined and attached to individual HTTPRoute resources, allowing for fine-grained control and the ability to override global settings for sensitive endpoints or specific application requirements. This hierarchical structure offers immense flexibility, enabling administrators to define broad organizational policies while allowing application teams to tailor security for their specific services.

The talk highlights the comprehensive suite of security features supported by these policies:

• CORS (Cross-Origin Resource Sharing): Security policies enable explicit definition of allowed origins, headers, and methods. This is crucial for front-end applications making API calls, preventing common browser-based security restrictions.
• Authentication: Envoy Gateway supports a range of authentication methods, catering to various use cases:
• HTTP Basic Authentication: A simple, though less secure, method often used for internal services.
• JWT (JSON Web Token) Token Authentication: A more modern and scalable approach, verifying tokens issued by trusted identity providers.
• API Key Authentication: Ideal for machine-to-machine communication or scenarios where simpler access control is needed.
• OIDC (OpenID Connect) Authentication: The focus of the demo, supporting integration with standard OIDC identity providers for robust user authentication.
• Authorization: Beyond just authenticating users, security policies allow for granular control over what authenticated users can do. Authorization can be based on:
• Client IP Address: Restricting access based on the source IP of the request.
• JWT Claims: Extracting information (claims) from a JWT (like sub for user ID or email) to make authorization decisions.
• Basic Authentication Username: Using the username from HTTP Basic Auth for access control.
• HTTP Headers and Methods: Defining rules based on specific HTTP headers or request methods (e.g., only allowing GET requests to a certain path).
• External Authorization (ExtAuth): For highly custom or complex authorization logic, security policies can be configured to hook into an external authorization system. This allows organizations to integrate their existing authorization services with Envoy Gateway, providing maximum flexibility.

A significant finding is the declarative simplicity. As demonstrated, even advanced configurations like OIDC authentication and authorization can be defined with a minimal amount of YAML—the speaker specifically mentions configurations as concise as "10 times of file" or "11 lines of YAML." This drastically reduces the configuration overhead compared to direct Envoy Proxy configuration, making advanced security features more accessible to a broader audience. These policies are then automatically translated by the Envoy Gateway control plane into the necessary Envoy Proxy configurations, abstracting away the underlying complexity.

Technical Deep Dive

▶ Watch: Overview of Envoy Gateway's security policy features (4:00)

The technical architecture underpinning Envoy Gateway's security policies involves a sophisticated interplay between the Kubernetes Gateway API, custom resources, and Envoy Proxy itself. At its core, Envoy Gateway acts as a control plane. When an administrator creates or updates Kubernetes Gateway API resources (like Gateway and HTTPRoute) or Envoy Gateway's custom resources (like SecurityPolicy and BackendTLSPolicy), the control plane intercepts these definitions. It then translates these high-level, declarative specifications into the precise, low-level configuration required by Envoy Proxy. This configuration is subsequently applied to the Envoy Proxy instances, which then handle the actual data plane traffic.

The Security Policy custom resource is central to this system. Its structure allows for comprehensive definition of security rules:

• targetRef: This field specifies the target resource (either a Gateway or an HTTPRoute) to which the policy applies. This is fundamental to the two-level policy application model, allowing policies to be scoped globally or to specific routes.
• cors: The CORS configuration block includes fields like allowOrigin, allowHeader, and allowMethod. These fields accept lists of values, enabling fine-grained control over which origins are permitted to make cross-origin requests, what HTTP headers can be included, and which HTTP methods are allowed.
• authentication: This block supports various authentication methods:
• basicAuth: Configured with a reference to a Kubernetes Secret containing base64-encoded username:password pairs.
• jwt: Defined by providers, each specifying an issuer URL, audiences (optional), and the location of the JSON Web Key Set (JWKS) for signature verification.
• apiKey: Configured to extractFrom a specific HTTP header and validated against secrets stored in a Kubernetes Secret.
• oidc: This is the most detailed authentication configuration, requiring provider (the OIDC issuer URL), clientID, clientSecret (from a Kubernetes Secret), redirectURL, scopes (e.g., openid, email, profile), and optional fields like cookieName (to store the ID token) and localPath (for internal OIDC redirects).

• authorization: This block defines rules for access control:
• rules: A list of rules, each containing matches (conditions based on client IP, JWT claims, HTTP headers, or methods) and an action (allow or deny). For JWT claims, specific claims like sub (subject) or email can be extracted and matched against expected values.
• externalAuth: Allows integration with an external authorization service by specifying a backendRef to a Kubernetes service or an external endpoint. This enables complex, custom authorization logic to be offloaded to a dedicated service.

The OIDC authentication flow demonstrated in the talk is a standard OAuth 2.0/OIDC Authorization Code Flow, orchestrated by Envoy Proxy:

1. An administrator deploys a SecurityPolicy with OIDC configuration, targeting an HTTPRoute.
2. Envoy Gateway translates this into Envoy Proxy configuration, setting up an OIDC filter.
3. A user's browser sends a request to the HTTPRoute without an OIDC token.
4. Envoy Proxy, detecting the lack of authentication, redirects the user's browser to the OIDC provider's (e.g., Amazon Cognito) login page.
5. The user authenticates with the OIDC provider.
6. Upon successful authentication, the OIDC provider redirects the user's browser back to Envoy Proxy with an authorization code.
7. Envoy Proxy, acting as a confidential client, uses this authorization code to communicate directly with the OIDC provider's token endpoint (a back-channel request).
8. The OIDC provider exchanges the authorization code for an ID Token (a JWT) and potentially an access token and refresh token.
9. Envoy Proxy validates the ID Token (signature, issuer, audience, expiry) and, if valid, authenticates the user. The ID Token (or claims derived from it) might be stored in a cookie.
10. The original request is then forwarded to the backend application. Crucially, this entire authentication process is transparent to the backend application, which receives an already authenticated request.

For authorization with JWT claims, the SecurityPolicy can define rules that inspect claims embedded within the ID Token. If the ID Token is stored in a cookie (as specified by cookieName in the OIDC configuration), Envoy Proxy can decrypt and parse it. Authorization rules then evaluate specific claims, such as sub (the unique user identifier) or email, against predefined values. For example, a rule might state: "If sub is 'user-123' AND email is '[email protected]', then allow access; otherwise, deny."

The talk also addresses scenarios involving self-signed certificates or custom identity providers like Keycloak. For these cases, Envoy Gateway introduces the Backend TLS Policy. This policy allows administrators to specify custom Certificate Authority (CA) certificates (from a Kubernetes ConfigMap) that Envoy Proxy should trust when establishing TLS connections with external or internal OIDC providers. This is particularly useful when the OIDC provider is not publicly trusted or uses internal CAs. The SecurityPolicy's OIDC configuration can reference a Backend resource, which defines the hostname and port of the OIDC provider, allowing for precise control over the communication path and TLS trust chain. This ensures secure communication even with non-standard identity provider setups.

Demo / Proof of Concept

▶ Watch: OIDC authentication flow with Amazon Cognito explained (8:50)

The talk featured a live demonstration showcasing the practical application of Envoy Gateway's security policies, running on a local kind (Kubernetes in Docker) cluster. The speaker pre-loaded the cluster with Envoy Gateway and a simple backend application named "Hello KubeCon" to save time, acknowledging potential conference Wi-Fi issues.

The demo environment consisted of:

• An Envoy Gateway instance installed in the envoy-gateway-system namespace, automatically setting up the Envoy Proxy data plane.
• A backend application deployed in the default namespace, serving a "Hello KubeCon" message.
• A Gateway resource configured to listen on port 443 for HTTPS traffic.
• An HTTPRoute named my-app under this gateway, routing traffic matching the /my-app prefix to the backend service.

Demo 1: OIDC Authentication with Amazon Cognito

The first part of the demo focused on implementing OIDC authentication using Amazon Cognito as the identity provider. A SecurityPolicy was created and applied to the my-app HTTPRoute. The configuration for OIDC authentication was remarkably concise, totaling only about 11 lines of YAML. This policy specified:

• The OIDC provider URL (Amazon Cognito's issuer endpoint).
• clientID and clientSecret (retrieved from a Kubernetes Secret).
• scopes (e.g., openid, email, profile).
• A cookieName to store the ID token after successful authentication.
• localPath for internal redirects.

When the speaker attempted to access the /my-app endpoint in the browser without an existing session, Envoy Gateway intercepted the request. As expected, it redirected the browser to the Amazon Cognito login page, demonstrating that authentication was required. After successfully entering credentials on the Cognito page, the user was redirected back to the /my-app endpoint, and the "Hello KubeCon" message was displayed, confirming successful OIDC authentication handled entirely by the gateway. The speaker expressed relief when this part of the demo worked flawlessly, highlighting the power of such a minimal configuration.

Demo 2: OIDC Authorization with JWT Claims

Building on the successful authentication, the second part of the demo extended the SecurityPolicy to include fine-grained authorization based on JWT claims. The existing OIDC policy was modified to add a jwt authorization section. This new section instructed Envoy Gateway to:

• Retrieve the ID token from the specified cookie (cookieName).
• Decrypt and parse the JWT to extract its claims.
• Apply authorization.rules based on these claims.

The speaker demonstrated a rule that allowed access only if the sub claim matched a specific unique ID and the email claim matched a particular email address (e.g., [email protected]). When the modified policy was applied, and the user, authenticated with the correct sub and email claims, accessed the /my-app route, access was granted. To prove the authorization was working, the speaker then modified the SecurityPolicy by changing the expected email address in the rule. Upon re-accessing the route, the request was denied, clearly illustrating that Envoy Gateway was enforcing authorization based on the specific JWT claims. The speaker also mentioned that authorization could combine other conditions, such as client IP addresses or JWT scopes.

Demo 3: OIDC with Keycloak and Self-Signed Certificates

The final part of the demo addressed a more advanced scenario: integrating OIDC with a self-hosted identity provider, specifically Keycloak, and handling self-signed certificates. Keycloak was deployed within the Kubernetes cluster. The SecurityPolicy was updated to point its OIDC provider to a Backend resource that represented the Keycloak service, rather than a direct URL.

Crucially, a Backend TLS Policy was used in conjunction with the Backend resource. This policy specified the CA certificate for Keycloak, stored in a Kubernetes ConfigMap. This allowed Envoy Gateway to establish a trusted TLS connection with Keycloak, which might be using a self-signed or internal certificate not trusted by public CAs. The speaker noted that while the Backend resource might not be strictly necessary if Keycloak is in the same cluster and reachable via service DNS, it's essential for external Keycloak instances or more complex network setups. The demo showed successful authentication through Keycloak, validating the capability to integrate with custom identity provider deployments and manage their TLS trust.

The speaker made the demo scripts and YAML files available in a GitHub repository, encouraging attendees to try it out, claiming it takes "like five minutes" to set up.

Defensive Implications

▶ Watch: Live demo: OIDC authentication with Envoy Gateway (11:00)

Envoy Gateway's advanced security policies offer significant defensive advantages for organizations operating cloud-native applications, particularly those leveraging Kubernetes.

First and foremost, the implementation provides centralized security enforcement at the gateway level. By pushing authentication and authorization logic to the edge, organizations can ensure that no unauthenticated or unauthorized traffic reaches their backend microservices. This creates a strong defensive perimeter, reducing the attack surface on individual application components.

Secondly, it offloads security responsibilities from application developers. Rather than each microservice needing to implement its own authentication and authorization mechanisms, Envoy Gateway handles these concerns transparently. This not only reduces the development burden but also minimizes the risk of security vulnerabilities arising from inconsistent or flawed implementations across different services. Developers can then focus on core business logic, knowing that the gateway is enforcing a consistent security posture.

The declarative nature of security policies—defined in concise YAML—enhances consistency and auditability. Security teams can review and manage policies across an entire fleet of applications from a single control plane, ensuring that security rules are uniformly applied. This reduces configuration drift and simplifies compliance efforts.

Envoy Gateway's support for a wide array of authentication methods, including OIDC, JWT, API Keys, and HTTP Basic Auth, provides flexibility to integrate with diverse identity management systems. This allows organizations to leverage existing identity providers (like Amazon Cognito, Auth0, Okta, or Keycloak) or to implement specific machine-to-machine authentication strategies, adapting to various operational requirements without compromising security.

The fine-grained authorization capabilities are a powerful defensive tool. By allowing rules based on client IP, JWT claims (e.g., user roles, group memberships), HTTP headers, and methods, organizations can implement highly specific access controls. This can prevent lateral movement by restricting access to sensitive endpoints based on user attributes or network context, adhering to the principle of least privilege. The ability to define policies at both the global (Gateway) and specific (HTTPRoute) levels allows for robust default security while enabling necessary overrides for particular use cases, such as an /admin endpoint accessible only by specific users or IPs.

Furthermore, the External Authorization (ExtAuth) feature offers an escape hatch for highly complex or custom authorization logic. If built-in policies are insufficient, organizations can integrate their own authorization services, allowing for sophisticated policy engines to be leveraged at the gateway, such as those based on Open Policy Agent (OPA).

Finally, the explicit configuration for CORS helps mitigate common web vulnerabilities related to cross-origin requests. By precisely defining allowed origins, headers, and methods, Envoy Gateway ensures that browsers enforce legitimate cross-domain interactions, preventing potential data leakage or unauthorized access attempts. The management of TLS trust for backend connections, especially with self-signed certificates via Backend TLS Policy, secures communication channels to internal identity providers, preventing man-in-the-middle attacks.

In summary, Envoy Gateway's security policies empower defenders by providing a robust, flexible, and manageable layer of security at the edge of their Kubernetes clusters, streamlining operations while significantly enhancing the overall security posture.

Key Takeaways

• Simplified Envoy Proxy Security: Envoy Gateway acts as a Kubernetes-native control plane that drastically simplifies the configuration of Envoy Proxy for API gateway security, abstracting away complex Envoy YAML with concise, declarative SecurityPolicy resources.
• Comprehensive Security Feature Set: Envoy Gateway's SecurityPolicy enables robust control over CORS, various authentication methods (OIDC, JWT, API Key, HTTP Basic Auth), and fine-grained authorization based on client IP, JWT claims, HTTP headers/methods, or external authorization systems.
• Flexible Policy Application: Security policies can be applied at two levels: globally to an entire Gateway or specifically to individual HTTPRoute resources, allowing for powerful default security settings with route-specific overrides.
• Seamless OIDC Integration: The platform provides a streamlined way to integrate with standard OIDC identity providers (like Amazon Cognito or Keycloak), handling the full OIDC authentication flow transparently at the gateway and supporting authorization based on JWT claims extracted from ID tokens.
• Custom Identity Provider Support: Envoy Gateway facilitates secure integration with self-hosted or non-publicly trusted OIDC providers (e.g., Keycloak with self-signed certificates) through Backend and Backend TLS Policy resources, enabling custom CA trust for TLS connections.
• Enhanced Defensive Posture: By centralizing and offloading authentication and authorization to the gateway, Envoy Gateway reduces the security burden on application developers, ensures consistent security enforcement, and strengthens the overall defensive perimeter of cloud-native applications.

About the Speaker(s)

Huabing (Robin) Zhao is a prominent figure in the cloud-native networking space. He is a maintainer of the Envoy Gateway project, playing a crucial role in guiding its development and ensuring its stability and feature set. In addition to his leadership on Envoy Gateway, Huabing is also a contributor to the broader Envoy Proxy project, demonstrating his deep expertise and commitment to the Envoy ecosystem. His work focuses on simplifying the adoption and management of Envoy in Kubernetes environments, making advanced traffic management and security capabilities more accessible to developers and operators.